Hierarchical Classification

#machine learning #supervised learning #classification #multilabel #multiclass

Hierarchical Classification Problem

Hierarchical classification labels involves hierarchical class labels. The hierarchical class labels maybe predefined or inferred. 1

Class Taxonomy

A hierarchical classification problem comes with a class taxonomy.

  • “IS-A” operator: $\prec$,
  • “IS-NOT-A” operator: $\nprec$

A IS-A relationship of the labels $c_a$ class set $C$ is

  • one root $R$ in the tree,
  • asymmetric, i.e., $c_i \prec c_j$ and $c_j\prec c_i$ can not be both true,
  • anti-reflexive, i.e., $c_i \nprec c_i$,
  • transitive, i.e., $c_i \prec c_j$ and $c_j\prec c_k$ $\Rightarrow$ $c_i \prec c_k$.

There are different representations of the hierarchical taxonomies.

Figure 2 in Silla2011, showing the difference between tree taxonomy and DAG taxonomy.

Figure 2 in Silla2011, showing the difference between tree taxonomy and DAG taxonomy.

  • A classifier doesn’t necessarily classify the labels to the leaves on the tree all the time.
  • A classifier can utilize the tree of DAG differently:
    • top-down: local classifiers that only utilizes partial hierarchical information, e.g., the prediction of the children will use the prediction of the parent as input; Classifier Chains Classifier chains is a method to predict hierarchical class labels ;
    • big-bang: global classifiers with each classifier utilizes the whole paths;
    • flat: classifiers ignores the hierarchical relations and predicting the leaves.

Flat Classification

Classify the leaves and forget about the hierarchies.

  1. The hierarchical information is not being used.

Global Classifiers

Also called Big-Bang approach, golbal classifiers can utilize

  1. Clusters in the taxnomony,
  2. Multi-label classification with full paths,
  3. Revise models, e.g., loss function calculation to include hierarchical information.

  1. Silla2011 Silla CN, Freitas AA. A survey of hierarchical classification across different application domains. Data Min Knowl Discov. 2011;22: 31–72. doi:10.1007/s10618-010-0175-9 ↩︎

Published: by ;

LM (2021). 'Hierarchical Classification', Datumorphism, 03 April. Available at: https://datumorphism.leima.is/wiki/machine-learning/classification/hierarchical-classification/.

Current Ref:

  • wiki/machine-learning/classification/hierarchical-classification.md