danieldux commited on
Commit
9428512
1 Parent(s): 00e1647

Refactor metric card instructions and fix formatting in README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -10
README.md CHANGED
@@ -14,21 +14,16 @@ pinned: false
14
 
15
  # Metric Card for ISCO-08 Hierarchical Accuracy Measure
16
 
17
- ***Module Card Instructions:*** *Fill out the following subsections. Feel free to take a look at existing metric cards if you'd like examples.*
18
-
19
  ## Metric Description
20
 
21
  The measure accounts for the hierarchical structure of the ISCO-08 classification scheme by rewarding partial correctness in classification, penalizing distant errors, and applying higher penalties for higher-level errors. It is designed to evaluate the accuracy of predictions within a hierarchical occupation classification system, such as the ISCO-08 scheme, which organizes occupations into a tree structure with four levels: major group, sub-major group, minor group, and unit group.
22
 
23
- 1. **Partial correctness in classification**: Misclassification within the same major group (e.g., within "Professionals") is penalized less than misclassification across major groups (e.g., from "Professionals" to "Managers"). This acknowledges the similarity in skill level and specialization within major groups.
24
-
25
- 2. **Penalization of distant errors**:
26
  2.1 Correct classification into a more specific group (e.g., "Medical doctors" within "Health professionals") is preferred over a broader classification (e.g., the entire "Health professionals" group).
27
  2.2 Conversely, incorrect classification into a more specific but unrelated group (e.g., "Nursing and midwifery professionals" when the correct classification is "Engineering professionals") is penalized more heavily than an incorrect but broader classification.
28
-
29
- 3. **Higher penalties for higher-level errors**: Misclassifications at the major group level are more severely penalized than errors within sub-major, minor, or unit groups, reflecting the greater disparity in skill level and specialization.
30
-
31
- 4. **Extension with ancestors in the hierarchy**: For the hierarchical precision ($hP$) and recall ($hR$) calculations, an occupation's classification is extended to include not only its specific group but also the parent groups to which it belongs, excluding the root level. This allows for an evaluation that recognizes partial correctness based on the occupation's position within the ISCO-08 hierarchy.
32
 
33
  ### Original description
34
 
@@ -39,7 +34,7 @@ The measure accounts for the hierarchical structure of the ISCO-08 classificatio
39
 
40
  The features described are accomplished by pairing hierarchical variants of precision ($hP$) and recall ($hR$) to form a hierarchical F1 ($hF_β$) score where each sample belongs not only to its class (e.g., a unit group level code), but also to all ancestors of the class in a hierarchical graph (i.e., the minor, sub-major, and major group level codes).
41
 
42
- Hierarchical precision can be computed with `$hP = \frac{| \v{C}_i ∩ \v{C}^′_i|} {|\v{C}^′_i |} = \frac{1}{2}$`
43
 
44
  Hierarchical recall can be computed with:
45
  `$hR = \frac{| \v{C}_i ∩ \v{C}^′_i|} {|\v{C}_i |} = \frac{1}{2}$`
 
14
 
15
  # Metric Card for ISCO-08 Hierarchical Accuracy Measure
16
 
 
 
17
  ## Metric Description
18
 
19
  The measure accounts for the hierarchical structure of the ISCO-08 classification scheme by rewarding partial correctness in classification, penalizing distant errors, and applying higher penalties for higher-level errors. It is designed to evaluate the accuracy of predictions within a hierarchical occupation classification system, such as the ISCO-08 scheme, which organizes occupations into a tree structure with four levels: major group, sub-major group, minor group, and unit group.
20
 
21
+ 1. Partial correctness in classification: Misclassification within the same major group (e.g., within "Professionals") is penalized less than misclassification across major groups (e.g., from "Professionals" to "Managers"). This acknowledges the similarity in skill level and specialization within major groups.
22
+ 2. Penalization of distant errors:
 
23
  2.1 Correct classification into a more specific group (e.g., "Medical doctors" within "Health professionals") is preferred over a broader classification (e.g., the entire "Health professionals" group).
24
  2.2 Conversely, incorrect classification into a more specific but unrelated group (e.g., "Nursing and midwifery professionals" when the correct classification is "Engineering professionals") is penalized more heavily than an incorrect but broader classification.
25
+ 3. Higher penalties for higher-level errors: Misclassifications at the major group level are more severely penalized than errors within sub-major, minor, or unit groups, reflecting the greater disparity in skill level and specialization.
26
+ 4. Extension with ancestors in the hierarchy: For the hierarchical precision ($hP$) and recall ($hR$) calculations, an occupation's classification is extended to include not only its specific group but also the parent groups to which it belongs, excluding the root level. This allows for an evaluation that recognizes partial correctness based on the occupation's position within the ISCO-08 hierarchy.
 
 
27
 
28
  ### Original description
29
 
 
34
 
35
  The features described are accomplished by pairing hierarchical variants of precision ($hP$) and recall ($hR$) to form a hierarchical F1 ($hF_β$) score where each sample belongs not only to its class (e.g., a unit group level code), but also to all ancestors of the class in a hierarchical graph (i.e., the minor, sub-major, and major group level codes).
36
 
37
+ Hierarchical precision can be computed with $hP=\frac{|\v{C}_i ∩\v{C}^′_i|}{|\v{C}^′_i|}=\frac{1}{2}$
38
 
39
  Hierarchical recall can be computed with:
40
  `$hR = \frac{| \v{C}_i ∩ \v{C}^′_i|} {|\v{C}_i |} = \frac{1}{2}$`