scfengv commited on
Commit
65cddc8
1 Parent(s): c83034c

Update README

Browse files
Files changed (1) hide show
  1. README.md +74 -14
README.md CHANGED
@@ -33,28 +33,86 @@ model-index:
33
  type: F1 score (Macro)
34
  value: 0.970818
35
  ---
36
- # Model Card for Model ID
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
37
 
38
- <!-- Provide a quick summary of what the model is/does. -->
39
 
40
- ## Model Details
 
 
 
 
 
 
41
 
42
- ### Model Description
43
 
44
- <!-- Provide a longer summary of what this model is. -->
 
 
 
 
 
 
 
 
 
 
45
 
46
- - **Developed by:** [scfengv](https://huggingface.co/scfengv)
47
- - **Model type:** BERT Multi-label Text Classification
48
- - **Language:** Chinese (Zh)
49
- - **Finetuned from model:** [google-bert/bert-base-chinese](https://huggingface.co/google-bert/bert-base-chinese)
50
 
51
- ### Model Sources
 
 
 
52
 
53
- - **Repository:** [scfengv/NLP-Topic-Modeling-for-TVL-livestream-comments](https://github.com/scfengv/NLP-Topic-Modeling-for-TVL-livestream-comments)
 
54
 
55
- ## How to Get Started with the Model
 
 
 
 
 
 
56
 
57
- Use the code below to get started with the model.
58
 
59
  ```python
60
  import torch
@@ -76,7 +134,9 @@ with torch.no_grad():
76
  print(predictions)
77
  ```
78
 
79
- ## Training Details
 
 
80
 
81
  - **Hardware Type:** NVIDIA Quadro RTX8000
82
  - **Library:** PyTorch
 
33
  type: F1 score (Macro)
34
  value: 0.970818
35
  ---
36
+ # Model Details of TVL_GeneralLayerClassifier
37
+
38
+ ## Base Model
39
+ This model is fine-tuned from [google-bert/bert-base-chinese](https://huggingface.co/google-bert/bert-base-chinese).
40
+
41
+ ## Model Architecture
42
+ - **Type**: BERT-based text classification model
43
+ - **Hidden Size**: 768
44
+ - **Number of Layers**: 12
45
+ - **Number of Attention Heads**: 12
46
+ - **Intermediate Size**: 3072
47
+ - **Max Sequence Length**: 512
48
+ - **Vocabulary Size**: 21,128
49
+
50
+ ## Key Components
51
+ 1. **Embeddings**
52
+ - Word Embeddings
53
+ - Position Embeddings
54
+ - Token Type Embeddings
55
+ - Layer Normalization
56
+
57
+ 2. **Encoder**
58
+ - 12 layers of:
59
+ - Self-Attention Mechanism
60
+ - Intermediate Dense Layer
61
+ - Output Dense Layer
62
+ - Layer Normalization
63
+
64
+ 3. **Pooler**
65
+ - Dense layer for sentence representation
66
+
67
+ 4. **Classifier**
68
+ - Output layer with 4 classes
69
+
70
+ ## Training Hyperparameters
71
 
72
+ The model was trained using the following hyperparameters:
73
 
74
+ ```
75
+ Learning rate: 1e-05
76
+ Batch size: 32
77
+ Number of epochs: 10
78
+ Optimizer: Adam
79
+ Loss function: torch.nn.BCEWithLogitsLoss()
80
+ ```
81
 
82
+ ## Training Infrastructure
83
 
84
+ - **Hardware Type:** NVIDIA Quadro RTX8000
85
+ - **Library:** PyTorch
86
+ - **Hours used:** 2hr 56mins
87
+
88
+ ## Model Parameters
89
+ - Total parameters: ~102M (estimated)
90
+ - All parameters are in 32-bit floating point (F32) format
91
+
92
+ ## Input Processing
93
+ - Uses BERT tokenization
94
+ - Supports sequences up to 512 tokens
95
 
96
+ ## Output
97
+ - 4-class multi-label classification
 
 
98
 
99
+ ## Performance Metrics
100
+ - Accuracy score: 0.952902
101
+ - F1 score (Micro): 0.968717
102
+ - F1 score (Macro): 0.970818
103
 
104
+ ## Training Dataset
105
+ This model was trained on the [scfengv/TVL-general-layer-dataset](https://huggingface.co/datasets/scfengv/TVL-general-layer-dataset).
106
 
107
+ ## Testing Dataset
108
+
109
+ - [scfengv/TVL-general-layer-dataset](https://huggingface.co/datasets/scfengv/TVL-general-layer-dataset)
110
+ - validation
111
+ - Remove Emoji
112
+ - Emoji2Desc
113
+ - Remove Punctuation
114
 
115
+ ## Usage
116
 
117
  ```python
118
  import torch
 
134
  print(predictions)
135
  ```
136
 
137
+ ## Additional Notes
138
+ - This model is specifically designed for TVL general layer classification tasks.
139
+ - It's based on the Chinese BERT model, indicating it's optimized for Chinese text.
140
 
141
  - **Hardware Type:** NVIDIA Quadro RTX8000
142
  - **Library:** PyTorch