Antony Kalloniatis
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -9,23 +9,27 @@ pipeline_tag: text-classification
|
|
9 |
---
|
10 |
#
|
11 |
#
|
12 |
-
# This model uses the Llama-3 model ("meta-llama/Meta-Llama-3-8B") fine-tuned with 4 bit quantization
|
13 |
-
# Parameter Efficient Fine Tuning - PEFT training, using LoRA and QLoRA adaptations for the task
|
14 |
-
# of Humor Recognition in Greek language.
|
15 |
#
|
16 |
|
17 |
|
18 |
|
19 |
## Model Details
|
20 |
|
21 |
-
The model was pre-trained on Greek Humorous Dataset
|
22 |
-
|
|
|
|
|
|
|
|
|
23 |
## Pre-processing details
|
24 |
|
25 |
-
The text needs to be pre-processed by
|
26 |
-
|
|
|
27 |
|
28 |
## Load Pretrained Model
|
|
|
29 |
|
30 |
```python
|
31 |
from transformers import AutoTokenizer, AutoModelForSequenceClassification
|
|
|
9 |
---
|
10 |
#
|
11 |
#
|
12 |
+
# This model uses the Llama-3 model ("meta-llama/Meta-Llama-3-8B") fine-tuned with 4 bit quantization Parameter Efficient Fine Tuning - PEFT training, using LoRA and QLoRA adaptations for the task of Humor Recognition in Greek language.
|
|
|
|
|
13 |
#
|
14 |
|
15 |
|
16 |
|
17 |
## Model Details
|
18 |
|
19 |
+
The model was pre-trained on Greek Humorous Dataset
|
20 |
+
|
21 |
+
## PEFT Configs
|
22 |
+
* Bits and bytes config for quantization - QLoRA
|
23 |
+
* LoRA config for LoRA adaptation
|
24 |
+
|
25 |
## Pre-processing details
|
26 |
|
27 |
+
The text needs to be pre-processed by:
|
28 |
+
* removing all greek diacritics and punctuations
|
29 |
+
* converting all letters to lowercase
|
30 |
|
31 |
## Load Pretrained Model
|
32 |
+
pad_token needs to be handle since Llama-3 pre-training doesn't have eos_token
|
33 |
|
34 |
```python
|
35 |
from transformers import AutoTokenizer, AutoModelForSequenceClassification
|