Update README.md
Browse files
README.md
CHANGED
@@ -22,45 +22,6 @@ tags:
|
|
22 |
- Availability: The model checkpoint can be accessed on Hugging Face: Tamnemtf/llama-2-7b-vi-oscar_mini
|
23 |
- Model trên được train dựa trên model gốc là ngoan/Llama-2-7b-vietnamese-20k
|
24 |
## How to Use
|
25 |
-
```python
|
26 |
-
# Activate 4-bit precision base model loading
|
27 |
-
use_4bit = True
|
28 |
-
|
29 |
-
# Compute dtype for 4-bit base models
|
30 |
-
bnb_4bit_compute_dtype = "float16"
|
31 |
-
|
32 |
-
# Quantization type (fp4 or nf4)
|
33 |
-
bnb_4bit_quant_type = "nf4"
|
34 |
-
|
35 |
-
# Activate nested quantization for 4-bit base models (double quantization)
|
36 |
-
use_nested_quant = False
|
37 |
-
|
38 |
-
# Load the entire model on the GPU 0
|
39 |
-
device_map = {"": 0}
|
40 |
-
```
|
41 |
-
|
42 |
-
```python
|
43 |
-
compute_dtype = getattr(torch, bnb_4bit_compute_dtype)
|
44 |
-
bnb_config = BitsAndBytesConfig(
|
45 |
-
load_in_4bit=use_4bit,
|
46 |
-
bnb_4bit_quant_type=bnb_4bit_quant_type,
|
47 |
-
bnb_4bit_compute_dtype=compute_dtype,
|
48 |
-
bnb_4bit_use_double_quant=use_nested_quant,
|
49 |
-
)
|
50 |
-
```
|
51 |
-
```python
|
52 |
-
model = AutoModelForCausalLM.from_pretrained(
|
53 |
-
'Tamnemtf/llama-2-7b-vi-oscar_mini',
|
54 |
-
quantization_config=bnb_config,
|
55 |
-
device_map=device_map
|
56 |
-
)
|
57 |
-
model.config.use_cache = False
|
58 |
-
model.config.pretraining_tp = 1
|
59 |
-
|
60 |
-
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
|
61 |
-
tokenizer.pad_token = tokenizer.eos_token
|
62 |
-
tokenizer.padding_side = "right" # Fix weird overflow issue with fp16 training
|
63 |
-
```
|
64 |
```python
|
65 |
# Run text generation pipeline with our next model
|
66 |
prompt = "Canh chua cá lau là món gì ?"
|
@@ -69,9 +30,6 @@ tags:
|
|
69 |
print(result[0]['generated_text'])
|
70 |
```
|
71 |
|
72 |
-
Để ưu tiên cho việc dễ dàng tiếp cận với các sinh viên dưới đây là mẫu ví dụ chạy thử model trên colab bằng T4
|
73 |
-
https://colab.research.google.com/drive/1ME_k-gUKSY2NbB7GQRk3sqz56CKsSV5C?usp=sharing
|
74 |
-
|
75 |
## Conntact
|
76 |
nguyndantdm6@gmail.com
|
77 |
|
|
|
22 |
- Availability: The model checkpoint can be accessed on Hugging Face: Tamnemtf/llama-2-7b-vi-oscar_mini
|
23 |
- Model trên được train dựa trên model gốc là ngoan/Llama-2-7b-vietnamese-20k
|
24 |
## How to Use
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
25 |
```python
|
26 |
# Run text generation pipeline with our next model
|
27 |
prompt = "Canh chua cá lau là món gì ?"
|
|
|
30 |
print(result[0]['generated_text'])
|
31 |
```
|
32 |
|
|
|
|
|
|
|
33 |
## Conntact
|
34 |
nguyndantdm6@gmail.com
|
35 |
|