VietnamAIHub
commited on
Commit
•
a5519c6
1
Parent(s):
e9a5245
update_readme
Browse files
README.md
CHANGED
@@ -1,28 +1,22 @@
|
|
1 |
-
# Llama-30b with LoRA Adapters
|
2 |
|
3 |
-
[Llama-30b with LoRA Adapters]
|
4 |
|
5 |
-
This repository contains a Llama-30b model fine-tuned with QLoRA (Quantization Low-Rank Adapter) adapters. The adapter is a plug-and-play tool that enables the LLaMa model to perform well in many Vietnamese NLP tasks.
|
|
|
|
|
6 |
|
7 |
## Model Overview
|
8 |
|
9 |
-
The Llama-
|
10 |
|
11 |
## Dataset and Fine-Tuning
|
12 |
|
13 |
The LLaMa model was fine-tuned on over 200K instructions from various sources to improve its ability to understand and generate text for different tasks. The instruction dataset comprises data from the following sources:
|
14 |
-
|
15 |
-
- Alpaca 52
|
16 |
-
- LiMA 1K
|
17 |
-
- Dolly 15K
|
18 |
-
- VietHealth
|
19 |
-
- WikiHow
|
20 |
-
- GPT4ALL
|
21 |
-
- VietQuAD
|
22 |
|
23 |
## Loading the Model
|
24 |
|
25 |
-
To load the fine-tuned Llama-30b model with LoRA adapters, follow the code snippet below:
|
26 |
|
27 |
```python
|
28 |
import torch
|
|
|
1 |
+
# Vietnamese Llama-30b with LoRA Adapters
|
2 |
|
|
|
3 |
|
4 |
+
This repository contains a Vietnamese Llama-30b model fine-tuned with QLoRA (Quantization Low-Rank Adapter) adapters. The adapter is a plug-and-play tool that enables the LLaMa model to perform well in many Vietnamese NLP tasks.
|
5 |
+
|
6 |
+
Project Github page: [Github](https://github.com/VietnamAIHub/Vietnamese_LLMs)
|
7 |
|
8 |
## Model Overview
|
9 |
|
10 |
+
The Vietnamese Llama-30B model is a large language model capable of generating meaningful text and can be used in a wide variety of natural language processing tasks, including text generation, sentiment analysis, and more. By using LoRA adapters, the model achieves better performance on low-resource tasks and demonstrates improved generalization.
|
11 |
|
12 |
## Dataset and Fine-Tuning
|
13 |
|
14 |
The LLaMa model was fine-tuned on over 200K instructions from various sources to improve its ability to understand and generate text for different tasks. The instruction dataset comprises data from the following sources:
|
15 |
+
Dataset is Coming Soon
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
16 |
|
17 |
## Loading the Model
|
18 |
|
19 |
+
To load the fine-tuned Vietnamese Llama-30b model with LoRA adapters, follow the code snippet below:
|
20 |
|
21 |
```python
|
22 |
import torch
|