Hugging Face's logo

RichardErkhov
/
yentinglin_-_Taiwan-LLM-7B-v2.0-base-gguf

GGUF
Inference Endpoints
conversational
Model card Files Community
yentinglin_-_Taiwan-LLM-7B-v2.0-base-gguf
File size: 8,212 Bytes 
bbb1b2e
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
 
Quantization made by Richard Erkhov.

[Github](https://github.com/RichardErkhov)

[Discord](https://discord.gg/pvy7H8DZMG)

[Request more models](https://github.com/RichardErkhov/quant_request)


Taiwan-LLM-7B-v2.0-base - GGUF
- Model creator: https://huggingface.co/yentinglin/
- Original model: https://huggingface.co/yentinglin/Taiwan-LLM-7B-v2.0-base/


| Name | Quant method | Size |
| ---- | ---- | ---- |
| [Taiwan-LLM-7B-v2.0-base.Q2_K.gguf](https://huggingface.co/RichardErkhov/yentinglin_-_Taiwan-LLM-7B-v2.0-base-gguf/blob/main/Taiwan-LLM-7B-v2.0-base.Q2_K.gguf) | Q2_K | 2.36GB |
| [Taiwan-LLM-7B-v2.0-base.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/yentinglin_-_Taiwan-LLM-7B-v2.0-base-gguf/blob/main/Taiwan-LLM-7B-v2.0-base.IQ3_XS.gguf) | IQ3_XS | 2.6GB |
| [Taiwan-LLM-7B-v2.0-base.IQ3_S.gguf](https://huggingface.co/RichardErkhov/yentinglin_-_Taiwan-LLM-7B-v2.0-base-gguf/blob/main/Taiwan-LLM-7B-v2.0-base.IQ3_S.gguf) | IQ3_S | 2.75GB |
| [Taiwan-LLM-7B-v2.0-base.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/yentinglin_-_Taiwan-LLM-7B-v2.0-base-gguf/blob/main/Taiwan-LLM-7B-v2.0-base.Q3_K_S.gguf) | Q3_K_S | 2.75GB |
| [Taiwan-LLM-7B-v2.0-base.IQ3_M.gguf](https://huggingface.co/RichardErkhov/yentinglin_-_Taiwan-LLM-7B-v2.0-base-gguf/blob/main/Taiwan-LLM-7B-v2.0-base.IQ3_M.gguf) | IQ3_M | 2.9GB |
| [Taiwan-LLM-7B-v2.0-base.Q3_K.gguf](https://huggingface.co/RichardErkhov/yentinglin_-_Taiwan-LLM-7B-v2.0-base-gguf/blob/main/Taiwan-LLM-7B-v2.0-base.Q3_K.gguf) | Q3_K | 3.07GB |
| [Taiwan-LLM-7B-v2.0-base.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/yentinglin_-_Taiwan-LLM-7B-v2.0-base-gguf/blob/main/Taiwan-LLM-7B-v2.0-base.Q3_K_M.gguf) | Q3_K_M | 3.07GB |
| [Taiwan-LLM-7B-v2.0-base.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/yentinglin_-_Taiwan-LLM-7B-v2.0-base-gguf/blob/main/Taiwan-LLM-7B-v2.0-base.Q3_K_L.gguf) | Q3_K_L | 3.35GB |
| [Taiwan-LLM-7B-v2.0-base.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/yentinglin_-_Taiwan-LLM-7B-v2.0-base-gguf/blob/main/Taiwan-LLM-7B-v2.0-base.IQ4_XS.gguf) | IQ4_XS | 3.4GB |
| [Taiwan-LLM-7B-v2.0-base.Q4_0.gguf](https://huggingface.co/RichardErkhov/yentinglin_-_Taiwan-LLM-7B-v2.0-base-gguf/blob/main/Taiwan-LLM-7B-v2.0-base.Q4_0.gguf) | Q4_0 | 3.56GB |
| [Taiwan-LLM-7B-v2.0-base.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/yentinglin_-_Taiwan-LLM-7B-v2.0-base-gguf/blob/main/Taiwan-LLM-7B-v2.0-base.IQ4_NL.gguf) | IQ4_NL | 3.58GB |
| [Taiwan-LLM-7B-v2.0-base.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/yentinglin_-_Taiwan-LLM-7B-v2.0-base-gguf/blob/main/Taiwan-LLM-7B-v2.0-base.Q4_K_S.gguf) | Q4_K_S | 3.59GB |
| [Taiwan-LLM-7B-v2.0-base.Q4_K.gguf](https://huggingface.co/RichardErkhov/yentinglin_-_Taiwan-LLM-7B-v2.0-base-gguf/blob/main/Taiwan-LLM-7B-v2.0-base.Q4_K.gguf) | Q4_K | 3.8GB |
| [Taiwan-LLM-7B-v2.0-base.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/yentinglin_-_Taiwan-LLM-7B-v2.0-base-gguf/blob/main/Taiwan-LLM-7B-v2.0-base.Q4_K_M.gguf) | Q4_K_M | 3.8GB |
| [Taiwan-LLM-7B-v2.0-base.Q4_1.gguf](https://huggingface.co/RichardErkhov/yentinglin_-_Taiwan-LLM-7B-v2.0-base-gguf/blob/main/Taiwan-LLM-7B-v2.0-base.Q4_1.gguf) | Q4_1 | 3.95GB |
| [Taiwan-LLM-7B-v2.0-base.Q5_0.gguf](https://huggingface.co/RichardErkhov/yentinglin_-_Taiwan-LLM-7B-v2.0-base-gguf/blob/main/Taiwan-LLM-7B-v2.0-base.Q5_0.gguf) | Q5_0 | 4.33GB |
| [Taiwan-LLM-7B-v2.0-base.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/yentinglin_-_Taiwan-LLM-7B-v2.0-base-gguf/blob/main/Taiwan-LLM-7B-v2.0-base.Q5_K_S.gguf) | Q5_K_S | 4.33GB |
| [Taiwan-LLM-7B-v2.0-base.Q5_K.gguf](https://huggingface.co/RichardErkhov/yentinglin_-_Taiwan-LLM-7B-v2.0-base-gguf/blob/main/Taiwan-LLM-7B-v2.0-base.Q5_K.gguf) | Q5_K | 4.45GB |
| [Taiwan-LLM-7B-v2.0-base.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/yentinglin_-_Taiwan-LLM-7B-v2.0-base-gguf/blob/main/Taiwan-LLM-7B-v2.0-base.Q5_K_M.gguf) | Q5_K_M | 4.45GB |
| [Taiwan-LLM-7B-v2.0-base.Q5_1.gguf](https://huggingface.co/RichardErkhov/yentinglin_-_Taiwan-LLM-7B-v2.0-base-gguf/blob/main/Taiwan-LLM-7B-v2.0-base.Q5_1.gguf) | Q5_1 | 4.72GB |
| [Taiwan-LLM-7B-v2.0-base.Q6_K.gguf](https://huggingface.co/RichardErkhov/yentinglin_-_Taiwan-LLM-7B-v2.0-base-gguf/blob/main/Taiwan-LLM-7B-v2.0-base.Q6_K.gguf) | Q6_K | 5.15GB |
| [Taiwan-LLM-7B-v2.0-base.Q8_0.gguf](https://huggingface.co/RichardErkhov/yentinglin_-_Taiwan-LLM-7B-v2.0-base-gguf/blob/main/Taiwan-LLM-7B-v2.0-base.Q8_0.gguf) | Q8_0 | 6.67GB |




Original model description:
---
license: apache-2.0
language:
- zh
widget:
 - text: "A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions. USER: 你好，請問你可以幫我寫一封推薦信嗎？ ASSISTANT:"
library_name: transformers
pipeline_tag: text-generation
extra_gated_heading: Acknowledge license to accept the repository.
extra_gated_prompt: Please contact the author for access.
extra_gated_button_content: Acknowledge license 同意以上內容
extra_gated_fields:
  Name: text
  Mail: text
  Organization: text
  Country: text
  Any utilization of the Taiwan LLM repository mandates the explicit acknowledgment and attribution to the original author: checkbox
  使用Taiwan LLM必須明確地承認和歸功於優必達株式會社 Ubitus 以及原始作者: checkbox
---
<img src="https://cdn-uploads.huggingface.co/production/uploads/5df9c78eda6d0311fd3d541f/CmusIT5OlSXvFrbTJ7l-C.png" alt="Taiwan LLM Logo" width="800" style="margin-left:'auto' margin-right:'auto' display:'block'"/>

# 🌟 Checkout [Taiwan-LLM Demo Chat-UI](http://www.twllm.com) 🌟

# Model Card for Taiwan LLM 7B v2.0 base

Taiwan LLM is an advanced language model tailored for Traditional Chinese, focusing on the linguistic and cultural contexts of Taiwan. 
Developed from a large base model, it's enriched with diverse Taiwanese textual sources and refined through Supervised Fine-Tuning. 
This model excels in language understanding and generation, aligning closely with Taiwan's cultural nuances. 
It demonstrates improved performance on various benchmarks like TC-Eval, showcasing its contextual comprehension and cultural relevance. 
For detailed insights into Taiwan LLM's development and features, refer to our [technical report](https://github.com/MiuLab/Taiwan-LLaMa/blob/main/twllm_paper.pdf).


## Model description

- **Model type:** A 7B parameter GPT-like model fine-tuned on a mix of publicly available, synthetic datasets.
- **Language(s) (NLP):** Primarily Traditional Chinese (zh-tw)
- **Finetuned from model:** [meta-llama/Llama-2-7b-hf](https://huggingface.co/yentinglin/meta-llama/Llama-2-7b-hf)

### Model Sources

<!-- Provide the basic links for the model. -->

- **Repository:** https://github.com/MiuLab/Taiwan-LLaMa
- **Demo:** https://twllm.com/

## Performance


![image/png](https://cdn-uploads.huggingface.co/production/uploads/5df9c78eda6d0311fd3d541f/HTwIzw6RDha2-PhuWqSuI.png)

## Intended uses

You should fine-tuned this model for instruction-following / chat application.

### Training hyperparameters

![image/png](https://cdn-uploads.huggingface.co/production/uploads/5df9c78eda6d0311fd3d541f/MdvHwdUvH-c926qyRAw7K.png)

![image/png](https://cdn-uploads.huggingface.co/production/uploads/5df9c78eda6d0311fd3d541f/kKpkvxDzOEyiAoTqmzRYO.png)


![image/png](https://cdn-uploads.huggingface.co/production/uploads/5df9c78eda6d0311fd3d541f/FsnlJ_fkRxf7fn5RKZnjE.png)

The following hyperparameters were used during training:
- learning_rate: 5e-05
- distributed_type: multi-GPU
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.03
- num_epochs: 5.0

## Citation

If you find Taiwan LLM is useful in your work, please cite it with:

```
@misc{lin2023taiwan,
      title={Taiwan LLM: Bridging the Linguistic Divide with a Culturally Aligned Language Model}, 
      author={Yen-Ting Lin and Yun-Nung Chen},
      year={2023},
      eprint={2311.17487},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```

# Acknowledgement

Taiwan LLM v2 is conducted in collaboration with [Ubitus K.K.](http://ubitus.net). Ubitus provides valuable compute resources for the project.