Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,124 @@
|
|
1 |
-
---
|
2 |
-
license: apache-2.0
|
3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: apache-2.0
|
3 |
+
language:
|
4 |
+
- ko
|
5 |
+
- en
|
6 |
+
metrics:
|
7 |
+
- accuracy
|
8 |
+
base_model:
|
9 |
+
- openchat/openchat_3.5
|
10 |
+
pipeline_tag: text-generation
|
11 |
+
---
|
12 |
+
### β± ν΄λΉ λͺ¨λΈμμ openchat3.5λ₯Ό Foundation λͺ¨λΈλ‘ νλ νκ΅μ΄ λ° νκ΅μ λ€μν
|
13 |
+
### λ¬Ένμ μ μ©ν μ μλλ‘ νκΈ° μν΄
|
14 |
+
### κ°λ° λμμΌλ©° μ체 μ μν 53μμμ νκ΅μ΄ λ°μ΄ν°λ₯Ό νμ©νμ¬ νκ΅ μ¬ν κ°μΉμ
|
15 |
+
### λ¬Ένλ₯Ό μ΄ν΄νλ λͺ¨λΈ μ
λλ€. β
|
16 |
+
|
17 |
+
|
18 |
+
|
19 |
+
# βΆ λͺ¨λΈ μ€λͺ
|
20 |
+
- λͺ¨λΈλͺ
λ° μ£ΌμκΈ°λ₯:
|
21 |
+
ν΄λΉ λͺ¨λΈμμ OpenChat 3.5 λͺ¨λΈμ κΈ°λ°μΌλ‘ SFT λ°©μμΌλ‘ νμΈνλλ Mistral 7B / openchat3.5 κΈ°λ° λͺ¨λΈμ
λλ€.
|
22 |
+
νκ΅μ΄μ νκ΅μ λ€μν λ¬Ένμ λ§₯λ½μ μ΄ν΄νλλ‘ μ€κ³λμμΌλ©° β¨β¨, μ체 μ μν 135κ° μμμ νκ΅μ΄
|
23 |
+
λ°μ΄ν°λ₯Ό νμ©ν΄ νκ΅ μ¬νμ κ°μΉμ λ¬Ένλ₯Ό λ°μν©λλ€.
|
24 |
+
μ£Όμ κΈ°λ₯μΌλ‘λ ν
μ€νΈ μμ±, λν μΆλ‘ , λ¬Έμ μμ½, μ§μμλ΅, κ°μ λΆμ λ° μμ°μ΄ μ²λ¦¬ κ΄λ ¨ λ€μν μμ
μ μ§μνλ©°,
|
25 |
+
νμ© λΆμΌλ λ²λ₯ , μ¬λ¬΄, κ³Όν, κ΅μ‘, λΉμ¦λμ€, λ¬Έν μ°κ΅¬ λ± λ€μν λΆμΌμμ μμ©λ μ μμ΅λλ€.
|
26 |
+
- λͺ¨λΈ μν€ν
μ²:ν΄λΉ λͺ¨λΈμμ Mistral 7B λͺ¨λΈμ κΈ°λ°μΌλ‘, νλΌλ―Έν° μλ 70μ΅ κ°(7B)λ‘ κ΅¬μ±λ κ³ μ±λ₯ μΈμ΄ λͺ¨λΈμ
λλ€.
|
27 |
+
μ΄ λͺ¨λΈμ OpenChat 3.5λ₯Ό νμ΄λ°μ΄μ
λͺ¨λΈλ‘ μΌμ, SFT(μ§λ λ―ΈμΈ μ‘°μ ) λ°©μμ ν΅ν΄ νκ΅μ΄μ νκ΅ λ¬Ένμ νΉνλ μ±λ₯μ λ°ννλλ‘ νλ ¨λμμ΅λλ€.
|
28 |
+
Mistral 7Bμ κ²½λνλ ꡬ쑰λ λΉ λ₯Έ μΆλ‘ μλμ λ©λͺ¨λ¦¬ ν¨μ¨μ±μ 보μ₯νλ©°, λ€μν μμ°μ΄ μ²λ¦¬ μμ
μ μ ν©νκ² μ΅μ νλμ΄ μμ΅λλ€.
|
29 |
+
μ΄ μν€ν
μ²λ ν
μ€νΈ μμ±, μ§μμλ΅, λ¬Έμ μμ½, κ°μ λΆμκ³Ό κ°μ λ€μν μμ
μμ νμν μ±λ₯μ 보μ¬μ€λλ€.
|
30 |
+
|
31 |
+
# β· νμ΅ λ°μ΄ν°
|
32 |
+
- ν΄λΉ λͺ¨λΈμμ μ체 κ°λ°ν μ΄ 3.6GB ν¬κΈ°μ λ°μ΄ν°λ₯Ό λ°νμΌλ‘ νμ΅λμμ΅λλ€. λͺ¨λ 233λ§ κ±΄μ QnA, μμ½, λΆλ₯ λ± λ°μ΄ν°λ₯Ό ν¬ν¨νλ©°,
|
33 |
+
κ·Έ μ€ 133λ§ κ±΄μ 53κ° μμμ κ°κ΄μ λ¬Έμ λ‘ κ΅¬μ±λμμ΅λλ€. μ΄ μμμλ νκ΅μ¬, μ¬ν, μ¬λ¬΄, λ²λ₯ , μΈλ¬΄, μν, μλ¬Ό, 물리, νν λ±μ΄ ν¬ν¨λλ©°,
|
34 |
+
Chain of Thought λ°©μμΌλ‘ νμ΅λμμ΅λλ€. λν 130λ§ κ±΄μ μ£Όκ΄μ λ¬Έμ λ νκ΅μ¬, μ¬λ¬΄, λ²λ₯ , μΈλ¬΄, μν λ± 38κ° μμμ κ±Έμ³ νμ΅λμμ΅λλ€.
|
35 |
+
νμ΅ λ°μ΄ν° μ€ νκ΅μ μ¬ν κ°μΉμ μΈκ°μ κ°μ μ μ΄ν΄νκ³ μ§μν μ¬νμ λ°λΌ μΆλ ₯ν μ μλ λ°μ΄ν°λ₯Ό νμ΅νμμ΅λλ€.
|
36 |
+
- νμ΅ Instruction Datasets Format:
|
37 |
+
<pre><code>{"prompt": "prompt text", "completion": "ideal generated text"}</code></pre>
|
38 |
+
|
39 |
+
# βΈ μ¬μ© μ¬λ‘
|
40 |
+
ν΄λΉ λͺ¨λΈμ λ€μν μμ© λΆμΌμμ μ¬μ©λ μ μμ΅λλ€. μλ₯Ό λ€μ΄:
|
41 |
+
- κ΅μ‘ λΆμΌ: μμ¬, μν, κ³Όν λ± λ€μν νμ΅ μλ£μ λν μ§μμλ΅ λ° μ€λͺ
μμ±.
|
42 |
+
- λΉμ¦λμ€: λ²λ₯ , μ¬λ¬΄, μΈλ¬΄ κ΄λ ¨ μ§μμ λν λ΅λ³ μ 곡 λ° λ¬Έμ μμ½.
|
43 |
+
- μ°κ΅¬ λ° λ¬Έν: νκ΅ μ¬νμ λ¬Ένμ λ§μΆ μμ°μ΄ μ²λ¦¬ μμ
, κ°μ λΆμ, λ¬Έμ μμ± λ° λ²μ.
|
44 |
+
- κ³ κ° μλΉμ€: μ¬μ©μμμ λν μμ± λ° λ§μΆ€ν μλ΅ μ 곡.
|
45 |
+
- μ΄ λͺ¨λΈμ λ€μν μμ°μ΄ μ²λ¦¬ μμ
μμ λμ νμ©λλ₯Ό κ°μ§λλ€.
|
46 |
+
|
47 |
+
# βΉ νκ³ ββ
|
48 |
+
- ν΄λΉ λͺ¨λΈμ νκ΅μ΄μ νκ΅ λ¬Ένμ νΉνλμ΄ μμΌλ,
|
49 |
+
νΉμ μμ(μ: μ΅μ κ΅μ μλ£, μ λ¬Έ λΆμΌ)μ λ°μ΄ν° λΆμ‘±μΌλ‘ μΈν΄ λ€λ₯Έ μΈμ΄ λλ
|
50 |
+
λ¬Ένμ λν μλ΅μ μ νμ±μ΄ λ¨μ΄μ§ μ μμ΅λλ€.
|
51 |
+
λν, 볡μ‘ν λ
Όλ¦¬μ μ¬κ³ λ₯Ό μꡬνλ λ¬Έμ μ λν΄ μ νλ μΆλ‘ λ₯λ ₯μ λ³΄μΌ μ μμΌλ©°,
|
52 |
+
νΈν₯λ λ°μ΄ν°κ° ν¬ν¨λ κ²½μ° νΈν₯λ μλ΅μ΄ μμ±λ κ°λ₯μ±λ μ‘΄μ¬ν©λλ€.
|
53 |
+
|
54 |
+
# βΊ μ¬μ© λ°©λ²
|
55 |
+
<pre><code>
|
56 |
+
from transformers import AutoModel, AutoTokenizer
|
57 |
+
|
58 |
+
tokenizer = AutoTokenizer.from_pretrained("")
|
59 |
+
model = AutoModel.from_pretrained("")
|
60 |
+
|
61 |
+
inputs = tokenizer("μλ
νμΈμ", return_tensors="pt")
|
62 |
+
outputs = model(**inputs)
|
63 |
+
</code></pre>
|
64 |
+
|
65 |
+
|
66 |
+
---
|
67 |
+
Hereβs the English version of the provided text:
|
68 |
+
|
69 |
+
|
70 |
+
|
71 |
+
# βΆ Model Description
|
72 |
+
|
73 |
+
**Model Name and Key Features**:
|
74 |
+
KTDSbaseLM v0.11 is based on the OpenChat 3.5 model, fine-tuned using the SFT method on the Mistral 7B model.
|
75 |
+
It is designed to understand Korean and various cultural contexts, utilizing data from 135 domains in Korean society.
|
76 |
+
The model supports tasks such as text generation, conversation inference, document summarization,
|
77 |
+
question answering, sentiment analysis, and other NLP tasks.
|
78 |
+
Its applications span fields like law, finance, science, education, business, and cultural research.
|
79 |
+
|
80 |
+
**Model Architecture**:
|
81 |
+
KTDSBaseLM v0.11 is a high-performance language model with 7 billion parameters based on the Mistral 7B model.
|
82 |
+
It uses OpenChat 3.5 as the foundation and is fine-tuned using SFT to excel in Korean language and culture.
|
83 |
+
The streamlined Mistral 7B architecture ensures fast inference and memory efficiency,
|
84 |
+
optimized for various NLP tasks like text generation, question answering, document summarization, and sentiment analysis.
|
85 |
+
|
86 |
+
---
|
87 |
+
|
88 |
+
# β· Training Data
|
89 |
+
|
90 |
+
KTDSbaseLM v0.11 was trained on 3.6GB of data, comprising 2.33 million Q&A instances.
|
91 |
+
This includes 1.33 million multiple-choice questions across 53 domains such as history,
|
92 |
+
finance, law, tax, and science, trained with the Chain of Thought method. Additionally,
|
93 |
+
1.3 million short-answer questions cover 38 domains including history, finance, and law.
|
94 |
+
|
95 |
+
**Training Instruction Dataset Format**:
|
96 |
+
`{"prompt": "prompt text", "completion": "ideal generated text"}`
|
97 |
+
|
98 |
+
---
|
99 |
+
|
100 |
+
# βΈ Use Cases
|
101 |
+
|
102 |
+
KTDSbaseLM v0.11 can be used across multiple fields, such as:
|
103 |
+
|
104 |
+
- **Education**: Answering questions and generating explanations for subjects like history, math, and science.
|
105 |
+
- **Business**: Providing responses and summaries for legal, financial, and tax-related queries.
|
106 |
+
- **Research and Culture**: Performing NLP tasks, sentiment analysis, document generation, and translation.
|
107 |
+
- **Customer Service**: Generating conversations and personalized responses for users.
|
108 |
+
|
109 |
+
This model is highly versatile in various NLP tasks.
|
110 |
+
|
111 |
+
---
|
112 |
+
|
113 |
+
# βΉ Limitations
|
114 |
+
|
115 |
+
KTDSBaseLM v0.11 is specialized in Korean language and culture.
|
116 |
+
However, it may lack accuracy in responding to topics outside its scope,
|
117 |
+
such as international or specialized data.
|
118 |
+
Additionally, it may have limited reasoning ability for complex logical problems and
|
119 |
+
may produce biased responses if trained on biased data.
|
120 |
+
|
121 |
+
---
|
122 |
+
|
123 |
+
|
124 |
+
|