SEOKDONG commited on
Commit
cfb9e0c
β€’
1 Parent(s): b045d86

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +124 -3
README.md CHANGED
@@ -1,3 +1,124 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - ko
5
+ - en
6
+ metrics:
7
+ - accuracy
8
+ base_model:
9
+ - openchat/openchat_3.5
10
+ pipeline_tag: text-generation
11
+ ---
12
+ ### β›± ν•΄λ‹Ή λͺ¨λΈμ€μ€ openchat3.5λ₯Ό Foundation λͺ¨λΈλ‘œ ν•˜λŠ” ν•œκ΅­μ–΄ 및 ν•œκ΅­μ˜ λ‹€μ–‘ν•œ
13
+ ### 문화에 μ μš©ν•  수 μžˆλ„λ‘ ν•˜κΈ° μœ„ν•΄
14
+ ### 개발 λ˜μ—ˆμœΌλ©° 자체 μ œμž‘ν•œ 53μ˜μ—­μ˜ ν•œκ΅­μ–΄ 데이터λ₯Ό ν™œμš©ν•˜μ—¬ ν•œκ΅­ μ‚¬νšŒ κ°€μΉ˜μ™€
15
+ ### λ¬Έν™”λ₯Ό μ΄ν•΄ν•˜λŠ” λͺ¨λΈ μž…λ‹ˆλ‹€. ✌
16
+
17
+
18
+
19
+ # ❢ λͺ¨λΈ μ„€λͺ…
20
+ - λͺ¨λΈλͺ… 및 μ£Όμš”κΈ°λŠ₯:
21
+ ν•΄λ‹Ή λͺ¨λΈμ€μ€ OpenChat 3.5 λͺ¨λΈμ„ 기반으둜 SFT λ°©μ‹μœΌλ‘œ νŒŒμΈνŠœλ‹λœ Mistral 7B / openchat3.5 기반 λͺ¨λΈμž…λ‹ˆλ‹€.
22
+ ν•œκ΅­μ–΄μ™€ ν•œκ΅­μ˜ λ‹€μ–‘ν•œ 문화적 λ§₯락을 μ΄ν•΄ν•˜λ„λ‘ μ„€κ³„λ˜μ—ˆμœΌλ©° ✨✨, 자체 μ œμž‘ν•œ 135개 μ˜μ—­μ˜ ν•œκ΅­μ–΄
23
+ 데이터λ₯Ό ν™œμš©ν•΄ ν•œκ΅­ μ‚¬νšŒμ˜ κ°€μΉ˜μ™€ λ¬Έν™”λ₯Ό λ°˜μ˜ν•©λ‹ˆλ‹€.
24
+ μ£Όμš” κΈ°λŠ₯μœΌλ‘œλŠ” ν…μŠ€νŠΈ 생성, λŒ€ν™” μΆ”λ‘ , λ¬Έμ„œ μš”μ•½, μ§ˆμ˜μ‘λ‹΅, 감정 뢄석 및 μžμ—°μ–΄ 처리 κ΄€λ ¨ λ‹€μ–‘ν•œ μž‘μ—…μ„ μ§€μ›ν•˜λ©°,
25
+ ν™œμš© λΆ„μ•ΌλŠ” 법λ₯ , 재무, κ³Όν•™, ꡐ윑, λΉ„μ¦ˆλ‹ˆμŠ€, λ¬Έν™” 연ꡬ λ“± λ‹€μ–‘ν•œ λΆ„μ•Όμ—μ„œ μ‘μš©λ  수 μžˆμŠ΅λ‹ˆλ‹€.
26
+ - λͺ¨λΈ μ•„ν‚€ν…μ²˜:ν•΄λ‹Ή λͺ¨λΈμ€μ€ Mistral 7B λͺ¨λΈμ„ 기반으둜, νŒŒλΌλ―Έν„° μˆ˜λŠ” 70μ–΅ 개(7B)둜 κ΅¬μ„±λœ κ³ μ„±λŠ₯ μ–Έμ–΄ λͺ¨λΈμž…λ‹ˆλ‹€.
27
+ 이 λͺ¨λΈμ€ OpenChat 3.5λ₯Ό νŒŒμš΄λ°μ΄μ…˜ λͺ¨λΈλ‘œ μ‚Όμ•„, SFT(지도 λ―Έμ„Έ μ‘°μ •) 방식을 톡해 ν•œκ΅­μ–΄μ™€ ν•œκ΅­ 문화에 νŠΉν™”λœ μ„±λŠ₯을 λ°œνœ˜ν•˜λ„λ‘ ν›ˆλ ¨λ˜μ—ˆμŠ΅λ‹ˆλ‹€.
28
+ Mistral 7B의 κ²½λŸ‰ν™”λœ κ΅¬μ‘°λŠ” λΉ λ₯Έ μΆ”λ‘  속도와 λ©”λͺ¨λ¦¬ νš¨μœ¨μ„±μ„ 보μž₯ν•˜λ©°, λ‹€μ–‘ν•œ μžμ—°μ–΄ 처리 μž‘μ—…μ— μ ν•©ν•˜κ²Œ μ΅œμ ν™”λ˜μ–΄ μžˆμŠ΅λ‹ˆλ‹€.
29
+ 이 μ•„ν‚€ν…μ²˜λŠ” ν…μŠ€νŠΈ 생성, μ§ˆμ˜μ‘λ‹΅, λ¬Έμ„œ μš”μ•½, 감정 뢄석과 같은 λ‹€μ–‘ν•œ μž‘μ—…μ—μ„œ νƒμ›”ν•œ μ„±λŠ₯을 λ³΄μ—¬μ€λ‹ˆλ‹€.
30
+
31
+ # ❷ ν•™μŠ΅ 데이터
32
+ - ν•΄λ‹Ή λͺ¨λΈμ€μ€ 자체 κ°œλ°œν•œ 총 3.6GB 크기의 데이터λ₯Ό λ°”νƒ•μœΌλ‘œ ν•™μŠ΅λ˜μ—ˆμŠ΅λ‹ˆλ‹€. λͺ¨λ‘ 233만 건의 QnA, μš”μ•½, λΆ„λ₯˜ λ“± 데이터λ₯Ό ν¬ν•¨ν•˜λ©°,
33
+ κ·Έ 쀑 133만 건은 53개 μ˜μ—­μ˜ 객관식 문제둜 κ΅¬μ„±λ˜μ—ˆμŠ΅λ‹ˆλ‹€. 이 μ˜μ—­μ—λŠ” ν•œκ΅­μ‚¬, μ‚¬νšŒ, 재무, 법λ₯ , 세무, μˆ˜ν•™, 생물, 물리, ν™”ν•™ 등이 ν¬ν•¨λ˜λ©°,
34
+ Chain of Thought λ°©μ‹μœΌλ‘œ ν•™μŠ΅λ˜μ—ˆμŠ΅λ‹ˆλ‹€. λ˜ν•œ 130만 건의 주관식 λ¬Έμ œλŠ” ν•œκ΅­μ‚¬, 재무, 법λ₯ , 세무, μˆ˜ν•™ λ“± 38개 μ˜μ—­μ— 걸쳐 ν•™μŠ΅λ˜μ—ˆμŠ΅λ‹ˆλ‹€.
35
+ ν•™μŠ΅ 데이터 쀑 ν•œκ΅­μ˜ μ‚¬νšŒ κ°€μΉ˜μ™€ μΈκ°„μ˜ 감정을 μ΄ν•΄ν•˜κ³  μ§€μ‹œν•œ 사항에 따라 좜λ ₯ν•  수 μžˆλŠ” 데이터λ₯Ό ν•™μŠ΅ν•˜μ˜€μŠ΅λ‹ˆλ‹€.
36
+ - ν•™μŠ΅ Instruction Datasets Format:
37
+ <pre><code>{"prompt": "prompt text", "completion": "ideal generated text"}</code></pre>
38
+
39
+ # ❸ μ‚¬μš© 사둀
40
+ ν•΄λ‹Ή λͺ¨λΈμ€ λ‹€μ–‘ν•œ μ‘μš© λΆ„μ•Όμ—μ„œ μ‚¬μš©λ  수 μžˆμŠ΅λ‹ˆλ‹€. 예λ₯Ό λ“€μ–΄:
41
+ - ꡐ윑 λΆ„μ•Ό: 역사, μˆ˜ν•™, κ³Όν•™ λ“± λ‹€μ–‘ν•œ ν•™μŠ΅ μžλ£Œμ— λŒ€ν•œ μ§ˆμ˜μ‘λ‹΅ 및 μ„€λͺ… 생성.
42
+ - λΉ„μ¦ˆλ‹ˆμŠ€: 법λ₯ , 재무, 세무 κ΄€λ ¨ μ§ˆμ˜μ— λŒ€ν•œ λ‹΅λ³€ 제곡 및 λ¬Έμ„œ μš”μ•½.
43
+ - 연ꡬ 및 λ¬Έν™”: ν•œκ΅­ μ‚¬νšŒμ™€ 문화에 맞좘 μžμ—°μ–΄ 처리 μž‘μ—…, 감정 뢄석, λ¬Έμ„œ 생성 및 λ²ˆμ—­.
44
+ - 고객 μ„œλΉ„μŠ€: μ‚¬μš©μžμ™€μ˜ λŒ€ν™” 생성 및 λ§žμΆ€ν˜• 응닡 제곡.
45
+ - 이 λͺ¨λΈμ€ λ‹€μ–‘ν•œ μžμ—°μ–΄ 처리 μž‘μ—…μ—μ„œ 높은 ν™œμš©λ„λ₯Ό κ°€μ§‘λ‹ˆλ‹€.
46
+
47
+ # ❹ ν•œκ³„ β›ˆβ›ˆ
48
+ - ν•΄λ‹Ή λͺ¨λΈμ€ ν•œκ΅­μ–΄μ™€ ν•œκ΅­ 문화에 νŠΉν™”λ˜μ–΄ μžˆμœΌλ‚˜,
49
+ νŠΉμ • μ˜μ—­(예: μ΅œμ‹  ꡭ제 자료, μ „λ¬Έ λΆ„μ•Ό)의 데이터 λΆ€μ‘±μœΌλ‘œ 인해 λ‹€λ₯Έ μ–Έμ–΄ λ˜λŠ”
50
+ 문화에 λŒ€ν•œ μ‘λ‹΅μ˜ 정확성이 λ–¨μ–΄μ§ˆ 수 μžˆμŠ΅λ‹ˆλ‹€.
51
+ λ˜ν•œ, λ³΅μž‘ν•œ 논리적 사고λ₯Ό μš”κ΅¬ν•˜λŠ” λ¬Έμ œμ— λŒ€ν•΄ μ œν•œλœ μΆ”λ‘  λŠ₯λ ₯을 보일 수 있으며,
52
+ 편ν–₯된 데이터가 포함될 경우 편ν–₯된 응닡이 생성될 κ°€λŠ₯성도 μ‘΄μž¬ν•©λ‹ˆλ‹€.
53
+
54
+ # ❺ μ‚¬μš© 방법
55
+ <pre><code>
56
+ from transformers import AutoModel, AutoTokenizer
57
+
58
+ tokenizer = AutoTokenizer.from_pretrained("")
59
+ model = AutoModel.from_pretrained("")
60
+
61
+ inputs = tokenizer("μ•ˆλ…•ν•˜μ„Έμš”", return_tensors="pt")
62
+ outputs = model(**inputs)
63
+ </code></pre>
64
+
65
+
66
+ ---
67
+ Here’s the English version of the provided text:
68
+
69
+
70
+
71
+ # ❢ Model Description
72
+
73
+ **Model Name and Key Features**:
74
+ KTDSbaseLM v0.11 is based on the OpenChat 3.5 model, fine-tuned using the SFT method on the Mistral 7B model.
75
+ It is designed to understand Korean and various cultural contexts, utilizing data from 135 domains in Korean society.
76
+ The model supports tasks such as text generation, conversation inference, document summarization,
77
+ question answering, sentiment analysis, and other NLP tasks.
78
+ Its applications span fields like law, finance, science, education, business, and cultural research.
79
+
80
+ **Model Architecture**:
81
+ KTDSBaseLM v0.11 is a high-performance language model with 7 billion parameters based on the Mistral 7B model.
82
+ It uses OpenChat 3.5 as the foundation and is fine-tuned using SFT to excel in Korean language and culture.
83
+ The streamlined Mistral 7B architecture ensures fast inference and memory efficiency,
84
+ optimized for various NLP tasks like text generation, question answering, document summarization, and sentiment analysis.
85
+
86
+ ---
87
+
88
+ # ❷ Training Data
89
+
90
+ KTDSbaseLM v0.11 was trained on 3.6GB of data, comprising 2.33 million Q&A instances.
91
+ This includes 1.33 million multiple-choice questions across 53 domains such as history,
92
+ finance, law, tax, and science, trained with the Chain of Thought method. Additionally,
93
+ 1.3 million short-answer questions cover 38 domains including history, finance, and law.
94
+
95
+ **Training Instruction Dataset Format**:
96
+ `{"prompt": "prompt text", "completion": "ideal generated text"}`
97
+
98
+ ---
99
+
100
+ # ❸ Use Cases
101
+
102
+ KTDSbaseLM v0.11 can be used across multiple fields, such as:
103
+
104
+ - **Education**: Answering questions and generating explanations for subjects like history, math, and science.
105
+ - **Business**: Providing responses and summaries for legal, financial, and tax-related queries.
106
+ - **Research and Culture**: Performing NLP tasks, sentiment analysis, document generation, and translation.
107
+ - **Customer Service**: Generating conversations and personalized responses for users.
108
+
109
+ This model is highly versatile in various NLP tasks.
110
+
111
+ ---
112
+
113
+ # ❹ Limitations
114
+
115
+ KTDSBaseLM v0.11 is specialized in Korean language and culture.
116
+ However, it may lack accuracy in responding to topics outside its scope,
117
+ such as international or specialized data.
118
+ Additionally, it may have limited reasoning ability for complex logical problems and
119
+ may produce biased responses if trained on biased data.
120
+
121
+ ---
122
+
123
+
124
+