a2ran commited on
Commit
76d5d7b
ยท
1 Parent(s): f3f7f0f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +43 -9
README.md CHANGED
@@ -2,12 +2,37 @@
2
  library_name: peft
3
  ---
4
 
5
- # WIP
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6
 
7
  ## 1. ์‚ฌ์šฉ์ ˆ์ฐจ
8
 
9
  * Install model and PEFT parameters
10
 
 
 
 
 
 
11
  ```
12
  import torch
13
  from peft import PeftModel, PeftConfig
@@ -33,7 +58,7 @@ streamer = TextStreamer(tokenizer)
33
 
34
  # your input sentence๊ฐ€ ๋“ค์–ด๊ฐˆ ๊ณณ
35
  input = """
36
- ### input @ ๋ฏธ๊ตญ์˜ ํ–‰์ •์‹œ์Šคํ…œ์— ๋Œ€ํ•ด ์„ค๋ช…ํ•ด์ค˜.\n\n### response @"""
37
 
38
  output = tokenizer.decode(model.cuda().generate(
39
  **tokenizer(
@@ -55,19 +80,19 @@ output = tokenizer.decode(model.cuda().generate(
55
 
56
  ```
57
  output =
58
- <s> ๋ฏธ๊ตญ์˜ ํ–‰์ • ์‹œ์Šคํ…œ์€ ๋ณต์žกํ•˜๊ณ  ๋‹ค์–‘ํ•œ ๊ธฐ๊ด€๋“ค๋กœ ์ด๋ฃจ์–ด์ ธ ์žˆ์Šต๋‹ˆ๋‹ค. ์—ฌ๊ธฐ์„œ๋Š” ์ฃผ์š” ๊ฐœ๋…๊ณผ ๊ตฌ์กฐ๋ฅผ ์‚ดํŽด๋ณด๋„๋ก ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค.
59
 
60
- ์šฐ์„ , ๋ฏธ๊ตญ์˜ ํ–‰์ • ์‹œ์Šคํ…œ์€ ์—ฐ๋ฐฉ์ œ ๊ตฌ์กฐ๋กœ ๊ตฌ์„ฑ๋˜์–ด ์žˆ์Šต๋‹ˆ๋‹ค. ์ด๋Š” ์—ฐ๋ฐฉ๊ณผ ์ฃผ์˜ ๊ตฌ๋ถ„์„ ๊ฐ–๊ณ  ์žˆ๋Š” ๊ฒƒ์ž…๋‹ˆ๋‹ค. ์—ฐ๋ฐฉ ์ •๋ถ€๋Š” ์ „๊ตญ์ ์ธ ๋ฒ•๊ณผ ์ •์ฑ…์„ ์„ธ์šฐ๊ณ  ์™ธ๊ต, ๊ตฐ์‚ฌ, ๊ฒฝ์ œ ๋“ฑ์—์„œ ์ค‘์š”ํ•œ ์—ญํ• ์„ ํ•ฉ๋‹ˆ๋‹ค. ๊ทธ๋ฆฌ๊ณ  ์ฃผ๋Š” ์ง€์—ญ์ ์ธ ๋ฌธ์ œ๋ฅผ ์ฒ˜๋ฆฌํ•˜๋ฉฐ ์—ฐ๋ฐฉ ์ •๋ถ€์™€ ์ƒํ˜ธ์ž‘์šฉํ•˜์—ฌ ๊ณต๋™์˜ ๋ชฉํ‘œ๋ฅผ ๋‹ฌ์„ฑํ•ฉ๋‹ˆ๋‹ค.
61
 
62
- ๊ทธ ๋‹ค์Œ์œผ๋กœ, ๋ฏธ๊ตญ์˜ ํ–‰์ • ์‹œ์Šคํ…œ์€ ์„ธ ๊ฐ€์ง€ ๋Œ€ํŒ์‚ฌ๋กœ ๋‚˜๋‰˜์–ด ์žˆ์Šต๋‹ˆ๋‹ค: ์ง‘ํ–‰๋ถ€(Executive), ๋ฒ•์ฒญ(Judiciary), ์ž…๋ฒ•๋ถ€(Legislative). ์ด๋Ÿฌํ•œ ์„ธ ๊ฐ€์ง€ ๋Œ€ํŒ์‚ฌ๋Š” ๊ฐ๊ฐ ํ–‰์ •, ์‚ฌ๋ฒ•, ์ž…๋ฒ• ์—ญํ• ์„ ์ˆ˜ํ–‰ํ•˜๋Š” ์—ญํ• ์„ ๊ฐ€์ง€๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค.
63
 
64
- ์ง‘ํ–‰๋ถ€๋Š” ๊ตญ๊ฐ€๋ฅผ ์ง€๋„ํ•˜๊ณ  ์‚ฌํšŒ์˜ ์ผ๊ด€์„ฑ์„ ์œ ์ง€ํ•˜๋Š” ์—ญํ• ์„ ํ•ฉ๋‹ˆ๋‹ค. ํ˜„์žฌ ์ง‘ํ–‰๋ถ€๋ฅผ ์ฐจ์ง€ํ•˜๊ณ  ์žˆ๋Š” ์‚ฌ๋žŒ์€ ์˜ค๋ฐ”๋งˆ ๋Œ€ํ†ต๋ น์ž…๋‹ˆ๋‹ค. ์ง‘ํ–‰๋ถ€๋Š” ๋‚ด๊ฐ์„ ํ†ตํ•ด ๊ฐ ๋ถ€์„œ์˜ ์žฅ๊ด€๋“ค์„ ์ž„๋ช…ํ•˜๊ณ , ์ •์ฑ…์„ ์šด์˜ํ•˜๊ณ , ๊ตญ๊ฐ€์˜ ์•ˆ์ „๊ณผ ๊ณตํ—Œ์„ ์œ„ํ•œ ๊ตฐ๋Œ€๋ฅผ ์ง€ํœ˜ํ•ฉ๋‹ˆ๋‹ค.
65
 
66
- ๋ฒ•์ฒญ์€ ์‚ฌ๋ฒ• ๋ถ„์•ผ์—์„œ ์ตœ์ข…์ ์ธ ๊ฒฐ์ •์„ ๋‚ด๋ฆฌ๋Š” ์—ญํ• ์„ ํ•ฉ๋‹ˆ๋‹ค. ์ฃผ์š” ์„ธ ๊ฐœ์˜ ๋ฒ•์› ์ค‘ ํ•˜๋‚˜์ธ ๋ฏธ๊ตญ ์—ฐ๋ฐฉ๋ฒ•์›์ด ์ด ๋ฒ•์ฒญ์— ํ•ด๋‹นํ•ฉ๋‹ˆ๋‹ค. ์ด ๋ฒ•์›์€ ๋ฏธ๊ตญ์˜ ๋Œ€ํ†ต๋ น, ์ƒ์›, ํ•˜์›, ๋˜๋Š” ์†Œ์ˆ˜ ๋ถ„ํŒŒ๋ฅผ ๋Œ€์ƒ์œผ๋กœ ํ•  ์ˆ˜ ์žˆ๋Š” ํ—Œ๋ฒ• ๋ถ„์•ผ์—์„œ ๋ฒ•์ ์ธ ์ตœ์ข… ๊ฒฐ์ •์„ ๋‚ด๋ฆฌ๋Š” ๊ถŒํ•œ์„ ๊ฐ€์ง€๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค.
67
 
68
- ๋งˆ์ง€๋ง‰์œผ๋กœ, ์ž…๋ฒ•๋ถ€๋Š” ๊ตญ๊ฐ€์˜ ๋ฒ•์„ ๋งŒ๋“ค๊ณ  ๋ณ€๊ฒฝํ•˜๋Š” ์—ญํ• ์„ ํ•ฉ๋‹ˆ๋‹ค. ์˜ํšŒ๋Š” 2๊ณณ์˜ ์˜ํšŒ๋กœ ๊ตฌ์„ฑ๋˜์–ด ์žˆ์Šต๋‹ˆ๋‹ค: ์ƒ์›๊ณผ ํ•˜์›์ž…๋‹ˆ๋‹ค. ์ƒ์›์€ 50๊ฐœ ์ฃผ ์ค‘ ํ•˜๋‚˜์—์„œ ์„ ์ถœ๋˜๋ฉฐ, ํ•˜์›์€ ๊ฐ ๊ทผ๋ฌด 1๋…„์— ํ•˜์›์˜์› 435๋ช…์„ ์„ ์ถœํ•ฉ๋‹ˆ๋‹ค. ์ด๋“ค์€ ๋ฏธ๊ตญ ๋Œ€ํ†ต๋ น๊ณผ ๊ฐ ์ฃผ์˜ ์ž…๋ฒ• ํšŒ์˜์™€ ํ•จ๊ป˜ ํ•จ๊ป˜ ๊ตญ๊ฐ€๋ฅผ ์œ„ํ•œ ๋ฒ•์„ ์ž‘์„ฑํ•ฉ๋‹ˆ๋‹ค.
69
 
70
- ์ด์™€ ๊ฐ™์ด ๋ฏธ๊ตญ์˜ ํ–‰์ • ์‹œ์Šคํ…œ์€ ๋ณต์žกํ•˜๊ณ  ๋‹ค์–‘ํ•œ ๊ตฌ์กฐ๋กœ ์ด๋ฃจ์–ด์ ธ ์žˆ์Šต๋‹ˆ๋‹ค. ์ด ์‹œ์Šคํ…œ์€ ๊ฐ๊ฐ์˜ ์—ญํ• ๊ณผ ํŠน์ง•์— ๋”ฐ๋ผ ๊ตญ๊ฐ€์˜ ์šด์˜์„ ์ง€์†์ ์œผ๋กœ ๊ด€๋ฆฌํ•˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค.</๋></s>
71
  ```
72
 
73
  ## 2. Training procedure
@@ -95,3 +120,12 @@ The following `bitsandbytes` quantization config was used during training:
95
 
96
 
97
  - PEFT 0.6.0.dev0
 
 
 
 
 
 
 
 
 
 
2
  library_name: peft
3
  ---
4
 
5
+ # About GPTeacher
6
+
7
+ GPTeacher๋Š” ๋กœ๋ด‡(Bot)์ด ์ถœ๋ ฅํ•˜๋Š” ๋“ฏํ•œ ๊ธฐ์กด LLM ๋ชจ๋ธ์˜ ์ถœ๋ ฅ๊ฐ’์„, ๊ฐ•์˜์ž๊ฐ€ ์‹ค์ œ๋กœ ํ•ด๋‹น ๋‚ด์šฉ์— ๋Œ€ํ•ด ๊ฐ•์˜ํ•˜๋Š” ๊ฒƒ์ฒ˜๋Ÿผ ํ’€์ด๊ณผ์ •์„ ์ƒ์„ฑํ•˜๋Š” ๊ฒƒ์„ ๋ชฉํ‘œ๋กœ ํ•˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค.
8
+ GPTeacher ํ”„๋กœ์ ํŠธ๋ฅผ ํ†ตํ•ด ์ถ”๊ตฌํ•˜๊ณ ์ž ํ•˜๋Š” ๋ชฉํ‘œ๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค.
9
+
10
+ 1. ์˜คํ”ˆ ๋ฐ์ดํ„ฐ์…‹ ์ œ๊ณต : [kullm-v2](https://huggingface.co/datasets/nlpai-lab/kullm-v2), [ko-alpaca](https://huggingface.co/datasets/beomi/KoAlpaca-v1.1a) ๋ฐ์ดํ„ฐ์…‹์˜ output์„ ๊ฐ•์˜ ํ˜•ํƒœ๋กœ ๋ณ€ํ™˜ํ•˜์—ฌ, ๊ธฐ์กด ๋ฐ์ดํ„ฐ์…‹์˜ output๊ณผ ๋”๋ถˆ์–ด extended_output ์นผ๋Ÿผ์„ ์ถ”๊ฐ€ํ•ด ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค.
11
+ 2. PEFTmodel ํ˜น์€ ko-llama2 ๋ชจ๋ธ ์ œ๊ณต : ํ˜„์žฌ ๋ฒ„์ „์€ academic research purpose๋กœ ๋ณ€ํ˜• ๊ฐ€๋Šฅํ•œ [wizardLM](https://github.com/nlpxucan/WizardLM/tree/main)์˜ ํŒŒ๋ผ๋ฏธํ„ฐ๋ฅผ ์ปค์Šคํ…€ ๋ฐ์ดํ„ฐ์…‹์„ ์ ์šฉํ•œ PEFTmodel ํ•™์Šต๋ฐฉ์‹์„ ํ†ตํ•ด ํŒŒ์ธํŠœ๋‹ํ•œ ๋ฒ„์ „์ž…๋‹ˆ๋‹ค. ์—ฌ๋Ÿฌ ๋ฒ„์ „ ์—…๋ฐ์ดํŠธ๋ฅผ ํ†ตํ•ด, ๋‹ค๋ฅธ ๋ชจ๋ธ๋กœ๋ถ€ํ„ฐ ํŒŒ์ธํŠœ๋‹์„ ํ•  ์ˆ˜ ์žˆ๋Š” ํ”„๋ ˆ์ž„์›Œํฌ๋ฅผ ์ œ๊ณตํ•˜๊ฑฐ๋‚˜, GPTeacher-ko-llama2 ๋ชจ๋ธ์„ ์ œ์ž‘ํ•ด ์ œ๊ณตํ•˜๊ณ ์ž ํ•ฉ๋‹ˆ๋‹ค.
12
+
13
+ **ํ˜„์žฌ ์‚ฌ์šฉ๋ชจ๋ธ : WizardLM-13B-v1.2**
14
+ */*์˜์–ด๊ถŒ ๋ฐ์ดํ„ฐ๋ฅผ ์ค‘์‹ฌ์œผ๋กœ ํ•™์Šตํ•œ WizardLM ๋ชจ๋ธ์„ ํŒŒ์ธํŠœ๋‹ ํ•˜์˜€๊ธฐ์— generalํ•œ ํ•œ๊ตญ ์ •๋ณด์— ๊ด€ํ•œ ์งˆ๋ฌธ์— ๋ฏธ์ˆ™ํ•ฉ๋‹ˆ๋‹ค.*/*
15
+
16
+
17
+ | Model | Checkpoint | Paper | MT-Bench | AlpacaEval | GSM8k | HumanEval | Demo | License |
18
+ |----------------------|-------------|--------------|----------|------------|-------|-----------|------|----------------|
19
+ | WizardLM-70B-V1.0 | ๐Ÿค— HF Link | ๐Ÿ“ƒComing Soon| 7.78 | 92.91% | 77.6% | 50.6 | | Llama 2 License|
20
+ | WizardLM-13B-V1.2 | ๐Ÿค— HF Link | | 7.06 | 89.17% | 55.3% | 36.6 | Demo | Llama 2 License|
21
+
22
+ ํ–ฅํ›„ ์ง„ํ–‰ ๋ฐฉํ–ฅ์€ ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค
23
+ 1. more custom dataset์œผ๋กœ ํŒŒ์ธํŠœ๋‹
24
+ 2. ์—ฌ๋Ÿฌ ํŒŒ๋ผ๋ฏธํ„ฐ ๋ชจ๋ธ์— ๋Œ€ํ•ด PEFT learning ์ง„ํ–‰ (30B, 7B, 70B...)
25
+ 3. ko-llama2 ๋ชจ๋ธ์— ๋Œ€ํ•ด ํŒŒ์ธํŠœ๋‹
26
 
27
  ## 1. ์‚ฌ์šฉ์ ˆ์ฐจ
28
 
29
  * Install model and PEFT parameters
30
 
31
+ ```
32
+ !pip install -U peft transformers optimum
33
+ !pip install auto-gptq --extra-index-url https://huggingface.github.io/autogptq-index/whl/cu117/
34
+ ```
35
+
36
  ```
37
  import torch
38
  from peft import PeftModel, PeftConfig
 
58
 
59
  # your input sentence๊ฐ€ ๋“ค์–ด๊ฐˆ ๊ณณ
60
  input = """
61
+ ### input @ ์‚ฌ๊ณผ๊ฐ€ ๋ชธ์— ์ข‹์€ ์ด์œ ๋ฅผ ์•Œ๋ ค์ฃผ์„ธ์š”.\n\n### response @"""
62
 
63
  output = tokenizer.decode(model.cuda().generate(
64
  **tokenizer(
 
80
 
81
  ```
82
  output =
83
+ <s> ์•ˆ๋…•ํ•˜์„ธ์š”. ์˜ค๋Š˜์€ ์‚ฌ๊ณผ์˜ ์žฅ์ ์— ๋Œ€ํ•ด ๋ฐฐ์šฐ๊ฒ ์Šต๋‹ˆ๋‹ค. ์‚ฌ๊ณผ๋Š” ํฌ๋„์ฃผ, ์นด๋ฐ”์˜ˆ๋กœ, ์‹œํŠธ๋Ÿฌ์Šค, ํ”ผ๋ฅด๊ณ ํƒ€ํ†จ๋ฆฌ๊ทธ๋ฆญ์Šค ๋“ฑ ์—ฌ๋Ÿฌ ๋‹ค๋ฅธ ์‹๋ฌผ๋“ค๊ณผ ๋น„๊ตํ•˜๋ฉด์„œ ์šฐ๋ฆฌ์˜ ๊ฑด๊ฐ•์„ ์ตœ์ ํ™”ํ•˜๋Š” ์ค‘์š”ํ•œ ์‹๋ฌผ์ž…๋‹ˆ๋‹ค.
84
 
85
+ ์ด์ œ, ์šฐ์„  ์‚ฌ๊ณผ์˜ ์„ฑ๋ถ„์„ ์‚ดํŽด๋ณด๋„๋ก ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค. ์‚ฌ๊ณผ๋Š” ๊ณ ํ˜ˆ์••์„ ๋‚ฎ์ถ”๋Š” ์—˜๋ฆฌ๊ฐ€ํ†จ๋ฆฐ๊ณผ ํšจ๋ชจ์™€ ๊ฐ™์€ ๋‹จ๋ฐฑ์งˆ, ํ”ผํŠธ์‚ฐ, ํ”ผ๋กœ๋‚˜์‚ฐ ๋“ฑ ์˜์–‘์†Œ๋ฅผ ํ’๋ถ€ํ•˜๊ฒŒ ํ•จ์œ ํ•ฉ๋‹ˆ๋‹ค. ์ด๋Ÿฌํ•œ ์„ฑ๋ถ„๋“ค์€ ์šฐ๋ฆฌ์˜ ์ฒด๋‚ด ์กฐ์ง์— ํ•„์ˆ˜์ ์ธ ๋ฌผ์งˆ๋“ค์ด๋ฏ€๋กœ ์‚ฌ๊ณผ๋ฅผ ์„ญ์ทจํ•  ๋งŒํ•œ ์ข‹์€ ์ด์œ  ์ค‘ ํ•˜๋‚˜์ž…๋‹ˆ๋‹ค.
86
 
87
+ ๋˜ํ•œ, ์‚ฌ๊ณผ๋Š” ๊ณ ๊ธฐ ์†Œ๊ธˆ๊ณผ ํ•จ๊ป˜ ์‚ฌ์šฉํ•˜๋ฉด ๋‹จ๋ฐฑ์งˆ์˜ ์†Œํ™”๋ฅผ ๋†’์ผ ์ˆ˜ ์žˆ์–ด์š”. ๋˜ํ•œ ์ง€๋ฐฉ ์†Œ๊ธˆ๊ณผ ํ•จ๊ป˜ ์‚ฌ์šฉํ•˜๋ฉด ๋‹น์งˆ์˜ ํก์ˆ˜๋ฅผ ์ฆ๊ฐ€์‹œํ‚ฌ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์ด๋Š” ์šฐ๋ฆฌ์˜ ์‹ ์ฒด ๊ธฐ๋Šฅ์„ ๊ฐœ์„ ํ•˜๋Š” ๋ฐ ๋„์›€์ด ๋ฉ๋‹ˆ๋‹ค.
88
 
89
+ ๋˜ํ•œ, ์‚ฌ๊ณผ๋Š” ๋น ๋ฅด๊ฒŒ ์†Œํ™”๋˜์–ด ํƒ„์ˆ˜ํ™”๋ฌผ์„ ๊ณต๊ธ‰ํ•  ์ˆ˜ ์žˆ๋Š” ๊ฒƒ์ด ์žฅ์ ์ด์ฃ . ์ด๋Š” ์ฒด๋‚ด ์—๋„ˆ์ง€์›์œผ๋กœ ์ž‘๋™ํ•˜๋Š” ๋Šฅ๋ ฅ์„ ๋†’์—ฌ์ค๋‹ˆ๋‹ค. ์‚ฌ๊ณผ๋ฅผ ์ •๊ตํ•˜๊ฒŒ ๋จน์„ ๊ฒฝ์šฐ ๏ฟฝ๏ฟฝ๊ทœ์น™ํ•œ ํ”ผ์ž„์„ ๋ฐฉ์ง€ํ•˜๊ณ  ์‹์ด๋งž์€ ์šด๋™์„ ์œ ์ง€ํ•˜๋Š” ๋ฐ ๋„์›€์ด ๋ฉ๋‹ˆ๋‹ค.
90
 
91
+ ๋งˆ์ง€๋ง‰์œผ๋กœ, ์‚ฌ๊ณผ๋Š” ํ”ผ๋กœ๋‹น๊ณผ ์นดํด๋ผ๋‹ฌ๋ ˆ๊ฐ€ ๋งŽ์ด ๋“ค์–ด ์žˆ์–ด์š”. ์ด๋Š” ๊ทผ์œก์— ๋ฏธ์„ธํ•œ ํž˜์„ ์ œ๊ณตํ•˜์—ฌ ๋†๊ตฌ, ํ…Œ๋‹ˆ์Šค ๋“ฑ ๋‹ค์–‘ํ•œ ์ข…๋ฅ˜์˜ ์šด๋™์„ ์œ„ํ•ด ํ•„์š”ํ•œ ์š”์†Œ์ž…๋‹ˆ๋‹ค. ์ด๋Š” ์šฐ๋ฆฌ์˜ ์šด๋™์„ ๋” ์ž˜ ์ˆ˜ํ–‰ํ•˜๋Š” ๋ฐ ๋„์›€์ด ๋ฉ๋‹ˆ๋‹ค.
92
 
93
+ ๋”ฐ๋ผ์„œ, ์‚ฌ๊ณผ๋ฅผ ์„ญ์ทจํ•  ๋•Œ๋Š” ์ฃผ๋กœ ์•„์นจ์ด๋‚˜ ์•„์นจ ์‹์‚ฌ ์ „์— ๋จน๋Š” ๊ฒƒ์ด ์ข‹์Šต๋‹ˆ๋‹ค. ๋˜ํ•œ, ์‚ฌ๊ณผ๋Š” ๋งค์šฐ ํก์น˜๊ธฐ ์‰ฝ๊ณ  ๋‹ค์–‘ํ•œ ์žฌ๋ฃŒ๋ฅผ ๊ฐ€๋ณ€ํ•  ์ˆ˜ ์žˆ๊ธฐ ๋•Œ๋ฌธ์— ์ฐฝ๊ณ ์— ๋ณด๊ด€ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์˜ฌ๋ฐ”๋ฅธ ์กฐ๋ฆฌ ๋ฐฉ๋ฒ•์„ ์•Œ๊ณ  ์žˆ์–ด์•ผ ํ•˜๋Š”๋ฐ, ์ฃผ๋กœ ํ• ์ธ์œผ๋กœ ๊ฐ€๊ณตํ•˜์—ฌ ์ถฉ๋ถ„ํ•œ ์Œ์‹์„ ๋งŒ๋“ค ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
94
 
95
+ ์ด์ƒ์œผ๋กœ ์‚ฌ๊ณผ์— ๋Œ€ํ•ด ์•Œ๋ ค๋“œ๋ ธ์Šต๋‹ˆ๋‹ค. ์‚ฌ๊ณผ๋Š” ์šฐ๋ฆฌ์˜ ๊ฑด๊ฐ•์„ ์ตœ์ ํ™”ํ•˜๋Š” ๋ฐ ํ•„์ˆ˜์ ์ธ ์š”์†Œ์ž…๋‹ˆ๋‹ค. ๊ทธ๋ ‡๊ธฐ ๋•Œ๋ฌธ์— ์„ญ์ทจํ•˜๋Š” ๊ฒƒ์ด ์ค‘์š”ํ•ฉ๋‹ˆ๋‹ค. ๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค.</๋></s>
96
  ```
97
 
98
  ## 2. Training procedure
 
120
 
121
 
122
  - PEFT 0.6.0.dev0
123
+
124
+ @misc{xu2023wizardlm,
125
+ title={WizardLM: Empowering Large Language Models to Follow Complex Instructions},
126
+ author={Can Xu and Qingfeng Sun and Kai Zheng and Xiubo Geng and Pu Zhao and Jiazhan Feng and Chongyang Tao and Daxin Jiang},
127
+ year={2023},
128
+ eprint={2304.12244},
129
+ archivePrefix={arXiv},
130
+ primaryClass={cs.CL}
131
+ }