heegyu commited on
Commit
0c56a01
Β·
1 Parent(s): a5b8b7c

README update

Browse files
Files changed (1) hide show
  1. README.md +38 -0
README.md CHANGED
@@ -1,3 +1,41 @@
1
  ---
2
  license: cc-by-nc-sa-4.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: cc-by-nc-sa-4.0
3
  ---
4
+
5
+ [skt/kogpt2-base-v2](https://huggingface.co/skt/kogpt2-base-v2)λ₯Ό AIHub μΌμƒλŒ€ν™” λ°μ΄ν„°μ…‹μœΌλ‘œ νŒŒμΈνŠœλ‹ν•œ λͺ¨λΈμž…λ‹ˆλ‹€.<br/>
6
+ ν•™μŠ΅ μ½”λ“œ: https://github.com/HeegyuKim/open-domain-dialog<br/>
7
+ Streamlit Demo: https://heegyukim-open-domain-dialog-st-demo-1tzktp.streamlitapp.com/<br/>
8
+
9
+
10
+ ## μ‚¬μš©μ˜ˆμ‹œ
11
+ ```
12
+ tokenizer = AutoTokenizer.from_pretrained("heegyu/kodialogpt")
13
+ model = AutoModelForCausalLM.from_pretrained("heegyu/kodialogpt")
14
+ generator = pipeline("text-generation", model=model, tokenizer=tokenizer)
15
+
16
+ generation_args = dict(
17
+ num_beams=4,
18
+ repetition_penalty=2.0,
19
+ no_repeat_ngram_size=4,
20
+ eos_token_id=375, # \n
21
+ max_new_tokens=64,
22
+ do_sample=True,
23
+ top_k=50,
24
+ early_stopping=True
25
+ )
26
+ generator(
27
+ ["0 : **λŠ” κ²Œμž„ μ’‹μ•„ν•˜λ‹ˆ\n1 :",
28
+ "0 : μ–΄μ œ κ°•λ‚¨μ—μ„œ 살인사건 λ‚¬λŒ€ γ…œγ…œ λ„ˆλ¬΄ λ¬΄μ„œμ›Œ\n1 : 헐 μ™œ? 무슨 일 μžˆμ—ˆμ–΄?\n0 : μ‚¬μ§„λ³΄λ‹ˆκΉŒ 막 ν”Όν˜λ¦¬λŠ” μ‚¬λžŒμžˆκ³  경찰듀이 λ– μ„œ μ œμ••ν•˜κ³  λ‚œλ¦¬λ„ μ•„λ‹ˆμ—ˆλ‹€λ˜λ°??\n1 :",
29
+ "0 : μžκΈ°μ•Ό μ–΄μ œλŠ” λ‚˜ν•œν…Œ μ™œ κ·Έλž¬μ–΄?\n1 : λ­” 일 μžˆμ—ˆμ–΄?\n0 : μ–΄λ–»κ²Œ λ‚˜ν•œν…Œ 말도 없이 그럴 수 μžˆμ–΄? λ‚˜ μ§„μ§œ μ‹€λ§ν–ˆμ–΄\n1 : "],
30
+ **generation_args
31
+ )
32
+ ```
33
+ κ²°κ³Ό
34
+ - eos_token_idλ₯Ό '\n'으둜 ν•΄μ€˜μ„œ μ§€λ €μ•Όν•˜λŠ”λ° generation된거 λ³΄λ‹ˆ μ•ˆμ§€λ¦¬λ„€μš”... μ΄μœ κ°€ λ­˜μ§€ γ… 
35
+ ```
36
+ [[{'generated_text': '0 : **λŠ” κ²Œμž„ μ’‹μ•„ν•˜λ‹ˆ\n1 : ν‚€ν‚€ λ­” κ²Œμž„ μ’‹μ•„ν•˜λƒ₯\n'}],
37
+ [{'generated_text': '0 : μ–΄μ œ κ°•λ‚¨μ—μ„œ 살인사건 λ‚¬λŒ€ γ…œγ…œ λ„ˆλ¬΄ λ¬΄μ„œμ›Œ\n1 : 헐 μ™œ? 무슨 일 μžˆμ—ˆμ–΄?\n0 : μ‚¬μ§„λ³΄λ‹ˆκΉŒ 막 ν”Όν˜λ¦¬λŠ” μ‚¬λžŒμžˆκ³  경찰듀이 λ– μ„œ μ œμ••ν•˜κ³  λ‚œλ¦¬λ„ μ•„λ‹ˆμ—ˆλ‹€λ˜λ°??\n1 : 헐 λ―Έμ³€λ‹€... μ§„μ§œ λ¬΄μ„œμ› κ² λ‹€\n0 : 그치 근데 ν”Όν•΄μžκ°€ ν‰κΈ°λ‘œ 찌λ₯Έ 것 κ°™μ•„\n'}],
38
+ [{'generated_text': '0 : μžκΈ°μ•Ό μ–΄μ œλŠ” λ‚˜ν•œν…Œ μ™œ κ·Έλž¬μ–΄?\n1 : λ­” 일 μžˆμ—ˆμ–΄?\n0 : μ–΄λ–»κ²Œ λ‚˜ν•œν…Œ 말도 없이 그럴 수 μžˆμ–΄? λ‚˜ μ§„μ§œ μ‹€λ§ν–ˆμ–΄\n1 : 뭘 λ§ν–ˆλŠ”λ°?\n0 : λ‚΄κ°€ λ„ˆν•œν…Œ 뭐라고 ν–ˆμž–μ•„\n'}]]
39
+ ```
40
+
41
+ ν•™μŠ΅μ— μ‚¬μš©ν•œ ν•˜μ΄νΌνŒŒλΌλ―Έν„°