Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,55 @@
|
|
1 |
---
|
2 |
license: cc-by-3.0
|
|
|
|
|
|
|
3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
license: cc-by-3.0
|
3 |
+
language:
|
4 |
+
- ko
|
5 |
+
pipeline_tag: text-generation
|
6 |
---
|
7 |
+
# korean-gpt-neox-125M
|
8 |
+
|
9 |
+
## Model Details
|
10 |
+
|
11 |
+
### Model Description
|
12 |
+
|
13 |
+
<!-- Provide a longer summary of what this model is. -->
|
14 |
+
|
15 |
+
- **Developed by:** [cateto](http://github.com/cateto)
|
16 |
+
- **Model type:** [gpt-neox](https://github.com/EleutherAI/gpt-neox)
|
17 |
+
- **Language(s) (NLP):** Korean
|
18 |
+
|
19 |
+
## Uses
|
20 |
+
|
21 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
22 |
+
|
23 |
+
### Direct Use
|
24 |
+
|
25 |
+
```python
|
26 |
+
# Import the transformers library
|
27 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
28 |
+
|
29 |
+
tokenizer = AutoTokenizer.from_pretrained("cateto/korean-gpt-neox-125M")
|
30 |
+
|
31 |
+
model = AutoModelForCausalLM.from_pretrained("cateto/korean-gpt-neox-125M")
|
32 |
+
|
33 |
+
# Get user input
|
34 |
+
user_input = "์ฐ๋ฆฌ๋ ์์ผ๋ก ๋๋์ ๋ฏธ๋๋ฅผ"
|
35 |
+
|
36 |
+
# Encode the prompt using the tokenizer
|
37 |
+
input_ids = tokenizer.encode(user_input, return_tensors="pt")
|
38 |
+
|
39 |
+
# Generate chatbot output using the model
|
40 |
+
output_ids = model.generate(
|
41 |
+
input_ids,
|
42 |
+
num_beams=4,
|
43 |
+
repetition_penalty=1.5,
|
44 |
+
no_repeat_ngram_size=3
|
45 |
+
)
|
46 |
+
|
47 |
+
# Decode chatbot output ids as text
|
48 |
+
bot_output = tokenizer.decode(output_ids.tolist()[0], skip_special_tokens=True)
|
49 |
+
|
50 |
+
# Print chatbot output
|
51 |
+
print(f"์ถ๋ ฅ ## ", bot_output)
|
52 |
+
|
53 |
+
# ์ถ๋ ฅ ## ์ฐ๋ฆฌ๋ ์์ผ๋ก ๋๋์ ๋ฏธ๋๋ฅผ ํฅํด ๋์๊ฐ ์ ์๋ค.
|
54 |
+
```
|
55 |
+
|