metadata
tags:
- text-generation
language:
- ko
base_model: yanolja/KoSOLAR-10.7B-v0.1
pipeline_tag: text-generation
DataVortexS-10.7B-v0.2
License
Will be updated soon ...
Model Details
Base Model
Trained On
H100 80GB 1ea
Instruction format
It follows (No Input) Alpaca format.
Model Benchmark
Ko-LLM-Leaderboard
On Benchmarking...
Implementation Code
Since, chat_template already contains insturction format above. You can use the code below.
from transformers import AutoModelForCausalLM, AutoTokenizer
device = "cuda"
model = AutoModelForCausalLM.from_pretrained("Edentns/DataVortexS-10.7B-v0.2", device_map=device)
tokenizer = AutoTokenizer.from_pretrained("Edentns/DataVortexS-10.7B-v0.2")
messages = [
{ "role": "user", "content": "๋ํ๋ฏผ๊ตญ์ ์๋๋ ์ด๋์ผ?" }
]
encoded = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt",
return_token_type_ids=False
).to(device)
decoded = model.generate(
input_ids=encoded,
temperature=0.2,
top_p=0.9,
repetition_penalty=1.2,
do_sample=True,
max_length=4096,
eos_token_id=tokenizer.eos_token_id,
pad_token_id=tokenizer.eos_token_id
)
decoded = decoded[0][encoded.shape[1]:decoded[0].shape[-1]]
decoded_text = tokenizer.decode(decoded, skip_special_tokens=True)
print(decoded_text)
