README.md

metadata

language: ko
license: apache-2.0

1. KONI-Llama3-8B-Instruct-20240729

• KoRnDAlpaca is based on Korean and fine-tuned with 1 million instruction data (R&D Instruction dataset v1.3) generated from Korean national research reports.
• The base model of KoRnDAlpaca is EleutherAI/polyglot-en-12.8b.
• For more information about the training procedure and model, please contact gsjang@kisti.re.kr.

2. How to use the model

import transformers
import torch

model_id = "KISTI-KONI/KONI-Llama3-8B-Instruct-20240729"

pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device_map="auto",
)

pipeline.model.eval()

instruction = "CNN과 RNN 모델의 차이점을 상세히 설명하고, 각각 적용 가능한 분야에 대해서도 설명하시오."

messages = [
   {"role": "user", "content": f"{instruction}"}
    ]

prompt = pipeline.tokenizer.apply_chat_template(
        messages, 
        tokenize=False, 
        add_generation_prompt=True
)

terminators = [
    pipeline.tokenizer.eos_token_id,
    pipeline.tokenizer.convert_tokens_to_ids("<|eot_id|>")
]

outputs = pipeline(
    prompt,
    max_new_tokens=8192,
    eos_token_id=terminators,
    do_sample=True,
    temperature=0.7,
    top_p=0.9
)

print(outputs[0]["generated_text"][len(prompt):])

3. Data

• 약 1000개의 SFT데이터와 약 7000개의 DPO 데이터를 사용
• SFT 데이터는 자체 구축한 데이터, huggingface에 공개된 데이터를 사용하였으며, 영어의 경우 한국어로 번역하여 사용
• DPO 데이터는 argilla/dpo-mix-7k를 번역 및 검수한 데이터를 사용

4. Future plans

• 23.10: Release KoRnDAlpaca v2 (adds the ability to generate long-form technology trend information in Markdown format)
• 23.12: Release NITS-seachGPT module v1 (Retriever + KoRnDAlpaca v3)
  • ※ R&D-specific open-domain question answering module with "Retriever + Generator" structure
  • ※ NTIS-searchGPT v1 is an early edition, with anticipated performance improvements scheduled for 2024.
• 23.12: KoRnDAlpaca v2 will be applied to the chatbot of NTIS (www.ntis.go.kr)

5. Date of last update

• 2024.07.29

References

• https://huggingface.co/meta-llama/Meta-Llama-3-8B
• meta-llama/Meta-Llama-3-8B-Instruct