README.md

metadata

language:
  - ko
  - en

Model Details

Model Developers: Sogang University SGEconFinlab(<https://sc.sogang.ac.kr/aifinlab/)

Model Description

This model is a language model specialized in economics and finance. This was learned with various economic/finance-related data. The data sources are listed below, and we are not releasing the data that we trained on because it was used for research/policy purposes. If you wish to use the original data, please contact the original author directly for permission to use it.

• Developed by: Sogang University SGEconFinlab(https://sc.sogang.ac.kr/aifinlab/)
• License: cc-by-nc-4.0
• Base Model: SGEcon/KoSOLAR-10.7B-v0.2_fin_v4(https://huggingface.co/SGEcon/KoSOLAR-10.7B-v0.2_fin_v4)

Loading the Model

peft_model_id = "SGEcon/KoSOLAR-10.7B-v0.2_fin_v4_dpo"
config = PeftConfig.from_pretrained(peft_model_id)
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16
)
model = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path, quantization_config=bnb_config, device_map={"":0})
model = PeftModel.from_pretrained(model, peft_model_id)
tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)
model.eval()

Conducting Conversation

import re

def gen(x):
    inputs = tokenizer(f"### 질문: {x}\n\n### 답변:", return_tensors='pt', return_token_type_ids=False)

    # Move data to GPU (if available)
    inputs = {k: v.to(device="cuda" if torch.cuda.is_available() else "cpu") for k, v in inputs.items()}

    gened = model.generate(
        **inputs,
        max_new_tokens=256,  # Maximum number of new tokens to create
        early_stopping=True,
        num_return_sequences=1,  # Generate only one answer
        do_sample=True,  # Enable sampling to generate a variety of answers
        eos_token_id=tokenizer.eos_token_id,  # Using EOS Token IDs 
        temperature=0.9,  # This option is adjustable.
        top_p=0.8,  # This option is adjustable.
        top_k=100  # This option is adjustable.
    )

    # Decode the generated sequence and convert it to output text 
    decoded = tokenizer.decode(gened[0], skip_special_tokens=True).strip()

    # Extract only text after a string "### 답변:" 
    answer_start_idx = decoded.find("### 답변:") + len("### 답변:")
    complete_answer = decoded[answer_start_idx:].strip()

    # Find the first punctuation mark (. ? !) and extract only up to it
    match = re.search(r"[\.\?\!][^\.\?\!]*$", complete_answer)
    if match:
        complete_answer = complete_answer[:match.end()].strip()

    return complete_answer

Training Details

Training our model with PEFT, LoRA, DPO and Merge.

• Low-Rank Adaptation (LoRA) fixes the weights of the pretrained model and attaches learnable rank decomposition matrices to each layer of the transformer, updating only these when finetuning. In other words, LoRA is a methodology that uses low-dimensional intrinsic rank (the number of dimensions that best describe the data for a given layer or parameter) for finetuning.

• PEFT is a technique that does not tune all parameters of a model during fine-tuning, but only a small subset of parameters. By tuning only a few parameters while leaving others fixed, the model is less likely to suffer from catastrophic forgetting, where the model forgets previously learned tasks when it learns new ones. By tuning only a few parameters, models can be trained for different tasks such as QA, Summarize, and Generate PEFT.

• Direct Preference Optimization (DPO) is an alternative to Reinforcement Learning from Human Feedback (RLHF). RLHF creates a reward function with human-selected data from multiple LLMs' answers to the same question, and then performs reinforcement learning on the reward function to improve model performance. DPO also uses preference data but trains directly without a reward function. We selected relatively important data from the data learned by the base model, asked the base model, and extracted four answers. All 4 answers were rejected, and the original answer was selected to create the dpo dataset. Then, combine our dataset and mncai/orca_dpo_pairs_ko dataset which published on huggingface.

• Merge is a way to mix two or more models into a single model. Because merge is not training, it has the advantage of being very fast, requiring only CPU computation.

Training Data

1. our dpo dataset

• It is not to be used for commercial purposes. Therefore, it is licensed under the license CC-BY-NC-4.0.

2. mncai/orca_dpo_pairs_ko(https://huggingface.co/datasets/mncai/orca_dpo_pairs_ko)

Training Hyperparameters

Hyperparameter	SGEcon/KoSOLAR-10.7B-v0.2_fin_v4_dpo
Lora Method	Lora
load in 4 bit	True
learning rate	1e-5
lr scheduler	cosine
lora alpa	8
lora rank	32
lora dropout	0.05
optim	adamw_torch
target_modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj, lm_head

License

The language identification model is distributed under the Creative Commons Attribution-NonCommercial 4.0 International Public License.

Example

중앙은행의 역할에 대해서 설명해줄래?

중앙은행은 국가의 통화 및 금융 시스템을 관리하는 정부 기관입니다. 주요 기능은 다음과 같습니다: 1. 통화 공급 관리: 중앙은행은 통화 공급량을 조절하여 물가를 안정시키고 경제 성장을 촉진합니다. 이를 위해 기준 금리를 설정하고, 유동성을 조절하며, 자산 매입 또는 매각을 통해 통화 공급량을 조절합니다. 2. 경제 안정화: 중앙은행은 경제가 과열되거나 침체하는 것을 방지하기 위해 통화 정책을 조정합니다. 예를 들어, 경제가 과열되면 중앙은행은 기준 금리를 인상하여 과열을 억제할 수 있습니다. 반대로, 경제가 침체하면 중앙은행은 기준 금리를 인하하여 경제 성장을 촉진할 수 있습니다. 3. 금융 시스템 감독: 중앙은행은 금융 시스템의 안정성을 보장하기 위해 은행 및 기타 금융 기관을 감독하고 규제합니다. 이는 위험 관리, 자본 요구 사항 및 감독 요건을 설정하는 것을 포함합니다. 4. 외환 관리: 중앙은행은 외환 시장을 안정화하기 위해 외환 정책을 수립하고 시행합니다.