---
library_name: transformers
license: cc-by-nc-4.0
datasets:
- kyujinpy/KOR-OpenOrca-Platypus-v3
language:
- ko
- en
tags:
- Economic
- Finance
---
# Model Details
Model Developers: Sogang University SGEconFinlab(<)
### Model Description
This model is a language model specialized in economics and finance. This was learned with various economic/finance-related data.
The data sources are listed below, and we are not releasing the data that we trained on because it was used for research/policy purposes.
If you wish to use the original data, please contact the original author directly for permission to use it.
- **Developed by:** Sogang University SGEconFinlab()
- **License:** cc-by-nc-4.0
- **Base Model:** yanolja/KoSOLAR-10.7B-v0.2()
## Loading the Model
peft_model_id = "SGEcon/KoSOLAR-10.7B-v0.2_fin_v4"
config = PeftConfig.from_pretrained(peft_model_id)
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16
)
model = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path, quantization_config=bnb_config, device_map={"":0})
model = PeftModel.from_pretrained(model, peft_model_id)
tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)
model.eval()
## Conducting Conversation
import re
def gen(x):
inputs = tokenizer(f"### 질문: {x}\n\n### 답변:", return_tensors='pt', return_token_type_ids=False)
# 데이터를 GPU로 이동(사용 가능한 경우)
inputs = {k: v.to(device="cuda" if torch.cuda.is_available() else "cpu") for k, v in inputs.items()}
gened = model.generate(
**inputs,
max_new_tokens=256, # 새로 생성할 토큰의 최대 개수
early_stopping=True,
num_return_sequences=1, # 하나의 답변만 생성
do_sample=True, # 다양한 답변 생성을 위해 샘플링 활성화
eos_token_id=tokenizer.eos_token_id, # EOS 토큰 ID 사용
temperature=0.9, # 생성 다양성 조절을 위한 온도 설정
top_p=0.8, # nucleus sampling에서 사용할 p 값
top_k=50 # top-k sampling에서 사용할 k 값
)
# 생성된 시퀀스를 디코드하여 출력 텍스트로 변환
decoded = tokenizer.decode(gened[0], skip_special_tokens=True).strip()
# "### 답변:" 문자열 이후의 텍스트만 추출
answer_start_idx = decoded.find("### 답변:") + len("### 답변:")
complete_answer = decoded[answer_start_idx:].strip()
# 첫 번째 구두점(. ? !)을 찾아서 그 부분까지만 추출
match = re.search(r"[\.\?\!][^\.\?\!]*$", complete_answer)
if match:
complete_answer = complete_answer[:match.end()].strip()
return complete_answer
## Training Details
First, we loaded the base model quantized to 4 bits. It can significantly reduce the amount of memory required to store the model's weights and intermediate computation results, which is beneficial for deploying models in environments with limited memory resources. It can also provide faster inference speeds.
Then,
### Training Data
1. 한국은행: 경제금융용어 700선()
2. 금융감독원: 금융소비자 정보 포털 파인 금융용어사전()
3. KDI 경제정보센터: 시사 용어사전()
4. 한국경제신문/한경닷컴: 한경경제용어사전(), 오늘의 TESAT(), 오늘의 주니어 TESAT(), 생글생글한경()
5. 중소벤처기업부/대한민국정부: 중소벤처기업부 전문용어()
6. 고성삼/법문출판사: 회계·세무 용어사전()
7. 맨큐의 경제학 8판 Word Index
8. yanolja/KoSOLAR-10.7B-v0.2()
### Training Procedure
#### Training Hyperparameters
|Hyperparameter|SGEcon/KoSOLAR-10.7B-v0.2_fin_v4|
|------|---|
|Lora Method|Lora|
|load in 4 bit|True|
|learning rate|1e-5|
|lr scheduler|linear|
|lora alpa|16|
|lora rank|16|
|lora dropout|0.05|
|optim|paged_adamw_32bit|
|target_modules|q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj, lm_head|
## Evaluation
### Testing Data, Factors & Metrics
#### Testing Data
[More Information Needed]
### Results
[More Information Needed]
#### Summary
## Citation [optional]