Update README.md

README.md

@@ -8,27 +8,18 @@ pipeline_tag: text-generation
 # Model Details
 Model Developers: Sogang University SGEconFinlab
 
+
 ### Model Description
 
 This model is a language model specialized in economics and finance. This was learned with various economic/finance-related data.
 The data sources are listed below, and we are not releasing the data we trained on because it was used for research/policy purposes. 
 If you wish to use the original data rather than our training data, please contact the original author directly for permission to use it.
 
-- **Developed by:** Sogang University SGEconFinlab
+- **Developed by:** Sogang University SGEconFinlab(<https://sc.sogang.ac.kr/aifinlab/>)
 - **Language(s) (NLP):** Ko/En
 - **License:** apache-2.0
 - **Base Model:** yanolja/KoSOLAR-10.7B-v0.2
 
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
 
 ## How to Get Started with the Model
 
@@ -46,7 +37,7 @@ If you wish to use the original data rather than our training data, please conta
     tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)
     model.eval()
 
-
+-------
     import re
     def gen(x):
         inputs = tokenizer(f"### 질문: {x}\n\n### 답변:", return_tensors='pt', return_token_type_ids=False)
@@ -88,31 +79,33 @@ If you wish to use the original data rather than our training data, please conta
     
 ## Training Details
 
+
 ### Training Data
 
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+1. 한국은행: 경제금융용어 700선(<https://www.bok.or.kr/portal/bbs/B0000249/view.do?nttId=235017&menuNo=200765>)
+2. 금융감독원: 금융소비자 정보 포털 파인 금융용어사전(<https://fine.fss.or.kr/fine/fnctip/fncDicary/list.do?menuNo=900021>)
+3. KDI 경제정보센터: 시사 용어사전(<https://eiec.kdi.re.kr/material/wordDic.do>)
+4. 한국경제신문/한경닷컴: 한경경제용어사전(<https://terms.naver.com/list.naver?cid=42107&categoryId=42107>), 오늘의 TESAT(<https://www.tesat.or.kr/bbs.frm.list/tesat_study?s_cateno=1>), 오늘의 주니어 TESAT(<https://www.tesat.or.kr/bbs.frm.list/tesat_study?s_cateno=5>), 생글생글한경(<https://sgsg.hankyung.com/tesat/study>)
+5. 중소벤처기업부/대한민국정부: 중소벤처기업부 전문용어(<https://terms.naver.com/list.naver?cid=42103&categoryId=42103>)
+6. 고성삼/법문출판사: 회계·세무 용어사전(<https://terms.naver.com/list.naver?cid=51737&categoryId=51737>)
+7. 맨큐의 경제학 8판 Word Index
 
-[More Information Needed]
 
 ### Training Procedure 
 
 <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
 
-#### Preprocessing [optional]
-
-[More Information Needed]
-
 
 #### Training Hyperparameters
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
+- Lora
+1. r=16,
+    lora_alpha=16,
+    target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj", "lm_head"], # this is different by models 
+    lora_dropout=0.05,
+    bias="none",
+    task_type="CAUSAL_LM"
+   
 ## Evaluation
 
 <!-- This section describes the evaluation protocols and provides the results. -->
@@ -125,18 +118,6 @@ If you wish to use the original data rather than our training data, please conta
 
 [More Information Needed]
 
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
 ### Results
 
 [More Information Needed]