Upload README.md

README.md

@@ -0,0 +1,160 @@
+---
+base_model: llm-jp/llm-jp-3-13b
+library_name: peft
+tags:
+- text-generation-inference
+- llama
+- trl
+license: apache-2.0
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** togepi55 
+- **Funded by [optional]:** llm-jp/llm-jp-3-13b 
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** apache-2.0
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### 注意
+プロンプトは形式でのみ学習しています。
+~~~
+"<s>以下は、タスクを説明する指示です。要求を適切に満たす応答を書きなさい
+
+### 指示:
+{instruction}
+
+### 応答:"
+~~~
+
+### サンプルコード
+
+~~~python
+import torch
+from transformers import (
+    AutoTokenizer,
+    AutoModelForCausalLM,
+    BitsAndBytesConfig,
+)
+from transformers import TextStreamer
+
+
+BASE_MODEL = "togepi55/llm-jp-3-13b-it"
+
+tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
+bnb_config = BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_compute_dtype=torch.float16,
+    bnb_4bit_quant_type="nf4",
+    bnb_4bit_use_double_quant=False,
+)
+
+model = AutoModelForCausalLM.from_pretrained(
+            BASE_MODEL,
+            device_map="auto",
+            quantization_config=bnb_config,
+            #torch_dtype=torch.bfloat16,
+            torch_dtype="auto",
+            trust_remote_code=True,
+        )
+
+streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
+
+
+instruction = "東京は日本の"
+
+
+prompt = f"<s>以下は、タスクを説明する指示です。要求を適切に満たす応答を書きなさい\n\n### 指示:\n{instruction}\n\n### 応答:\n"
+print(prompt)
+model_input = tokenizer(prompt, return_tensors="pt").to(model.device)
+input_ids = model_input["input_ids"]
+
+model.eval()
+with torch.no_grad():
+    result = model.generate(
+              input_ids,
+              max_new_tokens=300,
+              attention_mask = model_input.attention_mask,
+              pad_token_id=tokenizer.pad_token_id,
+              eos_token_id=tokenizer.eos_token_id,
+              do_sample=False,
+              #num_return_sequences=3,
+              streamer=streamer,
+              repetition_penalty=1.02,
+          )
+    print("----"*20)
+    del input_ids
+~~~
+
+
+
+
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+RLHF，DPOを実施していないため不適切な表現が出力される可能性があります。
+
+## Training Details
+
+### Training Data
+
+指示チューニングデータとして下記のものを利用しました。
+* ichikara-instruction-003-001-1.json
+* ichikara-instruction-003-002-1.json 
+* elyza/ELYZA-tasks-100
+
+### SFTの概要
+* 4bit量子化
+* LoRAによるSFT
+* learning_rate = 2e-4
+* optim="adamw_torch_fused"
+* lr_scheduler_type="cosine"
+* weight_decay=0.01
+
+