Model Card for yuhkis/llm-jp-3-13b-finetune
Model Details
Model Description
This is a LoRA-tuned version of LLM-jp-3-13b, fine-tuned on the Ichikara Instruction dataset.
- Developed by: Yuhki Shiraishi
- Model type: Instruction-tuned Japanese Language Model
- Language: Japanese
- License: CC-BY-NC-SA
- Finetuned from model: llm-jp/llm-jp-3-13b
Uses
Output Generation and Format
Implementation Details
To generate output in the required JSONL format:
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch
from tqdm import tqdm
import json
# Load model and tokenizer
model_id = "yuhkis/llm-jp-3-13b-finetune"
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16,
)
model = AutoModelForCausalLM.from_pretrained(
model_id,
quantization_config=bnb_config,
device_map="auto",
token=HF_TOKEN
)
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True, token=HF_TOKEN)
# Generate outputs
results = []
for data in tqdm(datasets):
input = data["input"]
prompt = f"""### 指示
{input}
### 回答
"""
tokenized_input = tokenizer.encode(prompt, add_special_tokens=False, return_tensors="pt").to(model.device)
attention_mask = torch.ones_like(tokenized_input)
with torch.no_grad():
outputs = model.generate(
tokenized_input,
attention_mask=attention_mask,
max_new_tokens=100,
do_sample=False,
repetition_penalty=1.2,
pad_token_id=tokenizer.eos_token_id
)[0]
output = tokenizer.decode(outputs[tokenized_input.size(1):], skip_special_tokens=True)
results.append({"task_id": data["task_id"], "output": output})
# Save results to JSONL file
with open("results.jsonl", 'w', encoding='utf-8') as f:
for result in results:
json.dump(result, f, ensure_ascii=False)
f.write('\n')
Output Format Specification
Required fields in the JSONL output:
- task_id: Task identifier (integer)
- output: Generated response (string)
Example output format:
{"task_id": 0, "output": "応答テキスト"}
Note: While additional fields (e.g., input, eval_aspect) may be included, only task_id and output are required for submission.
### Out-of-Scope Use
This model should not be used for:
- Commercial applications due to license restrictions
- Critical decision-making without human oversight
- Applications requiring strict reliability guarantees
## Bias, Risks, and Limitations
- The model inherits biases from its training data
- Output quality may vary depending on input complexity
- The model should not be used for making critical decisions without human oversight
### Recommendations
Users should be aware of the model's limitations and verify outputs when used in applications.
## Training Details
### Training Data
- Dataset: Ichikara Instruction Dataset
### Training Procedure
- **Training regime:** bf16 mixed precision
- **Library:** 🤗 Transformers
- **Optimization:** LoRA (Low-Rank Adaptation)
## Technical Specifications
### Model Architecture
- Base model: LLM-jp-3-13b
- Adaptation method: LoRA
## Citation
**BibTeX:**
```bibtex
@misc{shiraishi2024llm,
title={LLM-jp-3-13b-finetune: Instruction-tuned Japanese Language Model},
author={Yuhki Shiraishi},
year={2024},
publisher={Hugging Face},
howpublished={\url{https://huggingface.co/yuhkis/llm-jp-3-13b-finetune}}
}
Base Model Citation:
@misc{llm-jp2024,
title={LLM-jp-3: Large Language Model for Japanese},
author={LLM-jp Project Team},
year={2024},
publisher={Hugging Face},
howpublished={\url{https://huggingface.co/llm-jp/llm-jp-3-13b}}
}
Training Data Citation:
関根聡, 安藤まや, 後藤美知子, 鈴木久美, 河原大輔, 井之上直也, 乾健太郎.
ichikara-instruction: LLMのための日本語インストラクションデータの構築.
言語処理学会第30回年次大会(2024)
Model Card Contact
Primary Contact:
- Name: Yuhki Shiraishi
- GitHub: @yuhkis
For questions regarding this model, please open an issue in the GitHub repository or contact via HuggingFace discussion forum.
Please include "LLM-jp-3-13b-finetune" in the subject line of any correspondence.
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API:
The model has no pipeline_tag.