---
library_name: transformers
tags:
- llama
- trl
datasets:
- elyza/ELYZA-tasks-100
language:
- ja
---

# Model Card for Model ID




## Model Details

### Model Description

- **Developed by:** maktag
- **Language(s) (NLP):** Japanese
- **Finetuned from model [optional]:** llm-jp/llm-jp-3-13b


## How to Get Started with the Model

```
from transformers import AutoTokenizer, AutoModelForCausalLM

# Load the fine-tuned model and tokenizer
model_id = "your-username/llm-jp-3-13b-finetune1"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

# Prepare your input
prompt = """### 指示
以下の文章を英語に翻訳してください:
猫はかわいいです
### 回答
"""

# Tokenize and generate
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
    inputs["input_ids"],
    max_new_tokens=100,
    do_sample=False,
    repetition_penalty=1.2,
    pad_token_id=tokenizer.eos_token_id
)

# Decode and print the response
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
```

[More Information Needed]

## Training Details

- Fine-Tuning Framework: LoRA-based PEFT (Parameter-Efficient Fine-Tuning).
- Dataset: Proprietary Japanese instruction-following dataset.
- Sequence Length: 512 tokens.
- Hyperparameters:
  - Batch size: 16
  - Learning rate: 2e-5
  - Epochs: 3

### Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

- [elyza/ELYZA-tasks-100](https://huggingface.co/datasets/elyza/ELYZA-tasks-100)
- [Ichikara Instruction](https://liat-aip.sakura.ne.jp/wp/llmのための日本語インストラクションデータ作成/llmのための日本語インストラクションデータ-公開/)