README.md

---
base_model: llm-jp/llm-jp-3-13b
library_name: peft
---

# Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->



## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->



- **Developed by:** [More Information Needed]
- **Funded by [optional]:** [More Information Needed]
- **Shared by [optional]:** [More Information Needed]
- **Model type:** [More Information Needed]
- **Language(s) (NLP):** [More Information Needed]
- **License:** 本モデルは、CC-BY-NC-SAライセンス下で利用可能なデータセットを用いて学習されています。そのため、本モデルを利用する際には、元データセットのライセンスに準拠する必要があります。
- **Finetuned from model [optional]:** [More Information Needed]

### Model Sources [optional]

<!-- Provide the basic links for the model. -->

- **Repository:** [More Information Needed]
- **Paper [optional]:** [More Information Needed]
- **Demo [optional]:** [More Information Needed]

## Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

### Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

[More Information Needed]

### Downstream Use [optional]

<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->

[More Information Needed]

### Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

[More Information Needed]

## Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

[More Information Needed]

### Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

## How to Get Started with the Model

Use the code below to get started with the model.

Google Colabで実行してください。
```
!pip install bitsandbytes

```

```
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
import json
from tqdm import tqdm

# 必要な設定
model_name = "n4/llm-jp-3-13b-finetune-10"
max_seq_length = 1024
load_in_4bit = True  # 4-bit量子化を有効化

# モデルとトークナイザーのロード
print("モデルをロード中...")
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
dtype = torch.float16 if load_in_4bit else None

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    device_map="auto",
    torch_dtype=dtype,
    load_in_4bit=load_in_4bit,
)
tokenizer = AutoTokenizer.from_pretrained(model_name)
print("モデルのロードが完了しました。")
```

```
# 推論用ファイルの用意
elyza-tasks-100-TV_0.jsonl を /content/elyza-tasks-100-TV_0.jsonl となるようにアップロードしておいてください。
```

```

# データセットの読み込み
datasets = []
with open("./elyza-tasks-100-TV_0.jsonl", "r") as f:
    item = ""
    for i, line in enumerate(f):
        line = line.strip()
        item += line
        if item.endswith("}"):
            data = json.loads(item)
            # task_id がない場合は行番号を追加
            if "task_id" not in data:
                data["task_id"] = i  # 0から始まる行番号
            datasets.append(data)
            item = ""

# 推論
results = []
print("推論を開始します...")
for dt in tqdm(datasets):
    input_text = dt["input"]

    # プロンプト作成
    prompt = f"<s>指示を読んで、質問内容を把握してください。把握した内容を回答してください。選択肢の並べ変えや、意味の理解など、多様な質問が想定されるので質問を注意深くみてください。</s><s>### 指示\n{input_text}\n\n\n### 回答\n"

    # トークナイズ（token_type_idsを削除）
    inputs = tokenizer(prompt, return_tensors="pt").to(device)
    inputs.pop("token_type_ids", None)  # 不要なキーを削除

    # 推論
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        use_cache=True,
        do_sample=False,
        repetition_penalty=1.2,
    )
    # 結果のデコード
    prediction = tokenizer.decode(outputs[0], skip_special_tokens=True).split('\n### 回答')[-1]

    # 結果を保存
    results.append({"task_id": dt["task_id"], "input": input_text, "output": prediction})

# 推論結果の保存
output_file = f"{model_name.replace('/', '_')}_output.jsonl"
with open(output_file, "w") as f:
    for result in results:
        f.write(json.dumps(result, ensure_ascii=False) + "\n")

print(f"推論が完了しました。結果は {output_file} に保存されました。")
```

## Training Details

### Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

- 本モデルは、CC-BY-NC-SAライセンス下で提供されているデータセットを用いて学習されています。  
  このライセンスは、非営利的利用及び同一条件での共有を求めるため、利用者はライセンス条件を必ず確認してください。  
  参照: [CC-BY-NC-SA ライセンス詳細](https://creativecommons.org/licenses/by-nc-sa/4.0/)


### Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

#### Preprocessing [optional]

[More Information Needed]


#### Training Hyperparameters

- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->

#### Speeds, Sizes, Times [optional]

<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->

[More Information Needed]

## Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

### Testing Data, Factors & Metrics

#### Testing Data

<!-- This should link to a Dataset Card if possible. -->

[More Information Needed]

#### Factors

<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->

[More Information Needed]

#### Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

[More Information Needed]

### Results

[More Information Needed]

#### Summary



## Model Examination [optional]

<!-- Relevant interpretability work for the model goes here -->

[More Information Needed]

## Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).

- **Hardware Type:** [More Information Needed]
- **Hours used:** [More Information Needed]
- **Cloud Provider:** [More Information Needed]
- **Compute Region:** [More Information Needed]
- **Carbon Emitted:** [More Information Needed]

## Technical Specifications [optional]

### Model Architecture and Objective

[More Information Needed]

### Compute Infrastructure

[More Information Needed]

#### Hardware

[More Information Needed]

#### Software

[More Information Needed]

## Citation [optional]

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

**BibTeX:**

[More Information Needed]

**APA:**

[More Information Needed]

## Glossary [optional]

<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->

[More Information Needed]

## More Information [optional]

[More Information Needed]

## Model Card Authors [optional]

[More Information Needed]

## Model Card Contact

[More Information Needed]
### Framework versions

- PEFT 0.13.2