File size: 2,231 Bytes
a94c516
 
 
 
 
a25d0a7
a94c516
a25d0a7
 
a94c516
a25d0a7
 
 
 
a94c516
0544092
 
a94c516
a25d0a7
a94c516
a25d0a7
 
a94c516
a25d0a7
a94c516
a25d0a7
 
a94c516
 
a25d0a7
 
 
a94c516
a25d0a7
 
a94c516
a25d0a7
 
 
 
a94c516
a25d0a7
 
 
 
 
 
a94c516
a25d0a7
a94c516
a25d0a7
a94c516
a25d0a7
a94c516
a25d0a7
a94c516
a25d0a7
a94c516
a25d0a7
a94c516
a25d0a7
a94c516
a25d0a7
a94c516
a25d0a7
a94c516
a25d0a7
a94c516
a25d0a7
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
---
library_name: transformers
tags: []
---

# How to use ・ 使い方

We recommend on running with at least 4 A100 cards
A100の4枚の環境がおすすめです

### Huggingface
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tokenizer = AutoTokenizer.from_pretrained("lightblue/ao-karasu-72B")
model = AutoModelForCausalLM.from_pretrained("lightblue/ao-karasu-72B", device_map="auto")

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)

messages = [{"role": "system", "content": "あなたはAIアシスタントです。"}]
messages.append({"role": "user", "content": "イギリスの首相は誰ですか?"})

prompt = tokenizer.apply_chat_template(conversation=messages, add_generation_prompt=True, tokenize=False)

pipe(prompt, max_new_tokens=100, do_sample=False, temperature=0.0, return_full_text=False)
```


### vLLM
```python
from vllm import LLM, SamplingParams

sampling_params = SamplingParams(temperature=0.0, max_tokens=100)
llm = LLM(model="lightblue/aokarasu-72B", tensor_parallel_size=4)

messages = [{"role": "system", "content": "あなたはAIアシスタントです。"}]
messages.append({"role": "user", "content": "イギリスの首相は誰ですか?"})
prompt = llm.llm_engine.tokenizer.tokenizer.apply_chat_template(conversation=messages, add_generation_prompt=True, tokenize=False)
prompts = [prompt]

outputs = llm.generate(prompts, sampling_params)
for output in outputs:
    prompt = output.prompt
    generated_text = output.outputs[0].text
    print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")
```

# Training details 学習詳細

[ブログ](https://editor.note.com/notes/n483d194d3614)

# Training data 学習データ

Roughly 20 million characters samples from a dataset of more than 1.1 billion characters, which was made up of:

~450 million characters from Wikipedia-based QA (same as Qarasu)

~200 million characters from technical blogs (new)

~200 million characters from Japanese QA site answers (new)

~100 million characters from LLM generated prompts and responses (same as Qarasu)

~70 million characters from news articles (new)

# Training schedule

Training for ~1 day on a A100 (80GB) GPU