metadata
language:
- zh
This checkpoint is a states tuning file from RWKV-6-7B. Please download the base model from https://huggingface.co/BlinkDL/rwkv-6-world/tree/main . It will give a judgement about the type of the sentence. Usage:
- update the latest rwkv package: pip install --upgrade rwkv
- Download the base model and the states file. You may download the states from the epoch_2 directory.
- Following the codes:
- Loading the model and states
from rwkv.model import RWKV
from rwkv.utils import PIPELINE, PIPELINE_ARGS
import torch
# download models: https://huggingface.co/BlinkDL
model = RWKV(model='/media/yueyulin/KINGSTON/models/rwkv6/RWKV-x060-World-7B-v2.1-20240507-ctx4096.pth', strategy='cuda fp16')
print(model.args)
pipeline = PIPELINE(model, "rwkv_vocab_v20230424") # 20B_tokenizer.json is in https://github.com/BlinkDL/ChatRWKV
# use pipeline = PIPELINE(model, "rwkv_vocab_v20230424") for rwkv "world" models
states_file = '/media/yueyulin/data_4t/models/states_tuning/custom_trainer/epoch_2/RWKV-x060-World-7B-v2.1-20240507-ctx4096.pth.pth'
states = torch.load(states_file)
states_value = []
device = 'cuda'
n_head = model.args.n_head
head_size = model.args.n_embd//model.args.n_head
for i in range(model.args.n_layer):
key = f'blocks.{i}.att.time_state'
value = states[key]
prev_x = torch.zeros(model.args.n_embd,device=device,dtype=torch.float16)
prev_states = value.clone().detach().to(device=device,dtype=torch.float16).transpose(1,2)
prev_ffn = torch.zeros(model.args.n_embd,device=device,dtype=torch.float16)
states_value.append(prev_x)
states_value.append(prev_states)
states_value.append(prev_ffn)
The typed schema is :
schema = ['事件', '自然科学', '建筑结构', '地理地区', '组织', '医学', '天文对象', '人造物件', '运输', '作品', '生物', '人物']
Try the following case:
cat_char = '🐱'
bot_char = '🤖'
instruction ='你是一个图谱实体知识结构化专家。请从input中抽取出符合schema定义的实体实例和其属性，不存在的属性不输出，属性存在多值就返回列表。请按照JSON字符串的格式回答。'
schema = schemas['人物']
input_text = "个人简介姓名：拉塞·维比 所属球队：布伦特福德 国籍：丹麦、法国、荷兰、法属圭亚那 出生日期：1987-02-22 身高：181cm 体重：73kg 场上位置：前锋 球衣号码：21 丹麦射手拉塞-维比，获得了2014赛季瑞超联赛金靴"
input_text = {'input': input_text, 'schema': schema}
input_text = json.dumps(input_text).decode('UTF-8')
ctx = f'{cat_char}:{instruction}\n{input_text}\n{bot_char}:'
print(ctx)
def my_print(s):
print(s, end='', flush=True)
# For alpha_frequency and alpha_presence, see "Frequency and presence penalties":
# https://platform.openai.com/docs/api-reference/parameter-details
args = PIPELINE_ARGS(temperature = 1.0, top_p = 0, top_k = 0, # top_k = 0 then ignore
alpha_frequency = 0.25,
alpha_presence = 0.25,
alpha_decay = 0.996, # gradually decay the penalty
token_ban = [0], # ban the generation of some tokens
token_stop = [0,1], # stop generation whenever you see any token here
chunk_len = 256) # split input into chunks to save VRAM (shorter -> slower)
pipeline.generate(ctx, token_count=200, args=args, callback=my_print,state=states_value)
print('\n')
The output should look like:
{"result":人物}