Spaces:

Dovakiins
/

qwerrwe

Build error

App Files Files Community

qwerrwe / tests /fixtures

100 contributors

History: 9 commits

winglian's picture

misc sharegpt fixes (#723)

f30afe4 unverified about 1 year ago

alpaca
fix packing so that concatenated sequences reset the attention over 1 year ago
conversation.json

9.03 kB

add unit test for sharegpt tokenization over 1 year ago
conversation.missingturns.json

8.41 kB

better handling and logging of empty sharegpt turns (#603) about 1 year ago
conversation.tokenized.json

28.8 kB

misc sharegpt fixes (#723) about 1 year ago
conversation.tokenized_llama2chat.json

77 kB

fix test fixture b/c hf trainer tokenization changed (#464) over 1 year ago