Spaces:

Dovakiins
/

qwerrwe

Build error

App Files Files Community

qwerrwe / tests /fixtures

100 contributors

History: 7 commits

winglian's picture

better handling and logging of empty sharegpt turns (#603)

a363604 unverified over 1 year ago

alpaca
fix packing so that concatenated sequences reset the attention over 1 year ago
conversation.json

9.03 kB

add unit test for sharegpt tokenization over 1 year ago
conversation.missingturns.json

8.41 kB

better handling and logging of empty sharegpt turns (#603) over 1 year ago
conversation.tokenized.json

28.5 kB

Fix pre-commit for rebased files over 1 year ago
conversation.tokenized_llama2chat.json

77 kB

fix test fixture b/c hf trainer tokenization changed (#464) over 1 year ago