Support loading datasets saved via save_to_disk (#1432) e634118 unverified fozziethebeat commited on Mar 29, 2024
make sure to capture non-null defaults from config validation (#1415) 601b77b unverified winglian commited on Mar 26, 2024
fix(dataset): normalize tokenizer config and change hash from tokenizer class to tokenizer path (#1298) ff939d8 unverified Nanobit commited on Mar 25, 2024
strip out hacky qlora-fsdp workarounds now that qlora-fsdp fixes are upstreamed (#1428) 2a1589f unverified winglian commited on Mar 21, 2024
Train parameters exclusively in specific ranges (#1390) 05bcc9e unverified seungduk commited on Mar 14, 2024
Add Glaive conversation format support (#1365) b7d8a7d unverified Brian Fitzgerald winglian commited on Mar 11, 2024
plain input/output prompt strategy w/o chat templates (#1346) 4d09b42 unverified winglian commited on Mar 4, 2024
fix for protected model_ namespace w pydantic (#1345) 6b3b271 unverified winglian commited on Feb 28, 2024
Scheduler implementation of Continual Pre-Training of Large Language Models: How to (re)warm your model? (#1273) 8430db2 unverified jinwonkim93 commited on Feb 13, 2024
relora: magnitude pruning of the optimizer (#1245) 8c2e05a unverified winglian commited on Feb 6, 2024
Support for additional_special_tokens (#1221) [skip ci] 25e037f unverified DreamGenX winglian commited on Jan 31, 2024
ADD: warning if hub_model_id ist set but not any save strategy (#1202) af29d81 unverified JohanWork winglian commited on Jan 26, 2024
Feat/chatml add system message (#1117) 98b4762 unverified mhenrichsen Mads Henrichsen winglian commited on Jan 25, 2024
Feat(test): Add tests for alpaca chatml prompt tokenizer (#1088) 5439707 unverified JohanWork Nanobit commited on Jan 23, 2024
set fp16 to false if bf16, update bf16: auto in example YAMLs (#1122) [skip ci] 782b6a4 unverified winglian Nanobit commited on Jan 22, 2024
Add shifted sparse attention (#973) [skip-ci] 1d70f24 unverified jrc joecummings winglian commited on Jan 18, 2024
Add `layers_to_transform` for `lora_config` (#1118) 8487b97 unverified xzuyn commited on Jan 16, 2024
Enable or disable bf16 support based on availability (#1116) 0865613 unverified Simon Hällqvist commited on Jan 14, 2024
add gptneox embeddings, fix phi2 inputs, also fix the casting (#1083) 78c5b19 unverified winglian commited on Jan 11, 2024
update sharegpt conversations when chatml chat template is set (#1075) [skip ci] 0ce1a65 unverified winglian commited on Jan 10, 2024
fix: `train_on_inputs: true` ignored for sharegpt (#1045) [skip ci] 043c386 unverified Nanobit winglian commited on Jan 10, 2024
be more robust about checking embedding modules for lora finetunes (#1074) [skip ci] 0f10080 unverified winglian commited on Jan 10, 2024
attempt to also run e2e tests that needs gpus (#1070) 788649f unverified winglian commited on Jan 10, 2024
fix double eos token for chatml (#1054) [skip ci] 651b7a3 unverified winglian commited on Jan 9, 2024
streaming multipack for pretraining dataset (#959) 553c80f unverified jinwonkim93 [email protected] winglian commited on Jan 6, 2024
bump transformers and update attention class map name (#1023) bcc78d8 unverified winglian commited on Jan 3, 2024
Feat: Warns to add to modules_to_save when adding tokens or switching special_tokens (#787) 1ffa386 unverified Nanobit commited on Dec 22, 2023
Respect sequence_len in config for `type: llama2_chat` (#926) f1de29d unverified hamel commited on Dec 12, 2023