Spaces:

Dovakiins
/

qwerrwe

Build error

App Files Files Community

qwerrwe / TODO.md

winglian's picture

fdsp config dict fix, todo list, add torchdistx support

ad2b48c over 1 year ago

|

history blame contribute delete

262 Bytes

todo list

[] Validation of parameters for combinations that won't work

things that are known not to work

FSDP offload and gradient_checkpointing - https://github.com/pytorch/pytorch/issues/82203
adamw_bnb_8bit doesn't play well with FSDP offload