(Minimal) Lightning -> Accelerate?
Hi, I’m surprised not to find any info on this yet, but… I guess I’m the first one to ask:  Is there any way to make Accelerate work with a PyTorch Lightning based code?  (Or a recommended way to convert from the latter to the former?)Up until posting this, I’ve been assuming the answer is “No”, and have begun “ripping out” all my Lightning stuff and converting mypl.Trainermodule to a straight-PyTorch module in order to match up with theAccelerateexamples I’ve seen, and have begun writing a “manual” PyTorch training loop to replace thetrainer.fit()from Lightning.But…is that the only/best way to do it?  Figured it was worth asking before I got too far along in this (kind of) “refactoring”.
2022-06-23T14:17:15Z
drscotthawley:But…is that the only/best way to do it? Figured it was worth asking before I got too far along in this (kind of) “refactoring”.PL has their own DDP they use, and they wrap a lot of magic so it is indeed incompatible.If you want a framework that has Accelerate in it, you can use fastai(so long as you’re doing multi-gpu for now)
2022-06-23T14:19:53Z
Cool thanks.  yea I figured the DDP part was incompatible just wasn’t sure if there was any kind of “shortcut”.muellerzr:If you want a framework that has Accelerate in it, you can use fastaiCooool (and of course, given you & Sylvain on HF staff!).I may just do that!  I had “inherited” the Lightning code from a colleague anyway.And knowing now that the notebook example is fixed, I can now go “full nbdev”, LOL.
2022-11-06T15:18:39Z
Just wanted to extend this discussion since you’ve tried working on getting pytorch-lightning to work with accelerate.On the surface, it would seem that pytorch-lightning modules should be compatible with accelerate, since they can also be treated as plain pytorch counterparts. So other than not interleaving accelerate and lightning for training and optimization, it seems they should be able to share components?