How can I speedup T5 load?
I currently load T5 with:self.t5_enc = T5EncoderModel.from_pretrained(T5_MODEL).eval().to(self.device)But, I’m not sure if this using optimizations liketorch.nn.utils.skip_init, and loading a FSDP sharded-checkpoint for maximum loading speed.How can I ensure these optimizations are being enabled?
2024-09-26T02:00:48Z
I’ve never used FDSP so I don’t know…huggingface.coFully Sharded Data ParallelWe’re on a journey to advance and democratize artificial intelligence through open source and open science.Medium – 17 Jun 24Unlock Multi-GPU Finetuning Secrets: Huggingface Models & PyTorch FSDP...Finetuning Pretrained Models from Huggingface With Torch FSDPReading time: 7 min read