Lazy model initialization
Hello. How can I create a model object and skip the random initialization of weights? The random initialization is time consuming and unnecessary for my case, as I want to load the weights usingtorch.load_state_dict. For instance, see the code below.config = BloomConfig.from_pretrained("bigscience/bloom")
block = BloomBlock(config)  # initializes weights randomly, which is time consuming
block.load_state_dict(torch.load("path_to_pytorch_bin"))
2024-05-07T07:56:26Z
same question
2024-05-08T22:12:46Z
Use theinit_empty_weightsContextManager fromacceleratefrom accelerate import init_empty_weights
with init_empty_weights():
   block = BloomBlock(config)
2024-05-08T22:27:34Z
You can setlow_cpu_mem_usage=Truewhich will skip that:Models