Load/save HF block sparse model
Hey everyone,I am exploringhttps://github.com/huggingface/pytorch_block_sparseproject. One of the issues that popped up almost immediatelly is loading a saved “sparsified” model. So, let’s say you sparsified Roberta using an exampleprovided. Now that the model has been sparsified (it’s linear layers replaced with BlockSparseLinear nn modules) how can I load previously saved model using HF ecosystem? All I can think of is that I again need to create a Roberta model with uninitialized weights, sparsify it, and the load weights with model.load_state_dict(torch.load(PATH))?Am I overlooking something obvious?
2020-10-21T13:47:50Z
No mechanism in place for loading as of now, which is ok. I sparsed the model again and loaded the weights manually via model.load_state_dict(torch.load(PATH)).