Saving standard BertModel english and BertModel multilingual have drastically different sizes?
When I dobert_config = BertConfig.from_pretrained(‘bert-base-multilingual-cased’)model = BertModel(bert_config)torch.save(model.state_dict(), “temp.p”)print(‘Size (MB):’, os.path.getsize(“temp.p”)/1e6)and I save this model my model size is around 711 MB. But if I do the same for ‘bert-base-uncased’ or ‘bert-base-cased’ which is actually expected due to 110M parameters. Is this expected/ anyone know what I might be potentially doing wrong.
2020-08-28T15:28:49Z
Hey@mkumar10,Thebert-base-multilingual-cased’model  has largevocab_size(119547), so the embedding matrix is bigger than standard English BERT model. Which is why it takes more memory.
2020-08-28T20:48:07Z
That makes sense and that’s what I suspected but 270 MB is really substantial so just wanted to confirm if that is the only reason?