Re-initialize decoder parameters of a pretrained model
What is the recommended way of re-initializing only the decoder weights of a pretrained model?In the case of Mask2Former I noticed that it has a differentweight initializing methoddepending on the layer.from transformers import AutoModel

mask2former_model = AutoModel.from_pretrained("facebook/mask2former-swin-tiny-cityscapes-panoptic")

# re-initialize decoder weights...