Potential discrepancy between the weights between Huggingface and Timm for google/vit-base-patch16-224-in21k
If I understand correctly, thegoogle/vit-base-patch16-224-in21kcorresponds to timm’svit_base_patch16_224.augreg_in21k.However, I found HuggingFace’s has a Pooler layer that timm’s doesn’t have.Besides, I checked some specific weights, e.g.,Huggingface:embeddings.patch_embeddings.projection.weighttimm:patch_embed.proj.weightThey are not equal.Other minor things could beepsofLayerNorm.I’m wondering if the correct weights have been converted.