ViTMAEModel With model.eval(), get two different representations?
My code as following,  how to get equal encodings from same image?import torch
from transformers import ViTMAEModel

pixel_value = torch.randn([1, 3, 224, 224])

model = ViTMAEModel.from_pretrained("facebook/vit-mae-base").eval()  # convert to evaluate mode.

encoding_a = model(pixel_value).last_hidden_state[:, 1:, :].mean(dim=1)
encoding_b = model(pixel_value).last_hidden_state[:, 1:, :].mean(dim=1)

# print(encoding_a == encoding_b )
assert torch.equal(encoding_a , encoding_b )  # not equal!
2022-08-10T09:23:03Z
I have handle it problem: just set ‘mask_ratio = 0’, I don’t known whether this way is apply to inference?model = ViTMAEModel.from_pretrained("facebook/vit-mae-base", mask_ratio=0.0).eval()  # convert to evaluate mode.
2022-08-10T09:27:56Z
Hi,The model internally generates a random boolean mask as seenhere.To make it reproducable, one can provide anoiseargument to the forward method (to make sure the same boolean mask is applied):import numpy as np
import torch
from transformers import ViTMAEModel

pixel_values = torch.randn([1, 3, 224, 224])

model = ViTMAEModel.from_pretrained("facebook/vit-mae-base")

noise = num_patches = int((model.config.image_size // model.config.patch_size) ** 2)
noise = np.random.uniform(size=(1, num_patches))

encoding_a = model(pixel_values, noise=torch.from_numpy(noise)).last_hidden_state[:, 1:, :].mean(dim=1)
encoding_b = model(pixel_values, noise=torch.from_numpy(noise)).last_hidden_state[:, 1:, :].mean(dim=1)
assert torch.equal(encoding_a , encoding_b)
2022-08-10T12:24:51Z
Thanks your reply.But I found that we just need to reload the model with parameter ‘mask_ratio = 0.0’( equates to let model see all patchs), then could get reproducable encoding from same image.