I have a few Dorfman Pacific hats. All have fit just fine. All large sized.
I have noticed that when I have looked at the same hat in stores, that the
stiffness of the brim has varied (Quality varies) It sounds like the problem
with hats not fitting in your normal size, may be a Quality issue. Not sure where
they are made, but Quality does vary for whatever reason.
They may just mislabel the hats?

Update. Great hat. Fits perfect.