Mixtral bad FP16 performance
I am currently testing Mixtralmistralai/Mixtral-8x7B-v0.1If I use it in FP32 I get a wikitextperplexity of ~4(only tested on 10% of the dataset) which matches the expectations of the modelIf I enable FP16 in deepspeed I get aperplexity of 258I cannot really find other reports of this performance drop. anyone else seen that or run FP16 successfully ?