model weights
optimizer states
gradients
forward activations saved for gradient computation
temporary buffers
functionality-specific memory

A typical model trained in mixed precision with AdamW requires 18 bytes per model parameter plus activation memory. For 
inference there are no optimizer states and gradients, so we can subtract those. And thus we end up with 6 bytes per 
model parameter for mixed precision inference, plus activation memory.
Let's look at the details.
Model Weights: