Spaces:
Running
Running
File size: 325 Bytes
b10121d |
1 2 3 4 5 6 7 8 9 10 11 12 |
{
"Model": "mistralai/Mixtral-8x7B-Instruct-v0.1",
"GPU": "NVIDIA H100 80GB HBM3",
"TP": 4,
"PP": 1,
"Energy/req (J)": 159.99826563921496,
"Avg TPOT (s)": 0.1823368797648745,
"Token tput (tok/s)": 1948.5403179120656,
"Avg Output Tokens": 376.256,
"Avg BS (reqs)": 509.4956730769231,
"Max BS (reqs)": 512
} |