transformers endpoints_compatible region:us
---
{}
---
TRT-LLM version: baseten_v0.6.1_20231208, config: 
{
    "repo": "mistralai/Mistral-7B-Instruct-v0.2",
    "args": {
        "max_input_len": 4096,
        "max_output_len": 1024,
        "max_batch_size": 64
    },
    "quant": "smooth_quant"
}