Mathstral compiled for Neuron It has been compiled to run on an inf2.24xlarge instance on AWS. Note that while the inf2.24xlarge has 12 cores, this compilation uses 12.

SEQUENCE_LENGTH = 4096

BATCH_SIZE = 4

NUM_CORES = 12

PRECISION = "bf16"

Downloads last month: 2

Inference API

Unable to determine this model's library. Check the docs .

Collection including nithiyn/mathstral-neuron

neuron-compile-jobs

Collection

5 items • Updated Oct 18, 2024