
llama-like architecture with cerebras-gpt-111m's hyperparameters @ 4k context

| data source | arc-c @ 25| truthfulqa @ 0 | winogrande @ 5 | hellaswag @ 10 | mmlu @ 5 | gsm8k @ 5 |
| --- | --- | --- | --- | --- | --- | --- |
| slimpajama | 21.08±1.19 | 47.54±1.53 | 50.75±1.41 | 27.78±0.45 | 25.1±0.44 | 0±0 |
{"language": ["en"], "datasets": ["cerebras/SlimPajama-627B"]}
transformers safetensors Transformer text-generation custom_code en dataset:cerebras/SlimPajama-627B autotrain_compatible region:us