transformers safetensors llama text-generation conversational dataset:bigcode/starcoderdata dataset:cerebras/SlimPajama-627B doi:10.57967/hf/1589 autotrain_compatible endpoints_compatible text-generation-inference region:us
---
datasets:
- bigcode/starcoderdata
- cerebras/SlimPajama-627B 
pipeline_tag: text-generation
---

# Ultron_storm_sft_20231210

Ultron is a series of LLMs ranging from 160M to 1.1B parameters. 

## Details of Ultron_storm_sft_20231210

**Parameters:** 1.1B parameters 

**Attention:** Grouped Query Attention 

**Sequence Length**: 2048 tokens

**Learning rate**: 4e-4 

**Dataset Size:** 950B tokens



**Note:** This model is just a placeholder and doesn't represent the final Ultron lineup.