---
datasets:
- bigcode/starcoderdata
- cerebras/SlimPajama-627B 
pipeline_tag: text-generation
---

# Ultron_storm_sft_20231210

Ultron is a series of LLMs ranging from 160M to 1.1B parameters. 

## Details of Ultron_storm_sft_20231210

**Parameters:** 1.1B parameters 

**Attention:** Grouped Query Attention 

**Sequence Length**: 2048 tokens

**Learning rate**: 4e-4 

**Dataset Size:** 950B tokens



**Note:** This model is just a placeholder and doesn't represent the final Ultron lineup.