metadata
license: apache-2.0
datasets:
- Open-Orca/SlimOrca
language:
- en
Model Details
- Model Description: This model is test for data ordering.
- Developed by: Jisu Kim
- Model Type: Large Language Model
Model Architecture
This model is based on falcon-7B. We fine-tuning this model for data ordering task.
falcon-7B is a transformer model, with the following architecture choices:
- Grouped-Query Attention
- Sliding-Window Attention
- Byte-fallback BPE tokenizer
Dataset
We random sample Open-Orca dataset. (We finetune the 100,000 dataset)
Guthub
License
Apache License 2.0