| license: other | |
| # xLSTM goes 7B | |
| This xLSTM was pre-trained on the DCLM and selected high-quality data for in a total of approx. 2.3 T tokens using the `xlstm-jax` framework. | |
| ## How to use it | |
| First, install `xlstm`, which now uses the `mlstm_kernels` package for triton kernels: | |
| ```bash | |
| pip install xlstm | |
| pip install mlstm_kernels | |
| ``` | |
| For now, install the transformers repositiory fork from NX-AI (until it is merged): | |
| ```bash | |
| pip install 'transformers @ git+ssh://git@github.com/NX-AI/transformers.git@integrate_xlstm' | |
| ``` | |
| Use this model as: | |
| ```python | |
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| xlstm = AutoModelForCausalLM.from_pretrained("NX-AI/xLSTM-7b", device_map="auto") | |
| # this is a fork of EleutherAI/gpt | |
| tokenizers = AutoTokenizer.from_pretrained("NX-AI/xLSTM-7b") | |
| xlstm(tokenizer("Hello xLSTM, how are you doing?")) | |
| ``` | |
| License: NXAI Community License (see `LICENSE` file) | |