chore: update readme
Browse files
README.md
CHANGED
@@ -32,4 +32,18 @@ StripedHyena is a hybrid architecture composed of multi-head, grouped-query atte
|
|
32 |
|
33 |
To use StripedHyena outside of the playground, you will need to install custom kernels. Please follow the instructions from the [standalone repository](https://github.com/togethercomputer/stripedhyena).
|
34 |
|
35 |
-
StripedHyena is a mixed precision model. Make sure to keep your `poles` and `residues` in `float32` precision, especially for longer prompts or training.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
32 |
|
33 |
To use StripedHyena outside of the playground, you will need to install custom kernels. Please follow the instructions from the [standalone repository](https://github.com/togethercomputer/stripedhyena).
|
34 |
|
35 |
+
StripedHyena is a mixed precision model. Make sure to keep your `poles` and `residues` in `float32` precision, especially for longer prompts or training.
|
36 |
+
|
37 |
+
## Cite
|
38 |
+
|
39 |
+
If you have found the pretrained models or architecture useful for you research or application, consider citing:
|
40 |
+
```
|
41 |
+
@software{stripedhyena,
|
42 |
+
title = {{StripedHyena: Moving Beyond Transformers with Hybrid Signal Processing Models}},
|
43 |
+
author = { Poli, Michael and Wang, Jue and Massaroli, Stefano and Nguyen, Eric and Thomas, Armin},
|
44 |
+
month = 12,
|
45 |
+
year = 2023,
|
46 |
+
url = { https://github.com/togethercomputer/stripedhyena },
|
47 |
+
doi = { 10.57967/hf/1595 },
|
48 |
+
}
|
49 |
+
```
|