metadata
license: apache-2.0
language:
- en
StripedHyena-Hessian-7B (SH-7B)
Model Architecture
The architecture of StripedHyena-Hessian-7B is quite different from traditional decoder-only Transformers.
StripedHyena is a hybrid architecture composed of multi-head, grouped-query attention and gated convolutions arranged in Hyena blocks.