Divyasreepat
commited on
Commit
•
dedf948
1
Parent(s):
80a3ab3
Update README.md with new model card content
Browse files
README.md
CHANGED
@@ -1,7 +1,7 @@
|
|
1 |
---
|
2 |
library_name: keras-hub
|
3 |
---
|
4 |
-
|
5 |
FNet is a set of language models published by Google as part of the paper [FNet: Mixing Tokens with Fourier Transforms](https://arxiv.org/abs/2105.03824). FNet replaces the self-attention of BERT with an unparameterized fourier transform, dramatically lowering the number of trainable parameters in the model. FNet achieves training at 92-97% accuracy of BERT counterparts on GLUE benchmark, with faster training and much smaller saved checkpoints.
|
6 |
|
7 |
Weights and Keras model code are released under the [Apache 2 License](https://github.com/keras-team/keras-hub/blob/master/LICENSE).
|
@@ -34,7 +34,7 @@ The following model checkpoints are provided by the Keras team. Full code exampl
|
|
34 |
| `f_net_base_en` | 82.86M | 12-layer FNet model where case is maintained. |
|
35 |
| `f_net_large_en` | 236.95M | 24-layer FNet model where case is maintained. |
|
36 |
|
37 |
-
|
38 |
```python
|
39 |
import keras
|
40 |
import keras_hub
|
|
|
1 |
---
|
2 |
library_name: keras-hub
|
3 |
---
|
4 |
+
## Model Overview
|
5 |
FNet is a set of language models published by Google as part of the paper [FNet: Mixing Tokens with Fourier Transforms](https://arxiv.org/abs/2105.03824). FNet replaces the self-attention of BERT with an unparameterized fourier transform, dramatically lowering the number of trainable parameters in the model. FNet achieves training at 92-97% accuracy of BERT counterparts on GLUE benchmark, with faster training and much smaller saved checkpoints.
|
6 |
|
7 |
Weights and Keras model code are released under the [Apache 2 License](https://github.com/keras-team/keras-hub/blob/master/LICENSE).
|
|
|
34 |
| `f_net_base_en` | 82.86M | 12-layer FNet model where case is maintained. |
|
35 |
| `f_net_large_en` | 236.95M | 24-layer FNet model where case is maintained. |
|
36 |
|
37 |
+
## Example Usage
|
38 |
```python
|
39 |
import keras
|
40 |
import keras_hub
|