nicholasKluge
commited on
Commit
•
4c1bc0e
1
Parent(s):
8f2aa0c
Update README.md
Browse files
README.md
CHANGED
@@ -75,7 +75,7 @@ This repository has the [source code](https://github.com/Nkluge-correa/Aira) use
|
|
75 |
| | num_attention_heads | 12 |
|
76 |
| | num_hidden_layers | 12 |
|
77 |
| | num_key_value_heads | 12 |
|
78 |
-
| | torch_dtype | "float32"
|
79 |
| Data args. | dataset_name | "nicholasKluge/portuguese-corpus-v3" |
|
80 |
| | dataset_split | "train" |
|
81 |
| | train_num_samples | 1831873 |
|
@@ -96,8 +96,7 @@ This repository has the [source code](https://github.com/Nkluge-correa/Aira) use
|
|
96 |
| | seed | 42 |
|
97 |
| | mixed_precision | 'no' |
|
98 |
| | checkpointing_steps | 22000 |
|
99 |
-
|
100 |
-
* With `tf32` enabled during training.
|
101 |
|
102 |
## Usage
|
103 |
|
|
|
75 |
| | num_attention_heads | 12 |
|
76 |
| | num_hidden_layers | 12 |
|
77 |
| | num_key_value_heads | 12 |
|
78 |
+
| | torch_dtype | "float32" |
|
79 |
| Data args. | dataset_name | "nicholasKluge/portuguese-corpus-v3" |
|
80 |
| | dataset_split | "train" |
|
81 |
| | train_num_samples | 1831873 |
|
|
|
96 |
| | seed | 42 |
|
97 |
| | mixed_precision | 'no' |
|
98 |
| | checkpointing_steps | 22000 |
|
99 |
+
| | tf32 | true |
|
|
|
100 |
|
101 |
## Usage
|
102 |
|