Update README.md
Browse files
README.md
CHANGED
@@ -25,8 +25,13 @@ You can find the paper here: https://arxiv.org/abs/2311.10770, and the abstract
|
|
25 |
|
26 |
This is the raw pretraining checkpoint. You can use this to fine-tune on a downstream task like GLUE as discussed in the paper. This model is provided only as sanity check for research purposes, it is untested and unfit for deployment.
|
27 |
|
28 |
-
### How to
|
29 |
|
|
|
|
|
|
|
|
|
|
|
30 |
|
31 |
```python
|
32 |
import cramming
|
@@ -40,6 +45,8 @@ encoded_input = tokenizer(text, return_tensors='pt')
|
|
40 |
output = model(**encoded_input)
|
41 |
```
|
42 |
|
|
|
|
|
43 |
|
44 |
### Limitations and bias
|
45 |
|
|
|
25 |
|
26 |
This is the raw pretraining checkpoint. You can use this to fine-tune on a downstream task like GLUE as discussed in the paper. This model is provided only as sanity check for research purposes, it is untested and unfit for deployment.
|
27 |
|
28 |
+
### How to get started
|
29 |
|
30 |
+
1. Create a new Python/conda environment, or simply use one that does not have any previous version of the original `cramming` project installed. If, by accident, you use the original cramming repository code instead of the one provided in the `/training` folder of this project, you will be warned by `transformers` that there are some extra weights (FFF weight) and that some weights are missing (the FF weights expected by the original `crammedBERT`).
|
31 |
+
2. `cd ./training`
|
32 |
+
3. `pip install .`
|
33 |
+
4. Create `minimal_example.py`
|
34 |
+
5. Paste the code below
|
35 |
|
36 |
```python
|
37 |
import cramming
|
|
|
45 |
output = model(**encoded_input)
|
46 |
```
|
47 |
|
48 |
+
6. Run `python minimal_example.py`.
|
49 |
+
|
50 |
|
51 |
### Limitations and bias
|
52 |
|