kiddothe2b
commited on
Commit
•
6151925
1
Parent(s):
162074d
Update README.md
Browse files
README.md
CHANGED
@@ -3,7 +3,7 @@ license: cc-by-nc-sa-4.0
|
|
3 |
pipeline_tag: fill-mask
|
4 |
language: en
|
5 |
tags:
|
6 |
-
-
|
7 |
datasets:
|
8 |
- c4
|
9 |
model-index:
|
@@ -34,9 +34,9 @@ Note that this model is primarily aimed at being fine-tuned on tasks that use th
|
|
34 |
You can use this model directly with a pipeline for masked language modeling:
|
35 |
|
36 |
```python
|
37 |
-
from transformers import
|
38 |
-
|
39 |
-
mlm_model(
|
40 |
```
|
41 |
|
42 |
You can also fine-tun it for SequenceClassification, SequentialSentenceClassification, and MultipleChoice down-stream tasks:
|
@@ -96,7 +96,8 @@ The following hyperparameters were used during training:
|
|
96 |
- Tokenizers 0.11.6
|
97 |
|
98 |
|
99 |
-
##Citing
|
|
|
100 |
If you use HAT in your research, please cite [An Exploration of Hierarchical Attention Transformers for Efficient Long Document Classification](https://arxiv.org/abs/xxx)
|
101 |
|
102 |
```
|
|
|
3 |
pipeline_tag: fill-mask
|
4 |
language: en
|
5 |
tags:
|
6 |
+
- long-documents
|
7 |
datasets:
|
8 |
- c4
|
9 |
model-index:
|
|
|
34 |
You can use this model directly with a pipeline for masked language modeling:
|
35 |
|
36 |
```python
|
37 |
+
from transformers import AutoTokenizer, AutoModelforForMaskedLM
|
38 |
+
tokenizer = AutoTokenizer.from_pretrained("kiddothe2b/hierarchical-transformer-base-4096", trust_remote_code=True)
|
39 |
+
mlm_model = AutoModelforForMaskedLM(model='kiddothe2b/hierarchical-transformer-base-4096', trust_remote_code=True)
|
40 |
```
|
41 |
|
42 |
You can also fine-tun it for SequenceClassification, SequentialSentenceClassification, and MultipleChoice down-stream tasks:
|
|
|
96 |
- Tokenizers 0.11.6
|
97 |
|
98 |
|
99 |
+
## Citing
|
100 |
+
|
101 |
If you use HAT in your research, please cite [An Exploration of Hierarchical Attention Transformers for Efficient Long Document Classification](https://arxiv.org/abs/xxx)
|
102 |
|
103 |
```
|