Update README.md
Browse files
README.md
CHANGED
@@ -8,6 +8,10 @@ tags:
|
|
8 |
---
|
9 |
This version of DNABERT2 has been changed to be able to output the attention too, for attention analysis.
|
10 |
|
|
|
|
|
|
|
|
|
11 |
Most of the modifications were done in Bert_Layer.py.
|
12 |
It has been modified especially for fine tuning and hasn't been tried for pretraining.
|
13 |
Before or next to each modification, you can find "JAANDOUI" so to see al modifications, search for "JAANDOUI".
|
@@ -19,8 +23,6 @@ activate the extraction of attention: ```output_attentions=True``` (and ```retur
|
|
19 |
You can now extract the attention in ```outputs.attentions```
|
20 |
Read more about model outputs here: https://huggingface.co/docs/transformers/v4.40.2/en/main_classes/output#transformers.utils.ModelOutput
|
21 |
|
22 |
-
To the author of DNABERT2, feel free to use those modifications.
|
23 |
-
|
24 |
The official link to DNABERT2 [DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genome
|
25 |
](https://arxiv.org/pdf/2306.15006.pdf).
|
26 |
|
|
|
8 |
---
|
9 |
This version of DNABERT2 has been changed to be able to output the attention too, for attention analysis.
|
10 |
|
11 |
+
**To the author of DNABERT2, feel free to use those modifications.**
|
12 |
+
|
13 |
+
Use ```--model_name_or_path jaandoui/DNABERT2_AttentionExtracted``` instead of the original repository to have access to the attention.
|
14 |
+
|
15 |
Most of the modifications were done in Bert_Layer.py.
|
16 |
It has been modified especially for fine tuning and hasn't been tried for pretraining.
|
17 |
Before or next to each modification, you can find "JAANDOUI" so to see al modifications, search for "JAANDOUI".
|
|
|
23 |
You can now extract the attention in ```outputs.attentions```
|
24 |
Read more about model outputs here: https://huggingface.co/docs/transformers/v4.40.2/en/main_classes/output#transformers.utils.ModelOutput
|
25 |
|
|
|
|
|
26 |
The official link to DNABERT2 [DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genome
|
27 |
](https://arxiv.org/pdf/2306.15006.pdf).
|
28 |
|