jaandoui commited on
Commit
40c6375
1 Parent(s): 49497cd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -2
README.md CHANGED
@@ -8,6 +8,10 @@ tags:
8
  ---
9
  This version of DNABERT2 has been changed to be able to output the attention too, for attention analysis.
10
 
 
 
 
 
11
  Most of the modifications were done in Bert_Layer.py.
12
  It has been modified especially for fine tuning and hasn't been tried for pretraining.
13
  Before or next to each modification, you can find "JAANDOUI" so to see al modifications, search for "JAANDOUI".
@@ -19,8 +23,6 @@ activate the extraction of attention: ```output_attentions=True``` (and ```retur
19
  You can now extract the attention in ```outputs.attentions```
20
  Read more about model outputs here: https://huggingface.co/docs/transformers/v4.40.2/en/main_classes/output#transformers.utils.ModelOutput
21
 
22
- To the author of DNABERT2, feel free to use those modifications.
23
-
24
  The official link to DNABERT2 [DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genome
25
  ](https://arxiv.org/pdf/2306.15006.pdf).
26
 
 
8
  ---
9
  This version of DNABERT2 has been changed to be able to output the attention too, for attention analysis.
10
 
11
+ **To the author of DNABERT2, feel free to use those modifications.**
12
+
13
+ Use ```--model_name_or_path jaandoui/DNABERT2_AttentionExtracted``` instead of the original repository to have access to the attention.
14
+
15
  Most of the modifications were done in Bert_Layer.py.
16
  It has been modified especially for fine tuning and hasn't been tried for pretraining.
17
  Before or next to each modification, you can find "JAANDOUI" so to see al modifications, search for "JAANDOUI".
 
23
  You can now extract the attention in ```outputs.attentions```
24
  Read more about model outputs here: https://huggingface.co/docs/transformers/v4.40.2/en/main_classes/output#transformers.utils.ModelOutput
25
 
 
 
26
  The official link to DNABERT2 [DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genome
27
  ](https://arxiv.org/pdf/2306.15006.pdf).
28