how do I output the attention scores from the last layer of the encoder?

#22

by jkb0722 - opened Apr 5

Apr 5

Hi,
When running dnabert2, I'm trying to output not only the CLS and hidden states but also the attentions.

I tried:

`db2_model = AutoModel.from_pretrained(dirname, output_attentions=True, trust_remote_code=True)

output = db2_model(input_ids)
`

but the output is only a tuple of length 2 (the hidden states and the CLS tensor).

Any help would be much appreciated!

Thanks!

May 14

Read this, I was finally able to extract attention. I hope this will help all of us :D

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment