

TITLE
What's the most straightforward way to initialise BertForSequenceClassification for different token rather than [CLS]?

COMMENTS
1

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
# ❓ Questions & Help

## Details
BertForSequenceClassification uses [CLS] token's representation to feed a linear classifier. I want to leverage another token (say [X] in the input sequence) rather than [CLS]. What's the most straightforward way to implement that in Transformers?

**A link to original question on the forum/Stack Overflow**: https://stackoverflow.com/questions/64094098/how-to-initialize-bertforsequenceclassification-for-different-input-rather-than