text: 
  
  TITLE
  Weighted Loss in BertForTokenClassification
  
  COMMENTS
  9
  
  REACTIONS
  +1: 1
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  # 🚀 Feature request
  
  BertForTokenClassification models can compute cross entropy loss currently is only weighted. The option to have different weights for different classes can be useful in several use cases, including but not restricted to the problem of unbalanced output classes
  
  ## Motivation
  
  Right now, although BertForTokenClassification models can compute cross entropy loss during the forward pass, there is no explicit way of weighting the different classes, which seems like a useful feature, as sequence tagging tasks often have unbalanced classes. I ran into the above problem during solving a academic problem. I looked at the code for the BertForTokenClassification model, and found that it should be quite easy to implement
  
  ## Your contribution
  
  Not sure if I can help, coz I am not really familiar with the codebase. However, I can point out how and where to add code to implement the weighted loss very easily