

TITLE
Dependency parsing head for pretrained models

COMMENTS
8

REACTIONS
+1: 1
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
# 🚀 Feature request

Add a new classification head for pretrained models, for dependency parsing.

## Motivation

Current heads, such as `AutoModelForTokenClassification`, does not work well for finetuning a pretrained model for dependency parsing. [There are now such heads available](https://openreview.net/pdf?id=Hk95PK9le), so adding such would make it a lot easier to finetune models for such tasks. A `PyTorch` implementation of that paper (and others) can be found [in this repo](https://github.com/yzhangcs/parser).

## Your contribution

I could assist with a PR, but for now I'd like to start the discussion to see if this is something that the HF team and others would be interested in.