metadata

license: cc-by-nc-sa-4.0
pipeline_tag: fill-mask
language: en
tags:
  - legal
  - long-documents

Legal Longformer (base)

This is a derivative model based on the LexLM (base) RoBERTa model. All model parameters where cloned from the original model, while the positional embeddings were extended by cloning the original embeddings multiple times following Beltagy et al. (2020) using a python script similar to this one (https://github.com/allenai/longformer/blob/master/scripts/convert_model_to_long.ipynb).