Update README.md

README.md

@@ -3,10 +3,10 @@ license: cc-by-nc-sa-4.0
 pipeline_tag: fill-mask
 language: en
 tags:
-- legal
+- biomedical
 - long-documents
 ---
 
-# Legal Longformer (base)
+# Biomedical Longformer (base)
 
-This is a derivative model based on the [LexLM (base)](https://huggingface.co/lexlms/roberta-base-cased) RoBERTa model. All model parameters where cloned from the original model, while the positional embeddings were extended by cloning the original embeddings multiple times following [Beltagy et al. (2020)](https://arxiv.org/abs/2004.05150) using a python script similar to this one (https://github.com/allenai/longformer/blob/master/scripts/convert_model_to_long.ipynb).
+This is a derivative model based on [microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract](https://huggingface.co/microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract) BERT model developed in the work "Fine-Tuning Large Neural Language Models for Biomedical Natural Language Processing" by [Tinn et al. (2021)](https://arxiv.org/abs/2112.07869). All model parameters where cloned from the original model, while the positional embeddings were extended by cloning the original embeddings multiple times following [Beltagy et al. (2020)](https://arxiv.org/abs/2004.05150) using a python script similar to this one (https://github.com/allenai/longformer/blob/master/scripts/convert_model_to_long.ipynb).