XLM-R Adapter-Based Multitask Model (customized-mwt-ner)
	
This repository contains a fine-tuned XLM-RoBERTa model enhanced with task-specific adapter layers for multilingual NLP tasks including:
- Part-of-Speech tagging (UPOS,XPOS)
- Morphological features tagging (FEATS)
- Dependency parsing (DEPREL,HEAD)
- Named Entity Recognition (NER)
Trained using adapter-based finetuning and multi-task supervision on Universal Dependencies and custom NER data.
Files
- customized-mwt-ner.tagger.mdl
 → PyTorch state dict containing task adapter weights and classification heads.
- customized-mwt-ner.vocabs.json
 → Token and label vocabulary mappings (used for preprocessing and decoding).
Model Architecture
- Base: xlm-roberta-base(frozen)
- Adapter configuration: Pfeiffer
- Injected adapter layer per Transformer block
- Task-specific heads for:- POS (upos,xpos)
- Morphological features (feats)
- Dependency parsing (unlabeled,deprel)
- Optional NER layer (if enabled)
 
- POS (
Usage (PyTorch)
You’ll need to:
- Load xlm-roberta-basefrom Hugging Face
- Load adapter weights from customized-mwt-ner.tagger.mdl
- Use the vocab file for decoding predictions
NOTE: This model is not directly plug-and-play with
transformersunless you recreate the original architecture and adapter insertion logic.
Refer to the original paper for complete documentation.
Variants
This model comes in four variants:
| Context | Label Granularity | Folder / Repo Name | 
|---|---|---|
| With Context | Coarse | with-context-coarse | 
| With Context | Fine-grain | with-context-finegrain | 
| Without Context | Coarse | without-context-coarse | 
| Without Context | Fine-grain | without-context-finegrain | 
License
APACHE 2.0 License. Please cite appropriately if used in research.
Citation
@misc{sandhan2023depnecti,
      title={DepNeCTI: Dependency-based Nested Compound Type Identification for Sanskrit}, 
      author={Jivnesh Sandhan and Yaswanth Narsupalli and Sreevatsa Muppirala and Sriram Krishnan and Pavankumar Satuluri and Amba Kulkarni and Pawan Goyal},
      year={2023},
      eprint={2310.09501},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
Link to original paper => HERE
Link to DepNeCT Github Repo => HERE
