initial commit

.gitattributes

@@ -0,0 +1,17 @@
+*.bin.* filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tar.gz filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.MD

@@ -0,0 +1,69 @@
+## Introduction
+Code for the paper [Exploring the zero-shot limit of FewRel](https://www.aclweb.org/anthology/2020.coling-main.124). This repository implements a zero-shot relation extractor.
+
+## Dataset
+The dataset FewRel 1.0 has been created in the paper 
+[ FewRel: A Large-Scale Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation](https://www.aclweb.org/anthology/D18-1514.pdf)
+and is available [here](https://github.com/thunlp/FewRel).
+
+## Run the Extractor from the notebook
+An example relation extraction is in this [notebook](/notebooks/extractor_examples.ipynb).
+The extractor needs a list of candidate relations in English
+```python
+relations = ['noble title', 'founding date', 'occupation of a person']
+extractor = RelationExtractor(model, tokenizer, relations)
+```
+Then the model ranks the surface forms by the belief that the relation 
+connects the entities in the text 
+```python
+extractor.rank(text='John Smith received an OBE', head='John Smith', tail='OBE')
+
+[('noble title', 0.9690611883997917),
+ ('occupation of a person', 0.0012609362602233887),
+ ('founding date', 0.00024014711380004883)]
+```
+
+## Training
+This repository contains 4 training scripts related to the 4 models in the paper.
+```bash
+train_bert_large_with_squad.py
+train_bert_large_without_squad.py
+train_distillbert_with_squad.py
+train_distillbert_without_squad.py
+```
+
+## Validation
+There are also 4 scripts for validation
+```bash
+test_bert_large_with_squad.py
+test_bert_large_without_squad.py
+test_distillbert_with_squad.py
+test_distillbert_without_squad.py
+```
+
+The results as in the paper are
+
+| Model                  | 0-shot 5-ways | 0-shot 10-ways |
+|------------------------|--------------|----------------|
+|(1) Distillbert         |70.1±0.5      | 55.9±0.6       |
+|(2) Bert Large          |80.8±0.4      | 69.6±0.5       |
+|(3) Distillbert + SQUAD |81.3±0.4      | 70.0±0.2       |
+|(4) Bert Large + SQUAD  |86.0±0.6      | 76.2±0.4       |
+
+## Cite as
+```bibtex
+@inproceedings{cetoli-2020-exploring,
+    title = "Exploring the zero-shot limit of {F}ew{R}el",
+    author = "Cetoli, Alberto",
+    booktitle = "Proceedings of the 28th International Conference on Computational Linguistics",
+    month = dec,
+    year = "2020",
+    address = "Barcelona, Spain (Online)",
+    publisher = "International Committee on Computational Linguistics",
+    url = "https://www.aclweb.org/anthology/2020.coling-main.124",
+    doi = "10.18653/v1/2020.coling-main.124",
+    pages = "1447--1451",
+    abstract = "This paper proposes a general purpose relation extractor that uses Wikidata descriptions to represent the relation{'}s surface form. The results are tested on the FewRel 1.0 dataset, which provides an excellent framework for training and evaluating the proposed zero-shot learning system in English. This relation extractor architecture exploits the implicit knowledge of a language model through a question-answering approach.",
+}
+```
+

config.json

@@ -0,0 +1,24 @@
+{
+  "_name_or_path": "bert-large-uncased-whole-word-masking-finetuned-squad",
+  "architectures": [
+    "BertForQuestionAnswering"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 1024,
+  "initializer_range": 0.02,
+  "intermediate_size": 4096,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 24,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "transformers_version": "4.9.1",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c1dc151ec0572af0e410699a57084c0ca32f0ae81765fc4ea63fa75a7f68a6b5
+size 1341556197

special_tokens_map.json

@@ -0,0 +1 @@
+{"unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]"}

tokenizer_config.json

@@ -0,0 +1 @@
+{"do_lower_case": true, "do_basic_tokenize": true, "never_split": null, "unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]", "tokenize_chinese_chars": true, "strip_accents": null, "model_max_length": 512, "special_tokens_map_file": null, "tokenizer_file": "/home/alce/.cache/huggingface/transformers/9b7535fe1c0da28aa7cc66b7f34529d984f535c401be8352f6adeb25f7870def.7f2721073f19841be16f41b0a70b600ca6b880c8f3df6f3535cbc704371bdfa4", "name_or_path": "bert-large-uncased-whole-word-masking-finetuned-squad", "tokenizer_class": "BertTokenizer"}