machineteacher
commited on
Commit
β’
3f1dcfd
1
Parent(s):
e84d086
Add citation info and editing modes.
Browse files
README.md
CHANGED
@@ -40,15 +40,21 @@ The `medit-xxl` model was obtained by fine-tuning the `MBZUAI/bactrian-x-llama-1
|
|
40 |
### Model Sources
|
41 |
|
42 |
- **Repository:** https://github.com/vipulraheja/medit
|
43 |
-
- **Paper:**
|
44 |
|
45 |
## How to use
|
46 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
47 |
### Instruction format
|
48 |
|
49 |
Adherence to the following instruction format is essential; failure to do so may result in the model producing less-than-ideal results.
|
50 |
|
51 |
-
|
52 |
```
|
53 |
instruction_tokens = [
|
54 |
"Instruction",
|
@@ -73,14 +79,16 @@ task_descriptions = [
|
|
73 |
"Umschreiben Sie den Satz", # <-- Paraphrasing
|
74 |
...
|
75 |
]
|
|
|
76 |
|
77 |
-
The entire list of possible
|
78 |
|
|
|
|
|
|
|
79 |
|
80 |
-
|
81 |
|
82 |
-
Note that the tokens and the task description need not be in the language of the input.
|
83 |
-
```
|
84 |
|
85 |
### Run the model
|
86 |
|
@@ -92,7 +100,7 @@ tokenizer = AutoTokenizer.from_pretrained(model_id)
|
|
92 |
|
93 |
model = AutoModelForCausalLM.from_pretrained(model_id)
|
94 |
|
95 |
-
# English GEC
|
96 |
prompt = '### ε½δ»€:\nζη« γζζ³ηγ«γγ\n### ε
₯ε:\nI has small cat ,\n### εΊε:\n\n'
|
97 |
|
98 |
inputs = tokenizer(prompt, return_tensors='pt')
|
@@ -103,10 +111,29 @@ print(tokenizer.decode(outputs[0], skip_special_tokens=True)
|
|
103 |
|
104 |
# --> I have a small cat ,
|
105 |
|
106 |
-
# German GEC
|
107 |
-
|
108 |
prompt = '### ε½δ»€:\nζη« γζζ³ηγ«γγ\n### ε
₯ε:\nIch haben eines kleines Katze ,\n### εΊε:\n\n'
|
109 |
|
110 |
# ...
|
111 |
# --> Ich habe eine kleine Katze ,
|
112 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
40 |
### Model Sources
|
41 |
|
42 |
- **Repository:** https://github.com/vipulraheja/medit
|
43 |
+
- **Paper:** https://arxiv.org/abs/2402.16472v1
|
44 |
|
45 |
## How to use
|
46 |
|
47 |
+
Given an edit instruction and an original text, our model can generate the edited version of the text.<br>
|
48 |
+
|
49 |
+
![task_specs](https://cdn-uploads.huggingface.co/production/uploads/60985a0547dc3dbf8a976607/816ZY2t0XPCpMMd6Z072K.png)
|
50 |
+
|
51 |
+
Specifically, our models support both multi-lingual and cross-lingual text revision. Note that the input and output texts are always in the same language. The monolingual
|
52 |
+
vs. cross-lingual setting is determined by comparing the language of the edit instruction in relation to the language of the input text.
|
53 |
+
|
54 |
### Instruction format
|
55 |
|
56 |
Adherence to the following instruction format is essential; failure to do so may result in the model producing less-than-ideal results.
|
57 |
|
|
|
58 |
```
|
59 |
instruction_tokens = [
|
60 |
"Instruction",
|
|
|
79 |
"Umschreiben Sie den Satz", # <-- Paraphrasing
|
80 |
...
|
81 |
]
|
82 |
+
```
|
83 |
|
84 |
+
**The entire list of possible instructions, input/output tokens, and task descriptions can be found in the Appendix of our paper.**
|
85 |
|
86 |
+
```
|
87 |
+
prompt_template = """### <instruction_token>:\n<task_description>\n### <input_token>:\n<input>\n### <output_token>:\n\n"""
|
88 |
+
```
|
89 |
|
90 |
+
Note that the tokens and the task description need not be in the language of the input (in the case of cross-lingual revision).
|
91 |
|
|
|
|
|
92 |
|
93 |
### Run the model
|
94 |
|
|
|
100 |
|
101 |
model = AutoModelForCausalLM.from_pretrained(model_id)
|
102 |
|
103 |
+
# English GEC using Japanese instructions
|
104 |
prompt = '### ε½δ»€:\nζη« γζζ³ηγ«γγ\n### ε
₯ε:\nI has small cat ,\n### εΊε:\n\n'
|
105 |
|
106 |
inputs = tokenizer(prompt, return_tensors='pt')
|
|
|
111 |
|
112 |
# --> I have a small cat ,
|
113 |
|
114 |
+
# German GEC using Japanese instructions
|
|
|
115 |
prompt = '### ε½δ»€:\nζη« γζζ³ηγ«γγ\n### ε
₯ε:\nIch haben eines kleines Katze ,\n### εΊε:\n\n'
|
116 |
|
117 |
# ...
|
118 |
# --> Ich habe eine kleine Katze ,
|
119 |
```
|
120 |
+
|
121 |
+
#### Software
|
122 |
+
https://github.com/vipulraheja/medit
|
123 |
+
|
124 |
+
## Citation
|
125 |
+
|
126 |
+
**BibTeX:**
|
127 |
+
```
|
128 |
+
@article{raheja2023medit,
|
129 |
+
title={mEdIT: mEdIT: Multilingual Text Editing via Instruction Tuning},
|
130 |
+
author={Vipul Raheja and Dimitris Alikaniotis and Vivek Kulkarni and Bashar Alhafni and Dhruv Kumar},
|
131 |
+
year={2024},
|
132 |
+
eprint={2402.16472v1},
|
133 |
+
archivePrefix={arXiv},
|
134 |
+
primaryClass={cs.CL}
|
135 |
+
}
|
136 |
+
```
|
137 |
+
|
138 |
+
**APA:**
|
139 |
+
Raheja, V., Alikaniotis, D., Kulkarni, V., Alhafni, B., & Kumar, D. (2024). MEdIT: Multilingual Text Editing via Instruction Tuning. ArXiv. /abs/2402.16472
|