prithivida
commited on
Commit
•
69fb016
1
Parent(s):
deafd8a
Update README.md
Browse files
README.md
CHANGED
@@ -1 +1,103 @@
|
|
1 |
-
**This model is part of the Gramformer library** please refer to https://github.com/PrithivirajDamodaran/Gramformer/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
**This model is part of the Gramformer library** please refer to https://github.com/PrithivirajDamodaran/Gramformer/
|
2 |
+
|
3 |
+
# Gramformer
|
4 |
+
Human and machine-generated text often suffer from grammatical and/or typographical errors. It can be spelling, punctuation, grammatical or word choice errors. Gramformer is a library that exposes 3 separate interfaces to a family of algorithms to **detect, highlight and correct** grammar errors. To make sure the corrections and highlights recommended are of high quality, it comes with a quality estimator. You can use Gramformer in one or more areas mentioned under the "use-cases" section below or any other use case as you see fit. Gramformer stands on the shoulders of giants, it combines some of the top-notch researches in grammar correction. *Note: It works at **sentence levels** and has been trained on 128 length sentences, so not (yet) suitable for long prose or paragraphs (stay tuned for upcoming releases)*
|
5 |
+
|
6 |
+
## Usecases for Gramformer
|
7 |
+
|
8 |
+
**Area 1: Post-processing machine-generated text**
|
9 |
+
|
10 |
+
Machine-Language generation is becoming mainstream, so will post-processing machine-generated text.
|
11 |
+
|
12 |
+
- Conditioned Text generation output(Text2Text generation).
|
13 |
+
- NMT: Machine Translated output.
|
14 |
+
- ASR or STT: Speech to text output.
|
15 |
+
- HTR: Handwritten text recognition output.
|
16 |
+
- Paraphrase generation output.
|
17 |
+
- Controlled Text generation output(Text generation with PPLM) **[TBD]**.
|
18 |
+
- Free-form text generation output(Text generation)**[TBD]**.
|
19 |
+
|
20 |
+
|
21 |
+
**Area 2:Human-In-The-Loop (HITL) text**
|
22 |
+
<ul>
|
23 |
+
<li>Most Supervised NLU (Chatbots and Conversational) systems need humans/experts to enter or edit text that needs to be grammatically correct otherwise the quality of HITL data can degrade the model over a period of time </li>
|
24 |
+
</ul>
|
25 |
+
|
26 |
+
**Area 3:Assisted writing for humans**
|
27 |
+
<ul>
|
28 |
+
<li>Integrating into custom Text editors of your Apps. (A Poor man's grammarly, if you will) </li>
|
29 |
+
</ul>
|
30 |
+
|
31 |
+
**Area 4:Custom Platform integration**
|
32 |
+
|
33 |
+
As of today grammatical safety nets for authoring social contents (Post or Comments) or text in messaging platforms is very little (word level correction) or non-existent.The onus is on the author to install tools like grammarly to proof read.
|
34 |
+
|
35 |
+
- Messaging platforms and Social platforms can highlight / correct grammtical errors automatically without altering the meaning or intent.
|
36 |
+
|
37 |
+
## Installation
|
38 |
+
```python
|
39 |
+
pip install git+https://github.com/PrithivirajDamodaran/Gramformer.git@v0.1
|
40 |
+
```
|
41 |
+
## Quick Start
|
42 |
+
|
43 |
+
### Correcter - [Available now]
|
44 |
+
```python
|
45 |
+
from gramformer import Gramformer
|
46 |
+
import torch
|
47 |
+
|
48 |
+
def set_seed(seed):
|
49 |
+
torch.manual_seed(seed)
|
50 |
+
if torch.cuda.is_available():
|
51 |
+
torch.cuda.manual_seed_all(seed)
|
52 |
+
|
53 |
+
set_seed(1212)
|
54 |
+
|
55 |
+
|
56 |
+
gf = Gramformer(models = 2, use_gpu=False) # 0=detector, 1=highlighter, 2=corrector, 3=all
|
57 |
+
|
58 |
+
influent_sentences = [
|
59 |
+
"Matt like fish",
|
60 |
+
"the collection of letters was original used by the ancient Romans",
|
61 |
+
"We enjoys horror movies",
|
62 |
+
"Anna and Mike is going skiing",
|
63 |
+
"I walk to the store and I bought milk",
|
64 |
+
"We all eat the fish and then made dessert",
|
65 |
+
"I will eat fish for dinner and drank milk",
|
66 |
+
"what be the reason for everyone leave the company",
|
67 |
+
]
|
68 |
+
|
69 |
+
for influent_sentence in influent_sentences:
|
70 |
+
corrected_sentence = gf.correct(influent_sentence)
|
71 |
+
print("[Input] ", influent_sentence)
|
72 |
+
print("[Correction] ",corrected_sentence[0])
|
73 |
+
print("-" *100)
|
74 |
+
```
|
75 |
+
|
76 |
+
```text
|
77 |
+
[Input] Matt like fish
|
78 |
+
[Correction] Matt likes fish
|
79 |
+
----------------------------------------------------------------------------------------------------
|
80 |
+
[Input] the collection of letters was original used by the ancient Romans
|
81 |
+
[Correction] The collection of letters was originally used by the ancient Romans.
|
82 |
+
----------------------------------------------------------------------------------------------------
|
83 |
+
[Input] We enjoys horror movies
|
84 |
+
[Correction] We enjoy horror movies
|
85 |
+
----------------------------------------------------------------------------------------------------
|
86 |
+
[Input] Anna and Mike is going skiing
|
87 |
+
[Correction] Anna and Mike are going skiing
|
88 |
+
----------------------------------------------------------------------------------------------------
|
89 |
+
[Input] I walk to the store and I bought milk
|
90 |
+
[Correction] I walked to the store and bought milk.
|
91 |
+
----------------------------------------------------------------------------------------------------
|
92 |
+
[Input] We all eat the fish and then made dessert
|
93 |
+
[Correction] We all ate the fish and then made dessert
|
94 |
+
----------------------------------------------------------------------------------------------------
|
95 |
+
[Input] I will eat fish for dinner and drank milk
|
96 |
+
[Correction] I'll eat fish for dinner and drink milk.
|
97 |
+
----------------------------------------------------------------------------------------------------
|
98 |
+
[Input] what be the reason for everyone leave the company
|
99 |
+
[Correction] what can be the reason for everyone to leave the company.
|
100 |
+
----------------------------------------------------------------------------------------------------
|
101 |
+
```
|
102 |
+
|
103 |
+
|