iioSnail commited on
Commit
6083b5f
·
1 Parent(s): 7436a15

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +74 -0
README.md CHANGED
@@ -1,3 +1,77 @@
1
  ---
2
  license: afl-3.0
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: afl-3.0
3
+ language:
4
+ - zh
5
+ tags:
6
+ - Chinese Spell Correction
7
+ - csc
8
+ - Chinese Spell Checking
9
  ---
10
+
11
+ # ReaLiSe-for-csc
12
+
13
+ 中文拼写纠错(Chinese Spell Checking, CSC)模型
14
+
15
+ 该模型源于ReaLiSe源码提供的模型
16
+
17
+ 原论文为:https://arxiv.org/abs/2105.12306
18
+
19
+ 原论文官方代码为:https://github.com/DaDaMrX/ReaLiSe
20
+
21
+ 本模型在SIGHAN2015上的表现如下:
22
+
23
+ | | Detect-Acc | Detect-Precision | Detect-Recall | Detect-F1 | Correct-Acc | Correct-Precision | Correct-Recall | Correct-F1 |
24
+ |--|--|--|--|--|--|--|--|--|
25
+ | Chararcter-level | - | - | - | 87.16 | - | - | - | 91.39 |
26
+ | Sentence-level | 84.7 | 77.3 | 81.3 | 79.3 | 84.0 | 75.9 | 79.9 | 77.8 |
27
+
28
+
29
+
30
+ # 模型使用方法
31
+
32
+ [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/iioSnail/SCOPE/blob/main/ChineseBERT-for-csc_Demo.ipynb)
33
+
34
+ ```
35
+ from transformers import AutoTokenizer, AutoModel
36
+
37
+ tokenizer = AutoTokenizer.from_pretrained("iioSnail/ReaLiSe-for-csc", trust_remote_code=True)
38
+ model = AutoModel.from_pretrained("iioSnail/ReaLiSe-for-csc", trust_remote_code=True)
39
+
40
+ inputs = tokenizer(["我是炼习时长两念半的个人练习生蔡徐坤"], return_tensors='pt')
41
+ output_hidden = model(**inputs).logits
42
+ print(''.join(tokenizer.convert_ids_to_tokens(output_hidden.argmax(-1)[0, 1:-1])))
43
+ ```
44
+
45
+ 输出:
46
+
47
+ ```
48
+ 我是练习时长两年半的个人练习生蔡徐坤
49
+ ```
50
+
51
+ 你也可以使用本模型封装的`predict`方法。
52
+
53
+ ```
54
+ from transformers import AutoTokenizer, AutoModel
55
+
56
+ tokenizer = AutoTokenizer.from_pretrained("iioSnail/ReaLiSe-for-csc", trust_remote_code=True)
57
+ model = AutoModel.from_pretrained("iioSnail/ReaLiSe-for-csc", trust_remote_code=True)
58
+
59
+ model.set_tokenizer(tokenizer) # 使用predict方法前,调用该方法
60
+ print(model.predict("我是练习时长两念半的鸽仁练习生蔡徐坤"))
61
+ ```
62
+
63
+ 输出:
64
+
65
+ ```
66
+ 我是练习时长两年半的鸽人练习生蔡徐坤
67
+ ```
68
+
69
+ # 常见问题
70
+
71
+ 1. 网络问题,例如:`Connection Error`
72
+
73
+ 解决方案:将模型下载到本地使用。批量下载方案可参考该[博客](https://blog.csdn.net/zhaohongfei_358/article/details/126222999)
74
+
75
+ 2. 将模型下载到本地使用时出现报错:`ModuleNotFoundError: No module named 'transformers_modules.iioSnail/ReaLiSe-for-csc'`
76
+
77
+ 解决方案:将 `iioSnail/ChineseBERT-for-csc` 改为 `iioSnail\ChineseBERT-for-csc`,或升级transformers