KoichiYasuoka commited on
Commit
8b160e2
1 Parent(s): b6984e9

re-initialize

Browse files
Files changed (1) hide show
  1. README.md +87 -0
README.md ADDED
@@ -0,0 +1,87 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - "lzh"
4
+ tags:
5
+ - "classical chinese"
6
+ - "literary chinese"
7
+ - "ancient chinese"
8
+ - "token-classification"
9
+ - "pos"
10
+ license: "mit"
11
+ pipeline_tag: "token-classification"
12
+ widget:
13
+ - text: "不入虎穴不得虎子"
14
+ ---
15
+
16
+ [![Current PyPI packages](https://badge.fury.io/py/suparkanbun.svg)](https://pypi.org/project/suparkanbun/)
17
+
18
+ # SuPar-Kanbun
19
+
20
+ Tokenizer, POS-Tagger and Dependency-Parser for Classical Chinese Texts (漢文/文言文) with [spaCy](https://spacy.io), [Transformers](https://huggingface.co/transformers/) and [SuPar](https://github.com/yzhangcs/parser).
21
+
22
+ ## Basic usage
23
+
24
+ ```py
25
+ >>> import suparkanbun
26
+ >>> nlp=suparkanbun.load()
27
+ >>> doc=nlp("不入虎穴不得虎子")
28
+ >>> print(type(doc))
29
+ <class 'spacy.tokens.doc.Doc'>
30
+ >>> print(suparkanbun.to_conllu(doc))
31
+ # text = 不入虎穴不得虎子
32
+ 1 不 不 ADV v,副詞,否定,無界 Polarity=Neg 2 advmod _ Gloss=not|SpaceAfter=No
33
+ 2 入 入 VERB v,動詞,行為,移動 _ 0 root _ Gloss=enter|SpaceAfter=No
34
+ 3 虎 虎 NOUN n,名詞,主体,動物 _ 4 nmod _ Gloss=tiger|SpaceAfter=No
35
+ 4 穴 穴 NOUN n,名詞,固定物,地形 Case=Loc 2 obj _ Gloss=cave|SpaceAfter=No
36
+ 5 不 不 ADV v,副詞,否定,無界 Polarity=Neg 6 advmod _ Gloss=not|SpaceAfter=No
37
+ 6 得 得 VERB v,動詞,行為,得失 _ 2 parataxis _ Gloss=get|SpaceAfter=No
38
+ 7 虎 虎 NOUN n,名詞,主体,動物 _ 8 nmod _ Gloss=tiger|SpaceAfter=No
39
+ 8 子 子 NOUN n,名詞,人,関係 _ 6 obj _ Gloss=child|SpaceAfter=No
40
+
41
+ >>> import deplacy
42
+ >>> deplacy.render(doc)
43
+ 不 ADV <════╗ advmod
44
+ 入 VERB ═══╗═╝═╗ ROOT
45
+ 虎 NOUN <╗ ║ ║ nmod
46
+ 穴 NOUN ═╝<╝ ║ obj
47
+ 不 ADV <════╗ ║ advmod
48
+ 得 VERB ═══╗═╝<╝ parataxis
49
+ 虎 NOUN <╗ ║ nmod
50
+ 子 NOUN ═╝<╝ obj
51
+ ```
52
+
53
+ `suparkanbun.load()` has two options `suparkanbun.load(BERT="roberta-classical-chinese-base-char",Danku=False)`. With the option `Danku=True` the pipeline tries to segment sentences automatically. Available `BERT` options are:
54
+
55
+ * `BERT="roberta-classical-chinese-base-char"` utilizes [roberta-classical-chinese-base-char](https://huggingface.co/KoichiYasuoka/roberta-classical-chinese-base-char) (default)
56
+ * `BERT="roberta-classical-chinese-large-char"` utilizes [roberta-classical-chinese-large-char](https://huggingface.co/KoichiYasuoka/roberta-classical-chinese-large-char)
57
+ * `BERT="guwenbert-base"` utilizes [GuwenBERT-base](https://huggingface.co/ethanyt/guwenbert-base)
58
+ * `BERT="guwenbert-large"` utilizes [GuwenBERT-large](https://huggingface.co/ethanyt/guwenbert-large)
59
+ * `BERT="sikubert"` utilizes [SikuBERT](https://huggingface.co/SIKU-BERT/sikubert)
60
+ * `BERT="sikuroberta"` utilizes [SikuRoBERTa](https://huggingface.co/SIKU-BERT/sikuroberta)
61
+
62
+ ## Installation for Linux
63
+
64
+ ```sh
65
+ pip3 install suparkanbun --user
66
+ ```
67
+
68
+ ## Installation for Cygwin64
69
+
70
+ Make sure to get `python37-devel` `python37-pip` `python37-cython` `python37-numpy` `python37-wheel` `gcc-g++` `mingw64-x86_64-gcc-g++` `git` `curl` `make` `cmake` packages, and then:
71
+ ```sh
72
+ curl -L https://raw.githubusercontent.com/KoichiYasuoka/CygTorch/master/installer/supar.sh | sh
73
+ pip3.7 install suparkanbun --no-build-isolation
74
+ ```
75
+
76
+ ## Installation for Jupyter Notebook (Google Colaboratory)
77
+
78
+ ```py
79
+ !pip install suparkanbun
80
+ ```
81
+
82
+ Try [notebook](https://colab.research.google.com/github/KoichiYasuoka/SuPar-Kanbun/blob/main/suparkanbun.ipynb) for Google Colaboratory.
83
+
84
+ ## Author
85
+
86
+ Koichi Yasuoka (安岡孝一)
87
+