arampacha's picture
add tokenizer
650f83d
raw
history blame contribute delete
415 Bytes
{"ա": 1, "բ": 2, "գ": 3, "դ": 4, "ե": 5, "զ": 6, "է": 7, "ը": 8, "թ": 9, "ժ": 10, "ի": 11, "լ": 12, "խ": 13, "ծ": 14, "կ": 15, "հ": 16, "ձ": 17, "ղ": 18, "ճ": 19, "մ": 20, "յ": 21, "ն": 22, "շ": 23, "ո": 24, "չ": 25, "պ": 26, "ջ": 27, "ռ": 28, "ս": 29, "վ": 30, "տ": 31, "ր": 32, "ց": 33, "ւ": 34, "փ": 35, "ք": 36, "օ": 37, "ֆ": 38, "և": 39, "|": 0, "[UNK]": 40, "[PAD]": 41}