README.md

metadata

language:
  - ru
  - uk
  - be
  - kk
  - az
  - hy
  - ka
  - he
  - en
  - de
tags:
  - language classification
datasets:
  - open_subtitles
  - tatoeba
  - oscar

Model for Single Language Classification in texts. Supports 10 languages: ru, uk, be, kk, az, hy, ka, he, en, de.

Model trained on small parts of Open Subtitles, Oscar and Tatoeba datasets (~9k samples per language).

The metrics obtained from validation on part of dataset (~1k samples per language).

eval_accuracy	eval_az_f1-score	eval_az_precision	eval_az_recall	eval_az_support	eval_be_f1-score	eval_be_precision	eval_be_recall	eval_be_support	eval_de_f1-score	eval_de_precision	eval_de_recall	eval_de_support	eval_en_f1-score	eval_en_precision	eval_en_recall	eval_en_support	eval_he_f1-score	eval_he_precision	eval_he_recall	eval_he_support	eval_hy_f1-score	eval_hy_precision	eval_hy_recall	eval_hy_support	eval_ka_f1-score	eval_ka_precision	eval_ka_recall	eval_ka_support	eval_kk_f1-score	eval_kk_precision	eval_kk_recall	eval_kk_support	eval_loss	eval_macro avg_f1-score	eval_macro avg_precision	eval_macro avg_recall	eval_macro avg_support	eval_ru_f1-score	eval_ru_precision	eval_ru_recall	eval_ru_support	eval_uk_f1-score	eval_uk_precision	eval_uk_recall	eval_uk_support	eval_weighted avg_f1-score	eval_weighted avg_precision	eval_weighted avg_recall	eval_weighted avg_support
0.99	0.99849774661993	0.997	1	997	0.9960079840319361	0.998	0.9940239043824701	1004	0.9762506316321374	0.966	0.9867211440245148	979	0.9762376237623762	0.986	0.9666666666666667	1020	0.9995002498750626	1	0.999000999000999	1001	0.9944806823883593	0.991	0.9979859013091642	993	0.999	0.999	0.999	1000	0.9955112219451371	0.998	0.9930348258706467	1005	0.04831727221608162	0.9899994666596248	0.99	0.9901305007950791	10000	0.9822425164890917	0.968	0.9969104016477858	971	0.9822660098522168	0.997	0.9679611650485437	1030	0.9900005333403753	0.9901326000000001	0.99	10000