Qishuai/distilbert_punctuator_en

Punctuator for Uncased English

The model is fine-tuned based on DistilBertForTokenClassification for adding punctuations to plain text (uncased English)

Usage

from transformers import DistilBertForTokenClassification, DistilBertTokenizerFast

model = DistilBertForTokenClassification.from_pretrained("Qishuai/distilbert_punctuator_en")
tokenizer = DistilBertTokenizerFast.from_pretrained("Qishuai/distilbert_punctuator_en")

Model Overview

Training data

Combination of following three dataset:

• BBC news: From BBC news website corresponding to stories in five topical areas from 2004-2005. Reference
• News articles: 20000 samples of short news articles scraped from Hindu, Indian times and Guardian between Feb 2017 and Aug 2017 Reference
• Ted talks: transcripts of over 4,000 TED talks between 2004 and 2019 Reference

Model Performance

• Validation with 500 samples of dataset scraped from https://www.thenews.com.pk website. Reference

• Metrics Report:

  	precision	recall	f1-score	support
  COMMA	0.66	0.55	0.60	7064
  EXLAMATIONMARK	1.00	0.00	0.00	5
  PERIOD	0.73	0.63	0.68	6573
  QUESTIONMARK	0.54	0.41	0.47	17
  micro avg	0.69	0.59	0.64	13659
  macro avg	0.73	0.40	0.44	13659
  weighted avg	0.69	0.59	0.64	13659

• Validation with 86 news ted talks of 2020 which are not included in training dataset Reference

• Metrics Report:

  	precision	recall	f1-score	support
  COMMA	0.71	0.56	0.63	10712
  EXLAMATIONMARK	0.45	0.07	0.12	75
  PERIOD	0.75	0.65	0.70	7921
  QUESTIONMARK	0.73	0.67	0.70	827
  micro avg	0.73	0.60	0.66	19535
  macro avg	0.66	0.49	0.53	19535
  weighted avg	0.73	0.60	0.66	19535