mamei16/chonky_distilbert_base_uncased_1.1

Model Description

This is a fine-tune of mirth/chonky_distilbert_base_uncased_1 with the goal being to see if it can be improved further by training it on more data.

Comparison with chonky_distilbert_base_uncased_1 and chonky_modernbert_base_1

The following diffs were produced by comparing the chunks produced by the two models to be compared using an example text. Each chunk is delimited by the string -------------------- on a separate line.

• mamei16/chonky_distilbert_base_uncased_1.1 vs mirth/chonky_distilbert_base_uncased_1: https://www.diffchecker.com/PQ1EOMQ7/

• mamei16/chonky_distilbert_base_uncased_1.1 vs mirth/chonky_modernbert_base_1: https://www.diffchecker.com/iO0l2Ox8/

Training Data, Code and Hardware

The model was fine-tuned for one epoch on mamei16/wikipedia_paragraphs. The training code can found here. Fine-tuning was run on an RTX 5090 for about 3 hours and 45 minutes.