|
--- |
|
title: README |
|
emoji: 🏃 |
|
colorFrom: indigo |
|
colorTo: blue |
|
sdk: static |
|
pinned: false |
|
--- |
|
|
|
# hmByT5 - Language Models |
|
|
|
Historical Multilingual and Monolingual ByT5 Models. Following languages are currently covered: |
|
|
|
* English (British Library Corpus - Books) |
|
* German (Europeana Newspaper) |
|
* French (Europeana Newspaper) |
|
* Finnish (Europeana Newspaper) |
|
* Swedish (Europeana Newspaper) |
|
* Dutch (Delpher Corpus) |
|
* Norwegian (NCC) |
|
|
|
More details can be found in [our GitHub repository](https://github.com/stefan-it/hmByT5). |
|
|
|
# Leaderboard |
|
|
|
We test our pretrained language models on various datasets from HIPE-2020, HIPE-2022 and Europeana. The following table |
|
shows an overview of used datasets. |
|
|
|
| Language | Dataset | Additional Dataset | |
|
|----------|--------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------| |
|
| English | [AjMC](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-ajmc.md) | - | |
|
| German | [AjMC](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-ajmc.md) | - | |
|
| French | [AjMC](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-ajmc.md) | [ICDAR-Europeana](https://github.com/stefan-it/historic-domain-adaptation-icdar) | |
|
| Finnish | [NewsEye](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-newseye.md) | - | |
|
| Swedish | [NewsEye](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-newseye.md) | - | |
|
| Dutch | [ICDAR-Europeana](https://github.com/stefan-it/historic-domain-adaptation-icdar) | - | |
|
|
|
Current best models: |
|
|
|
| Model | English AjMC | German AjMC | French AjMC | Finnish NewsEye | Swedish NewsEye | Dutch ICDAR | French ICDAR | Avg. | |
|
|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|--------------|--------------|-----------------|-----------------|--------------|--------------|------| |
|
| [`hmbyt5/byt5-small-english`](https://huggingface.co/hmbyt5/byt5-small-english) | 85.65 ± 1.21 | 87.27 ± 0.50 | 84.44 ± 0.79 | | | | | | |
|
| [`hmbyt5-preliminary/byt5-small-english-german`](https://huggingface.co/hmbyt5-preliminary/byt5-small-english-german) | 85.74 ± 0.72 | 87.45 ± 0.67 | 84.23 ± 0.65 | | | | | | |
|
| [`hmbyt5-preliminary/byt5-small-english-german-french`](https://huggingface.co/hmbyt5-preliminary/byt5-small-english-german-french) | 85.61 ± 0.96 | 87.24 ± 0.76 | 84.39 ± 0.68 | | | | | | |
|
| [`hmbyt5-preliminary/byt5-small-english-german-french-finnish`](https://huggingface.co/hmbyt5-preliminary/byt5-small-english-german-french-finnish) | 85.30 ± 1.14 | 87.37 ± 0.53 | 84.12 ± 0.42 | | | | | | |
|
| [`hmbyt5-preliminary/byt5-small-english-german-french-finnish-swedish`](https://huggingface.co/hmbyt5-preliminary/byt5-small-english-german-french-finnish-swedish) | 85.40 ± 0.78 | 87.12 ± 0.19 | 84.41 ± 0.34 | | | | | | |
|
| [`hmbyt5-preliminary/byt5-small-english-german-french-finnish-swedish-dutch`](https://huggingface.co/hmbyt5-preliminary/byt5-small-english-german-french-finnish-swedish-dutch) | 85.51 ± 0.68 | 87.58 ± 0.39 | 84.39 ± 0.83 | 55.46 ± 1.99 | 73.38 ± 2.45 | 84.80 ± 0.44 | 75.97 ± 0.55 | | |
|
| [`hmbyt5-preliminary/byt5-small-multilingual-4g`](https://huggingface.co/hmbyt5-preliminary/byt5-small-multilingual-4g) | 83.49 ± 0.96 | 87.65 ± 0.63 | 84.16 ± 0.90 | | | | | | |
|
| [`hmbyt5-preliminary/byt5-small-multilingual-4g-2e`](https://huggingface.co/hmbyt5-preliminary/byt5-small-multilingual-4g-2e) | 83.86 ± 0.61 | 87.54 ± 0.19 | 84.29 ± 0.41 | | | | | | |
|
| [`hmbyt5-preliminary/byt5-small-multilingual-4g-3e`](https://huggingface.co/hmbyt5-preliminary/byt5-small-multilingual-4g-3e) | 83.49 ± 0.99 | 87.38 ± 0.53 | 84.30 ± 0.51 | | | | | | |
|
| [`hmbyt5-preliminary/byt5-small-historic-multilingual-flax`](https://huggingface.co/hmbyt5-preliminary/byt5-small-historic-multilingual-flax) | 83.28 ± 1.67 | 86.98 ± 0.71 | 83.49 ± 1.06 | 76.96 ± 1.58 | 78.80 ± 1.89 | 86.47 ± 0.79 | 77.43 ± 0.51 | | |
|
| [`hmbyt5-preliminary/byt5-small-historic-multilingual-span20-flax`](https://huggingface.co/hmbyt5-preliminary/byt5-small-historic-multilingual-span20-flax) | 84.91 ± 0.86 | 88.02 ± 0.35 | 84.78 ± 0.75 | 77.77 ± 1.83 | 79.94 ± 0.60 | 86.85 ± 0.91 | 77.45 ± 0.54 | | |
|
|
|
More recent results on more datasets can be found in the [`hmLeaderboard`](https://huggingface.co/spaces/hmbench/hmLeaderboard). |
|
|
|
# Acknowledgements |
|
|
|
We thank [Luisa März](https://github.com/LuisaMaerz), [Katharina Schmid](https://github.com/schmika) and |
|
[Erion Çano](https://github.com/erionc) for their fruitful discussions about Historical Language Models. |
|
|
|
Research supported with Cloud TPUs from Google's [TPU Research Cloud](https://sites.research.google/trc/about/) (TRC). |
|
Many Thanks for providing access to the TPUs ❤️ |
|
|
