File size: 6,732 Bytes
1fb8d9f b500534 8b473de b500534 6c9199b fc57e3c b500534 8b473de b500534 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 |
---
title: README
emoji: 🏃
colorFrom: indigo
colorTo: blue
sdk: static
pinned: false
---
# hmByT5 - Language Models
Historical Multilingual and Monolingual ByT5 Models. Following languages are currently covered:
* English (British Library Corpus - Books)
* German (Europeana Newspaper)
* French (Europeana Newspaper)
* Finnish (Europeana Newspaper)
* Swedish (Europeana Newspaper)
* Dutch (Delpher Corpus)
* Norwegian (NCC)
More details can be found in [our GitHub repository](https://github.com/stefan-it/hmByT5).
# Leaderboard
We test our pretrained language models on various datasets from HIPE-2020, HIPE-2022 and Europeana. The following table
shows an overview of used datasets.
| Language | Dataset | Additional Dataset |
|----------|--------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------|
| English | [AjMC](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-ajmc.md) | - |
| German | [AjMC](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-ajmc.md) | - |
| French | [AjMC](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-ajmc.md) | [ICDAR-Europeana](https://github.com/stefan-it/historic-domain-adaptation-icdar) |
| Finnish | [NewsEye](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-newseye.md) | - |
| Swedish | [NewsEye](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-newseye.md) | - |
| Dutch | [ICDAR-Europeana](https://github.com/stefan-it/historic-domain-adaptation-icdar) | - |
Current best models:
| Model | English AjMC | German AjMC | French AjMC | Finnish NewsEye | Swedish NewsEye | Dutch ICDAR | French ICDAR | Avg. |
|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|--------------|--------------|-----------------|-----------------|--------------|--------------|------|
| [`hmbyt5/byt5-small-english`](https://huggingface.co/hmbyt5/byt5-small-english) | 85.65 ± 1.21 | 87.27 ± 0.50 | 84.44 ± 0.79 | | | | | |
| [`hmbyt5-preliminary/byt5-small-english-german`](https://huggingface.co/hmbyt5-preliminary/byt5-small-english-german) | 85.74 ± 0.72 | 87.45 ± 0.67 | 84.23 ± 0.65 | | | | | |
| [`hmbyt5-preliminary/byt5-small-english-german-french`](https://huggingface.co/hmbyt5-preliminary/byt5-small-english-german-french) | 85.61 ± 0.96 | 87.24 ± 0.76 | 84.39 ± 0.68 | | | | | |
| [`hmbyt5-preliminary/byt5-small-english-german-french-finnish`](https://huggingface.co/hmbyt5-preliminary/byt5-small-english-german-french-finnish) | 85.30 ± 1.14 | 87.37 ± 0.53 | 84.12 ± 0.42 | | | | | |
| [`hmbyt5-preliminary/byt5-small-english-german-french-finnish-swedish`](https://huggingface.co/hmbyt5-preliminary/byt5-small-english-german-french-finnish-swedish) | 85.40 ± 0.78 | 87.12 ± 0.19 | 84.41 ± 0.34 | | | | | |
| [`hmbyt5-preliminary/byt5-small-english-german-french-finnish-swedish-dutch`](https://huggingface.co/hmbyt5-preliminary/byt5-small-english-german-french-finnish-swedish-dutch) | 85.51 ± 0.68 | 87.58 ± 0.39 | 84.39 ± 0.83 | 55.46 ± 1.99 | 73.38 ± 2.45 | 84.80 ± 0.44 | 75.97 ± 0.55 | |
| [`hmbyt5-preliminary/byt5-small-multilingual-4g`](https://huggingface.co/hmbyt5-preliminary/byt5-small-multilingual-4g) | 83.49 ± 0.96 | 87.65 ± 0.63 | 84.16 ± 0.90 | | | | | |
| [`hmbyt5-preliminary/byt5-small-multilingual-4g-2e`](https://huggingface.co/hmbyt5-preliminary/byt5-small-multilingual-4g-2e) | 83.86 ± 0.61 | 87.54 ± 0.19 | 84.29 ± 0.41 | | | | | |
| [`hmbyt5-preliminary/byt5-small-multilingual-4g-3e`](https://huggingface.co/hmbyt5-preliminary/byt5-small-multilingual-4g-3e) | 83.49 ± 0.99 | 87.38 ± 0.53 | 84.30 ± 0.51 | | | | | |
| [`hmbyt5-preliminary/byt5-small-historic-multilingual-flax`](https://huggingface.co/hmbyt5-preliminary/byt5-small-historic-multilingual-flax) | 83.28 ± 1.67 | 86.98 ± 0.71 | 83.49 ± 1.06 | 76.96 ± 1.58 | 78.80 ± 1.89 | 86.47 ± 0.79 | 77.43 ± 0.51 | |
| [`hmbyt5-preliminary/byt5-small-historic-multilingual-span20-flax`](https://huggingface.co/hmbyt5-preliminary/byt5-small-historic-multilingual-span20-flax) | 84.91 ± 0.86 | 88.02 ± 0.35 | 84.78 ± 0.75 | 77.77 ± 1.83 | 79.94 ± 0.60 | 86.85 ± 0.91 | 77.45 ± 0.54 | |
More recent results on more datasets can be found in the [`hmLeaderboard`](https://huggingface.co/spaces/hmbench/hmLeaderboard).
# Acknowledgements
We thank [Luisa März](https://github.com/LuisaMaerz), [Katharina Schmid](https://github.com/schmika) and
[Erion Çano](https://github.com/erionc) for their fruitful discussions about Historical Language Models.
Research supported with Cloud TPUs from Google's [TPU Research Cloud](https://sites.research.google/trc/about/) (TRC).
Many Thanks for providing access to the TPUs ❤️
|