Update constants.py
Browse files- constants.py +27 -17
constants.py
CHANGED
@@ -1,6 +1,6 @@
|
|
1 |
from pathlib import Path
|
2 |
|
3 |
-
# Directory where
|
4 |
DIR_OUTPUT_REQUESTS = Path("requested_models")
|
5 |
EVAL_REQUESTS_PATH = Path("eval_requests")
|
6 |
|
@@ -8,17 +8,21 @@ EVAL_REQUESTS_PATH = Path("eval_requests")
|
|
8 |
# Text definitions #
|
9 |
##########################
|
10 |
|
11 |
-
banner_url = "https://huggingface.co/
|
12 |
BANNER = f'<div style="display: flex; justify-content: space-around;"><img src="{banner_url}" alt="Banner" style="width: 40vw; min-width: 300px; max-width: 600px;"> </div>'
|
13 |
|
14 |
-
INTRODUCTION_TEXT = "
|
15 |
-
on the Hugging Face Hub using
|
16 |
-
|
17 |
-
|
|
|
|
|
|
|
|
|
18 |
|
19 |
CITATION_TEXT = """@misc{persian-asr-leaderboard,
|
20 |
title = {Persian Automatic Speech Recognition Leaderboard},
|
21 |
-
author = {
|
22 |
year = 2024,
|
23 |
publisher = {Hugging Face},
|
24 |
howpublished = "\\url{https://huggingface.co/spaces/your-username/persian_asr_leaderboard}"
|
@@ -26,22 +30,28 @@ CITATION_TEXT = """@misc{persian-asr-leaderboard,
|
|
26 |
"""
|
27 |
|
28 |
METRICS_TAB_TEXT = """
|
29 |
-
# Metrics and
|
30 |
-
|
31 |
## Metrics
|
|
|
|
|
|
|
32 |
|
33 |
-
|
34 |
|
35 |
-
|
36 |
-
|
37 |
|
38 |
-
|
|
|
39 |
|
40 |
-
|
|
|
41 |
|
42 |
-
|
|
|
43 |
|
44 |
-
|
|
|
45 |
|
|
|
46 |
"""
|
47 |
-
|
|
|
1 |
from pathlib import Path
|
2 |
|
3 |
+
# Directory where model requests are stored
|
4 |
DIR_OUTPUT_REQUESTS = Path("requested_models")
|
5 |
EVAL_REQUESTS_PATH = Path("eval_requests")
|
6 |
|
|
|
8 |
# Text definitions #
|
9 |
##########################
|
10 |
|
11 |
+
banner_url = "https://cdn-thumbnails.huggingface.co/social-thumbnails/spaces/k2-fsa/automatic-speech-recognition.png"
|
12 |
BANNER = f'<div style="display: flex; justify-content: space-around;"><img src="{banner_url}" alt="Banner" style="width: 40vw; min-width: 300px; max-width: 600px;"> </div>'
|
13 |
|
14 |
+
INTRODUCTION_TEXT = """
|
15 |
+
π The π€ **Persian Automatic Speech Recognition (ASR) Leaderboard** serves as an authoritative ranking of speech recognition models hosted on the Hugging Face Hub, evaluated using multiple Persian speech datasets.
|
16 |
+
We report two key performance metrics: [Word Error Rate (WER)](https://huggingface.co/spaces/evaluate-metric/wer) and [Character Error Rate (CER)](https://huggingface.co/spaces/evaluate-metric/cer), where lower scores indicate better performance.
|
17 |
+
|
18 |
+
The leaderboard primarily ranks models based on WER, from lowest to highest. You can refer to the π **Metrics** tab for a detailed explanation of how these models are evaluated.
|
19 |
+
|
20 |
+
If there is a model you'd like to see ranked but is not listed here, you may submit a request for evaluation by following the instructions in the "Request a Model" tab βοΈβ¨.
|
21 |
+
"""
|
22 |
|
23 |
CITATION_TEXT = """@misc{persian-asr-leaderboard,
|
24 |
title = {Persian Automatic Speech Recognition Leaderboard},
|
25 |
+
author = {Navid},
|
26 |
year = 2024,
|
27 |
publisher = {Hugging Face},
|
28 |
howpublished = "\\url{https://huggingface.co/spaces/your-username/persian_asr_leaderboard}"
|
|
|
30 |
"""
|
31 |
|
32 |
METRICS_TAB_TEXT = """
|
33 |
+
# Evaluation Metrics and Datasets
|
|
|
34 |
## Metrics
|
35 |
+
We employ the following metrics to evaluate the performance of Automatic Speech Recognition (ASR) models:
|
36 |
+
- **Word Error Rate (WER)**: WER quantifies the proportion of incorrectly predicted words in a transcription. A lower WER reflects higher accuracy.
|
37 |
+
- **Character Error Rate (CER)**: CER measures errors at the character level, providing a more granular view of transcription accuracy, especially in morphologically rich languages such as Persian.
|
38 |
|
39 |
+
Both metrics are widely used in ASR evaluation, offering a comprehensive view of model performance.
|
40 |
|
41 |
+
## Datasets
|
42 |
+
The models on the Persian ASR Leaderboard are evaluated using a diverse range of datasets to ensure robust performance across different speech conditions:
|
43 |
|
44 |
+
1. **Persian Common Voice (Mozilla)**
|
45 |
+
Available [here](https://huggingface.co/datasets/mozilla-foundation/common_voice_17_0), this dataset is part of the broader Common Voice project and features speech data from various speakers, accents, and environments. It serves as a representative benchmark for Persian ASR.
|
46 |
|
47 |
+
2. **ASR Farsi YouTube Chunked 10 Seconds**
|
48 |
+
This dataset, available on Hugging Face [here](https://huggingface.co/datasets/pourmand1376/asr-farsi-youtube-chunked-10-seconds), consists of transcribed speech from Persian YouTube videos, split into 10-second segments. It introduces variability in audio quality and speaker demographics, adding to the challenge of accurate recognition.
|
49 |
|
50 |
+
3. **Persian-ASR-Test-Set (Private)**
|
51 |
+
This private dataset is designed for in-depth model testing and evaluation. It contains curated, real-world Persian speech data from various contexts and speaker backgrounds. Access to this dataset is restricted, ensuring models are evaluated on a controlled, high-quality speech corpus.
|
52 |
|
53 |
+
## How to Submit Your Model for Evaluation
|
54 |
+
To request that a model be included on this leaderboard, please submit its name in the following format: `username/model_name`. Models should be available on the Hugging Face Hub for public access.
|
55 |
|
56 |
+
Simply navigate to the "Request a Model" tab, enter the details, and your model will be evaluated at the next available opportunity.
|
57 |
"""
|
|