Added model(s) card
Browse files
README.md
CHANGED
@@ -1,3 +1,55 @@
|
|
1 |
---
|
2 |
license: cc-by-nc-sa-4.0
|
3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
license: cc-by-nc-sa-4.0
|
3 |
---
|
4 |
+
|
5 |
+
# Model Card: Pre-trained Audio Representation Models on AudioSet
|
6 |
+
|
7 |
+
## Overview
|
8 |
+
|
9 |
+
This model card presents information about pre-trained audio representation models released by ALM. These models are pre-trained on the full AudioSet dataset and are intended for general-purpose Audio Representation Learning (ARL) tasks.
|
10 |
+
|
11 |
+
## Models
|
12 |
+
|
13 |
+
### 1. [ALM/hubert-base-audioset](https://huggingface.co/ALM/hubert-base-audioset)
|
14 |
+
|
15 |
+
- **Architecture**: HuBERT (Hubert-Base) transformer-based model
|
16 |
+
- **Description**: This model is based on the HuBERT architecture, pre-trained on the full AudioSet dataset.
|
17 |
+
|
18 |
+
### 2. [ALM/hubert-large-audioset](https://huggingface.co/ALM/hubert-large-audioset)
|
19 |
+
|
20 |
+
- **Architecture**: HuBERT (Hubert-Large) transformer-based model
|
21 |
+
- **Description**: Similar to the hubert-base-audioset model, this variant is larger in size, providing increased capacity for capturing audio representations from the full AudioSet dataset.
|
22 |
+
|
23 |
+
### 3. [ALM/wav2vec2-base-audioset](https://huggingface.co/ALM/wav2vec2-base-audioset)
|
24 |
+
|
25 |
+
- **Architecture**: Wav2Vec 2.0 (Wav2Vec2-Base) transformer-based model
|
26 |
+
- **Description**: This model is based on the Wav2Vec 2.0 architecture, trained on the full AudioSet dataset using SSL with CPC. It offers a different approach to audio representation learning compared to the HuBERT models.
|
27 |
+
|
28 |
+
### 4. [ALM/wav2vec2-large-audioset](https://huggingface.co/ALM/wav2vec2-large-audioset)
|
29 |
+
|
30 |
+
- **Architecture**: Wav2Vec 2.0 (Wav2Vec2-Large) transformer-based model
|
31 |
+
- **Description**: Similar to the wav2vec2-base-audioset model, this variant is larger in size, providing enhanced capacity for learning audio representations from the full AudioSet dataset.
|
32 |
+
|
33 |
+
## Intended Use
|
34 |
+
|
35 |
+
These pre-trained models are intended for a wide range of ARL tasks, including but not limited to speech recognition, music classification, and acoustic event detection. They serve as powerful tools for feature extraction and can be fine-tuned on task-specific datasets for downstream applications.
|
36 |
+
It's important to note that while these models offer versatility across various audio domains, their performance in speech-related tasks may be relatively lower compared to specialized models such as the original Wav2Vec and HuBERT models.
|
37 |
+
This is due to the diverse nature of the AudioSet dataset used for pre-training, which includes a wide range of audio sources beyond speech.
|
38 |
+
|
39 |
+
## Limitations and Considerations
|
40 |
+
|
41 |
+
- The models are pre-trained on the full AudioSet dataset, which may not cover all possible audio domains comprehensively.
|
42 |
+
- Fine-tuning on domain-specific data may be necessary to achieve optimal performance for certain tasks.
|
43 |
+
- Computational resources may be required for deploying and fine-tuning these models, especially the larger variants.
|
44 |
+
|
45 |
+
## Citation
|
46 |
+
|
47 |
+
If you use these pre-trained models in your work, please cite the following:
|
48 |
+
|
49 |
+
```bib
|
50 |
+
@article{ARCH,
|
51 |
+
title={Benchmarking Representations for Speech, Music, and Acoustic Events},
|
52 |
+
author={La Quatra, Moreno and Koudounas, Alkis and Vaiani, Lorenzo and Baralis, Elena and Garza, Paolo and Cagliero, Luca, and Siniscalchi, Sabato Marco},
|
53 |
+
year={2024}
|
54 |
+
}
|
55 |
+
```
|