---
license: cc-by-nc-4.0
base_model: facebook/mms-1b-all
tags:
- generated_from_trainer
metrics:
- wer
model-index:
- name: mms-lug
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# mms-lug

This model is a fine-tuned version of [facebook/mms-1b-all](https://huggingface.co/facebook/mms-1b-all) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: inf
- Wer: 0.1745

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.001
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- num_epochs: 5

### Training results

| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| No log        | 0.02  | 20   | inf             | 1.0049 |
| No log        | 0.04  | 40   | inf             | 1.0    |
| No log        | 0.07  | 60   | inf             | 0.3009 |
| No log        | 0.09  | 80   | inf             | 0.1998 |
| 4.042         | 0.11  | 100  | inf             | 0.2110 |
| 4.042         | 0.13  | 120  | inf             | 0.2047 |
| 4.042         | 0.15  | 140  | inf             | 0.2040 |
| 4.042         | 0.17  | 160  | inf             | 0.1949 |
| 4.042         | 0.2   | 180  | inf             | 0.2026 |
| 0.3406        | 0.22  | 200  | inf             | 0.1833 |
| 0.3406        | 0.24  | 220  | inf             | 0.1875 |
| 0.3406        | 0.26  | 240  | inf             | 0.2128 |
| 0.3406        | 0.28  | 260  | inf             | 0.1910 |
| 0.3406        | 0.3   | 280  | inf             | 0.1798 |
| 0.2831        | 0.33  | 300  | inf             | 0.1749 |
| 0.2831        | 0.35  | 320  | inf             | 0.1875 |
| 0.2831        | 0.37  | 340  | inf             | 0.1780 |
| 0.2831        | 0.39  | 360  | inf             | 0.1907 |
| 0.2831        | 0.41  | 380  | inf             | 0.2005 |
| 0.2796        | 0.43  | 400  | inf             | 0.2037 |
| 0.2796        | 0.46  | 420  | inf             | 0.1973 |
| 0.2796        | 0.48  | 440  | inf             | 0.1917 |
| 0.2796        | 0.5   | 460  | inf             | 0.2015 |
| 0.2796        | 0.52  | 480  | inf             | 0.1784 |
| 0.3198        | 0.54  | 500  | inf             | 0.1812 |
| 0.3198        | 0.56  | 520  | inf             | 0.1808 |
| 0.3198        | 0.59  | 540  | inf             | 0.1987 |
| 0.3198        | 0.61  | 560  | inf             | 0.1696 |
| 0.3198        | 0.63  | 580  | inf             | 0.1710 |
| 0.3131        | 0.65  | 600  | inf             | 0.1794 |
| 0.3131        | 0.67  | 620  | inf             | 0.1773 |
| 0.3131        | 0.69  | 640  | inf             | 0.1777 |
| 0.3131        | 0.72  | 660  | inf             | 0.2005 |
| 0.3131        | 0.74  | 680  | inf             | 0.1886 |
| 0.2777        | 0.76  | 700  | inf             | 0.2058 |
| 0.2777        | 0.78  | 720  | inf             | 0.1770 |
| 0.2777        | 0.8   | 740  | inf             | 0.1801 |
| 0.2777        | 0.83  | 760  | inf             | 0.1914 |
| 0.2777        | 0.85  | 780  | inf             | 0.1770 |
| 0.2694        | 0.87  | 800  | inf             | 0.1815 |
| 0.2694        | 0.89  | 820  | inf             | 0.1717 |
| 0.2694        | 0.91  | 840  | inf             | 0.1777 |
| 0.2694        | 0.93  | 860  | inf             | 0.1671 |
| 0.2694        | 0.96  | 880  | inf             | 0.1706 |
| 0.3216        | 0.98  | 900  | inf             | 0.2381 |
| 0.3216        | 1.0   | 920  | inf             | 0.1766 |
| 0.3216        | 1.02  | 940  | inf             | 0.1826 |
| 0.3216        | 1.04  | 960  | inf             | 0.1749 |
| 0.3216        | 1.06  | 980  | inf             | 0.1836 |
| 0.3008        | 1.09  | 1000 | inf             | 0.1696 |
| 0.3008        | 1.11  | 1020 | inf             | 0.1703 |
| 0.3008        | 1.13  | 1040 | inf             | 0.1703 |
| 0.3008        | 1.15  | 1060 | inf             | 0.1942 |
| 0.3008        | 1.17  | 1080 | inf             | 0.1780 |
| 0.2621        | 1.19  | 1100 | inf             | 0.1626 |
| 0.2621        | 1.22  | 1120 | inf             | 0.1728 |
| 0.2621        | 1.24  | 1140 | inf             | 0.1738 |
| 0.2621        | 1.26  | 1160 | inf             | 0.1745 |
| 0.2621        | 1.28  | 1180 | inf             | 0.1766 |
| 0.2837        | 1.3   | 1200 | inf             | 0.1685 |
| 0.2837        | 1.32  | 1220 | inf             | 0.1647 |
| 0.2837        | 1.35  | 1240 | inf             | 0.1664 |
| 0.2837        | 1.37  | 1260 | inf             | 0.1784 |
| 0.2837        | 1.39  | 1280 | inf             | 0.1699 |
| 0.2648        | 1.41  | 1300 | inf             | 0.1713 |
| 0.2648        | 1.43  | 1320 | inf             | 0.1749 |
| 0.2648        | 1.45  | 1340 | inf             | 0.1692 |
| 0.2648        | 1.48  | 1360 | inf             | 0.1808 |
| 0.2648        | 1.5   | 1380 | inf             | 0.1724 |
| 0.3012        | 1.52  | 1400 | inf             | 0.1710 |
| 0.3012        | 1.54  | 1420 | inf             | 0.1668 |
| 0.3012        | 1.56  | 1440 | inf             | 0.1724 |
| 0.3012        | 1.59  | 1460 | inf             | 0.1763 |
| 0.3012        | 1.61  | 1480 | inf             | 0.1868 |
| 0.287         | 1.63  | 1500 | inf             | 0.1675 |
| 0.287         | 1.65  | 1520 | inf             | 0.1728 |
| 0.287         | 1.67  | 1540 | inf             | 0.1661 |
| 0.287         | 1.69  | 1560 | inf             | 0.1657 |
| 0.287         | 1.72  | 1580 | inf             | 0.1794 |
| 0.2042        | 1.74  | 1600 | inf             | 0.1685 |
| 0.2042        | 1.76  | 1620 | inf             | 0.1956 |
| 0.2042        | 1.78  | 1640 | inf             | 0.1594 |
| 0.2042        | 1.8   | 1660 | inf             | 0.1626 |
| 0.2042        | 1.82  | 1680 | inf             | 0.1559 |
| 0.3219        | 1.85  | 1700 | inf             | 0.1612 |
| 0.3219        | 1.87  | 1720 | inf             | 0.1759 |
| 0.3219        | 1.89  | 1740 | inf             | 0.1742 |
| 0.3219        | 1.91  | 1760 | inf             | 0.1738 |
| 0.3219        | 1.93  | 1780 | inf             | 0.1640 |
| 0.2695        | 1.95  | 1800 | inf             | 0.1678 |
| 0.2695        | 1.98  | 1820 | inf             | 0.1615 |
| 0.2695        | 2.0   | 1840 | inf             | 0.1685 |
| 0.2695        | 2.02  | 1860 | inf             | 0.1650 |
| 0.2695        | 2.04  | 1880 | inf             | 0.1682 |
| 0.2759        | 2.06  | 1900 | inf             | 0.1938 |
| 0.2759        | 2.08  | 1920 | inf             | 0.1629 |
| 0.2759        | 2.11  | 1940 | inf             | 0.1587 |
| 0.2759        | 2.13  | 1960 | inf             | 0.1566 |
| 0.2759        | 2.15  | 1980 | inf             | 0.1605 |
| 0.2584        | 2.17  | 2000 | inf             | 0.1699 |
| 0.2584        | 2.19  | 2020 | inf             | 0.1780 |
| 0.2584        | 2.21  | 2040 | inf             | 0.1664 |
| 0.2584        | 2.24  | 2060 | inf             | 0.1629 |
| 0.2584        | 2.26  | 2080 | inf             | 0.1699 |
| 0.2618        | 2.28  | 2100 | inf             | 0.1629 |
| 0.2618        | 2.3   | 2120 | inf             | 0.1692 |
| 0.2618        | 2.32  | 2140 | inf             | 0.1706 |
| 0.2618        | 2.35  | 2160 | inf             | 0.1675 |
| 0.2618        | 2.37  | 2180 | inf             | 0.1696 |
| 0.278         | 2.39  | 2200 | inf             | 0.1626 |
| 0.278         | 2.41  | 2220 | inf             | 0.1612 |
| 0.278         | 2.43  | 2240 | inf             | 0.1696 |
| 0.278         | 2.45  | 2260 | inf             | 0.1654 |
| 0.278         | 2.48  | 2280 | inf             | 0.1668 |
| 0.2213        | 2.5   | 2300 | inf             | 0.1678 |
| 0.2213        | 2.52  | 2320 | inf             | 0.1745 |
| 0.2213        | 2.54  | 2340 | inf             | 0.1640 |
| 0.2213        | 2.56  | 2360 | inf             | 0.1608 |
| 0.2213        | 2.58  | 2380 | inf             | 0.1573 |
| 0.2187        | 2.61  | 2400 | inf             | 0.1657 |
| 0.2187        | 2.63  | 2420 | inf             | 0.1587 |
| 0.2187        | 2.65  | 2440 | inf             | 0.1622 |
| 0.2187        | 2.67  | 2460 | inf             | 0.1717 |
| 0.2187        | 2.69  | 2480 | inf             | 0.1562 |
| 0.2585        | 2.71  | 2500 | inf             | 0.1573 |
| 0.2585        | 2.74  | 2520 | inf             | 0.1566 |
| 0.2585        | 2.76  | 2540 | inf             | 0.1742 |
| 0.2585        | 2.78  | 2560 | inf             | 0.1605 |
| 0.2585        | 2.8   | 2580 | inf             | 0.1605 |
| 0.2913        | 2.82  | 2600 | inf             | 0.1745 |
| 0.2913        | 2.84  | 2620 | inf             | 0.1608 |
| 0.2913        | 2.87  | 2640 | inf             | 0.1615 |
| 0.2913        | 2.89  | 2660 | inf             | 0.1675 |
| 0.2913        | 2.91  | 2680 | inf             | 0.1612 |
| 0.2743        | 2.93  | 2700 | inf             | 0.1577 |
| 0.2743        | 2.95  | 2720 | inf             | 0.1598 |
| 0.2743        | 2.98  | 2740 | inf             | 0.1566 |
| 0.2743        | 3.0   | 2760 | inf             | 0.1573 |
| 0.2743        | 3.02  | 2780 | inf             | 0.1594 |
| 0.2854        | 3.04  | 2800 | inf             | 0.1608 |
| 0.2854        | 3.06  | 2820 | inf             | 0.1650 |
| 0.2854        | 3.08  | 2840 | inf             | 0.1580 |
| 0.2854        | 3.11  | 2860 | inf             | 0.1640 |
| 0.2854        | 3.13  | 2880 | inf             | 0.1594 |
| 0.243         | 3.15  | 2900 | inf             | 0.1601 |
| 0.243         | 3.17  | 2920 | inf             | 0.1703 |
| 0.243         | 3.19  | 2940 | inf             | 0.1573 |
| 0.243         | 3.21  | 2960 | inf             | 0.1587 |
| 0.243         | 3.24  | 2980 | inf             | 0.1668 |
| 0.2563        | 3.26  | 3000 | inf             | 0.1713 |
| 0.2563        | 3.28  | 3020 | inf             | 0.1570 |
| 0.2563        | 3.3   | 3040 | inf             | 0.1601 |
| 0.2563        | 3.32  | 3060 | inf             | 0.1619 |
| 0.2563        | 3.34  | 3080 | inf             | 0.1541 |
| 0.228         | 3.37  | 3100 | inf             | 0.1552 |
| 0.228         | 3.39  | 3120 | inf             | 0.1566 |
| 0.228         | 3.41  | 3140 | inf             | 0.1552 |
| 0.228         | 3.43  | 3160 | inf             | 0.1619 |
| 0.228         | 3.45  | 3180 | inf             | 0.1657 |
| 0.2684        | 3.47  | 3200 | inf             | 0.1622 |
| 0.2684        | 3.5   | 3220 | inf             | 0.1548 |
| 0.2684        | 3.52  | 3240 | inf             | 0.1608 |
| 0.2684        | 3.54  | 3260 | inf             | 0.1664 |
| 0.2684        | 3.56  | 3280 | inf             | 0.1594 |
| 0.2623        | 3.58  | 3300 | inf             | 0.1650 |
| 0.2623        | 3.6   | 3320 | inf             | 0.1541 |
| 0.2623        | 3.63  | 3340 | inf             | 0.1555 |
| 0.2623        | 3.65  | 3360 | inf             | 0.1598 |
| 0.2623        | 3.67  | 3380 | inf             | 0.1629 |
| 0.2573        | 3.69  | 3400 | inf             | 0.1584 |
| 0.2573        | 3.71  | 3420 | inf             | 0.1650 |
| 0.2573        | 3.74  | 3440 | inf             | 0.1612 |
| 0.2573        | 3.76  | 3460 | inf             | 0.1615 |
| 0.2573        | 3.78  | 3480 | inf             | 0.1584 |
| 0.2698        | 3.8   | 3500 | inf             | 0.1591 |
| 0.2698        | 3.82  | 3520 | inf             | 0.1601 |
| 0.2698        | 3.84  | 3540 | inf             | 0.1626 |
| 0.2698        | 3.87  | 3560 | inf             | 0.1566 |
| 0.2698        | 3.89  | 3580 | inf             | 0.1699 |
| 0.2178        | 3.91  | 3600 | inf             | 0.1647 |
| 0.2178        | 3.93  | 3620 | inf             | 0.1608 |
| 0.2178        | 3.95  | 3640 | inf             | 0.1703 |
| 0.2178        | 3.97  | 3660 | inf             | 0.1717 |
| 0.2178        | 4.0   | 3680 | inf             | 0.1626 |
| 0.2964        | 4.02  | 3700 | inf             | 0.1612 |
| 0.2964        | 4.04  | 3720 | inf             | 0.1671 |
| 0.2964        | 4.06  | 3740 | inf             | 0.1682 |
| 0.2964        | 4.08  | 3760 | inf             | 0.1738 |
| 0.2964        | 4.1   | 3780 | inf             | 0.1728 |
| 0.2778        | 4.13  | 3800 | inf             | 0.1879 |
| 0.2778        | 4.15  | 3820 | inf             | 0.1633 |
| 0.2778        | 4.17  | 3840 | inf             | 0.1675 |
| 0.2778        | 4.19  | 3860 | inf             | 0.1657 |
| 0.2778        | 4.21  | 3880 | inf             | 0.1879 |
| 0.292         | 4.23  | 3900 | inf             | 0.1664 |
| 0.292         | 4.26  | 3920 | inf             | 0.1699 |
| 0.292         | 4.28  | 3940 | inf             | 0.1822 |
| 0.292         | 4.3   | 3960 | inf             | 0.1678 |
| 0.292         | 4.32  | 3980 | inf             | 0.1682 |
| 0.2267        | 4.34  | 4000 | inf             | 0.1654 |
| 0.2267        | 4.36  | 4020 | inf             | 0.1724 |
| 0.2267        | 4.39  | 4040 | inf             | 0.1777 |
| 0.2267        | 4.41  | 4060 | inf             | 0.1721 |
| 0.2267        | 4.43  | 4080 | inf             | 0.1770 |
| 0.2807        | 4.45  | 4100 | inf             | 0.1717 |
| 0.2807        | 4.47  | 4120 | inf             | 0.1773 |
| 0.2807        | 4.5   | 4140 | inf             | 0.1777 |
| 0.2807        | 4.52  | 4160 | inf             | 0.1657 |
| 0.2807        | 4.54  | 4180 | inf             | 0.1692 |
| 0.2953        | 4.56  | 4200 | inf             | 0.1699 |
| 0.2953        | 4.58  | 4220 | inf             | 0.1650 |
| 0.2953        | 4.6   | 4240 | inf             | 0.1731 |
| 0.2953        | 4.63  | 4260 | inf             | 0.1685 |
| 0.2953        | 4.65  | 4280 | inf             | 0.1721 |
| 0.2463        | 4.67  | 4300 | inf             | 0.1643 |
| 0.2463        | 4.69  | 4320 | inf             | 0.1636 |
| 0.2463        | 4.71  | 4340 | inf             | 0.1657 |
| 0.2463        | 4.73  | 4360 | inf             | 0.1678 |
| 0.2463        | 4.76  | 4380 | inf             | 0.1759 |
| 0.313         | 4.78  | 4400 | inf             | 0.1682 |
| 0.313         | 4.8   | 4420 | inf             | 0.1671 |
| 0.313         | 4.82  | 4440 | inf             | 0.1713 |
| 0.313         | 4.84  | 4460 | inf             | 0.1675 |
| 0.313         | 4.86  | 4480 | inf             | 0.1808 |
| 0.2748        | 4.89  | 4500 | inf             | 0.1713 |
| 0.2748        | 4.91  | 4520 | inf             | 0.1812 |
| 0.2748        | 4.93  | 4540 | inf             | 0.1731 |
| 0.2748        | 4.95  | 4560 | inf             | 0.1738 |
| 0.2748        | 4.97  | 4580 | inf             | 0.1728 |
| 0.279         | 4.99  | 4600 | inf             | 0.1745 |


### Framework versions

- Transformers 4.32.0.dev0
- Pytorch 2.0.1+cu117
- Datasets 2.13.0
- Tokenizers 0.13.3