Hugging Face's logo

Spaces:
argmaxinc
/
whisperkit-benchmarks
Running

App Files Community
ardaatahan commited on
Commit
f33de5a
•
1 Parent(s): b57c4cc

update version.json

Browse files
Files changed (4) hide show
  1. .github/scripts/process_report.py +1 -0
  2. .github/workflows/dataset_update.yml +1 -0
  3. dashboard_data/performance_data.json +0 -0
  4. dashboard_data/support_data.csv +23 -23
.github/scripts/process_report.py CHANGED
@@ -216,6 +216,7 @@ def generate_report():
216
  prev_version = json.load(f)
217
  with open("dashboard_data/version.json", "r") as f:
218
  curr_version = json.load(f)
 
219
 
220
  prev_releases = set(prev_version.get("releases", []))
221
  curr_releases = set(curr_version.get("releases", []))
 
216
  prev_version = json.load(f)
217
  with open("dashboard_data/version.json", "r") as f:
218
  curr_version = json.load(f)
219
+ print(curr_version)
220
 
221
  prev_releases = set(prev_version.get("releases", []))
222
  curr_releases = set(curr_version.get("releases", []))
.github/workflows/dataset_update.yml CHANGED
@@ -108,6 +108,7 @@ jobs:
108
  path: report_data
109
 
110
  - name: Install dependencies
 
111
  run: |
112
  python -m pip install --upgrade pip
113
  pip install pandas beautifulsoup4
 
108
  path: report_data
109
 
110
  - name: Install dependencies
111
+ if: steps.check.outputs.has_updates == 'true'
112
  run: |
113
  python -m pip install --upgrade pip
114
  pip install pandas beautifulsoup4
dashboard_data/performance_data.json CHANGED
The diff for this file is too large to render. See raw diff
 
dashboard_data/support_data.csv CHANGED
@@ -1,23 +1,23 @@
1
- ,Model,"Apple M2 Pro (Mac14,12)","Apple M3 Max (Mac15,10)","iPad Air 11-inch (M2) (iPad14,8)","iPhone 11 (iPhone12,1)","iPhone 13 Pro (iPhone14,2)"
2
- distil-whisper_distil-large-v3,distil-whisper_distil-large-v3,✅ macOS 15.0.1,✅ macOS 15.2,✅ iPadOS 17.6.1,Not Supported,Not Supported
3
- distil-whisper_distil-large-v3_594MB,distil-whisper_distil-large-v3_594MB,✅ macOS 15.0.1,✅ macOS 15.2,✅ iPadOS 17.6.1,Not Supported,✅ iOS 18.1
4
- distil-whisper_distil-large-v3_turbo,distil-whisper_distil-large-v3_turbo,✅ macOS 15.0.1,✅ macOS 15.2,Not Supported,Not Supported,Not Supported
5
- distil-whisper_distil-large-v3_turbo_600MB,distil-whisper_distil-large-v3_turbo_600MB,✅ macOS 15.0.1,✅ macOS 15.2,✅ iPadOS 17.6.1,Not Supported,✅ iOS 18.1
6
- openai_whisper-base,openai_whisper-base,✅ macOS 15.0.1,✅ macOS 15.2,✅ iPadOS 17.6.1,✅ iOS 17.6.1,✅ iOS 18.1
7
- openai_whisper-base.en,openai_whisper-base.en,✅ macOS 15.0.1,✅ macOS 15.2,✅ iPadOS 17.6.1,✅ iOS 17.6.1,✅ iOS 18.1
8
- openai_whisper-large-v2,openai_whisper-large-v2,✅ macOS 15.0.1,✅ macOS 15.2,Not Supported,Not Supported,Not Supported
9
- openai_whisper-large-v2_949MB,openai_whisper-large-v2_949MB,✅ macOS 15.0.1,✅ macOS 15.2,✅ iPadOS 17.6.1,Not Supported,✅ iOS 18.1
10
- openai_whisper-large-v2_turbo,openai_whisper-large-v2_turbo,✅ macOS 15.0.1,✅ macOS 15.2,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-11-07T010510_5254d82/iPad14%2C8_summary_2024-11-07T052705.json>iPadOS 17.6.1</a>,Not Supported,Not Supported
11
- openai_whisper-large-v2_turbo_955MB,openai_whisper-large-v2_turbo_955MB,✅ macOS 15.0.1,✅ macOS 15.2,✅ iPadOS 17.6.1,Not Supported,✅ iOS 18.1
12
- openai_whisper-large-v3,openai_whisper-large-v3,✅ macOS 15.0.1,✅ macOS 15.2,Not Supported,Not Supported,Not Supported
13
- openai_whisper-large-v3-v20240930,openai_whisper-large-v3-v20240930,✅ macOS 15.0.1,✅ macOS 15.2,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-11-07T010510_5254d82/iPad14%2C8_summary_2024-11-07T052705.json>iPadOS 17.6.1</a>,Not Supported,Not Supported
14
- openai_whisper-large-v3-v20240930_626MB,openai_whisper-large-v3-v20240930_626MB,✅ macOS 15.0.1,✅ macOS 15.2,✅ iPadOS 17.6.1,Not Supported,✅ iOS 18.1
15
- openai_whisper-large-v3-v20240930_turbo,openai_whisper-large-v3-v20240930_turbo,✅ macOS 15.0.1,✅ macOS 15.2,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-11-07T010510_5254d82/iPad14%2C8_summary_2024-11-07T052705.json>iPadOS 17.6.1</a>,Not Supported,Not Supported
16
- openai_whisper-large-v3-v20240930_turbo_632MB,openai_whisper-large-v3-v20240930_turbo_632MB,✅ macOS 15.0.1,✅ macOS 15.2,✅ iPadOS 17.6.1,Not Supported,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-11-07T010510_5254d82/iPhone14%2C2_summary_2024-11-07T052705.json>iOS 18.1</a>
17
- openai_whisper-large-v3_947MB,openai_whisper-large-v3_947MB,✅ macOS 15.0.1,✅ macOS 15.2,✅ iPadOS 17.6.1,Not Supported,✅ iOS 18.1
18
- openai_whisper-large-v3_turbo,openai_whisper-large-v3_turbo,✅ macOS 15.0.1,✅ macOS 15.2,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-11-07T010510_5254d82/iPad14%2C8_summary_2024-11-07T052705.json>iPadOS 17.6.1</a>,Not Supported,Not Supported
19
- openai_whisper-large-v3_turbo_954MB,openai_whisper-large-v3_turbo_954MB,✅ macOS 15.0.1,✅ macOS 15.2,✅ iPadOS 17.6.1,Not Supported,✅ iOS 18.1
20
- openai_whisper-small,openai_whisper-small,✅ macOS 15.0.1,✅ macOS 15.2,✅ iPadOS 17.6.1,Not Supported,✅ iOS 18.1
21
- openai_whisper-small.en,openai_whisper-small.en,✅ macOS 15.0.1,✅ macOS 15.2,✅ iPadOS 17.6.1,Not Supported,✅ iOS 18.1
22
- openai_whisper-tiny,openai_whisper-tiny,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-11-07T010510_5254d82/Mac14%2C12_summary_2024-11-07T052705.json>macOS 15.0.1</a>,✅ macOS 15.2,✅ iPadOS 17.6.1,Not Supported,✅ iOS 18.1
23
- openai_whisper-tiny.en,openai_whisper-tiny.en,✅ macOS 15.0.1,✅ macOS 15.2,✅ iPadOS 17.6.1,✅ iOS 17.6.1,✅ iOS 18.1
 
1
+ ,Model,"Apple M2 Pro (Mac14,12)","Apple M2 Ultra (Mac14,14)","Apple M2 (Mac14,2)","Apple M3 Max (Mac15,10)","Apple M3 Pro (Mac15,6)","Apple M3 Max (Mac15,9)","Apple M1 (MacBookAir10,1)","iPad Air (5th generation) (iPad13,16)","iPad Air 11-inch (M2) (iPad14,8)","iPad mini (A17 Pro) (iPad16,1)","iPad Pro 11-inch (M4) (iPad16,3)","iPhone 11 (iPhone12,1)","iPhone 12 mini (iPhone13,1)","iPhone 12 (iPhone13,2)","iPhone 12 Pro Max (iPhone13,4)","iPhone 13 Pro (iPhone14,2)","iPhone 13 (iPhone14,5)","iPhone 14 (iPhone14,7)","iPhone 15 Pro (iPhone16,1)","iPhone 15 Pro Max (iPhone16,2)","iPhone 16 Pro (iPhone17,1)","iPhone 16 (iPhone17,3)","iPhone 16 Plus (iPhone17,4)"
2
+ distil-whisper_distil-large-v3,distil-whisper_distil-large-v3,✅ macOS 15.0.1,✅ macOS 15.0.1,✅ macOS 15.1,✅ macOS 15.2,✅ macOS 15.1,✅ macOS 15.1,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/MacBookAir10%2C1_summary_2024-10-30T012535.json>macOS 15.1</a>,Not Supported,✅ iPadOS 17.6.1,Not Supported,✅ iPadOS 18.1,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported
3
+ distil-whisper_distil-large-v3_594MB,distil-whisper_distil-large-v3_594MB,✅ macOS 15.0.1,✅ macOS 15.0.1,✅ macOS 15.1,✅ macOS 15.2,✅ macOS 15.1,✅ macOS 15.1,✅ macOS 15.1,Not Supported,✅ iPadOS 17.6.1,✅ iPadOS 18.0.1,✅ iPadOS 18.1,Not Supported,Not Supported,Not Supported,Not Supported,✅ iOS 18.1,✅ iOS 17.3,✅ iOS 17.3,✅ iOS 18.1,✅ iOS 18.2,✅ iOS 18.0,✅ iOS 18.0.1,✅ iOS 18.0.1
4
+ distil-whisper_distil-large-v3_turbo,distil-whisper_distil-large-v3_turbo,✅ macOS 15.0.1,✅ macOS 15.0.1,✅ macOS 15.1,✅ macOS 15.2,✅ macOS 15.1,✅ macOS 15.1,Not Supported,Not Supported,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/iPad14%2C8_summary_2024-10-30T073931.json>iPadOS 17.6.1</a>,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/iPad16%2C1_summary_2024-10-29T234233.json>iPadOS 18.0.1</a>,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/iPad16%2C3_summary_2024-10-30T073931.json>iPadOS 18.1</a>,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported
5
+ distil-whisper_distil-large-v3_turbo_600MB,distil-whisper_distil-large-v3_turbo_600MB,✅ macOS 15.0.1,✅ macOS 15.0.1,✅ macOS 15.1,✅ macOS 15.2,✅ macOS 15.1,✅ macOS 15.1,Not Supported,Not Supported,✅ iPadOS 17.6.1,✅ iPadOS 18.0.1,✅ iPadOS 18.1,Not Supported,Not Supported,Not Supported,Not Supported,✅ iOS 18.1,✅ iOS 17.3,✅ iOS 17.3,✅ iOS 18.1,✅ iOS 18.2,✅ iOS 18.0,✅ iOS 18.0.1,✅ iOS 18.0.1
6
+ openai_whisper-base,openai_whisper-base,✅ macOS 15.0.1,✅ macOS 15.0.1,✅ macOS 15.1,✅ macOS 15.2,✅ macOS 15.1,✅ macOS 15.1,✅ macOS 15.1,✅ iPadOS 18.1,✅ iPadOS 17.6.1,✅ iPadOS 18.0.1,✅ iPadOS 18.1,✅ iOS 17.6.1,✅ iOS 17.7,✅ iOS 17.2.1,✅ iOS 18.1,✅ iOS 18.1,✅ iOS 17.3,✅ iOS 17.3,✅ iOS 18.1,✅ iOS 18.2,✅ iOS 18.0,✅ iOS 18.0.1,✅ iOS 18.0.1
7
+ openai_whisper-base.en,openai_whisper-base.en,✅ macOS 15.0.1,✅ macOS 15.0.1,✅ macOS 15.1,✅ macOS 15.2,✅ macOS 15.1,✅ macOS 15.1,✅ macOS 15.1,✅ iPadOS 18.1,✅ iPadOS 17.6.1,✅ iPadOS 18.0.1,✅ iPadOS 18.1,✅ iOS 17.6.1,✅ iOS 17.7,✅ iOS 17.2.1,✅ iOS 18.1,✅ iOS 18.1,✅ iOS 17.3,✅ iOS 17.3,✅ iOS 18.1,✅ iOS 18.2,✅ iOS 18.0,✅ iOS 18.0.1,✅ iOS 18.0.1
8
+ openai_whisper-large-v2,openai_whisper-large-v2,✅ macOS 15.0.1,✅ macOS 15.0.1,✅ macOS 15.1,✅ macOS 15.2,✅ macOS 15.1,✅ macOS 15.1,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/MacBookAir10%2C1_summary_2024-10-30T012535.json>macOS 15.1</a>,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported
9
+ openai_whisper-large-v2_949MB,openai_whisper-large-v2_949MB,✅ macOS 15.0.1,✅ macOS 15.0.1,✅ macOS 15.1,✅ macOS 15.2,✅ macOS 15.1,✅ macOS 15.1,✅ macOS 15.1,Not Supported,✅ iPadOS 17.6.1,✅ iPadOS 18.0.1,✅ iPadOS 18.1,Not Supported,Not Supported,Not Supported,Not Supported,✅ iOS 18.1,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/iPhone14%2C5_summary_2024-10-30T073931.json>iOS 17.3</a>,✅ iOS 17.3,✅ iOS 18.1,✅ iOS 18.2,✅ iOS 18.0,✅ iOS 18.0.1,✅ iOS 18.0.1
10
+ openai_whisper-large-v2_turbo,openai_whisper-large-v2_turbo,✅ macOS 15.0.1,✅ macOS 15.0.1,✅ macOS 15.1,✅ macOS 15.2,✅ macOS 15.1,✅ macOS 15.1,Not Supported,Not Supported,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/iPad14%2C8_summary_2024-10-30T073931.json>iPadOS 17.6.1</a>,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/iPad16%2C1_summary_2024-10-29T234233.json>iPadOS 18.0.1</a>,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/iPad16%2C3_summary_2024-10-30T073931.json>iPadOS 18.1</a>,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported
11
+ openai_whisper-large-v2_turbo_955MB,openai_whisper-large-v2_turbo_955MB,✅ macOS 15.0.1,✅ macOS 15.0.1,✅ macOS 15.1,✅ macOS 15.2,✅ macOS 15.1,✅ macOS 15.1,Not Supported,Not Supported,✅ iPadOS 17.6.1,✅ iPadOS 18.0.1,✅ iPadOS 18.1,Not Supported,Not Supported,Not Supported,Not Supported,✅ iOS 18.1,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/iPhone14%2C5_summary_2024-10-30T073931.json>iOS 17.3</a>,✅ iOS 17.3,✅ iOS 18.1,✅ iOS 18.2,✅ iOS 18.0,✅ iOS 18.0.1,✅ iOS 18.0.1
12
+ openai_whisper-large-v3,openai_whisper-large-v3,✅ macOS 15.0.1,✅ macOS 15.0.1,✅ macOS 15.1,✅ macOS 15.2,✅ macOS 15.1,✅ macOS 15.1,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/MacBookAir10%2C1_summary_2024-10-30T012535.json>macOS 15.1</a>,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported
13
+ openai_whisper-large-v3-v20240930,openai_whisper-large-v3-v20240930,✅ macOS 15.0.1,✅ macOS 15.0.1,✅ macOS 15.1,✅ macOS 15.2,✅ macOS 15.1,✅ macOS 15.1,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/MacBookAir10%2C1_summary_2024-10-30T012535.json>macOS 15.1</a>,Not Supported,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/iPad14%2C8_summary_2024-10-30T073931.json>iPadOS 17.6.1</a>,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/iPad16%2C1_summary_2024-10-29T234233.json>iPadOS 18.0.1</a>,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/iPad16%2C3_summary_2024-10-30T073931.json>iPadOS 18.1</a>,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported
14
+ openai_whisper-large-v3-v20240930_626MB,openai_whisper-large-v3-v20240930_626MB,✅ macOS 15.0.1,✅ macOS 15.0.1,✅ macOS 15.1,✅ macOS 15.2,✅ macOS 15.1,✅ macOS 15.1,✅ macOS 15.1,Not Supported,✅ iPadOS 17.6.1,✅ iPadOS 18.0.1,✅ iPadOS 18.1,Not Supported,Not Supported,Not Supported,Not Supported,✅ iOS 18.1,✅ iOS 17.3,✅ iOS 17.3,✅ iOS 18.1,✅ iOS 18.2,✅ iOS 18.0,✅ iOS 18.0.1,✅ iOS 18.0.1
15
+ openai_whisper-large-v3-v20240930_turbo,openai_whisper-large-v3-v20240930_turbo,✅ macOS 15.0.1,✅ macOS 15.0.1,✅ macOS 15.1,✅ macOS 15.2,✅ macOS 15.1,✅ macOS 15.1,Not Supported,Not Supported,Not Supported,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/iPad16%2C1_summary_2024-10-29T234233.json>iPadOS 18.0.1</a>,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/iPad16%2C3_summary_2024-10-30T073931.json>iPadOS 18.1</a>,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported
16
+ openai_whisper-large-v3-v20240930_turbo_632MB,openai_whisper-large-v3-v20240930_turbo_632MB,✅ macOS 15.0.1,✅ macOS 15.0.1,✅ macOS 15.1,✅ macOS 15.2,✅ macOS 15.1,✅ macOS 15.1,Not Supported,Not Supported,✅ iPadOS 17.6.1,✅ iPadOS 18.0.1,✅ iPadOS 18.1,Not Supported,Not Supported,Not Supported,Not Supported,✅ iOS 18.1,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/iPhone14%2C5_summary_2024-10-30T073931.json>iOS 17.3</a>,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/iPhone14%2C7_summary_2024-10-30T073931.json>iOS 17.3</a>,✅ iOS 18.1,✅ iOS 18.2,✅ iOS 18.0,✅ iOS 18.0.1,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/iPhone17%2C4_summary_2024-10-30T132649.json>iOS 18.0.1</a>
17
+ openai_whisper-large-v3_947MB,openai_whisper-large-v3_947MB,✅ macOS 15.0.1,✅ macOS 15.0.1,✅ macOS 15.1,✅ macOS 15.2,✅ macOS 15.1,✅ macOS 15.1,✅ macOS 15.1,Not Supported,✅ iPadOS 17.6.1,✅ iPadOS 18.0.1,✅ iPadOS 18.1,Not Supported,Not Supported,Not Supported,Not Supported,✅ iOS 18.1,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/iPhone14%2C5_summary_2024-10-30T073931.json>iOS 17.3</a>,✅ iOS 17.3,✅ iOS 18.1,✅ iOS 18.2,✅ iOS 18.0,✅ iOS 18.0.1,✅ iOS 18.0.1
18
+ openai_whisper-large-v3_turbo,openai_whisper-large-v3_turbo,✅ macOS 15.0.1,✅ macOS 15.0.1,✅ macOS 15.1,✅ macOS 15.2,✅ macOS 15.1,✅ macOS 15.1,Not Supported,Not Supported,Not Supported,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/iPad16%2C1_summary_2024-10-29T234233.json>iPadOS 18.0.1</a>,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/iPad16%2C3_summary_2024-10-30T073931.json>iPadOS 18.1</a>,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported,Not Supported
19
+ openai_whisper-large-v3_turbo_954MB,openai_whisper-large-v3_turbo_954MB,✅ macOS 15.0.1,✅ macOS 15.0.1,✅ macOS 15.1,✅ macOS 15.2,✅ macOS 15.1,✅ macOS 15.1,Not Supported,Not Supported,✅ iPadOS 17.6.1,✅ iPadOS 18.0.1,✅ iPadOS 18.1,Not Supported,Not Supported,Not Supported,Not Supported,✅ iOS 18.1,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/iPhone14%2C5_summary_2024-10-30T073931.json>iOS 17.3</a>,✅ iOS 17.3,✅ iOS 18.1,✅ iOS 18.2,✅ iOS 18.0,✅ iOS 18.0.1,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/iPhone17%2C4_summary_2024-10-30T132649.json>iOS 18.0.1</a>
20
+ openai_whisper-small,openai_whisper-small,✅ macOS 15.0.1,✅ macOS 15.0.1,✅ macOS 15.1,✅ macOS 15.2,✅ macOS 15.1,✅ macOS 15.1,✅ macOS 15.1,✅ iPadOS 18.1,✅ iPadOS 17.6.1,✅ iPadOS 18.0.1,✅ iPadOS 18.1,Not Supported,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/iPhone13%2C1_summary_2024-10-30T022739.json>iOS 17.7</a>,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/iPhone13%2C2_summary_2024-10-30T152729.json>iOS 17.2.1</a>,✅ iOS 18.1,✅ iOS 18.1,✅ iOS 17.3,✅ iOS 17.3,✅ iOS 18.1,✅ iOS 18.2,✅ iOS 18.0,✅ iOS 18.0.1,✅ iOS 18.0.1
21
+ openai_whisper-small.en,openai_whisper-small.en,✅ macOS 15.0.1,✅ macOS 15.0.1,✅ macOS 15.1,✅ macOS 15.2,✅ macOS 15.1,✅ macOS 15.1,✅ macOS 15.1,✅ iPadOS 18.1,✅ iPadOS 17.6.1,✅ iPadOS 18.0.1,✅ iPadOS 18.1,Not Supported,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/iPhone13%2C1_summary_2024-10-30T022739.json>iOS 17.7</a>,⚠️ <a style='color: #3B82F6; text-decoration: underline; text-decoration-style: dotted;' href=https://huggingface.co/datasets/argmaxinc/whisperkit-evals-dataset/blob/main/benchmark_data/2024-10-29T190053_a9b92c4/iPhone13%2C2_summary_2024-10-30T152729.json>iOS 17.2.1</a>,✅ iOS 18.1,✅ iOS 18.1,✅ iOS 17.3,✅ iOS 17.3,✅ iOS 18.1,✅ iOS 18.2,✅ iOS 18.0,✅ iOS 18.0.1,✅ iOS 18.0.1
22
+ openai_whisper-tiny,openai_whisper-tiny,✅ macOS 15.0.1,✅ macOS 15.0.1,✅ macOS 15.1,✅ macOS 15.2,✅ macOS 15.1,✅ macOS 15.1,✅ macOS 15.1,✅ iPadOS 18.1,✅ iPadOS 17.6.1,✅ iPadOS 18.0.1,✅ iPadOS 18.1,Not Supported,✅ iOS 17.7,✅ iOS 17.2.1,✅ iOS 18.1,✅ iOS 18.1,✅ iOS 17.3,✅ iOS 17.3,✅ iOS 18.1,✅ iOS 18.2,✅ iOS 18.0,✅ iOS 18.0.1,✅ iOS 18.0.1
23
+ openai_whisper-tiny.en,openai_whisper-tiny.en,✅ macOS 15.0.1,✅ macOS 15.0.1,✅ macOS 15.1,✅ macOS 15.2,✅ macOS 15.1,✅ macOS 15.1,✅ macOS 15.1,✅ iPadOS 18.1,✅ iPadOS 17.6.1,✅ iPadOS 18.0.1,✅ iPadOS 18.1,✅ iOS 17.6.1,✅ iOS 17.7,✅ iOS 17.2.1,✅ iOS 18.1,✅ iOS 18.1,✅ iOS 17.3,✅ iOS 17.3,✅ iOS 18.1,✅ iOS 18.2,✅ iOS 18.0,✅ iOS 18.0.1,✅ iOS 18.0.1