STT Benchmark

O'zbek tilidagi nutqni matnga o'girish (transkripsiya) bo'yicha LLM va maxsus STT modellari reytingi. Har bir model 20 ta audio parcha bo'yicha 5 martadan sinovdan o'tkaziladi.

22
Modellar
20
Audio parchalar
2200
Baholangan transkripsiyalar
Gemini 3.8 Flash
Eng yaxshi LLM · WER 25.3%
KotibAI (uzbek_stt_v1)
Eng yaxshi STT · WER 4.8%
# Model O'xshashlik WER CER Vaqt Progress
1
Gemini 3.8 Flash API
google/gemini-3.8-flash
90.8%
± 14.5
 25.3% 12.6% 5.87s
100/100
2
Gemini 3.7 Flash API
google/gemini-3.7-flash
89.9%
± 17.3
 27.1% 14.8% 6.09s
100/100
3
Gemini 3.5 Flash API
google/gemini-3.5-flash
86.7%
± 18.9
 31.3% 15.7% 15.78s
100/100
4
Gemini 2.5 Flash API
google/gemini-2.5-flash
85.5%
± 24.2
 67.0% 55.9% 4.26s
100/100
5
Gemma 4 E4B (Local Q4_K_M) Local
local/gemma-4-E4B-it-Q4_K_M
85.3%
± 17.6
 61.9% 39.1% 1.69s
100/100
6
Gemini 3.6 Flash API
google/gemini-3.6-flash
85.2%
± 16.7
 35.5% 18.0% 4.35s
100/100
7
Gemini 3.1 Flash Lite API
google/gemini-3.1-flash-lite
84.5%
± 23.8
 30.6% 13.0% 3.53s
100/100
8
Gemma 4 E2B (Local Q4_0) Local
local/gemma-4-E2B_q4_0-it
82.7%
± 20.0
 70.7% 48.8% 0.91s
100/100
9
Gemini 3.5 Flash Lite API
google/gemini-3.5-flash-lite
81.5%
± 24.0
 37.5% 18.9% 3.25s
100/100
10
Gemini 2.5 Flash Lite API
google/gemini-2.5-flash-lite
81.3%
± 25.7
 63.2% 41.5% 2.04s
100/100
11
Nemotron 3 Nano Omni 30B (Local Q4_K_M) Local
local/NVIDIA-Nemotron-3-Nano-Omni-30B-A3B-Reasoning-UD-Q4_K_M
58.5%
± 18.3
 121.3% 72.1% 5.0s
100/100
12
Gemma 4 12B (Local Q4_0) Local
local/gemma-4-12B-it-QAT-Q4_0
44.0%
± 25.0
 139.4% 130.8% 3.82s
100/100
# Model O'xshashlik WER CER Vaqt Progress
1
KotibAI (uzbek_stt_v1) Local
Kotib/uzbek_stt_v1
98.5%
± 3.8
 4.8% 2.5% 1.0s
100/100
2
Uzbek MuteVoice-STT (BaseLayer) Local
BaseLayer/Uzbek_MuteVoice-STT
95.8%
± 7.0
 14.7% 7.1% 0.94s
100/100
3
Rubai STT v2 Medium (islomov) Local
islomov/rubaistt_v2_medium
95.4%
± 8.1
 16.2% 8.7% 0.97s
100/100
4
Whisper Large Tashkent (IbratDO) Local
IbratDO/whisper-large-linkdata-tashkent-dialect
92.7%
± 8.2
 25.5% 12.8% 1.47s
100/100
5
Qwen3 ASR 0.6B (Saidakmal) Local
Saidakmal/ASR_Qwen_0.6B_UZ
89.7%
± 10.8
 25.9% 11.5% 2.13s
100/100
6
FastConformer Uzbek (Saidakmal) Local
Saidakmal/ASR_nvidia_fastconformer
86.0%
± 19.4
 42.5% 20.1% 0.15s
100/100
7
Oyqiz (uzbek_stt) Local
oyqiz/uzbek_stt
82.4%
± 22.8
 61.5% 41.9% 0.08s
100/100
8
SarahAI (uzbek-stt-3) Local
sarahai/uzbek-stt-3
81.2%
± 24.2
 55.7% 28.1% 0.08s
100/100
9
Gemini 3.5 Transcribe API
google/gemini-3.5-transcribe
76.2%
± 31.2
 40.2% 27.7% 1.49s
100/100
10
Whisper Large v3 Turbo (OpenAI) Local
openai/whisper-large-v3-turbo
54.5%
± 26.9
 222.2% 177.6% 0.43s
100/100
# Model O'xshashlik WER CER Vaqt Progress
1
KotibAI (uzbek_stt_v1) Local STT
Kotib/uzbek_stt_v1
98.5%
± 3.8
 4.8% 2.5% 1.0s
100/100
2
Uzbek MuteVoice-STT (BaseLayer) Local STT
BaseLayer/Uzbek_MuteVoice-STT
95.8%
± 7.0
 14.7% 7.1% 0.94s
100/100
3
Rubai STT v2 Medium (islomov) Local STT
islomov/rubaistt_v2_medium
95.4%
± 8.1
 16.2% 8.7% 0.97s
100/100
4
Whisper Large Tashkent (IbratDO) Local STT
IbratDO/whisper-large-linkdata-tashkent-dialect
92.7%
± 8.2
 25.5% 12.8% 1.47s
100/100
5
Gemini 3.8 Flash API LLM
google/gemini-3.8-flash
90.8%
± 14.5
 25.3% 12.6% 5.87s
100/100
6
Gemini 3.7 Flash API LLM
google/gemini-3.7-flash
89.9%
± 17.3
 27.1% 14.8% 6.09s
100/100
7
Qwen3 ASR 0.6B (Saidakmal) Local STT
Saidakmal/ASR_Qwen_0.6B_UZ
89.7%
± 10.8
 25.9% 11.5% 2.13s
100/100
8
Gemini 3.5 Flash API LLM
google/gemini-3.5-flash
86.7%
± 18.9
 31.3% 15.7% 15.78s
100/100
9
FastConformer Uzbek (Saidakmal) Local STT
Saidakmal/ASR_nvidia_fastconformer
86.0%
± 19.4
 42.5% 20.1% 0.15s
100/100
10
Gemini 2.5 Flash API LLM
google/gemini-2.5-flash
85.5%
± 24.2
 67.0% 55.9% 4.26s
100/100
11
Gemma 4 E4B (Local Q4_K_M) Local LLM
local/gemma-4-E4B-it-Q4_K_M
85.3%
± 17.6
 61.9% 39.1% 1.69s
100/100
12
Gemini 3.6 Flash API LLM
google/gemini-3.6-flash
85.2%
± 16.7
 35.5% 18.0% 4.35s
100/100
13
Gemini 3.1 Flash Lite API LLM
google/gemini-3.1-flash-lite
84.5%
± 23.8
 30.6% 13.0% 3.53s
100/100
14
Gemma 4 E2B (Local Q4_0) Local LLM
local/gemma-4-E2B_q4_0-it
82.7%
± 20.0
 70.7% 48.8% 0.91s
100/100
15
Oyqiz (uzbek_stt) Local STT
oyqiz/uzbek_stt
82.4%
± 22.8
 61.5% 41.9% 0.08s
100/100
16
Gemini 3.5 Flash Lite API LLM
google/gemini-3.5-flash-lite
81.5%
± 24.0
 37.5% 18.9% 3.25s
100/100
17
Gemini 2.5 Flash Lite API LLM
google/gemini-2.5-flash-lite
81.3%
± 25.7
 63.2% 41.5% 2.04s
100/100
18
SarahAI (uzbek-stt-3) Local STT
sarahai/uzbek-stt-3
81.2%
± 24.2
 55.7% 28.1% 0.08s
100/100
19
Gemini 3.5 Transcribe API STT
google/gemini-3.5-transcribe
76.2%
± 31.2
 40.2% 27.7% 1.49s
100/100
20
Nemotron 3 Nano Omni 30B (Local Q4_K_M) Local LLM
local/NVIDIA-Nemotron-3-Nano-Omni-30B-A3B-Reasoning-UD-Q4_K_M
58.5%
± 18.3
 121.3% 72.1% 5.0s
100/100
21
Whisper Large v3 Turbo (OpenAI) Local STT
openai/whisper-large-v3-turbo
54.5%
± 26.9
 222.2% 177.6% 0.43s
100/100
22
Gemma 4 12B (Local Q4_0) Local LLM
local/gemma-4-12B-it-QAT-Q4_0
44.0%
± 25.0
 139.4% 130.8% 3.82s
100/100

Metodologiya

  • O'xshashlik — normallashtirilgan transkripsiya va haqiqiy matn orasidagi belgilar darajasidagi o'xshashlik (0–100%, yuqori — yaxshi). Reyting shu ko'rsatkich bo'yicha.
  • WER / CER — so'z / belgi xatolik darajasi (past — yaxshi). Matn kichik harfga o'tkaziladi, tinish belgilari olib tashlanadi.
  • ± qiymat — 5 ta takroriy sinov orasidagi standart og'ish (barqarorlik).
  • API modellari server tomonidan avtomatik baholanadi; Local modellar mahalliy kompyuterda ishga tushirilib, natijalari yuboriladi. LLMlar bir xil prompt bilan sinovdan o'tkaziladi.
  • Progress to'liq bo'lmagan modellar natijalari hali yakuniy emas.