---
title: kyrgyz-whisper-small
canonical_url: "https://www.modelscope.ai/models/nineninesix/kyrgyz-whisper-small"
md_url: "https://www.modelscope.ai/models/nineninesix/kyrgyz-whisper-small.md"
repository: nineninesix/kyrgyz-whisper-small
last_updated: 2026-11-05
license: apache-2.0
model_type:
  - whisper
architectures:
  - WhisperForConditionalGeneration
base_model:
  - openai/whisper-small
base_model_relation: finetune
parameters: 241.7M
tensor_type:
  - F32
library_name:
  - pytorch
  - transformer
  - safetensors
frameworks:
  - pytorch
language:
  - en
  - ru
  - ky
downloads: 31
stars: 0
tags:
  - asr
  - stt
  - whisper
---

# kyrgyz-whisper-small

> kyrgyz-whisper-small - An open-source model by nineninesix on ModelScope. 🇰🇬 Whisper Small - Kyrgyz, English, Russian Speech To Text model

nineninesix/kyrgyz-whisper-small is a 241.7M-parameter machine learning model on ModelScope. licensed under apache-2.0. derived from openai/whisper-small.

- **Repository**: nineninesix/kyrgyz-whisper-small
- **License**: apache-2.0
- **Parameters**: 241.7M
- **Base model**: openai/whisper-small
- **Tags**: asr, stt, whisper
- **Downloads**: 31
- **Stars**: 0
- **Last updated**: 2026-11-05

Source: https://www.modelscope.ai/models/nineninesix/kyrgyz-whisper-small

---

# 🇰🇬 Whisper Small - Kyrgyz, English, Russian Speech To Text model

> 🎙️ **Try it online: [AkylAI](https://akylai.com/)** — AI voice studio for Kyrgyz, Kazakh and Russian: [text-to-speech](https://akylai.com/ky/text-to-speech), [video dubbing](https://akylai.com/ky/video-dubbing), [voice cloning](https://akylai.com/ky/voice-cloning) and [speech-to-text](https://akylai.com/ky/speech-to-text). No install, works in the browser. · [Кыргызча](https://akylai.com/ky/) · [Қазақша](https://akylai.com/kk/) · [Русский](https://akylai.com/ru/)

## Model Description

**kyrgyz-whisper-small** is a fine-tuned multilingual speech recognition model based on OpenAI's [Whisper Small](https://huggingface.co/openai/whisper-small) architecture. This model adds **native Kyrgyz language support** while maintaining strong performance on English and Russian.

### Key Features
- **Kyrgyz language support** via custom `<|ky|>` token.
- **Multilingual**: Kyrgyz, English, and Russian.
- Trained on **~2,000 hours** of Kyrgyz audio + 40% English/Russian audio
- Ready for further improvement with **LoRA fine-tuning** (see [Colab Notebook](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Whisper.ipynb))
- Optimized for real-world, noisy audio conditions


## Performance

### WER Distributions on FLEURS Benchmark

The following visualization shows improvement after fine-tuning:

![plot_smal](https://cdn-uploads.huggingface.co/production/uploads/65ae44614b5cc40be0e4fe72/yjZlojcJnyHemrUj8a4sm.png)


**Key Observations:**
- **Kyrgyz**: Dramatic improvement from ~100% WER (unusable) → practical performance with peak around 0.2-0.4 WER
- **English & Russian**: Some performance degradation compared to base model as trade-off for Kyrgyz support
  - Distributions shifted right (higher WER)
  - This is expected when adding a new language to a fixed-capacity model
- **Multi-language trade-off**: The model sacrifices some accuracy on English/Russian to gain Kyrgyz capabilities
- [Benchmark Fleurs](https://huggingface.co/datasets/kyrgyz-ai/asr-fleurs-en-kg-ru-benchmark)

### Recommended Use Cases

- Kyrgyz media transcription
- Multilingual call centers
- Educational content in Kyrgyz
- Code-switching scenarios (common in Kyrgyzstan where people mix languages)
- Foundation model for LoRA fine-tuning on clean Kyrgyz data

## Technical Implementation

### Custom Tokenizer Integration
```python
from transformers import AutoTokenizer

# Load custom tokenizer with Kyrgyz support
tokenizer = AutoTokenizer.from_pretrained(
    "nineninesix/kyrgyz-whisper-small",
    trust_remote_code=True, ### !!! important !!!
    language="kyrgyz",
    task="transcribe"
)
```

### Kyrgyz Token Initialization
The `<|ky|>` token was initialized as an average of embeddings from linguistically similar languages:
```python
embedding_ky = (embedding_ru + embedding_kk + embedding_tr) / 3
```

---

## Usage

### Pipeline Usage

```python

import torch
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline, WhisperFeatureExtractor, AutoTokenizer

device = "cuda:0" if torch.cuda.is_available() else "cpu"
torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32

model_id = "nineninesix/kyrgyz-whisper-small"

model = AutoModelForSpeechSeq2Seq.from_pretrained(
    model_id, torch_dtype=torch_dtype, low_cpu_mem_usage=True, use_safetensors=True)
model.to(device)

feature_extractor = WhisperFeatureExtractor.from_pretrained(model_id)

tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    trust_remote_code=True, language="kyrgyz", task="transcribe")

pipe = pipeline(
    "automatic-speech-recognition",
    model=model,
    tokenizer=tokenizer,
    feature_extractor=feature_extractor,
    torch_dtype=torch_dtype,
    device=device
)

result = pipe("audio.mp3")
print(result['text'])

```



## Further Fine-tuning with LoRA

This model serves as a foundation for domain-specific fine-tuning using **LoRA** (Low-Rank Adaptation).

Unsloth integration example: see this [Google Colab](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Whisper.ipynb)




Benefits of LoRA fine-tuning:
-  Adapt to specific domains (medical, legal, conversational)
-  Memory-efficient training
-  Faster training than full fine-tuning
-  Improved accuracy on clean datasets

## Limitations

- Trained on noisy data - may have higher WER on clean benchmarks vs. clean-trained models
- Best performance on Kyrgyz, English, and Russian (other languages not supported)
- Requires custom tokenizer for Kyrgyz language support
- May require domain-specific fine-tuning for specialized applications

## Citation
```bibtex
@misc{kyrgyz-whisper-small,
  author = {nineninesix},
  title = {Whisper Small - Kyrgyz, English, Russian},
  year = {2025},
  publisher = {HuggingFace},
  url = {https://huggingface.co/nineninesix/kyrgyz-whisper-small}
}

@misc{radford2022whisper,
  doi = {10.48550/ARXIV.2212.04356},
  url = {https://arxiv.org/abs/2212.04356},
  author = {Radford, Alec and Kim, Jong Wook and Xu, Tao and Brockman, Greg and McLeavey, Christine and Sutskever, Ilya},
  title = {Robust Speech Recognition via Large-Scale Weak Supervision},
  publisher = {arXiv},
  year = {2022},
  copyright = {arXiv.org perpetual, non-exclusive license}
}
```

## Acknowledgments

- Based on OpenAI's Whisper architecture
- Kyrgyz tokenizer: `kyrgyz-ai/whisper_tokenizer_ky`
- Training datasets: Kyrgyz ASR community contributions
- Inspired by multilingual ASR research

## License

Apache 2.0 - see LICENSE file for details.