--- title: Index-Echo-S2ST-9B-GGUF canonical_url: "https://www.modelscope.ai/models/IndexTeam/Index-Echo-S2ST-9B-GGUF" md_url: "https://www.modelscope.ai/models/IndexTeam/Index-Echo-S2ST-9B-GGUF.md" repository: IndexTeam/Index-Echo-S2ST-9B-GGUF last_updated: 2026-10-19 license: apache-2.0 base_model: - IndexTeam/Index-Echo-S2ST-9B base_model_relation: quantized library_name: - gguf - pytorch frameworks: - pytorch downloads: 137 stars: 0 tags: - gguf - translation - index --- # Index-Echo-S2ST-9B-GGUF > Index-Echo-S2ST-9B-GGUF - An open-source model by IndexTeam on ModelScope. Index-Echo-S2ST-9B-GGUF IndexTeam/Index-Echo-S2ST-9B-GGUF is a machine learning model on ModelScope. licensed under apache-2.0. derived from IndexTeam/Index-Echo-S2ST-9B. - **Repository**: IndexTeam/Index-Echo-S2ST-9B-GGUF - **License**: apache-2.0 - **Base model**: IndexTeam/Index-Echo-S2ST-9B - **Tags**: gguf, translation, index - **Downloads**: 137 - **Stars**: 0 - **Last updated**: 2026-10-19 Source: https://www.modelscope.ai/models/IndexTeam/Index-Echo-S2ST-9B-GGUF --- # Index-Echo-S2ST-9B-GGUF Official GGUF conversion of the **text LLM backbone** (`stlm_llm/`) of [IndexTeam/Index-Echo-S2ST-9B](https://huggingface.co/IndexTeam/Index-Echo-S2ST-9B) (**v4rft**), part of the **Index-Echo** speech-to-speech translation (S2ST) model family by bilibili. The v4rft package updates the multilingual translator and covers **twelve directions**: Chinese, Japanese, or English speech input, dubbed into English, Spanish, Japanese, Korean, or Chinese. > **Note: LLM backbone only.** The audio tower, connector and speech-synthesis stack are **not** included - llama.cpp cannot run the full speech pipeline. For full speech-to-speech translation (S2ST) inference, use the original checkpoint or the FP8 build in [Index-Echo-S2ST-9B-FP8](https://huggingface.co/IndexTeam/Index-Echo-S2ST-9B-FP8), which ships the complete pipeline with an FP8-quantized LLM. Converted with [llama.cpp](https://github.com/ggml-org/llama.cpp) (v0.26.0, 2026-10-15); static post-training quantization, all bit-widths in this single repository. ## Provided files (sorted by size) | File | Size | Notes | |:-----|-----:|:------| | [Q2_K](https://huggingface.co/IndexTeam/Index-Echo-S2ST-9B-GGUF/resolve/main/Index-Echo-S2ST-9B.Q2_K.gguf) | 3.83 GB | 2-bit, significant quality loss | | [Q3_K_S](https://huggingface.co/IndexTeam/Index-Echo-S2ST-9B-GGUF/resolve/main/Index-Echo-S2ST-9B.Q3_K_S.gguf) | 4.26 GB | 3-bit, noticeable quality loss | | [Q3_K_M](https://huggingface.co/IndexTeam/Index-Echo-S2ST-9B-GGUF/resolve/main/Index-Echo-S2ST-9B.Q3_K_M.gguf) | 4.62 GB | 3-bit, noticeable quality loss | | [Q3_K_L](https://huggingface.co/IndexTeam/Index-Echo-S2ST-9B-GGUF/resolve/main/Index-Echo-S2ST-9B.Q3_K_L.gguf) | 4.93 GB | 3-bit, noticeable quality loss | | [IQ4_XS](https://huggingface.co/IndexTeam/Index-Echo-S2ST-9B-GGUF/resolve/main/Index-Echo-S2ST-9B.IQ4_XS.gguf) | 5.23 GB | 4-bit I-quant, small | | [Q4_K_S](https://huggingface.co/IndexTeam/Index-Echo-S2ST-9B-GGUF/resolve/main/Index-Echo-S2ST-9B.Q4_K_S.gguf) | 5.35 GB | 4-bit, slightly smaller/faster | | [Q4_K_M](https://huggingface.co/IndexTeam/Index-Echo-S2ST-9B-GGUF/resolve/main/Index-Echo-S2ST-9B.Q4_K_M.gguf) | 5.63 GB | 4-bit, good balance, recommended | | [Q5_K_S](https://huggingface.co/IndexTeam/Index-Echo-S2ST-9B-GGUF/resolve/main/Index-Echo-S2ST-9B.Q5_K_S.gguf) | 6.31 GB | 5-bit, low quality loss | | [Q5_K_M](https://huggingface.co/IndexTeam/Index-Echo-S2ST-9B-GGUF/resolve/main/Index-Echo-S2ST-9B.Q5_K_M.gguf) | 6.47 GB | 5-bit, low quality loss | | [Q6_K](https://huggingface.co/IndexTeam/Index-Echo-S2ST-9B-GGUF/resolve/main/Index-Echo-S2ST-9B.Q6_K.gguf) | 7.36 GB | 6-bit, very low quality loss | | [Q8_0](https://huggingface.co/IndexTeam/Index-Echo-S2ST-9B-GGUF/resolve/main/Index-Echo-S2ST-9B.Q8_0.gguf) | 9.53 GB | 8-bit, near-lossless | | [f16](https://huggingface.co/IndexTeam/Index-Echo-S2ST-9B-GGUF/resolve/main/Index-Echo-S2ST-9B.f16.gguf) | 17.92 GB | 16-bit, lossless conversion baseline | ## Usage ```bash # Run the recommended Q4_K_M directly: llama serve -hf IndexTeam/Index-Echo-S2ST-9B-GGUF:Q4_K_M llama cli -hf IndexTeam/Index-Echo-S2ST-9B-GGUF:Q4_K_M ``` ## Consistency validation Before release, each quantization tier was validated against the F16 conversion: per-token KL divergence / RMS delta-p via `llama-perplexity` (CPU build of llama.cpp), plus greedy-generation spot checks. | Quant | PPL (fixed corpus) | Mean KLD | RMS Δp | Same top-1 | |:------|-------------------:|---------:|-------:|-----------:| | f16 | 3.2502 | (reference) | - | - | | Q8_0 | 3.2507 | 0.001272 | 1.23% | 99.02% | | Q6_K | 3.2467 | 0.012566 | 4.18% | 96.61% | | Q5_K_M | 3.2775 | 0.016146 | 4.45% | 96.09% | | Q5_K_S | 3.3047 | 0.025917 | 5.45% | 95.37% | | Q4_K_M | 3.2806 | 0.035922 | 6.90% | 94.39% | | Q4_K_S | 3.6296 | 0.090064 | 10.84% | 91.32% | | Q3_K_L | 3.4872 | 0.092859 | 10.21% | 91.00% | | Q3_K_M | 3.8014 | 0.151914 | 14.26% | 87.87% | | Q3_K_S | 5.8845 | 0.535761 | 30.10% | 80.36% | | Q2_K | 4.6526 | 0.596711 | 26.59% | 76.32% | | IQ4_XS | 3.4390 | 0.081273 | 10.17% | 91.52% | *Converted and published by the Index team, 2026-10-16.*