--- title: Ling-3.1-flash-base canonical_url: "https://www.modelscope.ai/models/inclusionAI/Ling-3.1-flash-base" md_url: "https://www.modelscope.ai/models/inclusionAI/Ling-3.1-flash-base.md" repository: inclusionAI/Ling-3.1-flash-base last_updated: 2026-10-27 license: mit pipeline_tag: text-generation tasks: - text-generation model_type: - bailing_hybrid architectures: - BailingMoeV3ForCausalLM library_name: - safetensors - pytorch frameworks: - pytorch downloads: 6 stars: 0 --- # Ling-3.1-flash-base > Ling-3.1-flash-base - An open-source model by inclusionAI on ModelScope. 🤗 Hugging Face    |   ðŸ¤– ModelScope inclusionAI/Ling-3.1-flash-base is a text-generation model on ModelScope. licensed under mit. - **Repository**: inclusionAI/Ling-3.1-flash-base - **License**: mit - **Tasks**: text-generation - **Downloads**: 6 - **Stars**: 0 - **Last updated**: 2026-10-27 Source: https://www.modelscope.ai/models/inclusionAI/Ling-3.1-flash-base ---

inclusionAI

🤗 Hugging Face   |   ðŸ¤– ModelScope

# Ling-3.1-flash-base ## Introduction We introduce **Ling-3.1-flash-base**, the base checkpoint of [Ling-3.1-flash](https://huggingface.co/inclusionAI/Ling-3.1-flash), released to support research and community-driven innovation. With **560B total parameters** and **25B active parameters per token**, the model provides a foundation for continued pretraining, domain adaptation, and post-training research. Ling-3.1-flash-base shares the same hybrid-linear Mixture-of-Experts (MoE) architecture as Ling-3.1-flash, combining KDA with Gated MLA. As a base checkpoint, it is intended for further training and adaptation. The model was trained using **FP8**. This release provides the **BF16 base checkpoint** produced after **Warmup-Stable and Merge (WSM)** checkpoint merging. BF16 refers to the precision of the released weights, while FP8 refers to the precision used during training. ## Model Overview ### Key Features - **Sparse MoE architecture:** 512 routed experts, with 8 routed experts and 1 shared expert activated per token, for a total of 25B active parameters per token. - **Hybrid linear attention:** 63 KDA layers and 9 Gated MLA layers combine linear attention with Gated MLA at a 7:1 ratio. ### Model Specifications | Specification | Value | | --- | --- | | Model Type | Base checkpoint | | Architecture | Hybrid-linear MoE | | Parameter Scale | Total 560B, Activated 25B | | Transformer Layers | 63 KDA + 9 Gated MLA (7:1) | | Number of Dense Layers | 4 | | Number of Routed Experts | 512 | | Number of Shared Experts | 1 | | Number of Activated Routed Experts | 8 | | Attention Heads | 64 | | Hidden Size | 4096 | | Expert Intermediate Size | 1280 | | Dense Intermediate Size | 10240 | | Vocabulary Size | 157184 | ### Architecture Diagram ![Ling-3.1-flash architecture](https://cdn-uploads.huggingface.co/production/uploads/678143d4ddb1f1ce16daca8e/qwmfVu_K08auI7R_ijbIX.png) ## Base Model Evaluation We evaluate **Ling-3.1-flash-base** on 47 benchmarks across eight domains: language understanding, knowledge, code, mathematics, reasoning, multilingual capabilities, medical knowledge, and long-context understanding. The results below compare the model with DeepSeek-V4-Pro-Base and MiMo-V2.5-Pro-Base under the shot configurations shown. **Bold** indicates the highest score in each row; underlining indicates the second-highest distinct score. Tied scores receive the same formatting. EM denotes exact match, and Acc denotes accuracy. Parameter scales are shown as total parameters and active parameters per token (A).
Domain Benchmark Shot Config Ling-3.1-flash-base DeepSeek-V4-Pro-Base MiMo-V2.5-Pro-Base
560B A25B 1.6T A49B 1.02T A42B
Language Understanding Race-High(EM) 0-shot 93.88 93.37 93.60
SquadV2(Acc) 1-shot 92.79 87.70 88.93
Belebele(EM) 0-shot 94.67 93.83 92.72
C3(EM) 0-shot 83.34 81.64 81.53
CCPM(EM) 0-shot 91.88 93.31 90.18
Knowledge ARC-c(EM) 0-shot 97.97 96.61 96.95
AGIEval(Acc) few-shot(0~5) 82.59 81.26 79.90
SimpleQA-Verified(Acc) 5-shot 52.40 52.10 32.60
C-SimpleQA(Acc) 5-shot 77.43 77.60 70.93
AA-Omniscience(Acc) 3-shot 37.67 48.16 36.45
MMLU-Pro(EM) 5-shot 76.71 75.28 71.26
CEval(EM) 5-shot 92.36 93.76 92.08
CMMLU(EM) 5-shot 91.60 90.52 88.33
GPQA-Diamond(EM) 0-shot 56.57 52.02 54.04
SuperGPQA(EM) 5-shot 55.84 53.19 48.65
Code HumanEval-Plus(Pass@1) 0-shot 79.88 57.93 82.93
CruxEval(Pass@1) 1-shot 80.50 69.06 71.25
MultiPL-E(Pass@1) 0-shot 70.55 55.94 64.74
MBPP-Plus(Pass@1) 3-shot 76.19 73.81 77.51
BigCodeBench(Pass@1) 0-shot 53.60 51.49 66.75
LCBench1(Pass@1) 3-shot 68.48 59.14 67.58
LCBench-cn1(Pass@1) 3-shot 60.83 57.35 63.36
Math MATH500(Acc) 4-shot 79.40 68.20 57.20
CollegeMath(Acc) 4-shot 73.41 63.15 66.61
FinanceReasoning(Acc) 3-shot 81.76 82.16 80.89
OlympiadBench(Acc) 3-shot 60.24 46.39 48.04
TheoremQA(Acc) 5-shot 63.56 61.92 60.27
OmniMath(Acc) 3-shot 65.31 34.89 40.13
Reasoning CommonSenseQA(EM) 5-shot 92.22 86.49 88.53
BBH(EM) 3-shot 90.25 89.98 89.80
KorBench(Acc) 3-shot 54.96 54.56 53.20
AutoLogi(Acc) 3-shot 67.13 64.81 66.23
ZebraLogic(Acc) 1-shot 30.80 20.30 28.90
Multilingual mARC(EM) 0-shot 92.98 89.10 88.55
MMMLU(EM) 0-shot 77.56 68.81 67.15
MultiGSM(Acc) 4-shot 87.66 85.87 86.33
HumanEvalXL(Pass@1) 0-shot 83.25 70.31 87.34
Med DiagnosisArena(EM) 0-shot 62.20 55.50 52.70
medmcqa(EM) 5-shot 78.08 80.33 77.50
medqa(EM) 5-shot 87.82 87.75 85.55
Long-context LongBench(Acc) 0-shot 51.90 17.70 15.66
LEval(Acc) 0-shot 85.09 58.34 61.49
ruler_16k(Acc) 1-shot 96.90 97.00 90.61
ruler_32k(Acc) 1-shot 95.87 95.57 89.10
ruler_64k(Acc) 1-shot 93.81 95.56 87.19
ruler_128k(Acc) 1-shot 88.64 95.06 85.79
ruler_256k(Acc) 1-shot 86.64 94.37 86.00
Note:
1 LCBench and LCBench-cn: January 2023–February 2025 (2301-2502).
## Intended Use Recommended use cases: - Continued pretraining. - Mid-training. - Supervised fine-tuning for domain adaptation. - Preference optimization and reinforcement-learning post-training. - Distillation research. - Long-context and MoE systems research. Not recommended as-is for: - Direct end-user chat deployment. - Safety-critical applications without additional alignment and evaluation. - Production use without post-training and task-specific validation. ## Usage For fine-tuning examples, please refer to our [ling-cookbook](https://github.com/inclusionAI/ling-cookbook/). ## FAQ For questions or feedback, please open a discussion in the model repository. ## License This model is released under the **MIT License**.