Official llama.cpp distribution of VeriLoop E2
BF16 reference · Q8_0 high fidelity · Q6_K overall sweet spot · Q5_K_M memory-quality sweet spot · IQ1_M minimum-footprint sweet spot
27B post-trained model for code, mathematics, and physics · 262K native context · Apache License 2.0
Developed by Tsinghua SIGS Robot Lab · Libo Wang
GGUF · Hugging Face · GGUF · ModelScope Global · GGUF · ModelScope China
Parent Model · Technical Report · Evaluation Evidence · Riemann ζ Artifact
--- ## Model variants | Use case | File | Main size | BF16-paired retention | |---|---|---:|---| | **Default / overall balance** | **`VeriLoop-E2-Q6_K.gguf`** | **20.566 GiB** | PPL parity within uncertainty; KLD **0.004409**; Same top-p **98.204%** | | **Memory-quality balance** | **`VeriLoop-E2-Q5_K_M.gguf`** | **18.965 GiB** | PPL **+0.4450%**; KLD **0.006919**; Same top-p **97.251%** | | **Minimum footprint** | **`VeriLoop-E2-IQ1_M.gguf`** | **16.790 GiB** | PPL **+0.3191%**; KLD **0.014357**; Same top-p **95.870%** | | Lower-KLD low-footprint alternative | `VeriLoop-E2-IQ2_S.gguf` | **16.799 GiB** | PPL **+0.3457%**; KLD **0.014023**; Same top-p **95.516%** | | Low-footprint alternative | `VeriLoop-E2-Q3_K_M.gguf` | **16.826 GiB** | PPL **+0.4090%**; KLD **0.014349**; Same top-p **95.919%** | | Balanced compact | `VeriLoop-E2-Q4_K_M.gguf` | **18.301 GiB** | PPL **+0.4821%**; KLD **0.009700**; Same top-p **96.786%** | | High fidelity | `VeriLoop-E2-Q8_0.gguf` | **26.632 GiB** | PPL **+0.0643%**; KLD **0.002176**; Same top-p **98.815%** | | Reference | `VeriLoop-E2-BF16.gguf` | **50.113 GiB** | Canonical BF16 reference | ### Recommended deployment points **Q6_K — overall quality / efficiency sweet spot.** It is **58.96% smaller than BF16** while remaining statistically consistent with BF16 PPL parity under the frozen paired protocol. **Q5_K_M — memory-quality sweet spot.** It reduces the main-file footprint to **18.965 GiB** while preserving wider KLD and Same-top margins than the sub-17 GiB variants. **IQ1_M — minimum-footprint sweet spot.** It is **16.790078 GiB**, **66.4955% smaller than BF16**, and passes the frozen hard gate, engineering-reserve gate, stock llama.cpp runtime validation, and real MTP engagement validation. IQ2_S remains the lower-KLD low-footprint alternative. > **Naming note:** `VeriLoop-E2-IQ1_M.gguf` is a mixed-precision artifact, not a uniform 1-bit model. Its measured tensor policy is **353 F32 + 1 IQ1_M + 2 IQ2_S + 64 Q4_K + 429 Q5_K + 2 Q6_K = 851 tensors**, with **5.36 effective BPW**. `VeriLoop-E2-IQ2_S.gguf` is likewise mixed precision rather than uniform 2-bit quantization. ## Precision ladder | Tier | Role | Main size | Reduction vs BF16 | Effective density | |---|---|---:|---:|---:| | **BF16** | Canonical reference | **53.808 GB / 50.113 GiB** | — | 16-bit-class | | **Q8_0** | High fidelity | **28.596 GB / 26.632 GiB** | **46.86%** | **8.50 BPW** | | **Q6_K** | **Overall sweet spot** | **22.083 GB / 20.566 GiB** | **58.96%** | **6.57 BPW** | | **Q5_K_M** | **Memory-quality sweet spot** | **20.364 GB / 18.965 GiB** | **62.16%** | **6.05 BPW** | | **Q4_K_M** | Balanced compact | **19.651 GB / 18.301 GiB** | **63.48%** | **5.84 BPW** | | **Q3_K_M** | Low-footprint alternative | **18.067 GB / 16.826 GiB** | **66.42%** | **5.37 BPW** | | **IQ2_S** | Lower-KLD low-footprint alternative | **18.037 GB / 16.799 GiB** | **66.48%** | **5.36 BPW** | | **IQ1_M** | **Minimum-footprint sweet spot** | **18.028 GB / 16.790 GiB** | **66.50%** | **5.36 BPW** | ## Quantization-retention benchmark All measured tiers use the same frozen BF16 logits and the same paired protocol. | Tier | Mean PPL | PPL ratio vs BF16 | Relative PPL change | Mean KLD | Same top-p | log-PPL correlation | |---|---:|---:|---:|---:|---:|---:| | **BF16** | **4.840423 ± 0.119931** | 1.000000 | — | 0 reference | 100% reference | 100% reference | | **Q8_0** | **4.843536 ± 0.120062** | **1.000643 ± 0.000754** | **+0.0643%** | **0.002176 ± 0.000668** | **98.815 ± 0.120%** | **99.95%** | | **Q6_K** | **4.838514 ± 0.119694** | **0.999605 ± 0.001222** | **−0.0395% observed; parity within uncertainty** | **0.004409 ± 0.000953** | **98.204 ± 0.147%** | **99.88%** | | **Q5_K_M** | **4.861965 ± 0.120630** | **1.004450 ± 0.001361** | **+0.4450%** | **0.006919 ± 0.000945** | **97.251 ± 0.181%** | **99.85%** | | **Q4_K_M** | **4.863760 ± 0.120720** | **1.004821 ± 0.001722** | **+0.4821%** | **0.009700 ± 0.001110** | **96.786 ± 0.195%** | **99.76%** | | **Q3_K_M** | **4.860222 ± 0.120606** | **1.004090 ± 0.002181** | **+0.4090%** | **0.014349 ± 0.001985** | **95.919 ± 0.219%** | **99.62%** | | **IQ2_S** | **4.857159 ± 0.120482** | **1.003457 ± 0.002100** | **+0.3457%** | **0.014023 ± 0.001408** | **95.516 ± 0.229%** | **99.64%** | | **IQ1_M** | **4.855870 ± 0.120480** | **1.003191 ± 0.002175** | **+0.3191%** | **0.014357 ± 0.001317** | **95.870 ± 0.220%** | **99.62%** | These figures measure **quantization retention against the BF16 reference**. They are not downstream benchmark-score loss percentages. The nine parent-model benchmarks were not independently rerun for every quantization tier. ## IQ1_M validation record IQ1_M changes exactly one tensor relative to the IQ2_S precision policy: `blk.1.ffn_down.weight` moves from IQ2_S to IQ1_M. All other tensor assignments remain unchanged. | Precision | Tensor assignment | Count | |---|---|---:| | F32 | Non-quantized tensors retained by GGUF conversion | **353** | | Q6_K | `output.weight`, `token_embd.weight` | **2** | | Q5_K | Remaining quantized internal tensors | **429** | | Q4_K | all `*.ffn_up.weight` | **64** | | IQ2_S | `blk.0.ffn_down.weight`, `blk.3.ffn_down.weight` | **2** | | IQ1_M | `blk.1.ffn_down.weight` | **1** | | **Total** | | **851** | ### Fidelity and runtime validation | Validation | Requirement | Measured | Result | |---|---|---|---| | PPL ratio hard / reserve | ≤ **1.0150** / ≤ **1.0135** | **1.003191** | **PASS / PASS** | | Mean KLD hard / reserve | ≤ **0.0150** / ≤ **0.0145** | **0.014357** | **PASS / PASS** | | Same top-p hard / reserve | ≥ **95.0%** / ≥ **95.5%** | **95.870%** | **PASS / PASS** | | Exact tensor structure | 851 tensors; declared policy | **851 tensors; 0 mismatch** | **PASS** | | Stock llama.cpp main runtime | Successful real generation | HTTP **200**, non-empty generation | **PASS** | | MTP runtime | Successful real generation | HTTP **200**, non-empty generation | **PASS** | | Real MTP engagement | generated > 0; accepted > 0 | **76 accepted / 104 generated (73.0769%)** | **PASS** | | Main vs MTP deterministic audit | Advisory | Identical output SHA256 | **IDENTICAL** | **IQ1_M artifact identity** | Property | Value | |---|---| | Filename | `VeriLoop-E2-IQ1_M.gguf` | | Exact bytes | **18,028,208,896** | | Binary size | **16.790078 GiB** | | SHA256 | `e4d395806994cdbcfecf7711e22456af7a663b3ea37c89ef15d5bab4571cf68b` | | Effective density | **5.36 BPW** | | Tensor count | **851** | | Quantizer-reported size | **17,182.55 MiB** | | Quantizer time | **167.77645 s** | | llama.cpp validation revision | `42916d83f4a225e56709f873aa8050ac11f5b6a4` | ### Low-footprint comparison | Metric | Q3_K_M | IQ2_S | IQ1_M | |---|---:|---:|---:| | Main size | **16.825745 GiB** | **16.798508 GiB** | **16.790078 GiB** | | PPL ratio | **1.004090** | **1.003457** | **1.003191** | | Mean KLD | **0.014349** | **0.014023** | **0.014357** | | Same top-p | **95.919%** | **95.516%** | **95.870%** | | RMS Δp | **3.515%** | **3.679%** | **3.691%** | IQ1_M is **8.633 MiB** smaller than IQ2_S and **36.523 MiB** smaller than Q3_K_M. IQ2_S retains the lowest Mean KLD of the three; IQ1_M has the smallest footprint, the lowest PPL ratio, and higher Same top-p than IQ2_S. The point-estimate differences remain small relative to the reported uncertainty scale. ## Frozen BF16-paired protocol | Item | Value | |---|---| | Reference model | Canonical VeriLoop E2 BF16 GGUF | | Corpus | WikiText-2 raw test | | Corpus SHA256 | `173c87a53759e0201f33e0ccf978e510c2042d7f2cb78229d9a50d79b9e7dd08` | | Context | **2,048** | | Chunks | **8** | | Seed | **42** | | GPU layers | **40** | | Flash Attention | **Off** | | KV cache | **F16/F16** | | Batch / micro-batch | **512 / 512** | | Evaluator | `llama-perplexity` | | Reference logits | `--kl-divergence-base` | | Quantized comparison | `--kl-divergence` | | BF16 logits reused across tiers | **Yes** | | llama.cpp revision | `42916d83f4a225e56709f873aa8050ac11f5b6a4` | ## Parent-model benchmark record The scores below describe the **parent VeriLoop E2 release** and are not relabeled as quantization-specific reruns. | Benchmark | VeriLoop E2 parent score | Public evidence | |---|---:|---| | SWE-bench Pro | **76.2** | [Evidence](https://huggingface.co/datasets/tsinghua-sigs-robot-lab/VeriLoop-E2-Evaluation-Evidence/tree/main/swe-bench-pro) | | Terminal-Bench 2.1 | **88.8** | [Evidence](https://huggingface.co/datasets/tsinghua-sigs-robot-lab/VeriLoop-E2-Evaluation-Evidence/tree/main/terminal-bench-2.1) | | DeepSWE v1.1 | **64.6** | [Evidence](https://huggingface.co/datasets/tsinghua-sigs-robot-lab/VeriLoop-E2-Evaluation-Evidence/tree/main/deepswe-1.1) | | Terminal-Bench 3.0 | **29.7** | [Evidence](https://huggingface.co/datasets/tsinghua-sigs-robot-lab/VeriLoop-E2-Evaluation-Evidence/tree/main/terminal-bench-3.0) | | Terminal-Bench 4.0 | **37.9** | [Evidence](https://huggingface.co/datasets/tsinghua-sigs-robot-lab/VeriLoop-E2-Evaluation-Evidence/tree/main/terminal-bench-4.0) | | SWE-Marathon v1.1 | **45.0** | [Parent release](https://huggingface.co/tsinghua-sigs-robot-lab/VeriLoop-E2) | | AIME 2026 | **98.3** | [Evidence](https://huggingface.co/datasets/tsinghua-sigs-robot-lab/VeriLoop-E2-Evaluation-Evidence/tree/main/aime-2026) | | GPQA Diamond | **93.9** | [Evidence](https://huggingface.co/datasets/tsinghua-sigs-robot-lab/VeriLoop-E2-Evaluation-Evidence/tree/main/gpqa-diamond) | | Apex 2025 | **89.6** | [Evidence](https://huggingface.co/datasets/tsinghua-sigs-robot-lab/VeriLoop-E2-Evaluation-Evidence/tree/main/apex-2025) | ## Quick start with llama.cpp ```bash # Overall sweet spot hf download tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF VeriLoop-E2-Q6_K.gguf --local-dir . # Minimum-footprint sweet spot hf download tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF VeriLoop-E2-IQ1_M.gguf --local-dir . # Run llama-server \ -m ./VeriLoop-E2-IQ1_M.gguf \ -ngl 99 \ -c 32768 \ --host 127.0.0.1 \ --port 8080 ``` ## Memory guidance | Variant | Main file | Positioning | |---|---:|---| | BF16 | **50.113 GiB** | Reference | | Q8_0 | **26.632 GiB** | High fidelity | | **Q6_K** | **20.566 GiB** | **Overall sweet spot** | | **Q5_K_M** | **18.965 GiB** | **Memory-quality sweet spot** | | Q4_K_M | **18.301 GiB** | Balanced compact | | Q3_K_M | **16.826 GiB** | Low-footprint alternative | | IQ2_S | **16.799 GiB** | Lower-KLD low-footprint alternative | | **IQ1_M** | **16.790 GiB** | **Minimum-footprint sweet spot** | A **16.79 GiB model file does not imply full offload on a 16 GiB GPU**. Runtime memory also includes KV cache, compute buffers, allocator overhead, and optional speculative-decoding weights. A standalone Q8_0 `llama-bench` record exists (**pp512 3102.722776 tok/s; tg128 41.999677 tok/s**), but there is no frozen paired BF16 throughput campaign. No universal speedup percentage is claimed for the quantization ladder. ## Measurement boundaries - PPL, KLD, Same top-p, and token-probability statistics are **quantization-retention measurements**, not universal downstream capability-loss percentages. - IQ1_M and IQ2_S are mixed-precision artifacts; the tier name does not equal the model-wide effective bit width. - Native 262K context comes from the parent configuration; practical context depends on runtime memory. - MTP acceptance is prompt- and workload-dependent. - The model can still produce incorrect code, mathematics, scientific reasoning, or commands. ## Source model and evidence | Resource | Link | |---|---| | Parent model | [VeriLoop E2](https://huggingface.co/tsinghua-sigs-robot-lab/VeriLoop-E2) | | GGUF · Hugging Face | [VeriLoop E2 GGUF](https://huggingface.co/tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF) | | GGUF · ModelScope Global | [VeriLoop E2 GGUF](https://modelscope.ai/models/tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF) | | GGUF · ModelScope China | [VeriLoop E2 GGUF](https://modelscope.cn/models/tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF) | | Quantization quality record | [Quantization Quality](./QUANTIZATION_QUALITY.md) | | Artifact manifest | [Release Manifest](./RELEASE_MANIFEST.json) | | Technical report | [OpenReview](https://openreview.net/forum?id=P6FIQILHwX¬eId=P6FIQILHwX) | | Evaluation evidence | [VeriLoop E2 Evaluation Evidence](https://huggingface.co/datasets/tsinghua-sigs-robot-lab/VeriLoop-E2-Evaluation-Evidence) | | Riemann ζ artifact | [Public artifact](https://github.com/brucewang123456789/GeniusTrail/tree/VeriLoop-E2/riemann-hypothesis) | | llama.cpp | [ggml-org/llama.cpp](https://github.com/ggml-org/llama.cpp) | ## License The VeriLoop E2 model weights and this GGUF distribution are released under the **Apache License 2.0**, consistent with the parent model release. ## Citation ```bibtex @misc{wang2026veriloope2, title = {VeriLoop E2: A 27B Post-Trained Model for Code, Mathematics, and Scientific Reasoning}, author = {Wang, Libo}, year = {2026}, note = {Tsinghua Shenzhen International Graduate School (SIGS)}, howpublished = {Open model release}, url = {https://huggingface.co/tsinghua-sigs-robot-lab/VeriLoop-E2} } ``` For quantization-specific comparisons, identify the exact GGUF filename and corresponding manifest identity.