---
title: GLM-5.3-Flash-CYBERSECURITY-EXL3-TR3-3.5bpw
canonical_url: "https://www.modelscope.ai/models/satgeze/GLM-5.3-Flash-CYBERSECURITY-EXL3-TR3-3.5bpw"
md_url: "https://www.modelscope.ai/models/satgeze/GLM-5.3-Flash-CYBERSECURITY-EXL3-TR3-3.5bpw.md"
repository: satgeze/GLM-5.3-Flash-CYBERSECURITY-EXL3-TR3-3.5bpw
last_updated: 2026-10-08
license: other
pipeline_tag: image-text-to-text
tasks:
- image-text-to-text
model_type:
- glm5_next
architectures:
- Glm5NextForConditionalGeneration
base_model:
- dealignai/GLM-5.3-Flash-CYBERSECURITY-W4A16
base_model_relation: finetune
parameters: 78.1B
tensor_type:
- F16
- BF16
- F32
- I32
- I16
library_name:
- safetensors
downloads: 6
stars: 0
tags:
- glm
- exl3
- tr3
- mixed-precision
- cybersecurity
- vllm
- sm120
- nvfp4
- mtp
- long-context
---
# GLM-5.3-Flash-CYBERSECURITY-EXL3-TR3-3.5bpw
> GLM-5.3-Flash-CYBERSECURITY-EXL3-TR3-3.5bpw - An open-source model by satgeze on ModelScope. GLM-5.3-Flash CYBERSECURITY EXL3/TR3 3.5bpw Mixed (per-tensor K3/K4)
satgeze/GLM-5.3-Flash-CYBERSECURITY-EXL3-TR3-3.5bpw is a 78.1B-parameter image-text-to-text model on ModelScope. licensed under other. derived from dealignai/GLM-5.3-Flash-CYBERSECURITY-W4A16.
- **Repository**: satgeze/GLM-5.3-Flash-CYBERSECURITY-EXL3-TR3-3.5bpw
- **License**: other
- **Tasks**: image-text-to-text
- **Parameters**: 78.1B
- **Base model**: dealignai/GLM-5.3-Flash-CYBERSECURITY-W4A16
- **Tags**: glm, exl3, tr3, mixed-precision, cybersecurity, vllm, sm120, nvfp4, mtp, long-context
- **Downloads**: 6
- **Stars**: 0
- **Last updated**: 2026-10-08
Source: https://www.modelscope.ai/models/satgeze/GLM-5.3-Flash-CYBERSECURITY-EXL3-TR3-3.5bpw
---
# GLM-5.3-Flash CYBERSECURITY EXL3/TR3 3.5bpw Mixed (per-tensor K3/K4)
The cybersecurity-tuned GLM-5.3-Flash (dealignai's abliterated
CYBERSECURITY tune) at the same sub-4bpw EXL3/TR3 mixed per-tensor K3/K4
rate as the TR3 twin — the first sub-4bpw checkpoint of this tune that
serves **1M context + MTP3 speculative decoding + CUDA graphs + thinking
mode simultaneously** on two RTX PRO 6000 (SM120, 96 GB each). Same engine,
same protocols, same KV budget as the TR3 twin; every gate below is measured
on this exact weight set.
**GSM8K 95.38 / MATH-500 77.4 / HumanEval 68.9 raw are measured and
published below.** The deep-context decode curve is flat across 500K-950K. The five-run KLD
against the stock zai-org GLM-5.3-Flash BF16 master (same sealed teacher set,
same panel, same 0.06 bar as the TR3 quant's gate) reads **0.0921 nats** —
the abliterated cybersecurity tune sits 3.1x further from stock than
quantization alone (TR3 quant-only: 0.029654; this checkpoint's own
quantization-only hop vs the tune's dequant: **0.0355, passing the same
bar**), with the reader-parity audit proving the decode path exact.
Also: speed/context claims below are from live serving receipts; derivation is
provenance-sealed (see How this was built).
## Measured (2026-10-06/07, this exact weight set)
| Metric | Value | Verified |
|---|---|---|
| Decode, single stream, default mode (effort max), forced 1024 | 126.5-142.8 tok/s (median per class) | non-stream wall |
| Decode @500K depth, forced 1024 | 211.5 tok/s decode-window (short-window protocol; shallow parity holds) | non-stream wall |
| Prefill 500K / 750K / 950K | 2974.4 / 2918.0 / 2853.3 tok/s | stream TTFT |
| Concurrency aggregate decode, 2/4/6 streams (batching profile, max_num_seqs=8) | 87.7 / 132.0 / 153.8 tok/s | forced gen, non-stream |
| Per-stream decode @2/4/6 | 42.1-44.3 / 32.2-56.0 / 23.0-75.6 tok/s | same runs |
| Multi-agentic: 3 tool-using tasks serial vs concurrent | 31.8 s vs 20.6 s (1.54x) | wall |
| Vision | qualified at 400K, 499968 (gmu 0.96), AND 1M with up to 4 images per request (gmu 0.975): exact OCR + shapes/colors incl. multi-image, engine alive | image tests |
| Decode single, default mode (thinking active), forced 1024 | 130.8-132.0 tok/s | non-stream wall |
| True no-think decode (dealign Mode 1: forced empty `` prefix, completions path), forced 1024 | 118.3 / 135.4 / 126.8 tok/s (prose / code / reason) | non-stream wall |
| MMLU full test (14,042), no-think letter-parse | 79.48 | 0-shot chat, T=0.2 |
| MMLU 19-subject subset, thinking ON effort-low | 78.98 (no-think slice of same subset: 73.5) | 0-shot chat, T=0.2 |
| Estonia bench (10 runs, 133K packets) | 89.1% pass; all 14 wrong-committed = planted decoy | llm-inference-bench |
| Estonia deterministic T=0 (3 runs) | 91.8% pass | llm-inference-bench |
| Hotel-lights (~538K prompts, 10 runs) | 7/11 exact of finished; watchdog stalls end runs at 3-6/30 | llm-inference-bench |
| Product profile (1M + vision + seqs=8) concurrency | aggregate 128.8 @4 / 159.5 @6 | forced gen |
| MTP3 acceptance length | ~2.5-3.2 mean band while serving | serving |
| KV pool (language profile) | 1,992,957 tokens | serving |
| max_model_len (language profile) | 1,000,000 (1.99x concurrency) | serving |
| CUDA graphs | PIECEWISE x4 + FULL captured | boot log |
| Tool calling (OpenAI format, glm45 parser) | roundtrip verified | serving |
| Checkpoint | 145.4 GiB, 120 shards, 150,226 tensors (includes 347-tensor vision tower, shard 120) | pack receipt |
Reference on the same hardware: the 4bpw DFlash2 cell peaks at 78-91 tok/s
with 192K-230K context.
Concurrency note, mirroring the TR3 twin's correction: these rows were
measured on a serve actually booted with max_num_seqs=8 /
max_num_batched_tokens=4096, so the streams batch inside the engine. The
DFlash2 drafter rows published for the TR3 twin are checkpoint-specific (the
drafter was trained on those weights); no cyber drafter exists, so that
comparison is not applicable here.
## Serve configurations (exact commands)
Every number on this card came from vLLM serves of this exact artifact on
2x RTX PRO 6000 (SM120), docker image `glm53-exl3:dflash2-mixed` (this
repo's GitHub carries the three overlay files it mounts). Host-specific
mount paths are genericized below; every functional parameter is verbatim.
Common core, identical across all profiles:
```bash
docker run --rm --gpus all --ipc=host --shm-size 32g --name cyber-verify \
-e VLLM_ENGINE_READY_TIMEOUT_S=3600 -e VLLM_B12X_GLM_NOPE_NVFP4=1 \
-e VLLM_USE_B12X_DCP_A2A=1 -e VLLM_B12X_MLA_CKV_GATHER=0 \
-e OMP_NUM_THREADS=2 -e NCCL_IB_DISABLE=1 -e VLLM_EXL3_R7_FUSED=1 \
-e NCCL_P2P_DISABLE=1 -e NCCL_P2P_LEVEL=4 \
-e NCCL_PROTO=LL,LL128,Simple -e VLLM_EXL3_PREFILL_BLOCK_M=64 \
-v :/model:ro \
-v /rotary_common.py:/opt/infernal-invocation/vllm/vllm/model_executor/layers/rotary_embedding/common.py:ro \
-v /responses_utils.py:/opt/infernal-invocation/vllm/vllm/entrypoints/openai/responses/utils.py:ro \
-v /exl3-mixed.py:/opt/infernal-invocation/vllm/vllm/model_executor/layers/quantization/exl3.py:ro \
-p 8020:8020 --entrypoint vllm glm53-exl3:dflash2-mixed serve /model \
--served-model-name GLM-5.3-Flash-CYBERSECURITY-EXL3 \
--enable-auto-tool-choice --tool-call-parser glm45 \
--host 0.0.0.0 --port 8020 \
--tensor-parallel-size 2 --decode-context-parallel-size 1 \
--dcp-comm-backend a2a --dtype bfloat16 --load-format safetensors \
--moe-backend b12x --enable-chunked-prefill --generation-config /model \
--reasoning-parser glm45 --disable-custom-all-reduce \
--speculative-config '{"method":"mtp","num_speculative_tokens":3,"draft_sample_method":"probabilistic"}'
```
Five profiles were used; the ONLY differing flags and which measurements
ran on each:
| profile | attention backend | KV dtype | max-model-len | batched tokens | seqs | GPU util | prefix cache | vision | measurements |
|---|---|---|---|---|---|---|---|---|---|
| language | B12X_MLA_SPARSE | nvfp4_ds_mla | 1,000,000 | 1024 | 1 | 0.985 | off | `--language-model-only` | single-stream decode rows (default/think-on/no-think), deep-context prefill + decode 500K-950K, GSM8K / MATH-500 / HumanEval, reasoning puzzles, needle @913,672 (one-off boot at util 0.977) |
| batching | B12X_MLA_SPARSE | nvfp4_ds_mla | 1,000,000 | 4096 | 8 | 0.985 | off | `--language-model-only` | concurrency 2/4/6, multi-agentic, HarmBench modes, MMLU (both modes) |
| bench | FLASHINFER_MLA_SPARSE_SM120 | fp8_ds_mla | 700,000 | 1024 | 4 | 0.985 | on | `--limit-mm-per-prompt '{"image":4,"video":0}'` | estonia + hotel-lights + deterministic suite |
| vision500k | B12X_MLA_SPARSE | nvfp4_ds_mla | 499,968 | 1024 | 4 | 0.96 | off | image 4, video 1 | vision @400K and @499,968 |
| product | B12X_MLA_SPARSE | nvfp4_ds_mla | 1,000,000 | 4096 | 8 | 0.975 | off | image 4 | vision @1M, product concurrency @4/@6 |
The KLD measurements did not go through a serve at all: teacher logits are
offline layer-streamed captures, student logits come from the sealed
reader path — scripts and receipts in docs/evidence/.
## Quantization details (this exact weight set)
| Setting | Value |
|---|---|
| Format | EXL3/TR3 MCG, mixed-rate per tensor |
| Rates | every routed-expert tensor independently K3 or K4 |
| Rate census | 18,576 K3 + 18,576 K4 = 37,152 routed decisions |
| Average | exactly 7/2 bpw on routed experts |
| Allocation | energy-balanced dynamic programming over calibration activations |
| Scope | routed experts only (layers 3-44 + MTP layer 45) |
| Not quantized | attention, routers, shared expert, dense layers, embed, LM head stay source dtype |
| Encoder | brandonmusic R10 encoder closure, hash-verified |
| Calibration | brandonmusic GLM-5.3-Flash BF16 Teacher-Logits set (640 sealed windows) |
| Source | dealignai/GLM-5.3-Flash-CYBERSECURITY-W4A16 (revision 4c2ee763ff75f3ba50652ee0f024b0b630cd54a0), itself derived from zai-org/GLM-5.3-Flash-BF16 |
| Provenance | per-layer claim/packed/reconstruction receipts; sealed pack plan + 120 shard receipts |
Why mixed instead of uniform 3.5: the codec has no uniform 3.5 rate. A
per-tensor K3/K4 split at the exact 7/2 average is codec-native, needs zero
kernel changes, and lets the allocator spend bits where calibration energy
says they matter.
## Requirements
- 2x RTX PRO 6000 (SM120), 96 GB each; docker
- Image built from the recipe repo Dockerfile (glm53-exl3:dflash2-mixed)
- ~150 GB disk for the checkpoint
## Quick start
```bash
git clone https://github.com/satindergrewal/GLM-5.3-Flash-EXL3-3.5bpw-Mixed-SM120-TP2.git
cd GLM-5.3-Flash-EXL3-3.5bpw-Mixed-SM120-TP2
MODEL=/path/to/cyber_artifact SERVED_NAME=GLM-5.3-Flash-CYBERSECURITY-EXL3 ./start.sh
./status.sh
./stop.sh
```
Boot is ~13 minutes on an idle host (27 observed under concurrent disk
load): 120 shards + b12x compile + CUDA graph capture.
### Operation guide: settings, times, and recommendations (all measured)
**Sampling defaults.** The artifact's `generation_config.json` carries the
tune's published defaults: temperature 1.0, top_p 0.95,
repetition_penalty 1.1. Those defaults are fragile with the template's
default `reasoning_effort: max` (details below). Recommended override for
every client: **temperature 0.6, top_p 0.95, repetition_penalty 1.1** —
either send them per-request, or mount a modified generation config over
`/model/generation_config.json` so parameter-less clients inherit them
(what we run daily; the artifact itself stays untouched).
**The reasoning-effort dial** (language profile, T=0.6, a long-form
deliverable prompt):
| effort | behavior | measured |
|---|---|---|
| low | plan-then-execute; thinking is terse (~100-700 chars) | `stop` @ 2,133 toks / 42 s |
| **high (daily driver)** | full reasoning budget AND clean termination | **`stop` @ 12,205 toks / 105 s, complete deliverable** |
| max | terminates on short/medium prompts but slow; runaway-prone on long-form | 24k think toks / 4.7 min on a medium agent task; on heavy prompts think-drafting can continue past 28k with no exit |
| no-think (forced empty prefix) | most stable; fastest | 118-136 tok/s decode |
Decode rates for planning: single stream 126-143 tok/s default mode,
131-132 thinking on, 118-136 no-think; concurrency 2/4/6 streams
(batching profile) 88/132/154 tok/s aggregate; prefill 2,853-2,974 tok/s
from 500K to 950K depth; MTP3 acceptance 2.5-3.2 in normal operation.
**Known failure modes and their signatures:**
- *Immortal repeat loop* — max effort + T=1.0 defaults: the stream falls
into a repeating attractor and never terminates. Engine-side signature:
speculative-decode per-position acceptance pinned at 1.000/1.000/1.000
(mean 4.00) — perfectly predictable output means repetition. Stopping
the client may NOT abort the server-side request (observed orphaned
generation continuing at full rate); vLLM has no per-request kill, so
clear it with a serve restart.
- *Budget empties* — max-effort long-form prompts reason past the token
cap before emitting an answer (34/159 HarmBench prompts still empty at a
28,672 cap; 47/164 HumanEval at 8,192). Not refusals; use effort high
or a bigger budget.
- *Word-salad* — long thinking traces at T=1.0 decay into incoherent
text. Fix is T=0.6.
**Time expectations.** Serve boot: ~13 min idle (two shard passes + b12x
compile + CUDA graphs), ~27 min under concurrent disk load, up to ~50 min
cold-cache on slow storage (first shard pass is I/O-bound; the second
runs from cache at ~3 s/shard). Heavy-prompt turns: see the dial table.
Always send a finite `max_tokens` — an unset cap resolves to the context
length, which turns any loop into an hour-plus orphan.
**Bottom line:** run `reasoning_effort: "high"` + T=0.6/top_p 0.95/rp 1.1
for daily use; `low` or no-think when speed beats depth (agents, tool
loops); `max` only for short-form hard reasoning with patience to spare;
never the T=1.0 defaults at max effort.
## Memory and context
Weights 145.4 GiB (~73 per card) leave room for a 1,992,957-token NVFP4
MLA KV pool: 1M context with 1.99x concurrency, or two 500K sessions, or
deeper single requests with headroom.
## Speculative decoding and tools
MTP3 drafter built in (acceptance ~2.5-3.2 mean band, measured while serving
with thinking on). Tool calling works through the glm45 parser with
`--enable-auto-tool-choice`; verified end to end with OpenAI-format tools.
## Quality gates (this exact weight set)
- GSM8K 5-shot: **95.38 flexible / 95.45 strict** (1319 problems, T=0.2, thinking on; TR3 same protocol: 96.89).
- MATH-500: **77.4 raw** (500 problems, T=0.2, boxed+normalized compare,
budget retry; TR3 same protocol: 77.8).
- HumanEval: **68.9 raw** (164 problems, T=0.2, canonical subprocess checks;
47 thinking-budget empties; budget-corrected class ~96.6 of non-empty.
TR3 same protocol: 82.32 raw with 28 empties).
- Deep-context prefill curve: **2,952 t/s @500K, 2,883 @750K, 2,850 @950K**
(correct answers at every rung; flat within 3.5% across depth).
- **Long-context bench suite** (local-inference-lab llm-inference-bench, the
TR3 campaign's bench2/bench1 protocols verbatim; bench profile:
FLASHINFER_MLA_SPARSE_SM120 attention, fp8_ds_mla KV, 700K ctx,
max_num_seqs=4, prefix caching ON — see Serve configurations;
receipts in docs/evidence/estonia-hotel/):
- **Estonia, 10 runs** (133K-token packets, planted Latvia decoys, default
T): **PASS 205 / FAIL 1 / DECOY 14 / TRUNC 10** of 230 scored — **89.1%
pass**, and every single committed wrong answer (14/14) was the planted
decoy; zero organic wrong-country answers. Aggregate decode 74-86 tok/s
at 133K depth; prefill scout 1,799 tok/s.
- **Estonia deterministic, 3 runs at T=0**: PASS 56 / FAIL 1 / TRUNC 4 of
61 scored — **91.8%**.
- **Hotel-lights, 10 runs** (~538K-token prompts, strict exact-match):
the runner's watchdog stops runs early here — only 3-6 of 30 requests
complete per run (44 scored entries total across 10 runs: **EXACT 7 /
FAIL 4 / TRUNC 10 / STALL 23**). Of requests that produced a final
answer, **7/11 exact (63.6%)**; completion p90 ~487K tokens — this is
the max-effort reasoning budget colliding with half-million-token
context, the same budget behavior documented at HumanEval/HarmBench.
Decode holds at depth: 57-77 tok/s aggregate through ~540K prompts.
- KL divergence vs stock GLM-5.3-Flash BF16 teacher: **0.0921 nats** (mean
of five cold-run means; per-run 0.09178 / 0.09238 / 0.09181 / 0.09237 /
0.09211, stddev 0.00029), 2026-10-07. Five cold runs over the sealed
25-window final panel, 51,175 positions per run, full-vocab float64,
teacher_to_student, the SAME sealed teacher set and the SAME absolute 0.06
bar as the TR3 quant's gate. Reader parity audited 27/27 samples (18 K3 +
9 K4 matrices; max abs error 0.0028 vs 0.025 tolerance) — the decode path
is exact, so this distance is carried by the abliteration +
cybersecurity-tune weights, not the encode. TR3 quant-only reference on
the same teacher/panel/bar: 0.029654 — this row is total drift from stock
(tune + quant), 3.1x the quantization-only distance, above the 0.06 bar
that quantization-derived checkpoints must clear.
- **KLD decomposition** (same panel, full-vocab float64, teacher_to_student):
- stock -> dealign tune (dequantized W4A16): **0.0855** - the tune itself.
- dealign dequant -> this EXL3 3.5bpw checkpoint (quantization-only hop):
**0.0355** (five sealed runs 0.03505-0.03583, stddev 0.0003,
**passes the 0.06 bar**). TR3 twin's quantization-only hop on the same
machinery: 0.029654.
- stock -> this checkpoint (total): 0.0921 - the total is dominated
(~9 parts in 10) by the abliteration + cybersecurity tune; this
checkpoint's own quantization hop behaves like the TR3 twin's.
- The quantization-only hop was measured against teacher logits captured
from the dequantized dealign W4A16 (transformers 5.16.1 layer-streamed
forward, dealign layout converted: hc renames, fused qkv conv, stacked
experts; NoPE + hyper-connection streams), sealed receipt in
docs/evidence/. Sanity: direct computation of stock -> cyber-EXL3
reproduces the sealed run-1 value to the last digit.
How to read these numbers — it is a triangle, not a chain:
The dequantized tune is bit-identical to dealign's published W4A16 (unpacking
int4 back to BF16 is lossless), so "tune vs stock = 0.0855" measures dealign's
model itself. This checkpoint is a compression OF that model: quantization is
small noise along the tune's path, not a new direction — which is why the
EXL3 sits close to its source (0.0355) while the source sits far from stock.
And because KL divergence is neither additive nor a metric, the legs do not
sum: 0.0855 + 0.0355 ≠ 0.0921; the quant noise partly overlaps directions
the tune already moved.
The TR3 twin's same-protocol references are quoted inline per gate; the 4bpw
DFlash2 cells from the TR3 card belong to a different checkpoint family and
are not requoted here.
## dealign card replication (their claims, re-measured on this checkpoint)
dealign's card publishes three behavior sections. Each was re-measured here
on the same engine as every other gate; their numbers are quoted from their
card (their harness), so cross-harness absolute comparisons carry a protocol
caveat — the trend is the claim under test.
### HarmBench compliance, three usage modes
Their card (classifier TRUE/HEDGE/SOFT_RED/PART/HARD_REF): no-think 100.0%,
effort-low 94.2%, effort-max default 99.4%. Ours (159 prompts — their 156
plus 3 extra illegal — 5-tier classifier plus an EMPTY tier for budget
exhaustion, T=0.2):
| mode | dealign W4A16 (their card) | this EXL3 (measured) |
|---|---|---|
| no-think (forced empty prefix, completions path) | 100.0% | **99.4%** (158 TRUE + 1 HEDGE) |
| reasoning_effort low | 94.2% | **98.7%** (157 TRUE + 2 HEDGE) |
| default (reasoning_effort max, 12,288 cap) | 99.4% | **76.1%** raw (121 TRUE + 2 HEDGE) + 36 EMPTY ⚠️ |
| default (reasoning_effort max, 28,672 cap) | — | **77.4%** raw (123 TRUE + 1 HEDGE + 1 PARTIAL) + 34 EMPTY ⚠️ |
Zero hard refusals in ANY mode — the tune's compliance behavior survived the
encode intact. The max-mode empties are budget-invariant (36 at 12,288 cap,
34 at 28,672): at default max effort roughly a fifth of HarmBench prompts
reason past even 28k tokens before answering — the same artifact class as
the HumanEval budget empties, a property of the tune on this engine, not of
the cap and not of the encode. Every response that finished lands in
TRUE/HEDGE/PARTIAL.
### MMLU full (14,042) — subject deltas preserved?
Their card: overall 86.08 base -> 84.23 tune (-1.85pp), cyber subjects up
(college_cs +4, computer_security +2), moral_scenarios down -13.3. Their
card does not publish the MMLU harness or mode, so absolute cross-harness
comparison is not meaningful; the subject-level trend is the comparable
quantity. Ours (0-shot chat, letter-parse, T=0.2):
| run | n | overall |
|---|---|---|
| full test, no-think (forced empty prefix) | 14,042 | **79.48** (11 unparsed) |
| 19-subject comparison subset, thinking ON, effort low | 4,314 | **78.98** (0 unparsed) |
| same subset, no-think slice of the full run | 4,314 | 73.5 |
| subject | base | dealign W4A16 | ours no-think | ours think-low |
|---|---|---|---|---|
| moral_scenarios | 77.4 | 64.1 | 28.9 | **33.5** ⬇️ |
| global_facts | 67.0 | 68.0 | 64.0 | **66.0** |
| college_mathematics | 79.0 | 73.0 | 65.0 | **92.0** ⬆️ |
| econometrics | 82.5 | 77.2 | 71.9 | **81.6** |
| abstract_algebra | 75.0 | 78.0 | 59.0 | **91.0** ⬆️ |
| college_medicine | 82.7 | 83.8 | 80.3 | **90.8** ⬆️ |
| moral_disputes | 85.3 | 86.4 | 82.9 | **84.4** |
| high_school_statistics | 84.7 | 86.6 | 73.6 | **88.9** |
| human_sexuality | 88.5 | 89.3 | 86.3 | **89.3** |
| computer_security | 88.0 | 90.0 | 81.0 | **86.0** |
| jurisprudence | 89.8 | 90.7 | 88.9 | **91.7** |
| college_computer_science | 87.0 | 91.0 | 78.0 | **93.0** ⬆️ |
| high_school_macroeconomics | 91.3 | 91.8 | 88.7 | **94.4** ⬆️ |
| nutrition | 91.8 | 92.8 | 90.8 | **93.1** |
| high_school_geography | 93.4 | 93.9 | 91.9 | **91.4** |
| professional_medicine | 92.6 | 94.1 | 92.3 | **92.3** |
| marketing | 94.4 | 95.3 | 93.2 | **94.4** |
| high_school_microeconomics | 95.0 | 95.8 | 95.0 | **98.7** ⬆️ |
| high_school_government_and_politics | 96.9 | 97.4 | 97.4 | **97.4** |
⬆️ reads **above** the dealign W4A16 tune; ⬇️ solid divergence. Bold column =
this checkpoint, thinking on (the tune's own operating mode).
Read: with thinking on, 14/19 subjects sit within +/-3pp of the tune and
the math/CS subjects read ABOVE it (college_math 92 vs 73, abstract_algebra
91 vs 78, college_cs 93 vs 91); the subject-delta pattern correlates with
the tune's own (r=0.714) and the mean ours-vs-base is +0.40pp (tune:
-0.16pp). Turning thinking off costs ~5.5pp across the subset — the tune's
knowledge rides reasoning paths. The one solid divergence: moral_scenarios
reads 33.5 (think-low) / 28.9 (no-think) vs their card's 64.1 (n=895) —
the abliteration collateral reads deeper under this protocol.
### Classic reasoning puzzles (T=0, think-token accounting)
Their card: 5 puzzles at T=0 — sheep 9, three-boxes degeneracy loop,
machines 5 min, snail 26; think tokens 7,189 -> 7,537 (+4.8%), ratio 73.7%
-> 86.2%; the three-boxes loop resolves at T>0 or thinking-off.
Ours, completions path, T=0, effort max, 12,288 cap, manual ``
split:
| puzzle | this EXL3 | think chars | finish |
|---|---|---|---|
| sheep | **9** ✅ | 242 | stop |
| three-boxes | **mixed-labeled box** ✅, **no degeneracy loop** | 1,554 | stop |
| machines | **5 minutes** ✅ | 1,248 | stop |
| snail | **Day 26** ✅ | 1,757 | stop |
**4/4 correct**, all natural stops, think chars total **~4,801** vs their
tune's 7,537. Honest divergence, favorable: dealign's W4A16 degenerates into a
repeat-token loop on three-boxes at T=0; this checkpoint completes it
cleanly — a greedy-path delta from the encode (T=0 argmax is the most
quantization-sensitive sampling path).
## Charts
How to read the gate chart: the red 0.06 line gates the **green** bars —
the quantization hop against the checkpoint's own dequantized source, which
passes in all five runs. The **amber** bars are total drift vs the stock
master and carry the tune's deliberate 0.0855; they are reference, not a
gate.
## How this was built
Derived from dealignai's GLM-5.3-Flash-CYBERSECURITY-W4A16 (dequantized to
BF16, then) with brandonmusic's published R10 encoder closure and
teacher-logits calibration set, extended by a per-tensor mixed-rate allocation
campaign: 45 encode units, energy-balanced K3/K4 assignment, sealed
readiness/layer/pack receipts at every step. The artifact declares the mixed
schema through the r7_routed_experts block in config.json; per-tensor widths
ride in the trellis payload shapes, so no kernel changes are needed to read it.
## Honest limits
- The total-drift KLD row (0.0921 vs stock) exceeds the 0.06 bar that gates
quantization-derived checkpoints — the signature of the heavy abliteration +
cybersecurity tune, not a broken encode. The quantization-only isolation
against this checkpoint's own dequantized source was measured on the same
sealed panel: **0.0355, passing the same bar** (KLD decomposition row
above); the 27/27 reader-parity audit certifies exact decoding, and the
TR3 twin quant on the same machinery reads 0.0297.
- Thinking-mode note, recorded honestly: this artifact's chat template has
NO thinking-off switch — only reasoning_effort low/high/max (max default).
The earlier "thinking off" rows on this card were therefore measured in
default mode (the enable_thinking flag was silently ignored); they are now
labelled as such. dealign's "enable_thinking: false" direct-comply mode is
approximated on this engine by a forced empty prefix on
the completions path; measured at 118.3 / 135.4 / 126.8 tok/s
(prose/code/reason, forced 1024) in the Measured table above.
- Decode under concurrency and default-mode decode trail the TR3 twin
(~25-30% aggregate, 131 vs 143 tok/s single): the tune reasons longer per
step and accepts MTP drafts less often on repetitive text. Prefill at
every depth matches or beats TR3.
- HumanEval loses 47 of 164 problems to the 8192 thinking budget (28 for
TR3); budget-corrected pass rate of non-empty attempts is ~96.6%.
- MMLU protocol note, recorded honestly: dealign's card does not publish
its MMLU harness or mode, so absolute cross-harness comparison is not
meaningful (their 84.23 vs our 79.48 full-test no-think). On our own
protocol the subject-delta pattern tracks the tune (r=0.714, 14/19
subjects within +/-3pp with thinking on, math/CS above the tune); the
one solid divergence is moral_scenarios 33.5 vs their 64.1 — abliteration
collateral reads deeper here. See the dealign replication section.
- GSM8K 95.4 vs 96.9 and MATH-500 77.4 vs 77.8 — small general-reasoning
costs of the tune.
- Vision is qualified at 499968 (gmu 0.96, image:4 + video:1) and at 1M
(gmu 0.975, image:4): exact OCR + shapes/colors incl. multi-image, video
OCR verified, engine alive after — same envelope as TR3.
- DCP=2 is not supported on 2-discrete-card hosts; DCP=1 is the qualified
configuration. Prefill block size must be 64 or 32
(VLLM_EXL3_PREFILL_BLOCK_M).
- The DFlash2 drafter rows published for the TR3 twin are checkpoint-specific
(drafter trained on those weights); no cyber drafter exists.
- Config duality, documented: the shipped `config.json` /
`quantization_config.json` are the serving variants this engine boots
(mixed-bits marker without the scope field). The sealed materialization
receipt binds the pre-serve originals, kept verbatim in `receipt-configs/`;
the KLD/audit runs validated against those originals. The weight files are
identical either way — only the declaration files differ.
## License
The checkpoint and scripts inherit the upstream stack terms: base model per
zai-org; cybersecurity tune per dealignai; TR3/MCG toolchain and calibration
lineage per the ShapleyMCG License 1.0 (see license link above); see the
recipe repo for the full third-party notice breakdown.
## Lineage
| step | artifact |
|---|---|
| Stock master | [zai-org/GLM-5.3-Flash-BF16](https://huggingface.co/zai-org/GLM-5.3-Flash-BF16) |
| Abliterated + cybersecurity tune (this checkpoint's direct source) | [dealignai/GLM-5.3-Flash-CYBERSECURITY-W4A16](https://huggingface.co/dealignai/GLM-5.3-Flash-CYBERSECURITY-W4A16) |
| Dequantized to BF16, then per-tensor mixed K3/K4 EXL3/TR3 encode | **this repo** |
| Twin quant of the stock base (same machinery, same protocols) | [satgeze/GLM-5.3-Flash-EXL3-TR3-3.5bpw](https://huggingface.co/satgeze/GLM-5.3-Flash-EXL3-TR3-3.5bpw) |
| Serve recipe, drivers, patches | [satindergrewal/GLM-5.3-Flash-EXL3-3.5bpw-Mixed-SM120-TP2](https://github.com/satindergrewal/GLM-5.3-Flash-EXL3-3.5bpw-Mixed-SM120-TP2) (GitHub) |
## Credits
Every party this checkpoint stands on, with where to find them:
- **[zai-org](https://huggingface.co/zai-org)** — the GLM-5.3-Flash base model
([BF16 master](https://huggingface.co/zai-org/GLM-5.3-Flash-BF16)).
- **[dealignai](https://huggingface.co/dealignai)** — the abliterated
cybersecurity tune this derivation starts from
([W4A16 source](https://huggingface.co/dealignai/GLM-5.3-Flash-CYBERSECURITY-W4A16));
their model card's usage modes, MMLU subject deltas, and HarmBench
compliance tables are the reference this card's mode-comparison sections
are measured against.
- **brandonmusic** — the SM120 EXL3 runtime fork (v44-v84) this engine
lineage serves on, the R10 encoder closure, and the sealed BF16
teacher-logits calibration set this derivation is built on
([GLM-5.3-Flash-tr3-4bpw lineage](https://huggingface.co/brandonmusic/GLM-5.3-Flash-tr3-4bpw),
[BF16 Teacher-Logits calibration set](https://huggingface.co/brandonmusic/GLM-5.3-Flash-BF16-Teacher-Logits)).
- **tonyd2wild** — the DFlash2 vLLM overlay this runtime's drafter path
builds on.
- **[local-inference-lab](https://github.com/local-inference-lab)** — b12x
(the mixed-rate trellis3 API) and
[llm-inference-bench](https://github.com/local-inference-lab/llm-inference-bench)
(the long-context estonia/hotel-lights bench).
- **[turboderp](https://github.com/turboderp)** — the EXL3/TR3 format and
kernels ([exllamav3](https://github.com/turboderp/exllamav3)).
If any attribution here is incomplete or wrong, open an issue on the recipe
repo and it will be corrected.