---
title: GLM-5.3-Flash-GGUF-DGX-Spark
canonical_url: "https://www.modelscope.ai/models/AutoTrust-Labs/GLM-5.3-Flash-GGUF-DGX-Spark"
md_url: "https://www.modelscope.ai/models/AutoTrust-Labs/GLM-5.3-Flash-GGUF-DGX-Spark.md"
repository: AutoTrust-Labs/GLM-5.3-Flash-GGUF-DGX-Spark
last_updated: 2026-10-12
license: mit
pipeline_tag: text-generation
tasks:
  - text-generation
base_model:
  - zai-org/GLM-5.3-Flash
base_model_relation: quantized
library_name:
  - gguf
  - pytorch
frameworks:
  - pytorch
language:
  - en
  - zh
downloads: 42
stars: 0
tags:
  - gguf
  - llama.cpp
  - dgx-spark
  - gb10
  - glm5-next
  - moe
  - expert-pruned
  - iq2_xxs
---

# GLM-5.3-Flash-GGUF-DGX-Spark

> GLM-5.3-Flash-GGUF-DGX-Spark - An open-source model by AutoTrust-Labs on ModelScope. GLM-5.3-Flash-GGUF-DGX-Spark

AutoTrust-Labs/GLM-5.3-Flash-GGUF-DGX-Spark is a text-generation model on ModelScope. licensed under mit. derived from zai-org/GLM-5.3-Flash.

- **Repository**: AutoTrust-Labs/GLM-5.3-Flash-GGUF-DGX-Spark
- **License**: mit
- **Tasks**: text-generation
- **Base model**: zai-org/GLM-5.3-Flash
- **Tags**: gguf, llama.cpp, dgx-spark, gb10, glm5-next, moe, expert-pruned, iq2_xxs
- **Downloads**: 42
- **Stars**: 0
- **Last updated**: 2026-10-12

Source: https://www.modelscope.ai/models/AutoTrust-Labs/GLM-5.3-Flash-GGUF-DGX-Spark

---

# GLM-5.3-Flash-GGUF-DGX-Spark

[![AutoTrust/GuruSearch recommendation & search live demo: news.guru.so](https://huggingface.co/autotrust/JEV-27B/resolve/main/assets/autotrust_gurusearch_demo.png)](https://news.guru.so/ui)

## 🔴 Live: AutoTrust/GuruSearch recommendation & search demo → [news.guru.so](https://news.guru.so/ui)

> [!IMPORTANT]
> **Try AutoTrust/GuruSearch live (new, 11 October 2026).** A recommendation and search demo in which every ranking is a
> calibrated System 1 decision, made in about half a second:
> * **Recommendation:** the latest headlines from 6 news feeds, ranked by importance.
> * **Search:** results from Google News, Bing News and Yahoo News re-ranked by relevance, side by side with the
>   search engines' own order, plus a short answer with citations.

**GLM 5.3 Flash, 12 % smaller — Sparse-Squared compression with AutoTrust SLIM-Q, resident on a single 128 GB DGX Spark with room to spare.**

This repository is the llama.cpp GGUF build of the SLIM-Q-compressed GLM 5.3 Flash: 79.1 GiB instead of the stock 90 GiB Q2, with 32 of 288 routed experts removed per layer (256 retained — the **E256** member of the SLIM-Q family) and everything else untouched. It is the single-Spark companion to [GLM-5.3-GGUF-DGX-Spark](https://huggingface.co/autotrust/GLM-5.3-GGUF-DGX-Spark), the 744B SLIM-Q E192 build whose NVFP4 edition powers Guru Turbo 2.0. Runs in llama.cpp (glm5-next), so you get the OpenAI-compatible llama-server, continuous batching, tool calling and `reasoning_content` on the Spark.

---

## SLIM-Q: AutoTrust's Sparse-Squared compression recipe

**SLIM-Q** — **S**elective expert pruning + **L**ow-bit quantization for **I**nference of **M**oE — is AutoTrust's two-stage pipeline for post-training compression of frontier open-source MoE models. The idea in one line is **Sparse-Squared (Sparse²) compression**: a MoE is already sparse at inference (each token activates only the top-8 of 288 routed experts); SLIM-Q adds a second, structural sparsity axis — permanently removing the experts the router rarely uses — and then drives the surviving experts to low precision.

**Stage 1 — Selective expert pruning (SLIM).** Expert selection was profiled on a bilingual code / agent / science / maths calibration mix, and the 32 least-used of 288 routed experts per layer (11 %) were structurally removed. Attention (KDA + DSA), dense layers, shared experts, router, tokenizer and chat template are unchanged. Unlike dynamic expert-skipping, structural pruning permanently shrinks the weight footprint — on a 128 GB Spark, that is the difference between ~25 GiB and ~40 GiB of free unified memory after weights, i.e. between a 16–32 K context and a comfortable 64 K (or 4 × 16 K sessions).

**Stage 2 — Low-bit quantization (Q).** The pruned checkpoint is quantized aggressively on the routed experts (IQ2_XXS gate/up, Q2_K down) while attention, dense layers, shared experts, and routers stay at high precision — preserving the routing behavior that MoE quality depends on.

As on the 744B build, the per-token cost is untouched: routing still selects the top-8 of the remaining 256 experts, so compute and memory traffic per token equal the stock Flash Q2. Only the footprint drops.

| | Stock GLM-5.3-Flash Q2 GGUF | This model |
|---|---|---|
| Size | 90 GiB | **79.1 GiB (−12 %)** |
| Routed experts per layer (active per token) | 288 (8) | 256 (8) |
| Free unified memory on a 128 GB Spark after weights | ~25 GiB | **~40 GiB** |
| Context that fits comfortably on a Spark | 16–32 K | **64 K, at most 1M** |
| Decode speed | reference | same class (identical work per token) |
| Quality (see below) | reference | 4-bit: HumanEval 97.6, C-Eval 89.4, GPQA-D 77.3; 2-bit harness A/B on par or better |

---

## Download

```bash
hf download autotrust/GLM-5.3-Flash-GGUF-DGX-Spark --local-dir ./GLM-5.3-Flash-GGUF-DGX-Spark
```


The model is stored as two GGUF shards (Hugging Face's 50 GB per-file limit); llama.cpp loads them together when pointed at the first one: `GLM-5.3-Flash-Q2-DGX-Spark-00001-of-00002.gguf` (41.9 GiB) + `-00002-of-00002.gguf` (37.2 GiB), 79.1 GiB total. Checksums in `GLM-5.3-Flash-Q2-DGX-Spark.sha256`.

---

## DGX Spark quick start

GLM-5.3-Flash support is in llama.cpp pull request [#27773](https://github.com/ggml-org/llama.cpp/pull/27773) (glm5-next; not merged at the time of writing — once it is, plain master works).

```bash
git clone https://github.com/ggml-org/llama.cpp.git && cd llama.cpp
git fetch origin pull/27773/head:glm5next && git checkout glm5next
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=121a-real     # GB10
cmake --build build --config Release -j
```

```bash
# OpenAI-compatible API on :8080, 4 sessions x 16K, continuous batching
./build/bin/llama-server -m GLM-5.3-Flash-Q2-DGX-Spark-00001-of-00002.gguf -ngl 99 -fa on \
    -c 65536 -np 4 --cont-batching --host 0.0.0.0 --port 8080

# single long-context chat
./build/bin/llama-cli -m GLM-5.3-Flash-Q2-DGX-Spark-00001-of-00002.gguf -ngl 99 -fa on -c 65536
```

- Keep the file on the internal NVMe; the first load reads 79 GiB. Stop other GPU work first.
- **Memory on the Spark:** 79.1 GiB weights + ~1 GiB per 16 K tokens of context (KDA layers keep a constant state; the DSA layers use a compact cache) + a few GiB of compute buffers. `-c 65536` leaves ~30 GiB free; `-np 4 -c 65536` (4 × 16 K) is a good multi-user setting.
- Thinking is on by default (the GLM template opens `<think>`). `--reasoning-budget 0` disables it, `--reasoning-budget 4096` caps it, `--chat-template-kwargs '{"reasoning_effort":"low"}'` selects the template's low / high / max effort. The server returns thinking separately as `reasoning_content`; tool calls come back as OpenAI `tool_calls`.
- **Expected decode speed on a Spark:** the same class as the stock Flash Q2 — the per-token work is identical (8 experts + the same 8-bit attention). Decode is bound by the 273 GB/s LPDDR5X (about 11 GB of weights per token), so roughly 15–20 t/s single-stream; batching several sessions gives more aggregate throughput. Not measured by the author on a Spark; see the B200 table below for relative numbers.
- Two Sparks (ConnectX link) are not needed for this file; it is a single-Spark model. For the 744B two-Spark build, see [GLM-5.3-GGUF-DGX-Spark](https://huggingface.co/autotrust/GLM-5.3-GGUF-DGX-Spark).
- **Also runs on:** 128 GB Apple Silicon (`cmake -B build` without CUDA), discrete NVIDIA GPUs with ≥ 90 GB, or partially offloaded (`-ngl N`) on smaller cards.

---

## Speed reference

llama.cpp, CUDA, one B200, model resident (`llama-bench` / `llama-batched-bench`, 256-token prompts, 128 generated tokens, flash attention):

| | tokens/s |
|---|---|
| Prompt processing (pp512 / pp2048) | 1037 / 1086 |
| Generation, 1 sequence | 54 |
| Generation, 4 / 8 / 16 / 32 parallel sequences (aggregate) | 130 / 170 / 207 / 253 |

---

## Quality

**SLIM stage (4-bit build with the same 256-expert selection, under vLLM on one B200):** HumanEval 97.6 · C-Eval 89.4 · GPQA-Diamond 77.3 (low effort) · AIME 2025 74.2 · MMMU 76.1 · BFCL Non-Live 87.7 / Live 80.5 / multi-turn 73–75 — the pruning itself costs little.

**Q stage (this 2-bit file)** — DwarfStar's ds4-eval harness (GPQA Diamond, SuperGPQA, AIME 2025 interleaved; thinking on, 16 000-token budget, greedy), first 40 core cases, compared with the 2-bit 744B SLIM-Q E192 build on the same cases:

| 2-bit GGUF | Passed | Wrong | Out of budget |
|---|---|---|---|
| **GLM-5.3-Flash-Q2-DGX-Spark (this file, 79 GiB; 256 experts)** | **35 / 40** | 3 | 2 |
| GLM-5.3-Q2-DGX-Spark (744B SLIM-Q E192, 150 GiB; [autotrust/GLM-5.3-GGUF-DGX-Spark](https://huggingface.co/autotrust/GLM-5.3-GGUF-DGX-Spark)) | 30 / 40 | 1 | 9 |

An imatrix-guided build of the same layout scored 31 / 40 with twice the reasoning tokens, so this weight-energy-importance build is the one published. Qualitative checks: correct bilingual common-sense, medical and coding answers; no Chinese–English mixing.

**Choosing between the two Spark builds:** this Flash build closes more reasoning cases within budget and fits one Spark; the 744B E192 build keeps a clear edge on language modelling of agent/SWE trajectories and code (held-out PPL 6.2 vs 12.4 on agent traces, 3.1 vs 4.6 on SWE traces, 2.96 vs 3.17 on code) and needs two Sparks, a 180 GB GPU, or a big Mac.

---

## What is in the file

| Role | Type | Bytes |
|---|---|---|
| Routed experts gate / up (42 MoE layers × 256) | IQ2_XXS | 46.5 GB |
| Routed experts down | Q2_K | 29.6 GB |
| KDA linear attention, DSA attention, dense FFN, shared experts | Q8_0 (1.3 GB Q4_K) | 8.2 GB |
| Embedding, output head | Q8_0 | 1.3 GB |
| Norms, routers, hyper-connections, indexer, k-pool compressor | F32 | 0.5 GB |

45 layers (3 dense + 42 MoE) · KDA linear attention with DSA every fourth layer (k-pool indexer) · hyper-connections · top-8 of 256 routed experts + 1 shared · 154 880-token vocabulary · GLM 5.3 chat template with tool calling. Plain llama.cpp GGUF (architecture glm5-next). No MTP block; text model (no vision projector).

### Shards

| File | Bytes |
|---|---|
| GLM-5.3-Flash-Q2-DGX-Spark-00001-of-00002.gguf | 44,984,916,000 |
| GLM-5.3-Flash-Q2-DGX-Spark-00002-of-00002.gguf | 39,944,534,144 |
| GLM-5.3-Flash-Q2-DGX-Spark.sha256 | checksums of both shards |

Total 84,929,450,144 bytes (79.1 GiB). Split with `llama-gguf-split --split-max-size 45G`; to get a single file: `llama-gguf-split --merge GLM-5.3-Flash-Q2-DGX-Spark-00001-of-00002.gguf out.gguf`.

---

## Limitations

- **2-bit routed experts on top of an 11 % expert pruning.** Chat, coding and agent use are the target; expect a drop on the hardest maths/science reasoning versus the 4-bit deployment.
- Needs the llama.cpp glm5-next branch until PR #27773 is merged. Two quirks of that branch are patched in `llamacpp-pr27773-glm5next-lenient.patch` (in the tooling repository): tolerate u64 metadata, and return a reply verbatim instead of HTTP 500 when `max_tokens` cuts it mid-UTF-8-character. Neither is required to run the model.
- No MTP head, no vision projector in this file.

---

## License and credits

- **Weights:** MIT (base model); derivative of `zai-org/GLM-5.3-Flash`.
- **SLIM-Q pipeline and expert pruning (E256 selection):** AutoTrust AI — the same Sparse-Squared compression recipe as the 744B [GLM-5.3-SLIM-E192](https://huggingface.co/autotrust/GLM-5.3-SLIM-E192) behind Guru Turbo 2.0.
- **2-bit quantization recipe and quantizer:** DwarfStar (antirez/ds4), on llama.cpp / GGML; glm5-next support by the llama.cpp PR #27773 authors.
- **Tooling and this build:** https://github.com/yuhai-china/ds4-glm-slim