# Gemma4-12B Uncensored: 1M Context + MTP + Vision
[HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced](https://huggingface.co/HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced) (12B dense, Google QAT checkpoint) with a **1,048,576-token context baked in** (4x the native 262,144), shipping with its MTP speculative-decoding draft head and vision tower. All numbers below were measured on these exact files.
| Capability | Status |
|---|---|
| 1M context | Certified: 10/10 at every rung from 64K to 1M, f16 KV, on a single RTX 5090 |
| MTP speculative decoding | 144.9 to 218.4 tok/s (+51%), acceptance 0.723 (measured on this trunk, RTX 5090) |
| Vision | Verified July 6, 2026: reads image text and identifies objects |
| Uncensored | HauhauCS Balanced abliteration; trunk weights bit-identical to the source release |
The first Gemma 4 we know of certified needle-perfect at 1,048,576 tokens. Thanks to Gemma's 5:1 sliding-window attention the KV cache stays small enough that the entire 1M certification ran on a 32 GB RTX 5090 at f16 KV. One cell (786K, depth 15 percent) missed on the first seed and passed 10/10 on a second seed; both runs are in `results.jsonl`.
## MTP speculative decoding
The draft head predicts ahead and the trunk verifies every token, so output is identical to standard decoding, only faster. Measured speedup on this uncensored trunk beats the ~35 percent claimed upstream.
## RULER at 131,072 tokens (NVIDIA's benchmark, their scorer)
NIAH proves retrieval; [RULER](https://github.com/NVIDIA/RULER) is the harder, industry-standard suite (multi-key retrieval, variable tracking, aggregation). We ran five RULER tasks at 131K on this exact GGUF, 25 samples per task, greedy-free vendor sampling (temp 1.0, top_p 0.95, top_k 64, seed 42), scored end to end by NVIDIA's own `evaluate.py` (RULER commit `38da79d`). The bridge script and full reproduction recipe are published in [aviary-1m/tools/ruler](https://github.com/satindergrewal/aviary-1m).
| Task | thinking ON @131K | thinking OFF @131K | thinking OFF @262K |
|---|---|---|---|
| niah_multivalue | 68.0 | **88.0** | 83.0 |
| niah_multiquery | 96.0 | **100.0** | 100.0 |
| variable tracking (vt) | 65.6 | **95.2** | 91.2 |
| common words extraction (cwe) | 0.0 | **80.0** | 46.8 |
| frequent words extraction (fwe) | 0.0 | **96.0** | 86.7 |
| **Average** | 45.9 | **91.8** | **81.5** |
The ladder is climbing: at 262K (double the standard RULER maximum) retrieval and variable tracking barely move, and the model answered 122 of 125 samples. The cost of distance concentrates in aggregation (cwe). 524K and 1M rungs are queued; each publishes here as it lands, pass or fail.
**The honest headline: thinking mode halves this model's RULER score at 131K.** The failure mode is specific and reproducible: on aggregation tasks (cwe/fwe) and some hard retrieval samples, the model reasons in circles and exhausts any generation budget (tested to 12K tokens) without ever emitting an answer. At temperature 0 the same samples loop deterministically; vendor sampling reduces but does not eliminate it. Those runs score 0 by RULER's rules and we report them as such, nulls and all.
**Practical guidance:** for long-context retrieval and aggregation work with this model, run thinking OFF. 91.8 average at 131K with zero unanswered samples is the model's real capability; thinking mode is the liability, not the context window. Raw prediction files, both conditions, are in the repo history for anyone who wants to re-score.
## Files
| File | Size | Role |
|---|---|---|
| `gemma4-12b-uncensored-1M-Q4.gguf` | 7.4 GB | Trunk, 1M baked, QAT 4-bit |
| `mtp-gemma-12b.gguf` | 254 MB | MTP draft head, pair with `-md` |
| `mmproj-gemma12b-hauhau.gguf` | 175 MB | Vision tower, pair with `--mmproj` |
| `niah_heatmap.png`, `mtp_speedup.png`, `results.jsonl` | small | Verification evidence |
## Every file, every mirror
Nothing was discontinued: every quant is one click away. Hugging Face carries the curated picks, ModelScope always carries everything, and Ollama serves ready-to-run tags.
On Ollama every tag ships with the vision tower bundled and the 1M rope metadata baked in.
| File | Size | Hugging Face | ModelScope | Ollama |
|---|---|---|---|---|
| `gemma4-12b-uncensored-1M-Q4.gguf` | 7.4 GB | [download](https://huggingface.co/satgeze/Gemma4-12B-Uncensored-HauhauCS-1M-GGUF/resolve/main/gemma4-12b-uncensored-1M-Q4.gguf) | [download](https://www.modelscope.ai/models/satgeze/Gemma4-12B-Uncensored-HauhauCS-1M-GGUF/resolve/master/gemma4-12b-uncensored-1M-Q4.gguf) | [`ollama run satgeze/gemma4-12b-uncensored-1m`](https://ollama.com/satgeze/gemma4-12b-uncensored-1m) |
| `mmproj-gemma12b-hauhau.gguf` | 175 MB | [download](https://huggingface.co/satgeze/Gemma4-12B-Uncensored-HauhauCS-1M-GGUF/resolve/main/mmproj-gemma12b-hauhau.gguf) | [download](https://www.modelscope.ai/models/satgeze/Gemma4-12B-Uncensored-HauhauCS-1M-GGUF/resolve/master/mmproj-gemma12b-hauhau.gguf) | bundled in every tag |
| `mtp-gemma-12b.gguf` | 254 MB | [download](https://huggingface.co/satgeze/Gemma4-12B-Uncensored-HauhauCS-1M-GGUF/resolve/main/mtp-gemma-12b.gguf) | [download](https://www.modelscope.ai/models/satgeze/Gemma4-12B-Uncensored-HauhauCS-1M-GGUF/resolve/master/mtp-gemma-12b.gguf) | - |
## Run it
llama.cpp, everything on:
```bash
llama-server -m gemma4-12b-uncensored-1M-Q4.gguf \
-c 1048576 -np 1 --jinja \
-md mtp-gemma-12b.gguf --spec-type draft-mtp --spec-draft-n-max 3 \
--mmproj mmproj-gemma12b-hauhau.gguf
```
Ollama (1M and vision work; Ollama has no speculative decoding yet, so the MTP head adds no speed there):
```
FROM ./gemma4-12b-uncensored-1M-Q4.gguf
RENDERER gemma4
PARSER gemma4
PARAMETER num_ctx 262144
```
The RENDERER and PARSER lines avoid imported-GGUF template bugs under tool-heavy use. Raise `num_ctx` as memory allows.
## How this was built
YaRN rope-scaling metadata (factor 4.0 over native 262,144) baked into the GGUF header with gguf-py; weights are bit-identical to the HauhauCS release, no fine-tuning. Gemma 4's dual-rope design takes YaRN on its global-attention layers. Certification harness: 10 needles per rung at depths 5 to 95 percent, temperature 0, seeded prompts, f16 KV only. Method and tooling: [github.com/satindergrewal/aviary-1m](https://github.com/satindergrewal/aviary-1m).
For base capability benchmarks see Google's official Gemma 4 cards; uncensoring quality versus the official trunk has not been independently benchmarked here.
## Credits
Base model and QAT: Google (Gemma license; its terms flow down to these files). Uncensoring and packaging: [HauhauCS](https://huggingface.co/HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced). MTP head: Unsloth (via the HauhauCS repo). 1M YaRN extension, benchmarking, and certification: [SatGeze](https://huggingface.co/satgeze).
Sister repos: [12B](https://huggingface.co/satgeze/Gemma4-12B-Uncensored-HauhauCS-1M-GGUF) | [26B-A4B](https://huggingface.co/satgeze/Gemma4-26B-A4B-Uncensored-HauhauCS-1M-GGUF) | [31B](https://huggingface.co/satgeze/Gemma4-31B-Uncensored-HauhauCS-1M-GGUF) | [Qwen3.6-35B](https://huggingface.co/satgeze/Qwen3.6-35B-Uncensored-HauhauCS-1M-GGUF)
Mirrors: Hugging Face | [ModelScope](https://www.modelscope.ai/models/satgeze/Gemma4-12B-Uncensored-HauhauCS-1M-GGUF)