# Gemma4-12B Uncensored 1.5M: the longest needle-certified context we know of
The same [HauhauCS uncensored Gemma-4-12B](https://huggingface.co/HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced) trunk as our [1M repo](https://huggingface.co/satgeze/Gemma4-12B-Uncensored-HauhauCS-1M-GGUF), baked with YaRN factor 6 for a **1,572,864-token maximum context**, and needle-certified far past the million mark. Every number below was measured on a single RTX 5090 (32GB); raw per-needle records ship in this repo.
| Tier | Context | Result |
|---|---|---|
| Certified (f16 KV) | 262K / 1M / 1,310,720 | 10/10 at every rung, including 1.31M |
| Budget (q8_0 KV) | 1M | 10/10 |
| Budget (q8_0 KV) | 1,572,864 | 9/10 (one miss, published) |
10 needles per rung at depths 5 to 95 percent, temperature 0, seeded haystacks. The f16 tier is the certification; the q8 KV tier is the labeled budget config that fits the higher rungs into 32GB. At 1.57M with f16 KV the cache no longer fits a 32GB card (that rung runs on larger memory; result will be added).
## Where the ceiling is, honestly
We also baked and tested a factor-8 variant (2.1M max) so nobody has to wonder:
| Factor | 1M | 1.57M | 2M |
|---|---|---|---|
| 6x (this repo) | 10/10 | 9/10 (q8 KV) | not reachable by design |
| 8x (tested, not shipped) | 9/10 | 9/10 (q8 KV) | **6/10 (q8 KV)** |
Two findings worth stealing: **the YaRN factor itself costs quality** (8x drops a needle at 1M where 6x is clean, so bake the smallest factor that reaches your target), and **2M is where retrieval bends** (6/10, degradation spread across depths). The 2M line of work is closed: 6/10 does not meet our bar, the 8x build stays unshipped, and the data is published instead. All four raw ladders are in the results jsonl files, misses included.
Next for this build: **NVIDIA's RULER benchmark** (multikey NIAH, variable tracking, aggregation) at lengths up to and beyond 1M. Results will be published here either way, sample counts stated.
## Files
| File | Size | Role |
|---|---|---|
| `gemma4-12b-uncensored-1.5M-Q4.gguf` | 7.4 GB | Trunk, 1.57M max context baked (Google QAT 4-bit) |
| `mtp-gemma-12b.gguf` | 254 MB | MTP draft head, pair with `-md` (+51 percent decode measured on this trunk) |
| `mmproj-gemma12b-hauhau.gguf` | 175 MB | Vision tower, pair with `--mmproj` |
| `niah_heatmap.png`, `results-*.jsonl` | small | Verification evidence, all four ladders |
## Run it
```bash
llama-server -m gemma4-12b-uncensored-1.5M-Q4.gguf \
-c 1310720 -np 1 --jinja \
-md mtp-gemma-12b.gguf --spec-type draft-mtp --spec-draft-n-max 3 \
--mmproj mmproj-gemma12b-hauhau.gguf
```
For the certified experience use up to `-c 1310720` with default f16 KV. To reach the full 1.57M on a 32GB card add `--cache-type-k q8_0 --cache-type-v q8_0` (the budget tier above, 9/10 measured). Ollama:
```
FROM ./gemma4-12b-uncensored-1.5M-Q4.gguf
RENDERER gemma4
PARSER gemma4
PARAMETER num_ctx 262144
```
Raise `num_ctx` as desired; Gemma's 5:1 sliding-window attention keeps the KV cache small enough that this entire certification ran inside a 32GB GPU.
## How this was built
YaRN rope-scaling metadata (factor 6.0 over native 262,144) written into the GGUF header with gguf-py; weights bit-identical to the HauhauCS release, no fine-tuning. Certification harness: 10 needles per rung, depths 5 to 95 percent, temperature 0, f16 KV for certified rungs, q8 KV clearly labeled. Method and tooling: [github.com/satindergrewal/aviary-1m](https://github.com/satindergrewal/aviary-1m). Sister repo with the 4x/1M build and full ladder: [Gemma4-12B-Uncensored-HauhauCS-1M-GGUF](https://huggingface.co/satgeze/Gemma4-12B-Uncensored-HauhauCS-1M-GGUF).
## Credits
Base model and QAT: Google (Gemma license; terms flow down). Uncensoring: [HauhauCS](https://huggingface.co/HauhauCS). MTP head: Unsloth (via the HauhauCS repo). YaRN 1.5M extension, beyond-1M certification, and the 2M ceiling study: [SatGeze](https://huggingface.co/satgeze).
Mirrors: [Hugging Face](https://huggingface.co/satgeze/Gemma4-12B-Uncensored-HauhauCS-1.5M-GGUF) | [ModelScope](https://www.modelscope.ai/models/satgeze/Gemma4-12B-Uncensored-HauhauCS-1.5M-GGUF)