# Ornith-1.0-35B: 1M Context + MTP + Vision
[-181717?logo=github)](https://github.com/satindergrewal/aviary-1m)
[](https://opensource.org/licenses/MIT)
[](https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B)
[]()
[]()
[]()
**Mirrors:** [Hugging Face](https://huggingface.co/satgeze/Ornith-1.0-35B-1M-GGUF) | [ModelScope](https://www.modelscope.ai/models/satgeze/Ornith-1.0-35B-1M-GGUF) (full quant ladder always available on ModelScope)
[DeepReinforce's Ornith-1.0-35B](https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B) (35B MoE, 3B active, Qwen3.5 family) with YaRN RoPE scaling **baked into the GGUF metadata** for a **1,048,576-token context window**, needle-certified through the full range, now shipping MTP speculative decoding and a vision tower. Weights are bit-identical to the source builds; only rope metadata differs, so llama.cpp and Ollama apply the extension with no extra flags.
Native 262,144, YaRN factor 4 to 1,048,576: 10/10 needles at every rung through 1M, f16 KV, temperature 0 (`results.jsonl`). The APEX-Compact build was ladder-tested separately: 50/50 from 64K through 524K on the RTX 5090 (`results-apex.jsonl`); its 786K and 1M rungs are running on a 128 GB Mac at publish time and this card will update when they land.
## MTP: same output, more tokens per second
Ornith's base family (Qwen3.5) ships a multi-token-prediction layer in the official checkpoints; Ornith's own GGUF releases dropped it, and community builds by [wang-yang](https://huggingface.co/wang-yang/Ornith-1.0-35B-MTP-GGUF) (Q6_K) and [SC117](https://huggingface.co/SC117/Ornith-1.0-35B-MTP-APEX-GGUF) (APEX) restored it. The trunk verifies every drafted token, so output is identical to standard decoding, only faster.
Measured here (RTX 5090, 16K ctx): Q6_K 267.9 to 382.9 tok/s (+43 percent). APEX-Compact gains +14 percent; its custom imatrix shifts token probabilities away from the draft head, lowering acceptance, and its strength is instead the 17 GB footprint. Both builds were re-baked to 1M, coherence-gated, and NIAH-spot-checked here before shipping.
```bash
llama-server -m ornith-1.0-35b-1M-MTP-Q6_K.gguf \
-c 1048576 -np 1 --jinja \
--spec-type draft-mtp --spec-draft-n-max 3 \
--mmproj mmproj-ornith-35b-f16.gguf
```
Ollama (1M and vision work; Ollama has no speculative decoding yet, so MTP adds no speed there):
```
FROM ./ornith-1.0-35b-1M-MTP-Q6_K.gguf
RENDERER qwen3.5
PARSER qwen3.5
PARAMETER num_ctx 262144
```
## How this was built
1. **1M context**: YaRN rope-scaling metadata (factor 4.0, original context 262,144) written into the GGUF header with gguf-py. No weight changes, no fine-tuning.
2. **MTP**: adopted the community GGUFs carrying the official Qwen3.5 MTP layer, verified provenance and license, re-baked 1M metadata, then gated: coherence check, speed A/B, needle spot-checks with speculation active.
3. **Certification**: multi-needle harness (10 needles, depths 5 to 95 percent, temperature 0, seeded prompts) against llama-server, f16 KV only. The harness ships in this repo (`niah_test.py`) along with prefill timing data (`prefill_timing.jsonl`).
For base-model capability benchmarks see the [official Ornith-1.0-35B card](https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B); weights here are bit-identical apart from rope metadata, so base capability carries over modulo quantization.
## Credits
Ornith-1.0: [DeepReinforce](https://huggingface.co/deepreinforce-ai) (MIT), post-trained on Qwen 3.5. MTP layer: Qwen. MTP GGUF builds: wang-yang and SC117. 1M YaRN extension, vetting, and certification: [SatGeze](https://huggingface.co/satgeze).