--- title: gemma-4-E4B-it-heretic-QAT-GGUF canonical_url: "https://www.modelscope.ai/models/SC117/gemma-4-E4B-it-heretic-QAT-GGUF" md_url: "https://www.modelscope.ai/models/SC117/gemma-4-E4B-it-heretic-QAT-GGUF.md" repository: SC117/gemma-4-E4B-it-heretic-QAT-GGUF last_updated: 2026-10-04 license: apache-2.0 base_model: - coder3101/gemma-4-E4B-it-qat-q4_0-unquantized-heretic - google/gemma-4-E4B-it base_model_relation: quantized library_name: - gguf - pytorch frameworks: - pytorch language: - en - zh - multilingual downloads: 124 stars: 1 tags: - gguf - quantized - qat - heretic - uncensored - abliterated - gemma4 --- # gemma-4-E4B-it-heretic-QAT-GGUF > gemma-4-E4B-it-heretic-QAT-GGUF - An open-source model by SC117 on ModelScope. ⚡ Gemma 4 E4B Heretic QAT — Q40 GGUF Heretic ARA · QAT-Lossless Q40 · 4 GB · Mobile-Ready 📖 中文文档 Q40 4.5B Effective Heretic Uncensored 4 GB QAT Weights 📱 Mobile Uncensored… SC117/gemma-4-E4B-it-heretic-QAT-GGUF is a machine learning model on ModelScope. licensed under apache-2.0. derived from coder3101/gemma-4-E4B-it-qat-q4_0-unquantized-heretic、google/gemma-4-E4B-it. - **Repository**: SC117/gemma-4-E4B-it-heretic-QAT-GGUF - **License**: apache-2.0 - **Base model**: coder3101/gemma-4-E4B-it-qat-q4_0-unquantized-heretic, google/gemma-4-E4B-it - **Tags**: gguf, quantized, qat, heretic, uncensored, abliterated, gemma4 - **Downloads**: 124 - **Stars**: 1 - **Last updated**: 2026-10-04 Source: https://www.modelscope.ai/models/SC117/gemma-4-E4B-it-heretic-QAT-GGUF ---

⚡ Gemma 4 E4B Heretic QAT — Q4_0 GGUF

Heretic ARA · QAT-Lossless Q4_0 · 4 GB · Mobile-Ready

📖 中文文档

Q4_0 4.5B Effective Heretic Uncensored 4 GB QAT Weights 📱 Mobile

Uncensored version of Google Gemma 4 E4B IT (QAT), processed with Heretic ARA abliteration. Quantized to Q4_0 — only 4 GB, perfect for mobile and edge devices.

✂️ Heretic ARA Abliteration Parameters

Base: coder3101/heretic-QAT · Heretic v1.2.0 · ARA + Row-Norm

Parameter Value
start_layer_index20
end_layer_index36
preserve_good_behavior_weight0.5805
steer_bad_behavior_weight0.0023
overcorrect_relative_weight0.7764
neighbor_count13
Metric Heretic Original QAT
KL Divergence0.00650 (by definition)
Refusals5/10098/100
🏗️ Architecture
Base Modelgoogle/gemma-4-E4B-it
Parameters4.5B effective (8B with PLE embeddings)
ArchitectureDense with Per-Layer Embeddings (PLE) + multimodal
Layers42
Hidden Size2,560
Context Length128K tokens
Vocabulary262K, 140+ languages
ModalitiesText + Image + Audio + Video
QuantizationQ4_0 (matching Unsloth UD-Q4_K_XL layout), b9553 llama-quantize
📊 Quantization Details
FormatQ4_0 (QAT weights optimized for this precision)
File Size4 GB
Effective BPW4.54 (all weights Q4_0, norms F32)
Toolllama-quantize (b9553, CUDA 13.3)
QAT AdvantageQ4_0 + QAT weights achieves 98.5% Top-1 vs 90.9% naive Q4_0 (+7.6%)
⚙️ Recommended Sampling Parameters
Generaltemp=1.0, top_p=0.95, top_k=64
Codingtemp=0.6, top_p=0.95, top_k=64

Use --jinja flag with llama.cpp. Disable thinking: --chat-template-kwargs '{"enable_thinking":false}'.

📝 Usage

Only 4 GB — runs on any modern phone with 6GB+ RAM. Compatible with llama.cpp, LM Studio, Jan, MLC LLM, and mobile GGUF runtimes.

llama-server \
  -m gemma-4-E4B-it-heretic-QAT-UD-Q4_K_XL.gguf \
  --jinja -ngl 99 -c 4096 \
  --port 8001
🔗 Credits

Heretic Abliteration: coder3101 · Heretic v1.2.0 ARA + Row-Norm
QAT Weights: Google Gemma 4 E4B IT
Quantization Recipe: Unsloth UD-Q4_K_XL (Q4_0 layout)
Quantization Tool: llama.cpp b9553 · GitHub
Original Model: Google Gemma 4 E4B IT

Support SC117 on Ko-fi — support is optional and all releases remain freely available.