---
title: gemma-4-19b-a4b-it-REAP-GGUF
canonical_url: "https://www.modelscope.ai/models/barozp/gemma-4-19b-a4b-it-REAP-GGUF"
md_url: "https://www.modelscope.ai/models/barozp/gemma-4-19b-a4b-it-REAP-GGUF.md"
repository: barozp/gemma-4-19b-a4b-it-REAP-GGUF
last_updated: 2026-06-10
license: gemma
pipeline_tag: text-generation
tasks:
  - text-generation
base_model:
  - 0xSero/gemma-4-19b-a4b-it-REAP
base_model_relation: quantized
library_name:
  - gguf
  - pytorch
frameworks:
  - pytorch
language:
  - en
downloads: 99
stars: 0
tags:
  - gguf
  - gemma4
  - image-text-to-text
  - moe
  - pruning
  - reap
  - cerebras
  - expert-pruning
  - conversational
  - llama-cpp
  - quantized
---

# gemma-4-19b-a4b-it-REAP-GGUF

> gemma-4-19b-a4b-it-REAP-GGUF - An open-source model by barozp on ModelScope. gemma-4-19b-a4b-it-REAP — GGUF Quantizations

barozp/gemma-4-19b-a4b-it-REAP-GGUF is a text-generation model on ModelScope. licensed under gemma. derived from 0xSero/gemma-4-19b-a4b-it-REAP.

- **Repository**: barozp/gemma-4-19b-a4b-it-REAP-GGUF
- **License**: gemma
- **Tasks**: text-generation
- **Base model**: 0xSero/gemma-4-19b-a4b-it-REAP
- **Tags**: gguf, gemma4, image-text-to-text, moe, pruning, reap, cerebras, expert-pruning, conversational, llama-cpp, quantized
- **Downloads**: 99
- **Stars**: 0
- **Last updated**: 2026-06-10

Source: https://www.modelscope.ai/models/barozp/gemma-4-19b-a4b-it-REAP-GGUF

---

# gemma-4-19b-a4b-it-REAP — GGUF Quantizations

GGUF quantizations of [0xSero/gemma-4-19b-a4b-it-REAP](https://huggingface.co/0xSero/gemma-4-19b-a4b-it-REAP), a 30% expert-pruned variant of [google/gemma-4-26b-a4b-it](https://huggingface.co/google/gemma-4-26b-a4b-it) using the REAP (Router-weighted Expert Activation Pruning) method.

## Available Files

| File | Quant | Size | BPW | Description |
| --- | --- | --- | --- | --- |
| `gemma-4-19b-a4b-it-REAP-BF16.gguf` | BF16 | ~36 GB | 16.0 | Full precision, for re-quantization |
| `gemma-4-19b-a4b-it-REAP-Q8_0.gguf` | Q8_0 | ~19 GB | 8.0 | Near-lossless, large file |
| `gemma-4-19b-a4b-it-REAP-Q6_K.gguf` | Q6_K | ~15 GB | 6.56 | Near-lossless, recommended for high quality |
| `gemma-4-19b-a4b-it-REAP-Q5_K_M.gguf` | Q5_K_M | ~13 GB | 5.68 | High quality, larger size |
| `gemma-4-19b-a4b-it-REAP-Q5_K_S.gguf` | Q5_K_S | ~12 GB | 5.52 | High quality, slightly smaller |
| `gemma-4-19b-a4b-it-REAP-Q4_K_M.gguf` | Q4_K_M | ~12 GB | 4.89 | **Recommended** — best quality/size balance |
| `gemma-4-19b-a4b-it-REAP-Q4_K_S.gguf` | Q4_K_S | ~11 GB | 4.63 | 4-bit small |
| `gemma-4-19b-a4b-it-REAP-Q3_K_L.gguf` | Q3_K_L | ~9.5 GB | 4.27 | 3-bit large |
| `gemma-4-19b-a4b-it-REAP-Q3_K_M.gguf` | Q3_K_M | ~9 GB | 3.91 | 3-bit medium |
| `gemma-4-19b-a4b-it-REAP-Q3_K_S.gguf` | Q3_K_S | ~8.5 GB | 3.66 | 3-bit small |
| `gemma-4-19b-a4b-it-REAP-Q2_K.gguf` | Q2_K | ~7.5 GB | 2.96 | Smallest size, lowest quality |

## Model Details

| Property | Value |
| --- | --- |
| Architecture | Gemma 4 (hybrid sliding/full attention MoE) |
| Parameters | 19.02B total / ~4B active per token |
| Experts | 90 total / 8 active per token |
| Context Length | 262,144 tokens |
| Original dtype | BF16 |
| Quantization tool | llama.cpp |
| License | Gemma |

## Quantization Process
```bash
# 1. Convert BF16 SafeTensors → GGUF
python convert_hf_to_gguf.py 0xSero/gemma-4-19b-a4b-it-REAP \
  --outfile gemma-4-19b-a4b-it-REAP-BF16.gguf \
  --outtype bf16

# 2. Quantize (example: Q4_K_M)
llama-quantize gemma-4-19b-a4b-it-REAP-BF16.gguf \
  gemma-4-19b-a4b-it-REAP-Q4_K_M.gguf Q4_K_M
```

## Usage

### llama.cpp
```bash
llama-cli \
  -m gemma-4-19b-a4b-it-REAP-Q4_K_M.gguf \
  -ngl 99 -c 4096 \
  -p "Your prompt here"
```

### llama-server (OpenAI-compatible API)
```bash
llama-server \
  -m gemma-4-19b-a4b-it-REAP-Q4_K_M.gguf \
  -ngl 99 -c 4096 \
  --port 8080
```

### LM Studio / Jan / Ollama

Download the `.gguf` file and load it directly in your preferred local inference UI.

## Hardware Requirements

| Config | VRAM / RAM |
| --- | --- |
| Full GPU (Q4_K_M, recommended) | 14+ GB VRAM |
| Hybrid CPU+GPU (Q4_K_M) | 8 GB VRAM + 8 GB RAM |
| CPU only (Q4_K_M) | 16+ GB RAM |

## About the Original Model

[0xSero/gemma-4-19b-a4b-it-REAP](https://huggingface.co/0xSero/gemma-4-19b-a4b-it-REAP) applies REAP expert pruning ([arXiv:2510.13999](https://arxiv.org/abs/2510.13999)) to remove 30% of MoE experts (38 of 128 per layer) from Gemma 4 26B-A4B-it, while preserving routing behavior. Active parameters per token remain unchanged at ~4B. The result is a ~31% smaller model with near-identical generation quality across coding, math, and reasoning benchmarks.

## License

Gemma — see [Google's Gemma Terms of Use](https://ai.google.dev/gemma/terms).