---
title: Holo-3.1-35B-A3B-GGUF
canonical_url: "https://www.modelscope.ai/models/barozp/Holo-3.1-35B-A3B-GGUF"
md_url: "https://www.modelscope.ai/models/barozp/Holo-3.1-35B-A3B-GGUF.md"
repository: barozp/Holo-3.1-35B-A3B-GGUF
last_updated: 2026-06-10
license: apache-2.0
pipeline_tag: image-text-to-text
tasks:
  - image-text-to-text
base_model:
  - Hcompany/Holo-3.1-35B-A3B
base_model_relation: quantized
library_name:
  - gguf
  - pytorch
frameworks:
  - pytorch
language:
  - en
downloads: 166
stars: 0
tags:
  - gguf
  - quantized
  - llama.cpp
  - qwen3.5
  - moe
  - vision-language-model
  - computer-use
  - agent
---

# Holo-3.1-35B-A3B-GGUF

> Holo-3.1-35B-A3B-GGUF - An open-source model by barozp on ModelScope. Holo-3.1-35B-A3B — GGUF Quantizations

barozp/Holo-3.1-35B-A3B-GGUF is a image-text-to-text model on ModelScope. licensed under apache-2.0. derived from Hcompany/Holo-3.1-35B-A3B.

- **Repository**: barozp/Holo-3.1-35B-A3B-GGUF
- **License**: apache-2.0
- **Tasks**: image-text-to-text
- **Base model**: Hcompany/Holo-3.1-35B-A3B
- **Tags**: gguf, quantized, llama.cpp, qwen3.5, moe, vision-language-model, computer-use, agent
- **Downloads**: 166
- **Stars**: 0
- **Last updated**: 2026-06-10

Source: https://www.modelscope.ai/models/barozp/Holo-3.1-35B-A3B-GGUF

---

# Holo-3.1-35B-A3B — GGUF Quantizations

GGUF quantizations of [Hcompany/Holo-3.1-35B-A3B](https://huggingface.co/Hcompany/Holo-3.1-35B-A3B) for use with [llama.cpp](https://github.com/ggml-org/llama.cpp).

## About the Model

- **Architecture:** Qwen3.5 MoE (35B total, ~3B active parameters)
- **Type:** Vision-Language Model (VLM) for computer use agents
- **Context:** 262,144 tokens
- **License:** Apache 2.0

> **Note:** BF16 GGUF and imatrix sourced from [Hcompany/Holo-3.1-35B-A3B-GGUF](https://huggingface.co/Hcompany/Holo-3.1-35B-A3B-GGUF).

## Available Files

| Filename | Quant | Bits | Size (est.) |
|---|---|---|---|
| `Holo-3.1-35B-A3B-Q2_K.gguf` | Q2_K | 2 | ~12 GB |
| `Holo-3.1-35B-A3B-Q3_K_S.gguf` | Q3_K_S | 3 | ~15 GB |
| `Holo-3.1-35B-A3B-Q3_K_M.gguf` | Q3_K_M | 3 | ~17 GB |
| `Holo-3.1-35B-A3B-Q3_K_L.gguf` | Q3_K_L | 3 | ~18 GB |
| `Holo-3.1-35B-A3B-IQ3_XXS.gguf` | IQ3_XXS | 3 | ~14 GB |
| `Holo-3.1-35B-A3B-IQ3_S.gguf` | IQ3_S | 3 | ~15 GB |
| `Holo-3.1-35B-A3B-Q4_K_S.gguf` | Q4_K_S | 4 | ~20 GB |
| `Holo-3.1-35B-A3B-Q4_K_M.gguf` | Q4_K_M | 4 | ~22 GB |
| `Holo-3.1-35B-A3B-Q5_K_S.gguf` | Q5_K_S | 5 | ~24 GB |
| `Holo-3.1-35B-A3B-Q5_K_M.gguf` | Q5_K_M | 5 | ~26 GB |
| `Holo-3.1-35B-A3B-Q6_K.gguf` | Q6_K | 6 | ~30 GB |
| `Holo-3.1-35B-A3B-Q8_0.gguf` | Q8_0 | 8 | ~39 GB |
| `mmproj-Holo-3.1-35B-A3B-F16.gguf` | F16 | 16 | ~0.9 GB |

> BF16 GGUF (69.4 GB) available at [source repo](https://huggingface.co/Hcompany/Holo-3.1-35B-A3B-GGUF).

## Usage

```bash
# Text + Vision inference
huggingface-cli download barozp/Holo-3.1-35B-A3B-GGUF Holo-3.1-35B-A3B-Q4_K_M.gguf --local-dir ./
huggingface-cli download barozp/Holo-3.1-35B-A3B-GGUF mmproj-Holo-3.1-35B-A3B-F16.gguf --local-dir ./

llama-cli \
  -m Holo-3.1-35B-A3B-Q4_K_M.gguf \
  --mmproj mmproj-Holo-3.1-35B-A3B-F16.gguf \
  --jinja \
  -ngl 99 \
  -c 32768 \
  --image screenshot.png
```

## Quantization Details

- **BF16 GGUF source:** [Hcompany/Holo-3.1-35B-A3B-GGUF](https://huggingface.co/Hcompany/Holo-3.1-35B-A3B-GGUF)
- **imatrix:** Pre-computed, sourced from BF16 repo (192 MB)
- **Tool:** [llama.cpp](https://github.com/ggml-org/llama.cpp) `llama-quantize`
- **Quantized by:** [barozp](https://huggingface.co/barozp)