---
title: ZwZ-4B-GGUF
canonical_url: "https://www.modelscope.ai/models/inclusionAI/ZwZ-4B-GGUF"
md_url: "https://www.modelscope.ai/models/inclusionAI/ZwZ-4B-GGUF.md"
repository: inclusionAI/ZwZ-4B-GGUF
last_updated: 2026-03-07
license: apache-2.0
pipeline_tag: image-text-to-text
tasks:
  - image-text-to-text
base_model:
  - inclusionAI/ZwZ-4B
base_model_relation: quantized
library_name:
  - gguf
  - pytorch
frameworks:
  - pytorch
language:
  - en
downloads: 208
stars: 0
---

# ZwZ-4B-GGUF

> ZwZ-4B-GGUF - An open-source model by inclusionAI on ModelScope. This repository provides GGUF-format weights for ZwZ-4B, split into two components:

inclusionAI/ZwZ-4B-GGUF is a image-text-to-text model on ModelScope. licensed under apache-2.0. derived from inclusionAI/ZwZ-4B.

- **Repository**: inclusionAI/ZwZ-4B-GGUF
- **License**: apache-2.0
- **Tasks**: image-text-to-text
- **Base model**: inclusionAI/ZwZ-4B
- **Downloads**: 208
- **Stars**: 0
- **Last updated**: 2026-03-07

Source: https://www.modelscope.ai/models/inclusionAI/ZwZ-4B-GGUF

---

**ZwZ-4B-GGUF**

This repository provides GGUF-format weights for [ZwZ-4B](https://huggingface.co/inclusionAI/ZwZ-4B), split into two components:

 - Language model (LLM): FP16, Q8_0, Q4_K_M
 - Vision encoder (mmproj): FP16, Q8_0, Q4_K_M

These files are compatible with llama.cpp, Ollama, and other GGUF-based tools, supporting inference on CPU, NVIDIA GPU (CUDA), Apple Silicon (Metal), Intel GPUs (SYCL), and more. You can mix precision levels for the language and vision components based on your hardware and performance needs, and even perform custom quantization starting from the FP16 weights.

Enjoy running this multimodal model on your personal device! 🚀


## How to Use

To use these models with `llama.cpp`, please ensure you are using the **latest version**—either by [building from source](https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md) or downloading the most recent [release](https://github.com/ggml-org/llama.cpp/releases/tag/b6907) according to the devices.

You can run inference via the command line or through a web-based chat interface.

### CLI Inference (`llama-mtmd-cli`)

For example, to run ZwZ-4B with an Q8_0  vision encoder and Q8_0 quantized LLM:
```bash
llama-mtmd-cli \
  -m path/to/ZwZ-4B-Q8_0.gguf \
  --mmproj mmproj-ZwZ-4B-Q8_0.gguf\
  --image test.jpeg \
  -p "What is the publisher name of the newspaper?" \
  --temp 1.0 --top-k 20 --top-p 0.95 -n 1024
```
### Web Chat (using `llama-server`)
To serve ZwZ-4B via an OpenAI-compatible API with a web UI:
```bash
llama-server \
  -m path/to/ZwZ-4B-Q8_0.gguf  \
  --mmproj mmproj-ZwZ-4B-Q8_0.gguf
```

## Citation

```bibtex
@article{wei2026zooming,
  title={Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception},
  author={Wei, Lai and He, Liangbo and Lan, Jun and Dong, Lingzhong and Cai, Yutong and Li, Siyuan and Zhu, Huijia and Wang, Weiqiang and Kong, Linghe and Wang, Yue and Zhang, Zhuosheng and Huang, Weiran},
  journal={arXiv preprint arXiv:2602.11858},
  year={2026}
}
```

## License

This model follows the license of Apache 2.0 License.