--- title: Qwen3.5-4B-int4-ov canonical_url: "https://www.modelscope.ai/models/OpenVINO/Qwen3.5-4B-int4-ov" md_url: "https://www.modelscope.ai/models/OpenVINO/Qwen3.5-4B-int4-ov.md" repository: OpenVINO/Qwen3.5-4B-int4-ov last_updated: 2026-09-26 license: apache-2.0 pipeline_tag: image-text-to-text tasks: - image-text-to-text model_type: - qwen3_5 architectures: - Qwen3_5ForConditionalGeneration base_model: - Qwen/Qwen3.5-4B base_model_relation: quantized library_name: - transformer - openvino - pytorch frameworks: - pytorch downloads: 143 stars: 0 tags: - qwen3_5 - conversational --- # Qwen3.5-4B-int4-ov > Qwen3.5-4B-int4-ov - An open-source model by OpenVINO on ModelScope. Model creator: Qwen Original model: Qwen3.5-4B OpenVINO/Qwen3.5-4B-int4-ov is a image-text-to-text model on ModelScope. licensed under apache-2.0. derived from Qwen/Qwen3.5-4B. - **Repository**: OpenVINO/Qwen3.5-4B-int4-ov - **License**: apache-2.0 - **Tasks**: image-text-to-text - **Base model**: Qwen/Qwen3.5-4B - **Tags**: qwen3_5, conversational - **Downloads**: 143 - **Stars**: 0 - **Last updated**: 2026-09-26 Source: https://www.modelscope.ai/models/OpenVINO/Qwen3.5-4B-int4-ov --- # Qwen3.5-4B-int4-ov * Model creator: [Qwen](https://huggingface.co/Qwen) * Original model: [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) ## Description This is [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) model converted to the [OpenVINO™ IR](https://docs.openvino.ai/2026/documentation/openvino-ir-format.html) (Intermediate Representation) format with weights compressed to INT4 by [NNCF](https://github.com/openvinotoolkit/nncf). The model includes `openvino_mtp_model.xml` for built-in Multi-Token Prediction (MTP) speculative decoding. ## Quantization Parameters Weight compression was performed using `nncf.compress_weights` with the following parameters: * mode: **INT4_ASYM** * group_size: **128** * ratio: **1.0** For more information on quantization, check the [OpenVINO model optimization guide](https://docs.openvino.ai/2026/openvino-workflow/model-optimization-guide/weight-compression.html). ## Compatibility The provided OpenVINO™ IR model is compatible with: * OpenVINO version 2026.4.0 and higher * OpenVINO GenAI 2026.4.0.0 and higher * Optimum Intel 1.27.0 and higher ## Running Model Inference with [Optimum Intel](https://huggingface.co/docs/optimum/intel/index) 1. Install packages required for using [Optimum Intel](https://huggingface.co/docs/optimum/intel/index) integration with the OpenVINO backend: ``` pip install -U "git+https://github.com/huggingface/optimum-intel.git" torchvision Pillow --extra-index-url https://download.pytorch.org/whl/cpu pip install -U "openvino==2026.4.0" pip install -U "transformers==5.2.0" ``` 2. Run model inference: ```python import requests from PIL import Image from transformers import AutoProcessor from optimum.intel.openvino import OVModelForVisualCausalLM model_id = "OpenVINO/Qwen3.5-4B-int4-ov" processor = AutoProcessor.from_pretrained(model_id) model = OVModelForVisualCausalLM.from_pretrained(model_id) url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/ai2d-demo.jpg" image = Image.open(requests.get(url, stream=True).raw) messages = [ { "role": "user", "content": [ {"type": "image"}, {"type": "text", "text": "Describe this image."}, ], } ] text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) inputs = processor(text=[text], images=[image], return_tensors="pt") outputs = model.generate(**inputs, max_new_tokens=200) print(processor.batch_decode(outputs[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)[0]) ``` For more examples and possible optimizations, refer to the [Inference with Optimum Intel](https://docs.openvino.ai/2026/openvino-workflow-generative/inference-with-optimum-intel.html). ## Running Model Inference with [OpenVINO GenAI](https://github.com/openvinotoolkit/openvino.genai) 1. Install packages required for using OpenVINO GenAI. ``` pip install huggingface_hub "Pillow" pip install -U "openvino==2026.4.0" "openvino-tokenizers==2026.4.0.0" "openvino-genai==2026.4.0.0" ``` 2. Download model from HuggingFace Hub ```python import huggingface_hub as hf_hub model_id = "OpenVINO/Qwen3.5-4B-int4-ov" model_path = "Qwen3.5-4B-int4-ov" hf_hub.snapshot_download(model_id, local_dir=model_path) ``` 3. Run model inference: ```python import numpy as np import openvino as ov import openvino_genai as ov_genai import requests from PIL import Image device = "CPU" scheduler_config = ov_genai.SchedulerConfig() scheduler_config.enable_prefix_caching = False draft = ov_genai.draft_model(model_path, device) pipe = ov_genai.VLMPipeline( model_path, device, draft_model=draft, scheduler_config=scheduler_config, ) url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/ai2d-demo.jpg" image = Image.open(requests.get(url, stream=True).raw).convert("RGB") image_tensor = ov.Tensor(np.array(image)[None]) generation_config = ov_genai.GenerationConfig() generation_config.max_new_tokens = 200 generation_config.do_sample = False generation_config.num_return_sequences = 1 generation_config.num_assistant_tokens = 2 print(pipe.generate("Describe this image.", image=image_tensor, generation_config=generation_config)) ``` OpenVINO GenAI currently supports this MTP path with greedy decoding and prefix caching disabled, as demonstrated in the [Qwen3.8 MTP notebook](https://openvinotoolkit.github.io/openvino_notebooks/?search=qwen3.8-mtp). More GenAI usage examples can be found in OpenVINO GenAI library [docs](https://docs.openvino.ai/2026/openvino-workflow-generative/inference-with-genai.html) and [samples](https://github.com/openvinotoolkit/openvino.genai?tab=readme-ov-file#openvino-genai-samples) You can find more detailed usage examples in OpenVINO Notebooks: - [Qwen3-VL multimodal chatbot](https://openvinotoolkit.github.io/openvino_notebooks/?search=qwen3-vl) - [Visual-language assistant](https://openvinotoolkit.github.io/openvino_notebooks/?tasks=Image-to-Text) ## Limitations Check the original [model card](https://huggingface.co/Qwen/Qwen3.5-4B) for limitations. ## Legal information The original model is distributed under [Apache License Version 2.0](https://huggingface.co/Qwen/Qwen3.5-4B/blob/main/LICENSE) license. More details can be found in [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B). ## Disclaimer Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See [Intel's Global Human Rights Principles](https://www.intel.com/content/dam/www/central-libraries/us/en/documents/policy-human-rights.pdf). Intel's products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.