---
title: Kimi-K3-NVFP4
canonical_url: "https://www.modelscope.ai/models/RedHatAI/Kimi-K3-NVFP4"
md_url: "https://www.modelscope.ai/models/RedHatAI/Kimi-K3-NVFP4.md"
repository: RedHatAI/Kimi-K3-NVFP4
last_updated: 2026-09-17
license: other
pipeline_tag: image-text-to-text
tasks:
  - image-text-to-text
model_type:
  - kimi_k3
architectures:
  - KimiK3ForConditionalGeneration
base_model:
  - moonshotai/Kimi-K3
base_model_relation: quantized
parameters: 1588.7B
tensor_type:
  - F8_E4M3
  - U8
  - BF16
  - F32
library_name:
  - safetensors
  - pytorch
frameworks:
  - pytorch
downloads: 84
stars: 1
tags:
  - nvfp4
  - fp4
  - vllm
  - llm-compressor
  - compressed-tensors
---

# Kimi-K3-NVFP4

> Kimi-K3-NVFP4 - An open-source model by RedHatAI on ModelScope. Model Overview Model Architecture: KimiK3ForConditionalGeneration Input: Text / Image Output: Text Model Optimizations: Weight quantization: FP4 Activation quantization: FP4 Release Date:…

RedHatAI/Kimi-K3-NVFP4 is a 1588.7B-parameter image-text-to-text model on ModelScope. licensed under other. derived from moonshotai/Kimi-K3.

- **Repository**: RedHatAI/Kimi-K3-NVFP4
- **License**: other
- **Tasks**: image-text-to-text
- **Parameters**: 1588.7B
- **Base model**: moonshotai/Kimi-K3
- **Tags**: nvfp4, fp4, vllm, llm-compressor, compressed-tensors
- **Downloads**: 84
- **Stars**: 1
- **Last updated**: 2026-09-17

Source: https://www.modelscope.ai/models/RedHatAI/Kimi-K3-NVFP4

---

# Kimi-K3-NVFP4

## Model Overview
- **Model Architecture:** KimiK3ForConditionalGeneration
  - **Input:** Text / Image
  - **Output:** Text
- **Model Optimizations:**
  - **Weight quantization:** FP4
  - **Activation quantization:** FP4
- **Release Date:** 2026-07-30
- **Version:** 1.0
- **Model Developers:** RedHatAI

This model is a quantized version of [moonshotai/Kimi-K3](https://huggingface.co/moonshotai/Kimi-K3).
It was evaluated on several tasks to assess its quality in comparison to the unquantized model.

### Model Optimizations

This model was obtained by quantizing the MoE expert weights and activations of [moonshotai/Kimi-K3](https://huggingface.co/moonshotai/Kimi-K3) to FP4 (NVFP4) data type, ready for inference with vLLM. Attention, shared experts, and other non-quantized layers keep their original precision.

This optimization reduces the number of bits per parameter in the quantized MoE layers from 16 to 4, reducing the disk size and GPU memory requirements of those layers by approximately 75%.

Only the weights and activations of the linear operators in the MoE experts are quantized using [LLM Compressor](https://github.com/vllm-project/llm-compressor).

## Deployment

### vLLM Serving

```bash
vllm serve RedHatAI/Kimi-K3-NVFP4 \
  --tensor-parallel-size 8 \
  --trust_remote_code \
  --load-format instanttensor \
  --reasoning-parser kimi_k3 \
  --language-model-only  # optional
```

This model requires [vllm-project/vllm#50500](https://github.com/vllm-project/vllm/pull/50500).

## Creation

This model was created by applying [LLM Compressor](https://github.com/vllm-project/llm-compressor) with the NVFP4 scheme, exported in compressed-tensors format.

## Evaluation

This model was evaluated on GPQA Diamond, served with vLLM (OpenAI-compatible API). Recovery is computed against the unquantized model.

### Accuracy

<table>
  <thead>
    <tr>
      <th>Category</th>
      <th>Benchmark</th>
      <th>moonshotai/Kimi-K3</th>
      <th>RedHatAI/Kimi-K3-NVFP4</th>
      <th>Recovery</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><b>Reasoning</b></td>
      <td>GPQA Diamond</td>
      <td>93.5</td>
      <td>91.0</td>
      <td>97.33%</td>
    </tr>
  </tbody>
</table>