---
title: GLM-5.2-MXFP4xMXFP8
canonical_url: "https://www.modelscope.ai/models/RedHatAI/GLM-5.2-MXFP4xMXFP8"
md_url: "https://www.modelscope.ai/models/RedHatAI/GLM-5.2-MXFP4xMXFP8.md"
repository: RedHatAI/GLM-5.2-MXFP4xMXFP8
last_updated: 2026-09-17
pipeline_tag: text-generation
tasks:
  - text-generation
model_type:
  - glm_moe_dsa
architectures:
  - GlmMoeDsaForCausalLM
base_model:
  - zai-org/GLM-5.2
base_model_relation: quantized
library_name:
  - safetensors
  - pytorch
frameworks:
  - pytorch
language:
  - en
  - zh
downloads: 92
stars: 0
tags:
  - mxfp4
  - mxfp8
  - vllm
  - llm-compressor
  - compressed-tensors
---

# GLM-5.2-MXFP4xMXFP8

> GLM-5.2-MXFP4xMXFP8 - An open-source model by RedHatAI on ModelScope. Model Overview Model Architecture: GlmMoeDsaForCausalLM Input: Text Output: Text Model Optimizations: Weight quantization: MXFP4 Activation quantization: MXFP8 Release Date: 2026-09-15…

RedHatAI/GLM-5.2-MXFP4xMXFP8 is a text-generation model on ModelScope. derived from zai-org/GLM-5.2.

- **Repository**: RedHatAI/GLM-5.2-MXFP4xMXFP8
- **Tasks**: text-generation
- **Base model**: zai-org/GLM-5.2
- **Tags**: mxfp4, mxfp8, vllm, llm-compressor, compressed-tensors
- **Downloads**: 92
- **Stars**: 0
- **Last updated**: 2026-09-17

Source: https://www.modelscope.ai/models/RedHatAI/GLM-5.2-MXFP4xMXFP8

---

# GLM-5.2-MXFP4xMXFP8

## Model Overview
- **Model Architecture:** GlmMoeDsaForCausalLM
  - **Input:** Text
  - **Output:** Text
- **Model Optimizations:**
  - **Weight quantization:** MXFP4
  - **Activation quantization:** MXFP8
- **Release Date:** 2026-09-15
- **Version:** 1.0
- **Model Developers:** RedHatAI

This model is a quantized version of [zai-org/GLM-5.2](https://huggingface.co/zai-org/GLM-5.2).

### Model Optimizations

This model was obtained by quantizing the MoE linears (experts and shared experts) of [zai-org/GLM-5.2](https://huggingface.co/zai-org/GLM-5.2) to 4-bit MXFP4 weights with dynamic 8-bit MXFP8 activation scaling, ready for inference with vLLM.

This optimization reduces the number of bits per parameter from 16 to an average of ~4.5 bits, shrinking the on-disk model size from ~1.47 TB (BF16) to ~424 GB and reducing disk and GPU memory requirements by approximately 72%.

Only the linear operators within transformer blocks are quantized using [LLM Compressor](https://github.com/vllm-project/llm-compressor).

Note: this quantized checkpoint is experimentally supported in vLLM.

## Deployment

### Use with vLLM

```
vllm serve RedHatAI/GLM-5.2-MXFP4xMXFP8 \
  --tensor-parallel-size 8 \
  --kv-cache-dtype fp8_e4m3 \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 5}' \
  --tool-call-parser glm47 \
  --reasoning-parser glm45 \
  --enable-auto-tool-choice
```

## Creation

This model was created by applying [LLM Compressor](https://github.com/vllm-project/llm-compressor) with the quantization scheme described above (RTN, no calibration data), exported in compressed-tensors format.