--- title: MiniMax-H3-Turbo-Lora-Diffusers canonical_url: "https://www.modelscope.ai/models/InstantX/MiniMax-H3-Turbo-Lora-Diffusers" md_url: "https://www.modelscope.ai/models/InstantX/MiniMax-H3-Turbo-Lora-Diffusers.md" repository: InstantX/MiniMax-H3-Turbo-Lora-Diffusers last_updated: 2026-08-07 license: apache-2.0 pipeline_tag: text-to-video-synthesis tasks: - text-to-video-synthesis base_model: - MiniMaxAI/MiniMax-H3 base_model_relation: adapter parameters: 851.4M tensor_type: - BF16 library_name: - pytorch - lora - diffusers - peft - safetensors frameworks: - pytorch downloads: 115 stars: 0 tags: - text-to-video - text-to-audio - audio-video - lora - minimax-h3 - diffusers - peft --- # MiniMax-H3-Turbo-Lora-Diffusers > MiniMax-H3-Turbo-Lora-Diffusers - An open-source model by InstantX on ModelScope. MiniMax-H3 Turbo LoRA — Diffusers InstantX/MiniMax-H3-Turbo-Lora-Diffusers is a 851.4M-parameter text-to-video-synthesis model on ModelScope. licensed under apache-2.0. derived from MiniMaxAI/MiniMax-H3. - **Repository**: InstantX/MiniMax-H3-Turbo-Lora-Diffusers - **License**: apache-2.0 - **Tasks**: text-to-video-synthesis - **Parameters**: 851.4M - **Base model**: MiniMaxAI/MiniMax-H3 - **Tags**: text-to-video, text-to-audio, audio-video, lora, minimax-h3, diffusers, peft - **Downloads**: 115 - **Stars**: 0 - **Last updated**: 2026-08-07 Source: https://www.modelscope.ai/models/InstantX/MiniMax-H3-Turbo-Lora-Diffusers --- # MiniMax-H3 Turbo LoRA — Diffusers Diffusers / PEFT conversion of [`larryvrh/MiniMax-H3-Turbo-Lora`](https://huggingface.co/larryvrh/MiniMax-H3-Turbo-Lora): a LoRA that lets [MiniMax-H3](https://huggingface.co/MiniMaxAI/MiniMax-H3) render joint **video + synchronized stereo audio** in about **4 sampling steps** instead of the usual ~20. This repo ships: - converted LoRA weights in Diffusers PEFT layout (`transformer.*.lora_A/B.weight`) - `convert.py` to turn the original ComfyUI / `generate.py` safetensors into that layout > ⚠️ **Early prototype.** Same caveat as the upstream release: under-trained preview weights, not production quality. They already beat the base model at 4 steps (sharper detail, cleaner / better-synced audio), but treat this as a work-in-progress taste, not a finished product. Prefer the non-EMA `ckpt500` weights by default. ## Weights Converted from the upstream Turbo LoRA (bf16, `W_eff = W + lora_B @ lora_A`, **alpha = rank** so scale is 1). QKV is split into `to_q` / `to_k` / `to_v`, and SwiGLU `fc1` halves are swapped to match Diffusers' `[value; gate]` layout. | file | source (ComfyUI layout) | notes | |---|---|---| | `minimax_h3_turbo_4step_ckpt500_diffusers.safetensors` | `minimax_h3_turbo_4step_ckpt500.safetensors` | **recommended default** — newest non-EMA @ ~500 steps, usually sharpest | | `minimax_h3_turbo_4step_ema_ckpt500_diffusers.safetensors` | `minimax_h3_turbo_4step_ema_ckpt500.safetensors` | EMA @ ~500 steps — smoother, but early EMA can show **ghosting / motion smear** | Ranks: attention / MLP = 64, AdaLN = 16. Keys are prefixed with `transformer.` for `MiniMaxH3Transformer3DModel.load_lora_adapter`. ## Requirements MiniMax-H3 is not in a released Diffusers build yet. Install Diffusers from `main`, plus PEFT: ```bash pip install torch torchvision --index-url https://download.pytorch.org/whl/cu126 pip install -r requirements.txt pip install git+https://github.com/huggingface/diffusers.git ``` Base weights: Diffusers-format [MiniMaxAI/MiniMax-H3](https://huggingface.co/MiniMaxAI/MiniMax-H3) (or your local conversion). Use a **non-pruned** DiT; pruned time-conditioning layouts are **not** compatible with this LoRA (same restriction as upstream). ## Quick start (Diffusers) ```python import torch from diffusers import ComponentsManager, ModularPipeline from diffusers.utils.export_utils import encode_video from huggingface_hub import hf_hub_download from safetensors.torch import load_file def network_alphas_alpha_eq_rank(state_dict): # Turbo LoRA: alpha == rank. Required when ranks differ (attn/mlp=64, adaln=16). alphas = {} for key, tensor in state_dict.items(): if key.endswith(".lora_B.weight") and tensor.ndim > 1: base = key[: -len(".lora_B.weight")] alphas[f"{base}.alpha"] = float(tensor.shape[1]) return alphas lora_path = hf_hub_download( "InstantX/MiniMax-H3-Turbo-Lora-Diffusers", "minimax_h3_turbo_4step_ckpt500_diffusers.safetensors", ) manager = ComponentsManager() pipe = ModularPipeline.from_pretrained("MiniMaxAI/MiniMax-H3", components_manager=manager) pipe.load_components(dtype=torch.bfloat16) lora_sd = load_file(lora_path, device="cpu") pipe.transformer.load_lora_adapter( lora_sd, prefix="transformer", adapter_name="turbo_4step", network_alphas=network_alphas_alpha_eq_rank(lora_sd), ) # Load LoRA *before* enabling offload so PEFT injects into resident modules. manager.enable_auto_cpu_offload(device="cuda", memory_reserve_margin="12GB") # Optional: FlashAttention-3 on Hopper (kernels from the Hub). try: pipe.transformer.set_attention_backend("_flash_3_hub") except Exception: pipe.transformer.set_attention_backend("native") # MiniMaxH3Scheduler: num_inference_steps is the sigma grid length *including* terminal 0, # so it drives (num_inference_steps - 1) model evals. # 5 -> 4 evals (matches upstream generate.py --steps 4) # 7–9 -> 6–8 evals (upstream comfort zone for sharpness at this early checkpoint) results = pipe( prompt="A corgi in a chef hat flipping a pancake, sizzling sounds and a cheerful bark.", num_frames=124, # 17*k+5, ~5.17s @ 24fps height=768, width=1344, num_inference_steps=5, generator=torch.Generator().manual_seed(42), output=["videos", "audio", "sampling_rate"], ) encode_video( results["videos"][0], fps=24, output_path="out.mp4", audio=results["audio"][0], audio_sample_rate=results["sampling_rate"], ) ``` Diffusers already runs **dual video / audio schedules** (`scheduler` shift 12, `audio_scheduler` shift 3). You do **not** need the ComfyUI Turbo custom sampler node; a wrong single-schedule sampler is what blows up audio at 4 steps in ComfyUI. ## Convert from the original Turbo LoRA Original weights live in [`larryvrh/MiniMax-H3-Turbo-Lora`](https://huggingface.co/larryvrh/MiniMax-H3-Turbo-Lora) (ComfyUI module names, fused `qkv_proj` / `mlp.fc1`). ```bash pip install safetensors torch python convert.py \ --input minimax_h3_turbo_4step_ckpt500.safetensors \ --output minimax_h3_turbo_4step_ckpt500_diffusers.safetensors ``` What `convert.py` does: 1. Renames ComfyUI paths onto `MiniMaxH3Transformer3DModel` (`blocks.*` → `transformer_blocks.*`, `mlp.fc*` → `ff.net.*`, `final_layer.adaln_proj` → `norm_out.linear`, …). 2. Splits fused `attn.qkv_proj` LoRA into `to_q` / `to_k` / `to_v` (shared `A`, row-split `B` in `[q_all; k_all; v_all]` layout). 3. Swaps `mlp.fc1` LoRA halves from `[gate; value]` to Diffusers SwiGLU `[value; gate]`. 4. Writes keys with a `transformer.` prefix for `load_lora_adapter`. ## Notes - **Steps**: 4 model evals (`num_inference_steps=5`) works; at this early checkpoint **6–8 evals** (`num_inference_steps=7…9`) are usually sharper. Any count ≥ 4 evals is valid; more steps look better. - **Resolution / duration**: `height` / `width` multiples of 32 (short edge typically 768). `num_frames` at 24 fps snaps up to the video VAE’s `17·k+5` grid (124 ≈ 5 s). Validated roughly 5–15 s. - **VRAM**: the base DiT is ~33B. An 80–96 GB GPU is comfortable with `ComponentsManager.enable_auto_cpu_offload`; smaller cards need quantization / group offload as in the [MiniMax-H3 Diffusers docs](https://huggingface.co/docs/diffusers/main/en/api/pipelines/minimax_h3). - **Audio**: 32 kHz stereo aligned to the video; video and audio ride different flow schedules inside one transformer call. - **ComfyUI**: for the original graph / custom Turbo sampler, use the [upstream repo](https://huggingface.co/larryvrh/MiniMax-H3-Turbo-Lora) and [Larryvrh/ComfyUI-MiniMax-H3-Turbo](https://github.com/Larryvrh/ComfyUI-MiniMax-H3-Turbo). ## Credit - Turbo LoRA training & original release: [`larryvrh/MiniMax-H3-Turbo-Lora`](https://huggingface.co/larryvrh/MiniMax-H3-Turbo-Lora) - Base model: [`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3) - Diffusers MiniMax-H3 integration: Hugging Face Diffusers (modular pipeline)