---
title: kinrt-mind2act
canonical_url: "https://www.modelscope.ai/models/iamzwxyy/kinrt-mind2act"
md_url: "https://www.modelscope.ai/models/iamzwxyy/kinrt-mind2act.md"
repository: iamzwxyy/kinrt-mind2act
last_updated: 2026-11-06
license: other
downloads: 0
stars: 0
tags:
  - robotics
  - vision-language-action
  - kinrt
  - mixture-of-experts
  - mind2act
  - jax
---

# kinrt-mind2act

> kinrt-mind2act - An open-source model by iamzwxyy on ModelScope. KinRT Mind2Act final non-EMA inference weights, four experts, step264305

iamzwxyy/kinrt-mind2act is a machine learning model on ModelScope. licensed under other.

- **Repository**: iamzwxyy/kinrt-mind2act
- **License**: other
- **Tags**: robotics, vision-language-action, kinrt, mixture-of-experts, mind2act, jax
- **Downloads**: 0
- **Stars**: 0
- **Last updated**: 2026-11-06

Source: https://www.modelscope.ai/models/iamzwxyy/kinrt-mind2act

---

# KinRT · Mind2Act bench30

Final trained **KinRT / Pi0.5** inference weights for **ARX X5 absolute joint control**. This release contains the four MoE experts, the learned observation-conditioned router, and the normalization statistics used during training. The training configuration does **not use parameter EMA**; these are the final optimizer-updated parameters at step **264305**.

## Model and training

| Item | Value |
| --- | --- |
| Implementation | [KinRT](https://github.com/gleeacast/KinRT), revision `590d52802cde804cdc2d0ccb672c1a3a90d76f91` |
| Base initialization | `gs://openpi-assets/checkpoints/pi05_base` |
| Final optimizer step | 264305 |
| Training budget | 8.26 equivalent epochs |
| Data | Mind2Act bench30: 30 tasks, 90 task/difficulty groups, 4500 episodes, 8191528 frames, 30 Hz |
| Global batch | 256, trained on 32 GPUs |
| Experts / routing | Four experts, global Top-1 router, pooled observation input |
| Router supervision coefficient | 0.05 |
| EMA | Disabled |
| Inputs | Three RGB cameras, 14D robot state, English instruction |
| Output | 50 × 14 absolute joint/gripper commands |
| Token limit | 256 |
| Learning rate | 1000-step warmup, peak2.5e-5, cosine to2.5e-6 |

The twelve arm joints were trained as deltas relative to the current state, while grippers remained absolute. The supplied inference loader performs the corresponding output conversion and returns **absolute commands**. Do not convert those outputs to absolute coordinates again. `adapt_to_pi=False` matches training.

Router targets were fitted for this Mind2Act dataset using `chunk_velocity` features over50-step absolute-action chunks, a uniform200,000-frame fitting subset, PCA64 and four-class KMeans (`n_init=20`,seed0). Labels were then predicted for the full dataset. Frame counts were947827,4508829,2091317,643555; training sampled with inverse-square-root class weighting. **No clustering labels or future actions are required for inference**: the learned router runs from the current observation/state.

## Files

```text
264305/
  params/                         # Native JAX Orbax/Zarr v2 parameters, including router/experts
  assets/mind2act30/norm_stats.json
inference.py                      # Matching model configuration and transforms
model_config.json
inference_environment.json
parameter_validation.json
release_validation.json
SHA256SUMS
LICENSE.md                        # Gemma Terms of Use for derived weights
LICENSE_OPENPI                    # Apache-2.0 code license
NOTICE
```

Optimizer state and raw training data are excluded. Download the complete tree, including hidden `.zarray` files and Orbax metadata. This is an inference release, not a checkpoint for optimizer-state training resumption, and not a Transformers `save_pretrained` directory.

## Download

```bash
pip install modelscope-hub
```

```python
from modelscope_hub import HubApi
HubApi(endpoint="https://modelscope.ai").download_repo(
    "iamzwxyy/kinrt-mind2act", "model", local_dir="./kinrt-mind2act"
)
```

Verify from the downloaded model directory:

```bash
sha256sum -c SHA256SUMS
```

## Environment and inference

Install the **KinRT OpenPI fork** at the revision above:

```bash
git clone https://github.com/gleeacast/KinRT.git
cd KinRT
git checkout 590d52802cde804cdc2d0ccb672c1a3a90d76f91
cd policy/pi05
uv sync --frozen
source .venv/bin/activate
```

Validated packages are recorded in `inference_environment.json`: Python3.11, JAX/JAXlib0.5.0, Flax0.10.2, Orbax0.11.1, TensorStore0.1.74. Use a CUDA-compatible NVIDIA GPU; inference validation used an A10080GB. ModelScope provides file hosting; model execution uses KinRT locally.

In that environment, run the following from the downloaded `kinrt-mind2act` directory:

```python
import numpy as np
from inference import load_policy

policy = load_policy(".")
observation = {
    "images": {
        "cam_high": head_rgb.transpose(2, 0, 1),
        "cam_left_wrist": left_wrist_rgb.transpose(2, 0, 1),
        "cam_right_wrist": right_wrist_rgb.transpose(2, 0, 1),
    },
    "state": np.asarray(robot_state_14d, dtype=np.float32),
    "prompt": instruction,
}
actions = policy.infer(observation)["actions"]  # (50, 14), absolute commands
```

Provide HWC `uint8` RGB camera arrays before the transposes shown above. State/output ordering is left-arm six joints, left gripper, right-arm six joints, right gripper. Joint coordinates follow the training robot convention; grippers use0=closed,1=open. The tokenizer may be downloaded by OpenPI on first use.

When reading XPolicyLab HDF5, use its official `decode_image_bit` helper to obtain RGB: legacy and marked JPEG encodings coexist in the dataset. Do not add an unconditional channel swap.

## Validation and provenance

All62 parameter arrays were read and checked for finite values; all286 expected Zarr data chunks exist. Final training counters are264305 and normalization matches the training assets. A real Mind2Act observation passed through the released loader on GPU and produced a finite50×14 action sequence. See the validation JSON files for scope and results. No held-out task-success score is claimed, and the original training checkpoint remains preserved.

The weights are modified from Pi0.5 by KinRT MoE initialization and Mind2Act fine-tuning. The per-file SHA-256 manifest identifies the exact released parameters.

The PaliGemma/Gemma-derived weights are subject to the **Gemma Terms of Use**, including Section3.2 restrictions and the incorporated [Gemma Prohibited Use Policy](https://ai.google.dev/gemma/prohibited_use_policy). A copy of the terms is included in `LICENSE.md`; subsequent users must comply with those restrictions. KinRT's OpenPI fork retains its Apache-2.0 code license. The custom license label reflects the inherited weight terms rather than labeling every component as Apache-only.

Attribution: [KinRT authors](https://github.com/gleeacast/KinRT), [Physical Intelligence OpenPI](https://github.com/Physical-Intelligence/openpi), and Mind2Act bench30 fine-tuning.