--- title: spotter-robotwin-50x10 canonical_url: "https://www.modelscope.ai/datasets/seamoke/spotter-robotwin-50x10" md_url: "https://www.modelscope.ai/datasets/seamoke/spotter-robotwin-50x10.md" repository: seamoke/spotter-robotwin-50x10 last_updated: 2026-10-13 license: Apache-2.0 storage_size: "26 GB" downloads: 47 stars: 0 --- # spotter-robotwin-50x10 > spotter-robotwin-50x10 - An open-source dataset by seamoke on ModelScope. Spotter on RoboTwin 2.0 — 50 tasks × 10 episodes seamoke/spotter-robotwin-50x10 is a dataset on ModelScope. totalling 26 GB. licensed under Apache-2.0. - **Repository**: seamoke/spotter-robotwin-50x10 - **License**: Apache-2.0 - **Storage size**: 26 GB - **Downloads**: 47 - **Stars**: 0 - **Last updated**: 2026-10-13 Source: https://www.modelscope.ai/datasets/seamoke/spotter-robotwin-50x10 --- # Spotter on RoboTwin 2.0 — 50 tasks × 10 episodes Policy-only control vs. Spotter with a Qwen judge, at two repair-turn thinking budgets, on the same 500 RoboTwin 2.0 episodes. | Arm | Success | Rate | |---|---:|---:| | Control (pi0.5 alone) | 240 / 500 | 48.0% | | Spotter-Qwen, thinking budget 2000 tokens | 252 / 500 | 50.4% | | Spotter-Qwen, thinking budget 5000 tokens | 267 / 500 | 53.4% | 180 (budget 2000) and 160 (budget 5000) of the 500 Spotter episodes had at least one intervention. ## Setup - **Simulator:** RoboTwin 2.0, aloha-agilex dual arm, `demo_randomized` scenes, 50 tasks × 10 episodes. Each episode maps to a fixed RoboTwin seed (column `robotwin_seed`); all three arms use the same seed per episode. - **Policy:** pi0.5, checkpoint `motus-robotics/pi0.5_robotwin2` (train config `pi05_robotwin2_clean_randomized`, step 40000). - **Step budget:** 1.8 × the official per-task limit for every arm. No rollbacks or resets. - **Judge:** Qwen3.8-27B-FP8. Screener and window verdicts do not think; repair turns think up to the budget given (`QWEN_THINK_BUDGET` = 2000 or 5000). Window of 2 policy chunks, screener every window, at most 5 interventions per episode, judge history 10 turns, context compaction at 40k tokens, 1-shot example from the RoboTwin example bank. - **Code:** github.com/zqc3117/Spotter (`bash spotter.sh run robotwin --arm treat --one-shot` for Spotter, `--arm ctrl` for the control). ## Comparing the arms pi0.5 on RoboTwin is not bit-reproducible across runs, so an episode the Spotter arm never touched can still end differently from the control. Compare the arms by aggregate success rate. ## Files | File | Content | |---|---| | `per_episode.csv` | One row per episode: task, episode, RoboTwin seed, instruction, success of each arm, interventions of the two Spotter arms | | `per_task.csv` | Successes per task for each arm | | `control_results.jsonl` | Raw driver result rows of the control arm | | `qwen_think2000_results.jsonl`, `qwen_think5000_results.jsonl` | Raw driver result rows of the two Spotter arms, including the per-window event log | Full records (per-chunk camera frames and telemetry, the screener's and judge's prompts and replies, repair plans and their execution, simulator truth files) are in `robotwin_rt500_full.tar.part-*`: check with `md5sum -c robotwin_rt500_full.md5`, then `cat robotwin_rt500_full.tar.part-* | tar -xf -`. Inside, `control/`, `qwen_think2000/` and `qwen_think5000/` hold one folder per episode, `jd-_s0_ep/`, plus `_truth/jd-_s0_ep/` with the simulator truth (the control arm keeps only its `ctrl/` frames). **Chat histories.** In the full records, the `qwen_session_*.json` files keep every prompt and every reply of the judge, but the pictures that were embedded in them (base64) are replaced by the text `[image omitted from this archive to save space; the judge's pictures are saved in this episode folder as w/wam_montage.png]`. The same pictures are stored once as PNG files in the episode folder (`w/wam_montage.png`, plus the per-camera frames), which keeps the archive at about half the size.