--- title: ftt-dual-adapter-tad-ocr-15raw canonical_url: "https://www.modelscope.ai/models/yubodata/ftt-dual-adapter-tad-ocr-15raw" md_url: "https://www.modelscope.ai/models/yubodata/ftt-dual-adapter-tad-ocr-15raw.md" repository: yubodata/ftt-dual-adapter-tad-ocr-15raw last_updated: 2026-11-11 license: apache-2.0 library_name: - onnx downloads: 2 stars: 0 tags: - temporal-action-detection - table-tennis - video - OCR --- # ftt-dual-adapter-tad-ocr-15raw > ftt-dual-adapter-tad-ocr-15raw - An open-source model by yubodata on ModelScope. FTT raw-video dual-adapter temporal detection and scoreboard OCR reproducibility package yubodata/ftt-dual-adapter-tad-ocr-15raw is a machine learning model on ModelScope. licensed under apache-2.0. - **Repository**: yubodata/ftt-dual-adapter-tad-ocr-15raw - **License**: apache-2.0 - **Tags**: temporal-action-detection, table-tennis, video, OCR - **Downloads**: 2 - **Stars**: 0 - **Last updated**: 2026-11-11 Source: https://www.modelscope.ai/models/yubodata/ftt-dual-adapter-tad-ocr-15raw --- # FTT raw-video dual-Adapter TAD and scoreboard OCR Public reproducibility package for the FTT seed-42, 35-epoch checkpoint used on `15_raw.MP4`. The video and ground-truth transcription are user-provided evaluation material. The TAD inference itself does **not** read the ground truth. This package contains three independent PyTorch weight files, inference source, an optional scoreboard OCR baseline and the three RapidOCR ONNX models. ## Files - `weights/ftt_dual_adapter_seed42_epoch35_trainable.pth`: trainable-only checkpoint (internal adapters, fusion, detection and relation layers). It is **not** a standalone backbone checkpoint. - `weights/vit-giant-p14_videomaev2-hybrid_pt_1200e_k710_ft_my.pth`: VideoMAEv2-G base weights. - `weights/vjepa2_1_vit_g_kdbd_encoder.pt`: V-JEPA2.1 ViT-G KDBD base encoder weights. - `code/`: inference source and dependencies snapshot. See `code/scripts/infer_ftt_single_video_20261110.py` for the model path. `code/run_pipeline.py` is the end-to-end CLI. `code/merge_rallies.py` is optional experimental grouping. - `source_code_and_ocr.tar.gz`: full `code/` and `ocr/` trees, including the bundled OpenTAD/V-JEPA2 snapshots. Extract this archive in the repository root after download; the main scripts are also available individually in the repository browser. - `ocr/`: scoreboard OCR code, result examples and `ocr/models/*.onnx`. The default crop is calibrated to a 1920×1080 broadcast layout. - `examples/15_raw.MP4`: sample input. `code/rally_gt_from_user_image.json` is for evaluation only. The model and source come from a research environment. Run TAD on Linux with NVIDIA CUDA and compatible PyTorch/OpenTAD dependencies; the captured environment used Python 3.11, PyTorch 2.0.1+cu118, MMEngine 0.10.7, MMAction2 1.1.0, Decord 0.6.0 and NumPy 1.23.5. OpenTAD source and `timm` compatibility snapshot are included under `code/`. FFmpeg/ffprobe must be on `PATH`. For the optional CPU OCR baseline, install `rapidocr-onnxruntime==1.4.4`, `onnx`, `opencv-python`, and `numpy`. The ONNX files are identical to the packaged RapidOCR models; the library loads them from its installed package by default. The three OCR files contain 1,171,841 detection, 2,690,407 recognition and 133,777 orientation ONNX constant elements, respectively (3,996,025 total; this is not a trainable-parameter count). Third-party backbone weights and source retain their original licenses; the repository license metadata does not relicense them. ## Run From the repository root, with the inference environment active: ```bash python code/run_pipeline.py --video examples/15_raw.MP4 --out results/15_raw --device cuda:0 ``` Add `--ocr` to run the independent 2 FPS scoreboard baseline. For another 1920×1080 broadcast layout, first verify the scoreboard location; for other layouts pass an `ffmpeg` crop as `--ocr-crop w:h:x:y`. The optional VLM-assisted scoreboard localization scripts in `ocr/` require a separate `MODELINK_API_KEY` environment variable and do not use the ModelScope token. The OCR baseline uses a 540×120 pixel scoreboard crop at `(x=85,y=915)` in the sample broadcast, a 0.50 magenta-overlay visibility threshold, a 0.50 OCR confidence threshold, and three stable samples within six 2 FPS observations. Its 4.5 s assumed rally duration is a heuristic, so long rallies may be truncated. Exploratory scripts under `ocr/` additionally include 10 FPS local score-change refinement and visual/court-start proposals; their prior sample-video results are documented in `ocr/REPORT_20261111.md` and `ocr/variants/`. These are not trained TAD predictions. Outputs are `results/15_raw/tad/predictions.json`, `rallies_all.csv`, `actions_all.csv`, `rallies_merged.{json,csv}`, and, with `--ocr`, `results/15_raw/ocr/{predictions.json,rallies.csv}`. The original model applies class-aware Soft-NMS across all windows. The optional rally merge uses a fixed highest-score seed and joins proposals with seed-to-candidate tIoU≥0.3 or a positive gap≤1.0 s; it preserves the seed score and unions time boundaries. It does not merge action classes and was explored on this sample video, so the raw NMS output should remain the primary result. The input length is unrestricted by the model window count: each window is 768 original frames, the stride is 384 frames, and the final window is padded. For the 13,001-frame sample video, this produces 33 windows. GPU memory and runtime rise with the number of windows. ## Evaluation and provenance The training used FTT's existing 343-segment development split (224 train, 56 validation, 63 test), not the paper's original 400-segment version. The checkpoint was selected at epoch 35; the validation rally/fine-action Avg mAP@0.3:0.7 was 89.85/36.07. These are **validation** metrics, not the sample video's or sealed test set's scores. The user image provides only approximate integer-second rally ground truth for the sample video; it has no fine-action labels. Run `code/evaluate_rallies.py` separately if desired. OpenTAD snapshot: `346d09d19e2091372cec48172dbe40f7b28bdee6`; V-JEPA2 code snapshot: `204698b45b3712590f06245fbfba32d3be539812`.