---
title: MarathonDocBench
canonical_url: "https://www.modelscope.ai/datasets/baidu/MarathonDocBench"
md_url: "https://www.modelscope.ai/datasets/baidu/MarathonDocBench.md"
repository: baidu/MarathonDocBench
last_updated: 2026-11-04
license: apache-2.0
storage_size: "11 GB"
downloads: 75
stars: 0
---

# MarathonDocBench

> MarathonDocBench - An open-source dataset by baidu on ModelScope. A New Benchmark for Ultra-Long Document Parsing: Targeting Hundred-Page-Scale Documents.

baidu/MarathonDocBench is a dataset on ModelScope. totalling 11 GB. licensed under apache-2.0.

- **Repository**: baidu/MarathonDocBench
- **License**: apache-2.0
- **Storage size**: 11 GB
- **Downloads**: 75
- **Stars**: 0
- **Last updated**: 2026-11-04

Source: https://www.modelscope.ai/datasets/baidu/MarathonDocBench

---

<div align="center">

<img src="assets/baidu.png" width="40%" alt="Baidu Inc." />

</div>

<hr>

<h1 align="center">MarathonDocBench</h1>

<div align="center">
  <a href="https://github.com/baidu/Unlimited-OCR">
    <img alt="GitHub" src="https://img.shields.io/badge/GitHub-Code-181717?logo=github&logoColor=white" />
  </a>  
  <a href="https://huggingface.co/baidu/Unlimited-OCR2">
    <img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-ffc107?color=ffc107&logoColor=white" />
  </a>
</div>

<div align="center">
    <a href="https://huggingface.co/baidu/Unlimited-OCR-2/blob/main/Unlimited-OCR-2.pdf">
    <img alt="arXiv" src="https://img.shields.io/badge/arXiv-Unlimited OCR Works 2-b31b1b?logo=arxiv&logoColor=white" />
  </a>
  <a href="https://x.com/Baidu_Inc" target="_blank">
    <img alt="Twitter Follow" src="https://img.shields.io/badge/Twitter-Baidu Inc.-white?logo=x&logoColor=white" />
  </a>
</div>



<h3 align="center">A New Benchmark for Ultra-Long Document Parsing: Targeting Hundred-Page-Scale Documents.</h3>

<!-- <div align="center">

![MarathonDocBench leaderboard](assets/leaderboard.png)

</div> -->

MarathonDocBench is the first benchmark specifically designed for **hundred-page document parsing**. It contains 100 documents spanning **100–300 pages** (~18k pages, ~205k annotated layout elements, mixed Chinese/English), with rich cross-page structures — cross-page text and tables appear frequently and densely across documents. Structural annotations (bounding boxes, block type, hierarchy, reading order, and cross-page continuations of text and tables) are initialized from a strong page-by-page parser and carefully refined by human annotators; content transcriptions are verified through an iterative VLM-based refinement pipeline, with flagged disagreements adjudicated by human annotators.


<div align="center">

<img src="assets/overview.png" width="1000" alt="MarathonDocBench overview" />

</div>

This repository ships the **data** (page images + ground-truth annotations).
Evaluation reuses the **MPDocBench-Parse** toolkit — see *Usage* below.

---

## 1. Dataset

```
MarathonDocBench.json        # ground-truth annotations (100 documents)
images/
├── PDF1/page_1.png … 
├── PDF1/page_2.png …  
├── PDF1/ …                  # one folder of page images per document
├── PDF2/page_1.png
└── …                        # ~18k PNG pages total
```


Each entry in `MarathonDocBench.json` follows the MPDocBench schema (`page_info` / `layout_dets` / `extra`): `layout_dets` carries per-element `category_type`, `text`/`html`/`latex`, reading `order`, heading `anno_id`, and `poly` boxes; `extra.relation` lists cross-page `truncated` merges.


---

## 2. Usage

Evaluate with the **MPDocBench-Parse** toolkit; this dataset is a drop-in ground-truth replacement (same schema, same eight metrics).

```bash
# 1. Get the toolkit
git clone https://github.com/Tongyi-Zhiwen/Qwen-Doc.git
cd Qwen-Doc/MPDocBench
pip install -r requirements.txt
pip install mmeval          # needed by the dataset loader, not pinned upstream

# 1b. Replace 3 evaluator files with the accelerated drop-in from THIS repo
#     (metric-identical; only adds doc-level parallelism + a configurable match
#     timeout — vanilla hardcodes a 30s serial match that under-scores ultra-long
#     docs and can take days). Overwrites dataset/end2end_dataset.py,
#     metrics/cal_metric.py and utils/data_preprocess.py:
cp -r /path/to/MarathonDocBench/utils/mpdocbench_patch/* .

# 2. Place this dataset where the config can find it, e.g.
#    data/MarathonDocBench.json   and   data/images/...

# 3. Generate predictions: one Markdown file per document, named <doc_id>.md
#    (e.g. mp3lan_0101.md). Figure tags must follow the required format:
#        ![](pageN_x1_y1_x2_y2.jpg)
#    (N = 1-based page index; x1,y1,x2,y2 = integer coords on a 1000x1000 grid.)
#    Any parser is fine, as long as figure tags follow this format.

# 4. Point configs/end2end.yaml at the dataset (ground_truth.data_path ->
#    data/MarathonDocBench.json, prediction.data_path -> predictions/your_model_md),
#    then run with the settings used for the leaderboard. MPDOC_TEDS_TIMEOUT is
#    required: a few tables have tens of thousands of cells and pure-Python TEDS
#    would run for hours on them; past the cap such a table scores 0.
MPDOC_EVAL_NPROC=32 MPDOC_MATCH_TIMEOUT=1800 MPDOC_DOC_BUDGET=0 \
MPDOC_TEDS_NPROC=32 MPDOC_TEDS_TIMEOUT=300 \
    python pdf_validation.py --config ./configs/end2end.yaml
```

Per-metric results are written to `result/`. The leaderboard `Overall` is the mean of the eight metrics — compute it from the per-metric outputs.

> - **Running a general VLM.** Whole-document inference can fail on provider caps: too many input images per request, request/JSON size limits, or the output-token ceiling (long docs truncate mid-table). Our competitor baselines send all pages in one request, progressively downscale images on oversize/timeout and retry, skip docs over the image cap, and leave failures as empty predictions so the denominator stays 100.
> - **Evaluation notes.** Headings must be real Markdown headings (`#`, `##`, …) — emphasis such as `**bold**` is NOT counted as a heading (it will lower HeadingTEDS). Figure page attribution is part of the score: a figure tagged with the wrong `pageN` is matched against the wrong page and penalized by FigureF1. The documents are very long; with the accelerated evaluator above (`MPDOC_EVAL_NPROC` + a large `MPDOC_MATCH_TIMEOUT`) a full 100-document run takes tens of minutes instead of hours.


---

## 3. Leaderboard

Sorted by `Overall` (the mean of the eight metrics), descending. All entries are end-to-end VLM-based parsers.

| Model | Overall | TextEdit | T-TextEdit | TableTEDS | T-TableTEDS | FormulaCDM | ReadOrderEdit | FigureF1 | HeadingTEDS |
|:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
| **Unlimited OCR Works 2** | **77.06** | **89.6** | **87.7** | **74.8** | **71.1** | **87.9** | **85.3** | **71.8** | **48.2** |
| Kimi-K3 | 33.7 | 52.5 | 33.6 | 27.8 | 26.3 | 40.5 | 38.9 | 21.4 | 28.9 |
| Gemini-3.8-Flash | 28.6 | 48.1 | 34.2 | 23.4 | 19.6 | 26.4 | 31.3 | 18.6 | 27.3 |
| Qwen-3.8-Flash-Next | 27.8 | 45.6 | 24.7 | 25.4 | 21.0 | 28.3 | 32.1 | 24.1 | 21.5 |
| Gemini-3.1-Pro Preview | 27.0 | 48.4 | 23.6 | 28.5 | 21.7 | 30.4 | 34.6 | 18.9 | 9.7 |
| Qwen-3.8-Max | 22.93 | 39.96 | 19.63 | 15.20 | 13.71 | 18.82 | 26.67 | 33.39 | 16.09 |
| Minimax-M3 | 17.97 | 34.09 | 18.22 | 17.79 | 14.84 | 14.20 | 23.73 | 3.67 | 17.16 |
| DeepSeek-4.1-Flash | 12.34 | 26.00 | 6.67 | 12.74 | 12.59 | 10.45 | 15.70 | 6.95 | 7.62 |
| Doubao-Seed-2.1-Turbo | 8.50 | 18.38 | 8.97 | 5.87 | 5.07 | 6.00 | 10.18 | 8.16 | 5.37 |
| GLM-5.3-Flash | 7.51 | 14.01 | 6.43 | 7.12 | 6.65 | 4.50 | 7.70 | 7.10 | 6.55 |
| Doubao-Seed-2.1-Pro | 5.54 | 10.75 | 4.25 | 3.15 | 4.76 | 2.82 | 7.13 | 7.52 | 3.92 |

---

## License & Copyright

Annotations and tooling references are released under Apache-2.0. The page images are derived from publicly accessible documents and are provided **for academic and research use only**; redistribution of the source documents is not granted by this license. If any content raises copyright concerns, contact the maintainers and it will be addressed promptly.

## Acknowledgement

We build on [MPDocBench-Parse](https://github.com/Tongyi-Zhiwen/Qwen-Doc) for the evaluation toolkit and annotation schema, and thank [OmniDocBench](https://github.com/opendatalab/OmniDocBench) and [CDM](https://github.com/opendatalab/UniMERNet/tree/main/cdm) for the metrics and ideas this benchmark reuses.

## Citation

```bibtex
@misc{shu2026unlimitedocrworks2,
      title={Unlimited OCR Works 2}, 
      author={Baidu ERNIE Team},
      year={2026},
      eprint={xx},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={xx}, 
}
```