Dr.DocBench Challenge

Overview

The Dr.DocBench Challenge evaluates expert-level document parsing on pages where strong systems still struggle. It is based on Dr.DocBench, a difficulty-aware benchmark developed by 2077AI and research collaborators, and is part of the DocInsights 2026 workshop at EMNLP 2026.

Challenge Task

Expert-Level Document Parsing

Participants convert difficult document images into structured Markdown while preserving:

Original-language text Structured HTML tables LaTeX formulas Reading order Document structure Chemistry notation Musical notation (exploratory content)

Musical notation appears in the challenge data as exploratory content. It does not receive a separate EvalAI metric.

Timeline

August 10, 2026

Competition opens

October 10, 2026 · 12:59 PM UTC

Submission deadline

October 11–23, 2026

Final evaluation period

DocInsights 2026

Winners announced at the workshop

Awards

US$5,000 Combined Prize Pool

Shared across the Dr. DocBench and DocSEM challenges. Award details and allocation will be announced soon.

Leaderboard

Public EvalAI results will be updated periodically during the competition. Official final rankings will be determined through the Private Test phase.

Ranking is descending by Overall under the official EvalAI scoring contract.

Expert-Level Document Parsing

Rank	Team	Text ED	Table TEDS	Formula CDM	Reading Order	Overall
Leaderboard opens with public EvalAI submissions.

Dataset

Challenge Data

The public Dr.DocBench dataset and annotations are available on Hugging Face for development and format reference.

Open on Hugging Face

Competition Evaluation

Official evaluation inputs, sample submissions, and validation tools are provided through the Dr.DocBench EvalAI challenge.

Open EvalAI Challenge

Evaluation

Submissions are evaluated using four component metrics plus one Overall aggregate. Final rankings are determined by the Overall score.

Text Edit Distance

Normalized Levenshtein distance over scored text categories, from 0–1. Lower is better.

Table TEDS

Mean tree-edit-distance similarity for valid tables, shown on a 0–100 scale. Higher is better.

Formula CDM

Mean CDM F1 for isolated display formulas, shown on a 0–100 scale. Higher is better.

Reading Order

(1 − normalized reading-order edit distance) × 100. Higher is better.

Overall

Available component scores are converted to a 0–100 scale and averaged for each scorable page, then averaged across all scorable pages. Missing components are excluded from the page denominator rather than scored as zero; non-scorable pages are excluded.

Submission

1. Prepare

Download the relevant evaluation package and generate one prediction for every released page identifier. The challenge uses a single-page prediction unit, unlike the default two-page window setting described in the Dr.DocBench paper.

submission.zip
└── predictions.jsonl
    └── required fields: subject, document_id, page, markdown

The evaluator applies strict checks for missing, extra, or duplicate predictions and invalid identifiers or submission structure.

A canonical mds/ Markdown tree mirroring the released hierarchy is also accepted.

2. Submit

Upload the validated ZIP through EvalAI. Final rankings will be determined through the Private Test phase.

Open EvalAI Challenge

Rules

Eligibility

Open to researchers and teams worldwide. There is no team-size limit.

Submissions

Maximum three submissions per day. Each team may select up to two submissions for final evaluation.

Requirements

Top-ranked teams must submit a technical report. All external data, models, and resources must be disclosed.