PDF→Markdown Evaluation

9 PDFs · 900 questions · Multiple parsers (Mathpix / TextIn / Reducto / Marker / Internal)

📘 Read the Report ⭐ GitHub

Last updated: 2025-08-12 22:39 UTC

Datasets (9)

23-0323_ego_stx4500_stx4500-fc_stringtrimmer_manual_en

Hanwha_Integration_Guide

SSA1200_EGO_SNOW-SHOVEL-ATTACHMENT_22-0519_EXPLOSION-DIAGRAM_VERSION-A

TP-MVD8MV2-rotated

ego_accessory_compatibility_matrix

feier_start_100_manual

zt4200s_ego_zero-turn-riding-mower_version-a

How this works

Convert each PDF with multiple parsers → store Markdown under docs/md/<parser>/<doc_id>/
100 PDF-specific questions per doc → questions/<doc_id>/questions.jsonl (mirrored to site)
Run RAG pipelines → write answers to runs/<parser>/<doc_id>/answers.jsonl
Use an LLM evaluator (or exact-match) against the original PDF to score