PDF to Markdown Before You Build RAG: Docling, Marker and MinerU, and a Harness That Scores All Three

A quarterly revenue table came out of conversion as a single string of numbers in reading order, and the retriever keeps answering with the wrong figures. The embedding model is fine. The PDF-to-text step quietly deleted the structure before retrieval ever saw it.

💡 What You Will Learn

A quarterly revenue table came out of conversion as a single string of numbers in reading order, and the retriever keeps answering with the wrong figures. The embedding model is fine. The PDF-to-text

📜 Table of Contents

• What "good conversion" means, concretely
• The three converters, with versions that can be checked
• Get a baseline before converting anything
• Run all three, then score them with one script
• How to read the four numbers without fooling yourself
• Scanned documents are a different problem, and OCR defaults will fool you
• The optional LLM pass: quality gain, data-flow decision
• Operational weight nobody mentions in the comparison posts
• Choosing by document type
• FAQ
• Which converter gives the best tables?
• Do I need a GPU?
• Can I skip conversion and embed the PDF directly?
• Why did my output come back with duplicated paragraphs?
• Run it on ten documents this week
• Related on this site

Ask a retrieval system why it answered wrong and the answer is often not in the retrieval logic. A quarterly table of segment revenue became a single paragraph of numbers in reading order, "Q1 2025 4.2 3.8 5.1", and no reranker can recover what the header row meant. The PDF conversion step decided the answer before the embedding model ever saw the text.

That step is also where teams spend the least time, usually by grabbing whichever library appeared first in a tutorial. Three open-source converters currently dominate the practical choices, and they fail on different documents. This is a harness for choosing between them with your own files instead of someone else's benchmark chart, plus what to look at when the numbers disagree.

What "good conversion" means, concretely

"Characters extracted" is a useless metric: a column-scrambling extractor produces plenty of characters. Four properties matter, and they map to different failure modes downstream:

• Reading order — multi-column papers and sidebars. Broken order produces sentences that are fluent and false, which embeddings cannot detect.
• Table structure — a markdown table with headers keeps meaning; flattened numbers do not. This is the single biggest source of confidently wrong answers over financial and scientific documents.
• Heading hierarchy — needed for chunking that respects section boundaries rather than cutting every 512 tokens.
• Figures and formulas — whether an equation becomes text, an image reference, or nothing at all.

Any harness worth running should report on those four, plus time. The code later in this article does exactly that, deliberately crudely.

The three converters, with versions that can be checked

Two rows deserve a second look before procurement signs anything. MinerU's license file opens by stating it is Apache-2.0 subject to additional terms — the terms are in the file and they are short, so read them rather than trusting the SPDX badge GitHub shows as "NOASSERTION". And Marker's optional LLM pass is not a local operation: it sends page content to a model provider, which is a data-flow decision, not a quality setting.

Get a baseline before converting anything

The baseline is the PDF's own text layer, extracted naively. Every converter should recover close to all of it; a big shortfall means content is being dropped, and a big surplus usually means the tool is inventing structure from OCR noise. With poppler installed, one command per file:

mkdir -p work/baseline work/out
pdftotext -layout report.pdf work/baseline/report.txt

Note what this baseline is not: it has no reading-order guarantee and no tables. It is a word inventory, which is exactly what makes it useful for a recall check — a converter that misses 15 percent of the words is not "slightly worse", it is deleting content.

Run all three, then score them with one script

Each converter has its own conventions, so run each into its own directory. Docling's CLI takes the file directly; Marker writes beside an output directory you name; MinerU emits JSON and markdown through its parse command, which defaults to the first ten pages unless you say otherwise — a detail that will quietly make it look fast and lossy on a 40-page report.

# Docling
docling report.pdf --output work/out/docling

# Marker
marker_single report.pdf --output_format markdown --output_dir work/out/marker

# MinerU: watch the page limit
mineru parse report.pdf --pages 1-40

The scoring script is deliberately small, because the point is to measure four things and not build a benchmarking platform. It compares word inventories against the baseline, counts markdown headings and table rows, and prints one row per output:

#!/usr/bin/env python3
# score.py BASELINE.txt OUT1.md OUT2.md ...
import collections, re, sys, os

WORD = re.compile(r"[A-Za-z0-9][A-Za-z0-9'._,%-]*")

def words(t):
    return collections.Counter(w.lower() for w in WORD.findall(t))

def main():
    base = words(open(sys.argv[1], encoding="utf-8", errors="replace").read())
    total = sum(base.values()) or 1
    print("%-30s %7s %7s %8s %6s %7s" % ("file", "chars", "words", "recall", "heads", "tblrow"))
    for p in sys.argv[2:]:
        md = open(p, encoding="utf-8", errors="replace").read()
        got = words(md)
        hit = sum(min(c, base[w]) for w, c in got.items())
        heads = sum(1 for l in md.splitlines() if re.match(r"^#{1,4}\s", l))
        rows = sum(1 for l in md.splitlines() if l.strip().startswith("|") and l.count("|") >= 2)
        print("%-30s %7d %7d %7.1f%% %6d %7d" % (
            os.path.basename(p)[:30], len(md), sum(got.values()),
            100.0 * hit / total, heads, rows))

main()

python3 score.py work/baseline/report.txt work/out/docling/report.md \
  work/out/marker/report/report.md work/out/mineru/report/report.md

Output paths differ per tool and change between versions, so the second command is the part you will have to adjust; use find work/out -name '*.md' once and fix the three arguments.

How to read the four numbers without fooling yourself

The scores are crude on purpose, and each one has a known way of lying:

• Recall near 100 percent with a low table-row count means the text is all there and the tables are gone. This is the most common result and the most damaging for retrieval, because the words are searchable and the relationships are not.
• Recall above 100 percent means duplicated content — page headers, footers, or table cells emitted twice. Harmless-looking, poisonous for chunking, since the same sentence lands in two chunks and dominates similarity scores.
• Many headings is only good if they are real. A tool that promotes every short line to a heading produces a table of contents full of page numbers.
• Table rows with no header separator (|---|---|) are just pipes in text. Count tables that have a separator line, not lines that contain a pipe.

None of this replaces reading five pages yourself. It replaces guessing across fifty documents.

Scanned documents are a different problem, and OCR defaults will fool you

On a born-digital PDF the text layer is already there, and OCR is wasted compute that can also corrupt clean text. On a scanned exhibit, no text layer exists and a converter that does not OCR will return an empty file that still looks like a successful run. Docling exposes the choice explicitly: --ocr to enable it, plus --ocr-engine, --ocr-lang and --ocr-mode to control it; the default pipeline assumes a digital document.

docling scanned-exhibit.pdf --ocr --ocr-lang eng --to md --output work/out/docling

A quick way to know which case you are in, before converting anything: pick a page in any reader, try to select text, and paste it into a terminal. Nothing selectable means scanned. Do that check across a sample of ten documents, because mixed collections are the norm — a folder of "PDFs" is often half generated, half photographed, and a pipeline tuned for one produces silent empty outputs on the other.

The optional LLM pass: quality gain, data-flow decision

Marker can hand difficult pages to a language model with --use_llm, which helps with tables that span pages, inline math, and form fields. The important detail is where that page goes: the default service is a hosted model, and the project lists Gemini, Claude, OpenAI-compatible endpoints, Azure, Vertex, OpenRouter and Ollama as options. If the documents are client material, pointing it at a local Ollama model keeps the flow inside your network at the cost of quality; if they are public papers, the hosted default is the pragmatic choice. Treat this as a compliance decision with a quality side effect, not the reverse.

marker_single contract.pdf --output_format markdown --use_llm
# local alternative: configure an Ollama endpoint rather than the hosted default

Docling offers a comparable option on the model side rather than by calling a chat API: a VLM pipeline with a chosen document model, invoked as docling --pipeline vlm --vlm-model NAME. Same trade: heavier local compute, no third-party data flow.

Operational weight nobody mentions in the comparison posts

All three are PyTorch-based pipelines. Installing one means a multi-gigabyte environment and a first run that downloads layout, OCR and table models before producing a single page. That has three consequences worth planning for:

• Disk and install time, not runtime, is the first cost. Provision for it on a build machine, then bake the environment into an image rather than reinstalling per job.
• Air-gapped installs need the model cache copied, exactly as with speech models. The conversion function itself is offline-capable; the first-run download is not.
• Pin the version. Converter output changes between minor releases, and a re-index that silently changes chunk boundaries will look like a retrieval regression months later. Record the version next to the index, the way you record an embedding model name.

Choosing by document type

FAQ

Which converter gives the best tables?

There is no universal answer, and that is the point of the harness: table handling depends on whether the table has ruling lines, spans cells across pages, or is an image of a table. Run five table-heavy documents from your own corpus through all three and count tables that survive with headers intact. That number is stable enough to decide with.

Do I need a GPU?

No, but you will want one for volume. All three run on CPU; layout and OCR models make that slow enough that a few hundred documents becomes an overnight job. Use --num-threads in Docling to control CPU parallelism, and measure pages per minute before promising a deadline.

Can I skip conversion and embed the PDF directly?

Some pipelines accept PDFs as images with a multimodal model. That works for questions about layout ("what does the chart show"), costs far more per page, and cannot be inspected when the answer is wrong. Text conversion gives you something you can diff, which matters when a retrieval failure needs debugging.

Why did my output come back with duplicated paragraphs?

Headers, footers and running titles are repeated on every page. A converter that emits them inline puts the same sentence into every chunk derived from those pages. Strip repeated lines that appear on more than half the pages before indexing, or the duplication will dominate similarity scores.

Run it on ten documents this week

Pick ten files that resemble the real corpus, generate the baseline text, convert with two tools, and read the four numbers plus five pages by hand. The decision usually becomes obvious within an hour, and the harness stays useful afterwards: when a new PDF arrives that breaks retrieval, you can tell within minutes whether the converter dropped a table or the retriever missed it. The conversion step is the part of a RAG pipeline that is cheapest to measure and most expensive to get wrong.

Related on this site

• Unstructured Data Parsing: PDFs and HTML into Clean RAG Input
• Document Parser GitHub: Seven Repos for Document AI
• LangChain RAG Tutorial: Rewriting the Pipeline After the chains Module Disappeared