非结构化数据解析2026:把PDF、邮件、HTML变成干净的RAG输入
RAG语料80%是PDF和邮件,粗暴提取会弄坏内容。Unstructured把杂乱文件变成干净的分区文档。
Unstructured: The Document Parser for Modern RAG
Unstructured (15,242 GitHub stars, Apache-2.0) is the open-source library that solves the boring-but-critical problem at the start of every RAG pipeline: getting content out of real-world files. PDFs with tables, emails with headers, HTML with navigation, PowerPoint decks - Unstructured partitions them into structured elements (Title, NarrativeText, Table, ListItem) that chunk cleanly and embed well.
Why Naive Extraction Fails
PyPDF-style extraction gives you a wall of text: tables lose their columns, headers mix with body content, and layout order scrambles. When those fragments get chunked and embedded, retrieval quality tanks silently - the embeddings are built from garbage. Unstructured fixes this with layout-aware partitioning: it detects titles, paragraphs, tables, and page structure, and outputs typed elements you can chunk intelligently (keep tables whole, merge short list items, etc.).
The API in Practice
from unstructured.partition.auto import partition
elements = partition("quarterly_report.pdf")
# elements: [Title('Q3 Report'), NarrativeText('...'), Table('| Q1 | Q2 |'), ...]
Partition by content type: partition_pdf, partition_email, partition_html, partition_pptx, partition_docx - all sharing the same element output model. For scanned PDFs, it hooks into OCR (via Tesseract) to recover text from images.
The RAG Pipeline Fit
Unstructured slots in right before chunking: files in, typed elements out, then chunk elements (a common pattern: group text by section, keep tables as standalone chunks), then embed and store. The element types also enable smarter retrieval - for example, boosting table elements when the query smells numeric.
Hosted vs Self-Hosted
The library is free and runs locally. The company also offers a hosted API with more format coverage (Excel, complex PDFs, and 25+ file types) and a platform for document workflows. For most teams, the open-source library covers PDFs, Office files, HTML, and email - the 90% case.
FAQ
Is Unstructured free? The library is Apache-2.0 open source; the hosted API is paid.
Does it handle scanned PDFs? Yes - with OCR enabled (Tesseract), it extracts text from scans.
Can it parse Chinese documents? Yes - text extraction works for CJK content; OCR quality depends on the OCR engine.
Unstructured vs LlamaParse? Unstructured is the open-source standard; LlamaParse is LlamaIndex's hosted parser with its own format coverage.
