Ingesting PDFs into LangChain and LlamaIndex Using ParseAI

LangChain's PyPDFLoader and LlamaIndex's SimpleDirectoryReader are the default starting points for most RAG pipelines. They work for clean, text-native PDFs with simple layouts. For anything more complex — multi-column layouts, tables, scanned documents, forms, mixed-language content — they produce text that does not chunk or embed well.

Adding ParseAI as a preprocessing step replaces the default loader with a cleaner, more reliable document representation. The rest of your pipeline — splitter, embedder, vector store, retriever — stays exactly the same.

What Default PDF Loaders Get Wrong

The core problem with PDF text extraction libraries is that they read PDFs as a stream of text objects, not as documents with semantic structure. The output reflects the character positions in the PDF file, not the logical organisation of the content.

In practice, this means:

• Tables: Column headers and values are extracted in row-by-row or cell-by-cell order, losing the column relationships. A table comparing product features across five columns becomes five disconnected lists of values.
• Multi-column text: A two-column document produces text that alternates between column one and column two, mid-sentence.
• Headers and sections: The distinction between a section heading and body text is not captured. All text appears at the same level.
• Scanned PDFs: No text at all. PyPDFLoader cannot read images; it returns empty strings for scanned pages.
• Mixed content: PDFs with a mix of native text and scanned sections produce partial extractions with gaps.

When this degraded text is chunked and embedded, the resulting retrieval is unreliable. Chunks are semantically incoherent. LLM answers are incorrect or incomplete despite the source material being present in the index.

The ParseAI Preprocessing Step

The Parse API accepts a document and returns clean structured markdown. The markdown preserves:

• Heading hierarchy (H1, H2, H3) as markdown heading levels
• Tables as properly formatted markdown tables with aligned columns
• Lists as markdown unordered and ordered lists
• Multi-column text in the correct reading order, without interleaving
• Scanned content processed through internal OCR, same output format as native PDFs

This markdown is what you pass into your splitter. The structural markers — headings, table blocks, list boundaries — give your splitter meaningful places to split.

LangChain Integration Pattern

The simplest integration replaces the PDF loader with a ParseAI API call and wraps the output in a LangChain Document:

import requests
from langchain.schema import Document

def load_with_parseai(file_path: str, api_key: str) -> Document:
    with open(file_path, "rb") as f:
        response = requests.post(
            "https://api.parseai.org/v1/parse",
            headers={"Authorization": f"Bearer {api_key}"},
            files={"file": f}
        )
    markdown = response.json()["markdown"]
    return Document(
        page_content=markdown,
        metadata={"source": file_path}
    )

Pass the returned Document to your text splitter. For markdown content, use MarkdownHeaderTextSplitter to split on heading boundaries — this produces semantically coherent chunks:

from langchain.text_splitter import MarkdownHeaderTextSplitter

splitter = MarkdownHeaderTextSplitter(
    headers_to_split_on=[
        ("#", "h1"), ("##", "h2"), ("###", "h3")
    ]
)
chunks = splitter.split_text(document.page_content)

Each chunk now corresponds to a section of the original document, not an arbitrary character count window.

LlamaIndex Integration Pattern

For LlamaIndex, replace SimpleDirectoryReader with a custom loader that calls Parse API per document:

import requests
from llama_index.core.schema import TextNode

def parse_document(file_path: str, api_key: str) -> TextNode:
    with open(file_path, "rb") as f:
        response = requests.post(
            "https://api.parseai.org/v1/parse",
            headers={"Authorization": f"Bearer {api_key}"},
            files={"file": f}
        )
    markdown = response.json()["markdown"]
    return TextNode(
        text=markdown,
        metadata={"file_path": file_path}
    )

Pass the nodes to your VectorStoreIndex as normal. LlamaIndex's MarkdownNodeParser can then split on heading boundaries.

Handling Tables in LangChain and LlamaIndex

Tables in ParseAI markdown output are standard markdown tables. The key consideration is to not split tables mid-table during chunking. Both frameworks have chunking options that respect table boundaries when the input is clean markdown.

A common pattern: when a chunk contains a markdown table, do not split it further by character count. Let the table be a complete chunk. Tables retrieved whole are more useful to an LLM than tables split across multiple chunks.

Performance Considerations

For large document ingestion jobs — thousands of PDFs — the Parse API call adds latency per document. Strategies for production:

• Cache the markdown output. If the same document is re-ingested, use the cached markdown rather than calling the API again. Store the markdown alongside the original file.
• Parallelise API calls. The Parse API is stateless; concurrent requests are safe. Use a thread pool or async client to process multiple documents simultaneously.
• Process incrementally. For ongoing ingestion pipelines, only process new or updated documents. Skip documents whose cached markdown is still current.