Split your PDF into token-budgeted chunks ready for embeddings, vector databases, and retrieval pipelines. No upload.
Retrieval-augmented generation (RAG) works best when documents are split into focused, token-budgeted chunks. Each chunk should be small enough to fit in a retrieval embedding call, but large enough to carry meaningful context.
This tool extracts text from your PDF, splits it into chunks based on page boundaries, paragraph breaks, or heading structure, counts the tokens in each chunk, and exports the result as JSONL or Markdown ready for your vector database or indexing pipeline.
Your PDF content stays on your device. No text, chunks, or embeddings are sent anywhere.
For most embedding models (OpenAI text-embedding-3, Cohere embed-v3), 512 tokens is a solid default. If your retrieval context window is larger, 1024 or 2048 can work. Smaller chunks give more precise retrieval; larger chunks give more context per hit.
Overlap repeats the last N tokens of each chunk at the start of the next one. This helps when relevant information crosses chunk boundaries. 32 to 64 tokens is typical.
Page mode is simplest and works for most documents. Paragraph mode gives finer splits. Heading mode tries to keep sections together, which helps for structured reports.