How chat-with-PDF tools work (and why they miss things)
Updated 13 October 2026 · ChatPDF guides · 6 sources
AI can miss or misstate things. Summaries and answers from AI tools, ChatPDF included, can leave out important details or get them wrong. Check anything that matters against the document itself, especially legal, medical, financial and exam material.
- Most chat-with-PDF tools pull the text out of the PDF, split it into passages and build a search index from it. 1,2
- For each question, the tool finds the passages that look most relevant and gives only those, with your question, to the AI model. This is called retrieval-augmented generation. 3
- If the passage that holds the answer is not picked, the model never sees it, so answers can miss facts that are elsewhere in the document.
- Extraction itself can garble text: a PDF stores where characters sit on a page, not clean words and paragraphs. 4
Chat-with-PDF tools feel like the AI has read your whole document. Usually it has not. Behind the chat box is a pipeline with three stages: get the text out of the PDF, find the parts of that text that match your question, and have a language model write an answer from those parts. Knowing the stages makes it much easier to ask good questions and to spot answers you should not trust.
Step 1: getting the text out of the PDF
A PDF is built to look the same on every screen and printer. Internally it records fonts and the position of each character on the page, not words, sentences or paragraphs as such 4. Software has to rebuild those from the positions, and it does not always get it right. A benchmark of PDF text extraction tools found common problems with word boundaries, paragraphs, reading order and stray text from headers, footers and captions 4.
In practice that means:
- Columns can be merged so that sentences from two columns are interleaved.
- Tables lose their grid: you get the numbers, but not always which row and column they belonged to (see working with tables and numbers).
- Scanned pages have no text at all unless someone has run OCR on them (see scanned PDFs and OCR).
Step 2: passages and a search index
Long documents do not fit comfortably into a single request, and sending everything every time would be slow and costly. So the text is split into passages, often a few hundred words or less each. Each passage is then turned into an embedding: a list of numbers that represents its meaning, so that passages about similar things end up close to each other 2. The embeddings are stored in a search index.
Splitting has a cost. A passage cut out of its surroundings can lose the context that made it meaningful. Anthropic's write-up on retrieval gives the example of a passage from a company filing that says revenue grew 3% over the previous quarter without naming the company or the quarter, which makes it hard to match to the right question 1.
Step 3: retrieval and the answer
When you ask a question, the tool turns your question into an embedding too and looks up the passages closest to it. Some tools also use ordinary keyword matching 1. The top few passages are added to a prompt together with your question, and the language model writes an answer from them. Researchers call this pattern retrieval-augmented generation 3,2.
Requests about the whole document, such as "summarize this" or "make a quiz", do not match any single passage well. Tools handle these differently: some send a broader sample of passages from across the document, some send as much text as fits. Either way, the model is working from a selection.
Why answers miss things
- The right passage was not retrieved. If your wording differs from the document's ("cancel" versus "terminate"), or the answer is spread over several pages, the search can pick the wrong passages.
- Context was cut off. A passage may hold the number but not the condition attached to it two paragraphs earlier.
- Long inputs are used unevenly. One study found language models were best at using information placed at the beginning or end of a long input and noticeably worse when it sat in the middle 5.
- The model fills gaps. When the passages do not contain an answer, a model may still produce a fluent one. Summarization research has documented this kind of unsupported content for years 6. See can you trust AI PDF summaries?
How to ask so retrieval works for you
- Use the document's own terms. If a contract says "termination", ask about termination.
- Ask one thing at a time. "What is the notice period?" retrieves better than a three-part question.
- Name the section if you know it: "In the methods section, what sample size was used?"
- Ask a follow-up when an answer looks thin: "Is there anything else in the document about fees?"
- Check the source text. Search the PDF for a key word from the answer and read the paragraph yourself (see how to search a PDF on iPhone).
More question patterns are in prompts that work well for PDFs and how to ask questions to a PDF.
Where ChatPDF fits
ChatPDF follows this same pattern. The app extracts the text on your iPhone or iPad, and our server splits it into passages and builds a search index with Cloudflare Workers AI embeddings. For a question, the most relevant passages go to the answer model with your question; for summaries, key takeaways, quizzes and flashcards, a broader sample of the document is sent. Answers are written by Anthropic's Claude models, with Meta Llama models run by Replicate as a backup. Because the model only sees the passages it is given, it can miss things, and its answers do not include page references, so checking the document yourself still matters. Details of what is sent where are in privacy of AI PDF tools and our privacy policy.
Common questions
Does the AI read my whole PDF?
Usually not for a single question. Most tools search the document and pass the model only the passages that look relevant. For whole-document requests some tools send a broader sample, but it is still a selection. 3,1
What is an embedding?
A list of numbers that represents the meaning of a piece of text, so that texts about similar things have similar numbers. Search indexes compare these numbers to find passages related to your question. 2
Why did the tool say the document does not mention something that is clearly in it?
The search probably picked other passages, often because your wording differs from the document or the information is split across sections. Rephrase with the document’s own terms, or search the PDF yourself. 1
Sources and further reading
- Anthropic (2024). Introducing Contextual Retrieval. anthropic.com
- Cloudflare Developers. Vector databases (Vectorize reference). developers.cloudflare.com
- Lewis P, Perez E, Piktus A, et al. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems 33 (arXiv:2005.11401). arxiv.org
- Bast H, Korzen C (2017). A benchmark and evaluation for text extraction from PDF. ACM/IEEE Joint Conference on Digital Libraries (JCDL) 2017. doi:10.1109/JCDL.2017.7991564 ad-publications.cs.uni-freiburg.de
- Liu NF, Lin K, Hewitt J, et al. (2024). Lost in the middle: how language models use long contexts. Transactions of the Association for Computational Linguistics 12:157–173. doi:10.1162/tacl_a_00638 aclanthology.org
- Maynez J, Narayan S, Bohnet B, McDonald R (2020). On faithfulness and factuality in abstractive summarization. Proceedings of ACL 2020:1906–1919. doi:10.18653/v1/2020.acl-main.173 aclanthology.org