Retrieval-augmented generation (RAG)
Retrieval-augmented generation (RAG) is a method that finds relevant source material at response time and supplies it to a language model so the model can answer with that context. It lets an AI phone agent use a maintained body of business information without placing every fact in its prompt.
A typical RAG system prepares documents for search by extracting text and dividing it into smaller sections. When a caller asks a question, the system forms a search query, retrieves likely relevant sections, and places selected material in the model's context. The model then produces a conversational answer based on the caller's question and the retrieved text.
RAG is not the same as training a model. The source material is looked up when it is needed, which means teams can update the underlying documents without retraining. It is also not a guarantee of correctness. Retrieval can miss the right passage, return outdated material, or surface text that does not actually answer the question. The model can still misread what it receives.
For phone calls, retrieval must balance relevance, speed, and speakability. Returning an entire manual may slow the response and bury the needed sentence. A strong setup uses clear source documents, useful section boundaries, narrow retrieval, and instructions for what to do when evidence is absent or conflicting. The spoken answer should be concise even when the source is detailed.
RAG works best for factual questions grounded in maintained material: service policies, product details, procedures, or operating information. Transactional data such as live availability or an individual account balance is often better obtained through a purpose-built tool. The distinction is practical: documents answer “what is the policy?” while a system lookup answers “what is true for this caller right now?”
In practice on ThunderPhone, the organization knowledge base accepts text, Markdown, CSV, PDF, and DOCX files, shows processing status, supports content search and per-agent document selection, and provides a built-in mid-call knowledge search tool. Current documented limits are 5 MB per text file and 50 MB per PDF or DOCX file. Uploaded documents do not apply automatically; they must be selected for each agent.
Evaluation should include questions with a clear answer, no answer, several similar answers, and outdated or conflicting sources. Review which passages were retrieved as well as what the agent said. That separates a search failure from a response-generation failure and makes the knowledge system easier to improve.