What RAG is in AI and how to use it with your company’s data

Bruno Díaz Bruno Díaz — Digital Marketing and AI Consultant
Updated:
what RAG in AI is, for answering with your business's data

Ask ChatGPT about your store’s returns policy and it’ll give you a reasonable answer that’s probably wrong, because it has never read your terms.

An AI that checks your documents before it answers

That’s what RAG fixes: retrieval-augmented generation. The name is scarier than the idea. Before answering, the system searches your documents for the passages related to the question, hands them to the model and asks it to answer based on that material. The model is the same; the difference is that it now has the right page of the manual in front of it.

The documents are prepared once. They’re split into chunks, each chunk is turned into a numerical representation of what it means, and everything is stored in a database that searches by similarity, known as a vector database. That’s why, if someone asks whether they can return a discounted product, the system finds the paragraph even if it says “promotional items”.

finding the answer in the company's own documents

For an SME the advantage is twofold: the answers come from your data and each one can say which document it comes from, so checking it takes seconds.

RAG, fine-tuning or pasting the document: which one suits you

With a contract or a short catalog, just pasting it into the conversation is enough: it fits in what the model reads in one go and there’s nothing to build. It stops working when you have hundreds of files or when every query means pasting them in again.

Fine-tuning retrains the model with your own examples. It’s useful for teaching it a way of doing things, a tone, a format, a way of sorting emails, and it’s a poor fit for data that changes, because every price update means retraining. For an SME it’s almost never the first thing we recommend.

RAG fits when there’s a lot of material and it changes often, because updating it means replacing the document and reindexing, without touching the model.

Building RAG at an SME: much of the time goes into the documents

You start with a narrow use case, never with the technology: customer questions about the catalog or internal questions about procedures. Then:

  1. Gather and clean up the documents, which is where much of the time goes, between scanned PDFs, duplicate versions and old price lists.
  2. Build the pipeline. n8n has nodes for loading documents into a vector database such as Supabase, Pinecone or Qdrant and querying it from an agent, with Claude, GPT or Gemini writing the answer via API.
  3. Test it with the questions that reached the support inbox over the last month before opening it up to anyone.
  4. Decide who uploads new documents and how often it’s reindexed, because with outdated data it answers wrongly and with great confidence.

The budget depends on the number and condition of the documents, on whether there are permissions to respect (so the sales rep can’t see the payroll), on the channel, whether internal chat, website or WhatsApp, and on the query volume, which sets the monthly API spend. As a rough guide, a pilot with one document type and one channel takes weeks, and one with several sources and per-user permissions, months.

Before commissioning one, try Claude, ChatGPT or Gemini Notebook

In Claude, projects are available even on the free account, up to a maximum of five. You upload the documents and every conversation in the project draws on them, and on paid plans, when the material approaches the limit of what the model can read, Claude switches to a RAG mode on its own.

ChatGPT has changed things: personal accounts (Free, Go, Plus and Pro) can no longer create new GPTs; that’s now reserved for Business, Enterprise and Edu workspaces. The equivalent for you is ChatGPT projects, with a cap of 5 files per project on the free plan, 25 on Go and Plus, and 40 on Pro and Business.

NotebookLM, which Google has renamed Gemini Notebook, allows 50 sources per notebook for free and 300 on Pro, and points to the passage each answer comes from.

That’s what we’ll tell you before quoting anything: run the test with the questions you actually get. You’ve outgrown it if the documents don’t fit, if they change so often that re-uploading them becomes a standing task, if it has to answer customers outside the tool, if each person should only see part of it, or if the answer comes from a live system such as the ERP or the online store. If none of that happens, you already have what you were looking for for the price of a subscription.

If any of those signs sound familiar, tell us about it and we’ll look at which part of your case calls for custom development.

About the author

Bruno Díaz — Digital Marketing and AI Consultant

Digital marketing consultant specializing in SEO, SEM and web projects. I have two masters in AI (the degree and years of hands-on experience with it). Now I apply that experience to create new opportunities and boost productivity for our clients.

Articles by Bruno

Let's talk

Do you have a project in mind?

Tell us about it and let's grow your business together.

Analyze my sales opportunities