“What does it cost and how long does it take?” is the first question in any conversation about a RAG system – and the one to which there is rarely a usable answer. Usually follows either a number that no one can prove, or an evasion into the non-binding. Neither helps with an investment decision. This article describes how a RAG project actually works, where the effort really lies and how you recognize an offer that has simply factored in the uncertainty. What we do is on our side RAG systems and AI agents.
The five phases – and what is at the end of each phase
A RAG project is divided into five sections, which can hardly be exchanged in order. Crucially, each of them delivers a verifiable result that you could break behind without losing everything.
1. Review of document stocks
Before anything is built, it must be clear what is being worked with: which storages are there, how many documents are in them, in which formats, how current they are and who has access today. This sounds trivial and never is – in most companies there is no complete answer. At the end of this phase, there is a reliable statement about whether the desired use case is sustainable at all, and a realistic estimate for everything else.
2. Proof of Concept
A narrow application case with a real, but manageable document stock and a fixed group of test users from the specialist department. The PoC answers exactly one question: Are the answers good enough for people to use the system voluntarily? Everything else – connection, authorizations, operation – is deliberately left out. Typically, this is a few weeks.
3. Construction
Only now is the system made production-ready: connection to the real source systems instead of an export, submission of access rights up to the search results, logging, deletion concept, operating processes. This section is the most elaborate and the one that offers are most often too small.
4. Rollout
Extension to other user groups and stocks – gradually, not all at once. A system that works great with fifty curated documents does not automatically get better by adding five thousand unchecked ones. It gets worse, and then the trust is gone.
5. Operation and care
The part that project plans like to leave out. Documents become obsolete, questions change, new sources are added. Without someone to decide which documents apply, the system loses quality within months – regardless of how well it was built.
Where the effort really lies
This is the biggest miscalculation. Most expect the lion’s share to go into technology. In fact, the technical core of an RAG system is now largely standard – the building blocks are available and well documented.
The effort lies in three other things: the preparation of the documents, the clarification of the authorizations and the professional evaluation of the answers. All three need your home, not just the service provider. A project is almost never delayed because a component does not work, but because no one decides which of the three versions of a manual is the valid one.
Rule of thumb from practice: If the department participation does not appear in the offer, someone has either underestimated the effort or plans to bill it to you later.
What determines the price
A serious number arises only after sighting. What can be said before are the factors on which it depends:
- Condition of the document stock. It is not the quantity that matters, but the order. Ten thousand clean PDFs are simpler than five hundred files in twelve versions with no recognizable date.
- Number of source systems. A file server is a connection. File servers plus ticket system plus wiki plus contract filing are four – each with its own rights logic.
- Authorization depth. A system that shows everyone the same documents is much simpler than one that has to pass the existing access rights through to the search results. However, the latter is not negotiable for personal or confidential content.
- Operational model. Completely in-house, in a private environment or as a supervised solution – this changes both the one-time effort and the running costs considerably.
- Quality requirements. An assistant for internal research may occasionally be wrong because the user notices it. A system that prepares information to customers requires significantly more testing and assurance.
We deliberately do not mention a lump sum before the first point is clarified. Who does this, calculates the uncertainty – at your expense.
What do you know about a weak offer
Four warning signs that reliably indicate problems in practice.
The first is a Fixed price commitment without prior sighting the document stock. No one can estimate the cost of processing without seeing what needs to be processed – anyone who does it anyway has priced in a buffer or will ask for it later.
The second is an offer in which no measurement of response quality occurs. Without separate evaluation of retrieval and response, it is not possible to say whether the system will be better or only different; how to do this is stated in our contribution to the Evaluation of RAG systems.
The third is a missing erasure concept. If a document is deleted, the vectors generated from it must also be deleted – this does not happen by itself and is difficult to retrofit. Why embeddings are to be treated as copies of the content, we explain in the article on Vector databases.
The fourth is a project that the Go-Live ends. Those who do not offer or at least describe the operation plan a system that no one uses after a year.
What you need to contribute yourself
The uncomfortable part: A RAG project cannot be completely outsourced. You need two roles in your own house. First, someone from the specialist department who decides which documents apply – no service provider can answer this question because it is technical and not technical. Secondly, someone from the IT or organization who can provide information about filings, rights and systems.
Together, these are not full-time positions, but they are reliable contacts over the project period. Where these roles are not filled, the schedule shifts – and not a bit.
Conclusion
A RAG project can be planned as soon as you accept that the reliable number is not at the beginning, but after sighting. The way there is short and favorable: an inventory, then a tightly cut proof of concept, the result of which you can judge for yourself. Only then will it be decided whether and to what extent to build. If you want to assess how your document stock stands and which first step is worthwhile, talk to us The first meeting costs nothing.
What use case actually carries and what the EU AI Act requires, we have in the contribution to the Introduction of a RAG system in SMEs described.
Would you like to implement this in your company? We support you pragmatically – from the idea to the operation.