What the EU AI Act actually asks of a retrieval system

A retrieval-augmented system draws on selected documents to produce an answer. If a person uses that answer in a consequential decision, a later review may need to establish which passages, versions and controls contributed to it. The architecture should support that review from the outset.

Regulatory assessment starts with the intended use, the organisation’s role and the system’s classification. A retrieval pipeline does not create one uniform set of obligations. For high-risk systems, Article 12 addresses automatic logging and traceability, while Article 14 addresses effective human oversight. Other obligations depend on the context.

Decide which evidence the organisation needs and how it will remain recoverable. This includes ownership, access and retention as well as the content of a trace. Adding logging later cannot restore records that were never captured.

The question an assessment needs to answer

For a retrieval system, a useful question is: when this system produced an output that mattered, can you show what it was working from? This is an architecture recommendation, rather than a complete statement of legal duties.

General model capabilities and average accuracy do not reconstruct a particular answer. For an operational review, identify the request, the system state at the time, the retrieved material and the action taken from the output.

The relevant record may include:

• Which documents were in the index at the time.
• Which version of each of those documents.
• Which passages were returned for the query, and in what order.
• What the model was given as context, after truncation and re-ranking.
• Which model version produced the answer.
• What the person on the other end did with it.

These records need owners, retention rules and access controls. Decide what must be recoverable before the system starts carrying consequential work. Adding a record later cannot recover history that was never captured.

Why this is architecture, not documentation

Consider an incorrect customer answer reviewed several months after it was issued. The review needs the historical system state rather than a fresh answer from the current service.

If source documents retain only their latest version and retrieval traces have been discarded, rerunning the query cannot establish what the original model received. Re-embedding or changing the index may further alter the result. State what can be recovered and avoid presenting a reconstruction as an original record.

The evidence requirement affects index versioning, storage, identifiers and retention. These are architecture decisions. They should be designed with security and privacy constraints rather than added as an unrestricted log of all user content.

An evidence trail is useful when it preserves the system state relevant to the decision and remains accessible for an authorised review.

Specify the evidence requirement early and include its storage and operating costs in the design. Retrofitting it after deployment can require changes across several components, and some missing history may remain unrecoverable. The extent and cost depend on the existing system.

What to decide before the first sprint

An architecture review should establish how each of the following records supports the assessment. This is a practical design guide, with requirements tailored to the use and the applicable legal duties.

Version the index, not just the documents. A change of embedding model can make existing vectors incompatible with new queries. Record the embedding model and index version so a later review can establish which configuration answered the question.

Persist the retrieval trace with the answer. Document IDs, versions, scores, and the final assembled context. Store it with the output, not in an application log that rotates after thirty days.

Record what was cut. Truncation and re-ranking decide what the model actually saw. The passages you dropped are part of the explanation for the answer you got.

Pin the model version in the record. “GPT-class model” is not a version. A provider’s silent upgrade is a change to your system’s behaviour that you did not make and cannot otherwise detect.

Capture the human decision. Record whether the person accepted, edited, escalated or overrode the answer. These records help assess how oversight operates in practice. They do not, on their own, establish that the legal requirements for oversight are met.

Keep the evaluation set under version control. It supports consistent comparisons across changes. Assign an owner, document its provenance and refresh it when the intended use or operating population changes.

Establish the obligations and preserve the evidence

These design practices can support an assessment whether or not a system is high-risk. They are not a full legal checklist. Intended use, classification and the organisation’s role determine which obligations need to be addressed.

Classification depends on the intended use and the system’s characteristics. An internal policy assistant and a system used to screen applicants may attract different obligations even if they share a retrieval architecture. Assess the deployed use rather than deriving the answer from the technology label.

Article 99 sets maximum administrative fine ceilings of €35,000,000 or 7% of an undertaking’s worldwide annual turnover for prohibited-practice infringements, and €15,000,000 or 3% for the obligations listed in paragraph 4. These are maximum ceilings, not predictions of a penalty. The higher calculation generally applies to undertakings. Small and medium-sized enterprises, including start-ups, use the lower amount or percentage. Under the 2026 amendments, small mid-cap enterprises also use the lower calculation for the €15,000,000 and €7,500,000 tiers, but not the prohibited-practice tier. National rules determine how fines apply to public authorities. Applicable obligations and the circumstances of an infringement must be assessed before drawing conclusions about exposure.

The limits of this guidance

These practices do not establish compliance on their own. Legal interpretation requires a review of the organisation’s role, the intended use and the applicable provisions. Architecture evidence supports that assessment without replacing it.

These design choices make an assessment more practical. Preserve the evidence trail alongside the sources, model versions and people who use the output. The next architecture review should establish what can be recovered today, what is missing and who owns the decision to address it.

illustrative diagram

A sourced answer has an operational path

Gateway

Check the user and request policy before retrieving permitted sources.

Provenance

Carry passages and source versions into the request and record answer validation.

Outcome

Return source references when validation passes; otherwise review or abstain.