An AI log you cannot retrieve is not an audit trail

A supplier’s audit screen can show events, timestamps and decisions without demonstrating the customer’s ability to retrieve them. Procurement should test an export from the intended deployment using the access rights the organisation will actually hold.

If retrieval depends on the supplier’s support process, record that dependency in the contract and operating design. An investigation may need evidence promptly, after a configuration change or at service exit. Establish the events created, the party controlling them and the tested path through which the deployer can obtain them.

A capability is not the same as possession

The EU AI Act’s high-risk rules apply from 2 December 2027 for Annex III systems and 2 August 2028 for systems embedded in Annex I regulated products, according to the Commission’s current implementation timeline. When applicable, they require high-risk AI systems to be designed so that events can be automatically recorded over their lifetime. Its record-keeping provision, Article 12, describes logging as a capability for tracing operation and supporting post-market monitoring. The Act also addresses who keeps logs. Under Article 26, deployers keep automatically generated logs to the extent they are under their control, for a period appropriate to purpose and, where that provision applies, at least six months unless another law provides otherwise.

The qualification separates product capability from customer control. Ask how automatically generated records become available to the organisation, in which format and under which rights. A questionnaire response confirming that logging exists cannot establish that the buyer can operate its evidence path.

Put the retrieval test in the buying process

Define a plausible investigation before sending the request for proposals. A disputed customer recommendation, for example, may require the system version, material input references, output and subsequent human action. Allocate those records across the provider, deployer and integration service, then test each retrieval boundary.

Ask for an export using the access level the firm will actually hold. It should cover the events, identifiers, timestamps, version references and retention settings needed for that investigation. An administrator demonstration is useful only if the customer will have that administrator access. If an event can be read only by filing a support ticket, document the response time, scope of retrieval and what happens if the contract ends.

Specify where logs go. A vendor dashboard is not necessarily the firm’s system of record. Decide whether records are sent to the firm’s logging estate, retained in the vendor service with a tested export, or duplicated in both. Each option changes access control, deletion, cost and the ability to preserve evidence while an incident is investigated. Personal data in logs also needs an appropriate retention and access decision. Keeping everything indefinitely is no substitute for knowing why it is kept.

Repeat the export after a model update, a configuration change and a simulated service exit. The log contract should name events, formats, delivery mechanisms, availability and handover responsibilities. Inspect the resulting records rather than accepting a general assurance of auditability.

Make the exercise specific enough to fail. Ask the supplier to reconstruct a single disputed recommendation from the customer-facing case identifier. The export should connect that identifier to the system version, input references, tool calls, output and any human approval. The buyer should time the request from its own account, then check whether a reviewer outside the supplier can read the result without a private explanation of the fields.

There are two distinct failure modes. The events may never have been recorded, or they may exist but be impossible for the deployer to locate in time. A dashboard screenshot cannot distinguish them. A sample export, field dictionary and tested retrieval procedure can. If the provider says it cannot export prompts or inputs for privacy reasons, agree which references or redacted records will still let the firm reconstruct the decision.

Run the same exercise at contract exit. Identify who can initiate an export, how long it remains available, which format the firm receives and how it verifies completeness. A right to request logs after the service has been switched off is weak if the identifiers needed to ask for them have already disappeared. The decision is whether the firm can operate that evidence path throughout the period it has chosen to keep the record.

The limit of the claim

This is not a claim that every AI purchase is a high-risk system under the EU AI Act, or that one retention period fits every system. Role, classification, timing and other law require a specific assessment. The buyer’s design question exists even where that Act does not apply: can the organisation recover the record on which its decision depends? Legal interpretation stays with counsel.

Make retrieval an acceptance condition owned jointly by procurement and the system owner. Obtain a usable export, assign responsibility for missing fields and test the evidence path through contract exit. Purchase against demonstrated access to the record the organisation will need.

illustrative fixture

Follow the record, not hidden reasoning

Hypothetical disputed customer recommendation

ReplayPausePrevious stepNext stepStep 6 of 6

Swipe or scroll for the full diagram →

RequestPermissionSource versionsModel versionValidationHuman handoff

Human handoff

The reviewer receives the evidence references and records an amend, approve or reject decision.

Request

Case R-104, received 09:00. Keep the customer-facing identifier.

Permission

Reviewer role checked against policy P-3. Restricted documents remain excluded.

Source versions

Record permitted document D-7, revision 2, and retrieval index I-4.

Model version

Record candidate model M-2 and template T-5 with the generated recommendation.

Validation

Keep the unsupported-claim check and its review-required result.

Human handoff

The reviewer receives the evidence references and records an amend, approve or reject decision.