An AI log you cannot retrieve is not an audit trail

A procurement team is shown a demonstration of an AI system’s audit screen. It contains event identifiers, timestamps and decisions. The buyer asks for an export from a particular deployment, with the inputs and version that produced an output. The answer is that support can request it from the provider.

That answer is a contract and architecture finding, not a minor usability issue. A log that exists only behind another organisation’s support process may be unavailable when the deployer needs to investigate a decision, respond to an incident or reconstruct what happened. The buyer should establish which events the system creates, which party controls them and how the deployer can retrieve them before signing.

A capability is not the same as possession

The EU AI Act’s high-risk rules apply from 2 December 2027 for Annex III systems and 2 August 2028 for systems embedded in Annex I regulated products, according to the Commission’s current implementation timeline. When applicable, they require high-risk AI systems to be designed so that events can be automatically recorded over their lifetime. Its record-keeping provision, Article 12, describes logging as a capability for tracing operation and supporting post-market monitoring. The Act also addresses who keeps logs. Under Article 26, deployers keep automatically generated logs to the extent they are under their control, for a period appropriate to purpose and, where that provision applies, at least six months unless another law provides otherwise.

The qualification matters. A provider can implement automatic logging while the customer lacks an API, export right or storage path that puts the evidence under the customer’s control. A procurement questionnaire that asks whether logging is supported will record yes. It will not discover whether the firm can meet its own operational need for the record.

Put the retrieval test in the buying process

Choose one plausible investigation before sending the request for proposals. For example, a customer disputes an adverse recommendation made last month. What record would show the system version, the material inputs, the outcome and the human action that followed? Which part belongs to the provider, the deployer and a separate integration service? Draw those boundaries as data flows, then test the supplier’s answer against them.

Ask for an export using the access level the firm will actually hold. It should cover the events, identifiers, timestamps, version references and retention settings needed for that investigation. An administrator demonstration is useful only if the customer will have that administrator access. If an event can be read only by filing a support ticket, document the response time, scope of retrieval and what happens if the contract ends.

Specify where logs go. A vendor dashboard is not necessarily the firm’s system of record. Decide whether records are sent to the firm’s logging estate, retained in the vendor service with a tested export, or duplicated in both. Each option changes access control, deletion, cost and the ability to preserve evidence while an incident is investigated. Personal data in logs also needs an appropriate retention and access decision. Keeping everything indefinitely is no substitute for knowing why it is kept.

Finally, test the lifecycle. Repeat the export after a model update, a supplier configuration change and a simulated exit. A useful log contract names the events, formats, delivery mechanism, availability and handover process. A general promise of auditability does not.

Make the exercise specific enough to fail. Ask the supplier to reconstruct a single disputed recommendation from the customer-facing case identifier. The export should connect that identifier to the system version, input references, tool calls, output and any human approval. The buyer should time the request from its own account, then check whether a reviewer outside the supplier can read the result without a private explanation of the fields.

There are two distinct failure modes. The events may never have been recorded, or they may exist but be impossible for the deployer to locate in time. A dashboard screenshot cannot distinguish them. A sample export, field dictionary and tested retrieval procedure can. If the provider says it cannot export prompts or inputs for privacy reasons, agree which references or redacted records will still let the firm reconstruct the decision.

Run the same exercise at contract exit. Identify who can initiate an export, how long it remains available, which format the firm receives and how it verifies completeness. A right to request logs after the service has been switched off is weak if the identifiers needed to ask for them have already disappeared. The decision is whether the firm can operate that evidence path throughout the period it has chosen to keep the record.

The limit of the claim

This is not a claim that every AI purchase is a high-risk system under the EU AI Act, or that one retention period fits every system. Role, classification, timing and other law require a specific assessment. The buyer’s design question exists even where that Act does not apply: can the organisation recover the record on which its decision depends? Legal interpretation stays with counsel.

The head of procurement and the system owner can make that question an acceptance test. Request a real export and name the party responsible for each missing field. The decision to buy then rests on an evidence path the firm can operate, not a screenshot of one it cannot reach.