The AI Supplier Questions Procurement Should Be Asking and Usually Is Not

Your supplier questionnaire is a good document. It asks where data is hosted, who has access, how long it is retained, what the encryption is, what the breach notification period is, and whether the vendor holds a certificate. It has caught real problems and it was written by somebody competent.

It was also written for software that behaves the same way in June as it did in March. That assumption is now false in one specific place, and the questionnaire has no idea.

The AI feature your accounting package added does not have a fixed behaviour. It has a behaviour today. The vendor can change the model behind it without changing a line of code you can see, without a release note reaching you, and without breaching a single clause in the contract you signed — because nothing in that contract contemplated it. Everything below is the set of questions that closes that gap, and the shape of the non-answers that should worry you. It applies whether or not you are pursuing a formal AI management system; the difference is only whether you have to evidence it.

First, decide which conversation you are having

Two very different things get called "an AI supplier", and asking one set of questions of the other wastes everybody's time.

The vendor whose product has an AI feature. Your ERP proposes a three-way match. Your CRM scores a lead. Your helpdesk drafts a reply. You did not build it, you cannot inspect it, you configure a little of it, and you are relying on it in some proportion. This is a supplier-risk conversation. It is also, by volume, ninety per cent of what any mid-market company actually faces.

The vendor supplying you a model or a platform you build on. An API, a hosted model, a fine-tuning service. Here you are the one shipping the decision to your own customers, and their exposure is your exposure.

The questions overlap heavily. The weight does not. For the first, the goal is a defensible register entry and two or three contract clauses. For the second, the goal is a supply chain you can stand behind under questioning, because you are now the one answering somebody else's questionnaire.

A third case is worth naming to dismiss it: an employee using a public chatbot to draft something they publish under their own name. That is an acceptable-use policy, not a supplier assessment. Treating it as the latter is how organisations generate a hundred register entries and govern none of them.

The question that matters most, and nobody asks it

What obliges you to tell us when the model changes, and how much notice do we get?

Ask this first. Ask it before hosting, before certificates, before anything on your existing form. It is the question with no counterpart anywhere in conventional supplier management, because conventional supplier management assumes change originates with a release you can see.

Here is the mechanism it defends against. You evaluate a feature, you validate it against a sample of your own work, you find it acceptable, you switch it on, and you build a process around it. Six months later the vendor swaps the underlying model — for cost, for capability, for a supplier change of their own. Your validation is stale and there is no event anywhere in your organisation that tells you so. The process continues, its outputs have drifted, and the first sign is a downstream error somebody blames on a person.

What you want in the contract is narrow and gettable: notice before a material change to the model or its provider, with a defined period, and a right to disable the feature without penalty if you do not accept the change. Vendors resist the second half harder than the first. The first half alone is worth a great deal.

And notice is useless without a receiver. Name a person on your side whose job it is to act on the notification, and connect it to your own change process so that a supplier notice triggers re-validation rather than an email nobody files. Otherwise you have bought a clause and not a control.

Data: three questions, not one

Your questionnaire asks where data is stored. That is one of three, and it is the least interesting.

What does the feature actually send, and where does it go? Not "our platform is hosted in region X" — what leaves your tenancy at the moment the feature runs. If the vendor calls a third-party model, your data is at that third party regardless of where the vendor's database sits. This is the single most common surprise in these assessments, and it is usually not concealment. The vendor's sales engineer genuinely does not know.

Is our data used to train or improve any model? Ask for the answer in the contract, not in a policy page that can be edited on a Tuesday. Ask separately about the vendor's own models and about any third party's, because the answers frequently differ. And ask what happens to that if you leave: a commitment not to train is worth little if the data already ingested stays ingested.

Who are the sub-processors, and what tells us when the list changes? A model provider is a sub-processor. A change of model provider is a change of sub-processor, and it is the change most likely to happen without anyone telling you — which is the same defect as the model-change question wearing different clothes. If your data protection terms already have a sub-processor notification mechanism, this is the cheapest question on the list, because you are asking them to apply a clause they already signed.

Rights: the question that survives the deal

On what basis was the model trained, and what happens to us if that is challenged?

You are not going to audit a training corpus and nobody expects you to. What you are looking for is whether the vendor will stand behind it — an indemnity, or at minimum a statement of position, covering claims arising from the training data or from the outputs the model produces.

The reason this is a procurement question rather than a legal curiosity: the exposure lands on the party that used the output, and that is you. If a model produces something that infringes and you published it, the first letter arrives at your address. What the contract says about who defends that letter is a commercial term, and it is negotiable at signature and almost never afterwards — which is the general property that makes reading a contract for its exit rather than its discount the only reading worth doing.

Ask the same question about your own data going the other way. If you fine-tune on your records, who owns the resulting weights, and can you take anything with you when the contract ends? The usual answer is that you cannot, which is fine as long as you knew before you spent eight months building a process on it.

Behaviour: what happens when it is wrong

Every AI feature is wrong sometimes. A supplier who will not describe how theirs is wrong has either not measured it or does not want to say, and both are answers.

What does it do when it does not know? The dangerous design is one that always produces something. A feature that returns a confident wrong answer is worse than one that abstains, because abstention is visible and a wrong answer is not. Ask what the low-confidence behaviour is, and whether you can see the confidence.

Can we tell, from the record, that the output came from the model? If an AI-proposed value and a human-entered value are indistinguishable in the database afterwards, you cannot measure the feature, you cannot audit it, and you cannot roll it back. This is a data model question dressed as a governance one, and it is the difference between a feature you can manage and one you can only trust.

What can it do without a person? The answer is a specification, not a reassurance. Which records it can write, up to what value, in which states, under whose credential. An agent operating inside your systems needs its own login, a write boundary and a value ceiling before it is switched on, and if the vendor cannot describe those three things for their own product, they have not designed them.

What did the last twelve months of incidents look like? Not "have you had a breach" — that is on your form already. Ask about incidents specific to the model: a bad update rolled back, a systematic error found in production, a customer-reported failure class. A vendor with a mature product has some and will describe them. A vendor with none is either very new or not looking.

Exit, which is where the real cost hides

What do we get back, and in what state?

The AI-specific version of a question you already ask. If the feature has been categorising, scoring or enriching your records for two years, those derived values are now embedded in your operational data. On exit you keep them and lose the ability to produce more, which is an asymmetry worth understanding before you depend on it. Ask whether the derived fields export, whether they export with their provenance, and what the record looks like without the vendor.

What is the cost curve? Usage-priced AI features have a different shape from per-user licensing: the bill follows adoption, and adoption is the thing you were trying to encourage. Ask for the pricing unit, what counts as one unit, and what has happened to that price historically. A vendor who will not commit to a price ceiling on a metered feature is asking you to sign a blank line.

Reading the answers

The answers matter less than their shape, and four shapes recur.

The specific answer. Names the model or the provider, names the region, quotes the contract clause, describes a failure. Whether or not you like it, you can act on it and it goes in the register as it stands.

The deflection to a certificate. We are ISO 27001 certified. That is a real and useful fact about information security management and it says nothing about model change, training data or output rights, because those are not what it certifies. Accepting it as an answer to any question in this article is the most common failure in AI supplier assessment, and it happens because both sides want the conversation to end.

The confident answer from someone who cannot know it. A sales engineer telling you no data leaves your tenancy, when the feature demonstrably calls something external. Not usually dishonest. The fix is procedural rather than adversarial: ask for it in writing from someone who can bind the company, and watch how quickly the confidence becomes a caveat.

The silence. No answer, or an answer to a question you did not ask. Record it in the register as a gap with the date you asked. A register with honest gaps in it is a working document. A register with no gaps has not been used.

What to do with all this

Two artefacts, and neither is large.

A register entry per AI feature in use: what it does, what data it sees, where that data goes, who the sub-processors are, what the change-notice position is, who owns it on your side, and when you last looked. This is the same register your existing supplier process already produces, with four columns added. Give it a review date and an owner, or it becomes a document that was true once.

A clause set you take to renewal: model change notice, sub-processor notification extended to model providers, a training-data position, an outputs indemnity or a stated position, and an exit description covering derived data. You will not get all of it. Ask for all of it, because the ones you get are the ones you asked for, and renewal is the only moment when a signed contract is open.

Then stop. The failure at the other end of this is a governance programme that treats a spell-checker as a scored AI system, generates a hundred assessments, and reviews none of them — the same collapse that turns connecting everything to everything into a permanent cost centre. Depth belongs where the decision is yours and the exposure is yours. For everything else, a named owner and four columns is a defensible position, and it is the position we would help you take before we would sell you anything larger.