AI testing for consultancies and integrators
If you build or integrate assistants for your clients, an independent test goes with the delivery, with criteria the client signs before it starts and a third-party report the client can take to its management.
AI testing for consultancies and integrators. If you build or integrate assistants for your clients, an independent test goes with the delivery, with criteria the client signs before it starts and a third-party report the client can take to its management.
What can go wrong
An invented example of how it can fail: One of the client’s users asks something that was not in the project’s test plan. The assistant answers confidently from a document that does not deal with that subject. The tests written by the team that built it measure what that team expected. An independent test also looks for what nobody foresaw, and the client receives it signed by a third party.
Questions the test would include:
- What does it answer when the question falls outside the client’s documents?
- When two versions of the same document coexist, does it cite the one in force?
- Does it keep the same answer when the user phrases the question another way?
What the AI Act means here
Whoever puts their name or trademark on a high-risk system already on the market takes on the provider’s obligations (Article 25(1)). The same applies to a substantial modification of such a system, or to changing a system’s purpose so that it becomes high-risk. It is worth stating in the contract who the provider is. Most document assistants are not high-risk, and then the test is good practice you can deliver with the project.
In any case, if the assistant talks with people, they must be told they are talking to an AI unless that is obvious (Article 50, from 2 August 2026). And whoever develops it or uses it in their organisation must take measures to support the AI literacy of the staff who work with it (Article 4). Whether a system is high-risk is a legal question for your adviser. I do not classify it; the test gives you evidence either way.
With the delivery, your client receives an independent third party’s evidence of how the assistant performs against the criteria the client signed.