How to evaluate an AI tool before you trust it with real work

How to evaluate an AI tool before you trust it with real work

Most AI tools are judged on their demo. Demos are chosen by the vendor, run on inputs the model handles well, and are evaluated by eye. That tells you what the tool can do on its best day. It does not tell you what it will do on yours.

This is a short protocol for the other question: should this thing touch work I get paid for? It takes about an hour and needs no infrastructure.

1. Pick a task whose answer you already know

Start from a real task you have already completed by hand, where you know what the correct output looks like. Then run the tool on the same inputs.

If the tool cannot reproduce work you have already finished, the interesting part is not the score — it is where it diverges. Write those divergences down; they become your review checklist for every future run.

2. Test the boring cases, not the demo cases

The demo case is a clean photo, a well-formed sentence, a tidy spreadsheet. Real input is: a filename with an ampersand in it, a document in a language the tool was not trained on, an image with rotation metadata, a file that is empty, a row where one field is missing.

Failure on boring cases is not a reason to reject a tool. Silent failure on boring cases is: if the tool claims success while producing garbage, you will not notice until a customer does.

3. Look at the failure mode, not the failure rate

Two tools with the same error rate can be very different suppliers. Ask:

• Does it fail loudly (error, empty output) or quietly (confident nonsense)?
• Is the failure recoverable — can you re-run one item, or is the whole batch lost?
• Does it leave the original input untouched?

A tool that fails loudly and touches nothing is usually safe to adopt, even if it fails often. A tool that fails quietly can be unusable even at a 2% error rate, because you must review 100% of the output to find the 2%.

4. Check where your data goes

"Inference happens on our servers" is a different product from "the model runs on your laptop", and the difference is not only privacy. Ask which of these is true for your data:

• files are uploaded and deleted after processing, or retained for training;
• output is generated in a shared service, or locally with your own compute;
• the tool works with the network unplugged, or stops working.

If the answer to the last one matters to you — regulated data, offline sites, slow connections — test it, do not read the marketing page. Pull the network cable and run one job.

5. Decide in advance what happens when it is wrong

Nothing here is about avoiding mistakes; it is about arranging them. Before you adopt the tool, write down:

• who reviews the output, and against what checklist;
• how you roll back (the cheapest rollback is not overwriting the input);
• what triggers abandoning the tool entirely.

For higher-stakes settings, the NIST AI Risk Management Framework and the OWASP list for LLM applications are good structures to borrow vocabulary from, even if you implement none of them formally.

What this changes

None of these steps requires a benchmark suite or a labelled dataset. They require an hour of honest use on inputs the vendor did not choose — which is the only kind of test that tells you anything about your own work. And it is why tools that let you inspect the intermediate steps, keep the originals and run offline are easier to trust: they make the failure mode visible instead of hiding it.