A practical test for whether an AI feature belongs in your product — and the three shapes that tend to survive contact with real users.
Adding a model to a product is easy now. Deciding whether it should be there is the part that still takes judgement, and it's where most AI features quietly fail: they demo beautifully and go unused.
The test
An AI feature earns its place when the task has three properties at once: the user knows what a good result looks like, producing it by hand is tedious, and being wrong occasionally is survivable.
Drop any one of those and it breaks. If the user can't judge the output, they can't trust it. If the manual version is already fast, the model is friction. If a wrong answer is expensive — money moved, a message sent, a record deleted — you need a human in the loop, which changes the design entirely.
Three shapes that tend to work
Almost every AI feature I've seen deliver real value falls into one of these:
- The blank page killer. The model produces a first draft the user edits. Value comes from removing the hardest 10% of the work, not from being right.
- The extractor. Unstructured input in, structured data out — invoices, emails, transcripts. Verifiable, boring, and enormously useful.
- The router. The model decides where something goes or what it is, and a deterministic system does the rest. Cheap to evaluate, easy to bound.
Notice what's missing: the open-ended chatbot bolted onto a product that already has a perfectly good interface.
Build the evaluation before the feature
The most common engineering mistake is shipping a prompt with no way to tell whether a change made things better. Prompts are code with no type system and no tests — unless you build them.
Before writing the feature, collect twenty real inputs and write down what a good output looks like for each. That set is your regression suite. It costs an afternoon and it's the difference between iterating and guessing.
// The cheapest useful eval: a fixed set of real cases,
// a scorer you trust, and a number that has to go up.
const cases = await loadCases("./evals/support-triage.jsonl");
const results = await Promise.all(
cases.map(async (c) => ({
id: c.id,
expected: c.expectedCategory,
actual: (await triage(c.input)).category,
})),
);
const accuracy =
results.filter((r) => r.actual === r.expected).length / results.length;
console.table(results.filter((r) => r.actual !== r.expected));
console.log(`accuracy: ${(accuracy * 100).toFixed(1)}%`);Design for being wrong
Every model output is a guess with a confidence you can't fully see. The interface has to make that honest: show the source, make the edit obvious and cheap, and never take an irreversible action on the model's word alone.
Products that get this right feel like a fast assistant. Products that get it wrong feel like a slot machine — and users stop pulling the lever.