Does Your Product Really Need AI?

Over the past few years most product teams have been asked the same question: where can we use AI?

The question has an ordering problem built into it. Starting with a solution and looking for a problem to fit it rarely works well in product development. The same approach was tried with mobile apps, chat interfaces, and blockchain; most of what it produced were features nobody used.

The more useful order runs the other way: what problem do you have, and what kind of answer does that problem require?

What kind of answer does the problem need?

Product problems divide roughly in two, and the division comes before any technology choice.

Some have one correct answer that can be computed. The total on an invoice, whether a user has permission, whether stock has fallen below a threshold. These are rules. Putting a language model in front of them only makes something certain probabilistic.

Others require judgement, and the input is unstructured: inferring intent from free text, summarising a long document, matching similar records, routing a request to the right team. There is no single correct answer here — there is an acceptable range.

Discussions held without making this distinction tend not to resolve, because the people in them are talking about different kinds of problem.

The deciding axis: tolerance for being wrong

Even a judgement problem is not automatically a good fit. What actually decides it is what happens when the output is wrong.

In some tasks a wrong answer is cheap. If a list of suggestions misses, the user ignores it; if a summary is thin, they open the source. The cost is small and reversible.

In others it is expensive: routing a payment, interpreting a contract clause, presenting medical or legal information, determining what a user is allowed to access. "Right most of the time" is not a sufficient standard there.

That axis settles much of the decision on its own. Where being wrong is cheap, a probabilistic approach is comfortable. Where being wrong is expensive, either a deterministic path is chosen or human review becomes part of the design rather than an afterthought.

Where the deterministic answer is better

If something a model can do is also achievable by a simpler method, the simpler method is almost always right.

Search is the most common example. A number of features positioned as "AI-powered search" are solved faster, more cheaply, and far more predictably by well-built filtering and ranking. Users are frequently not looking for a clever answer — they are looking for the right filter.

Classification is the second. Where the number of categories is small and the rules are clear, a rule set is both more accurate and explainable. Where the boundaries are genuinely too blurred to write down, a model starts to earn its place.

Generation is the third. Producing text a template could produce reduces consistency and raises cost. If the text genuinely has to vary and respond to context, the calculation changes.

Three costs that stay hidden

The cost of an AI feature is usually estimated as development time. The real line items sit elsewhere.

Data. A model-based feature needs context to work. If that context does not exist in your product — scattered, incomplete, or inaccessible — the quality of the feature is capped before it starts. On most projects the real work is not connecting a model but making the data usable.

Latency. A model's response is noticeably slower than a database query. When the wait sits in a search box, that difference is felt directly. This is the same subject as how performance shapes perceived quality; only the source is different.

Variability. The same input may not produce the same output twice. That complicates testing, debugging, and support. "It worked when I tried it" becomes a technical fact rather than an excuse.

What does "accurate enough" mean?

Most AI discussions proceed without anyone stating an accuracy target. The result is that nobody can declare success at the end of the project.

The most practical way to set one is comparison with what exists. If this work is done by hand today, how often is it wrong when done by hand? In most companies that number is unknown and assumed to be zero. In reality, a tired person classifying several thousand records makes mistakes too — they have simply never been measured.

Without that comparison the model is tested against an expectation of perfection, which it always loses. With it, the question becomes more honest: is this feature better than what the current process produces, and is the shape of its errors more manageable?

The second point is that error types are not equal. In classification, a false positive and a false negative usually cost different amounts. Wrongly flagging a request as urgent is not the same as missing one that is. A target should therefore be expressed not as a single percentage but as a statement about which kind of error is acceptable.

The third is that accuracy should not be measured once and abandoned. Building a small evaluation set from real usage is the only practical way to notice when a version change has broken something that previously worked.

The cost is not once, it is every use

With conventional features, cost sits mostly in development; once live, running them is comparatively cheap.

Model-based features invert that. Every call carries a price, and that price grows with usage. If the product succeeds, the cost grows with it — the opposite of the curve most teams are used to.

The practical consequence is that where and how often the feature is invoked becomes a design decision. A feature that runs on every keystroke and one that runs only when the user asks for it may differ little in usefulness and enormously in cost.

The same question applies to caching: does the same work genuinely need doing again for inputs that have been seen before?

Budgeting for it differs too. A fixed development cost is straightforward to approve; an operating cost that grows with usage produces surprises unless a ceiling is discussed at the outset.

Where is the value created?

The last and often most revealing question: is AI the value of your product, or the way that value is delivered?

In products where it is the value, model quality sits at the centre of competition, and investing there is correct.

That distinction also determines how it should be built. Where the value is the model, the model is what you work on. Where the value is the delivery, using an existing service is almost always right — competition does not live in that layer, and training your own brings far more maintenance than it resolves.

For most products it is not. AI is a layer that makes an existing job a little less effortful. That is legitimate and worth having — but if the core job still does not work well, a clever layer on top will not compensate. As with scoping an MVP, a feature added before the main flow is finished also ruins the measurement: did people dislike the product, or the feature?

In closing

Not using AI is not a step backwards; for many problems it is the correct engineering decision. Equally, refusing it where it fits is just stubbornness.

What separates the two is not a view about the technology but the shape of the problem: does the answer need to be certain, what does being wrong cost, and is there a simpler route?

In our AI integration work, the conversation usually opens with those three questions rather than with a model — because the answers also determine how complicated the rest of the project is going to be.