Insights/ Strategy
Not every business problem needs AI
MIT found 95% of enterprise generative-AI pilots deliver no measurable financial return. Why knowing where AI doesn't belong matters more than the technology.

Artificial intelligence has moved from a specialist concern to a boardroom default. Worldwide AI spending is forecast to reach roughly $1.5 trillion in 2025, and most organisations now report using AI in at least one business function. Yet the most-cited research of the past year tells a humbling counter-story: the majority of enterprise AI pilots deliver no measurable financial return. This article argues a deliberately unfashionable point: not every business issue should be solved with AI, and treating it as a universal solvent is one of the more reliable ways to waste money. The discipline behind that argument transfers directly to how we think about security awareness.
Data sourcesMIT Project NANDA, The GenAI Divide: State of AI in Business 2025 · Gartner, worldwide AI spending forecast (2025) · McKinsey, The State of AI (2025) · S&P Global Market Intelligence (2025) · IBM Institute for Business Value (2025) · LLM API pricing aggregators (2026)Figures reflect publicly reported research and vendor pricing as of mid-2026. Model prices change frequently. Verify current rates before budgeting.
01The unfashionable argument
The smartest AI decision an organisation makes is sometimes the decision not to use it
MIT's Project NANDA, in The GenAI Divide: State of AI in Business 2025, found that despite an estimated $30 to 40 billion in enterprise spending, roughly 95% of generative-AI pilots delivered no measurable impact on profit and loss. Separately, S&P Global reported that 42% of companies abandoned most of their AI initiatives in 2025, more than double the prior year.
The failures aren't primarily about weak models. They're about organisations applying AI to the wrong problems, in the wrong way, without integration into real workflows.
The question is rarely "is the AI good enough?" It's "should this problem have been handed to AI at all?" That's a discipline we recognise, because we make the same argument every day about a different category of tool. In security awareness, the reflex is to buy the platform and assume the problem is handled. It rarely is. The lesson transfers cleanly: the tool is not the answer. The judgement about where the tool belongs is the answer.
of generative-AI pilots showed no measurable P&L impact.
MIT NANDA, 2025of companies abandoned most of their AI initiatives in 2025, more than double the prior year.
S&P Globalsuccess rate for buying from specialist vendors, versus roughly a third of that for internal builds.
MIT NANDA, 202502Rules don't need a model
Simple automation doesn't need artificial intelligence
A large share of "AI projects" are, on inspection, deterministic problems wearing a fashionable label. If a task has fixed rules, predictable inputs, and a single correct output, a language model is usually the most expensive and least reliable way to perform it. The alternatives have existed for years:
The distinction is between deterministic and probabilistic systems. A rules engine returns the same answer for the same input by design. A language model produces a likely answer: a feature for open-ended language tasks, a liability for tasks that demand exact, repeatable outcomes.
A practical heuristic: if you can write down the rule on a single page, automate the rule. Reserve AI for the problems where the rule can't be written down: ambiguous language, unstructured documents, judgement, and genuine variability.
03A cost structure most haven't managed
How expensive, and how predictable?
Generative AI introduces a cost structure most organisations have never managed: consumption-based, per-token billing. A model charges for every unit of text it reads and produces, priced per million tokens, with output almost always more expensive than input. As of mid-2026, published rates ranged from well under $1 per million tokens for budget models to more than $100 per million output tokens for the most expensive reasoning models, a difference of two orders of magnitude or more for the same unit of work.
The most powerful model is usually overkill
Reasoning-grade models cost more per token and consume extra tokens internally as they "think," so real-world costs run several times higher than the headline rate. Using a premium model to classify a support ticket is the computational equivalent of hiring a senior consultant to file paperwork: the same 10,000 tickets can cost ten to a hundred times more on a frontier model than on a budget one.
Variability is itself a risk. A fixed licence is easy to budget. A per-token bill scales with usage, prompt length, and context resent on every call. Without controls, costs drift unpredictably, which is why finance and engineering increasingly track spend per task, not merely per month. Spending on AI without a predictable cost model is a signal of risk, not ambition.
04Right-size the tool
Match the model to the challenge
If a problem genuinely warrants AI, the next decision is which AI, and the dominant failure mode is reaching for the largest model by default. Premium tiers can cost several times their mid-tier siblings. The disciplined approach matches capability to the task:
High-volume, low-complexity tasks
Classification, extraction, routing, short replies. A budget or balanced model is almost always sufficient: the accuracy gain from a premium tier rarely justifies the cost multiple.
Conversational & customer-facing work
A balanced mid-tier model typically offers the best quality-to-cost ratio for interactive, customer-facing tasks.
Complex reasoning & high-stakes judgement
Reserve the frontier tier, and only where the accuracy gain genuinely justifies the cost, not as the default starting point.
This is where MIT's most actionable finding belongs: buying from specialised vendors and building partnerships succeeded roughly 67% of the time, while internal "build-it-ourselves" efforts succeeded at about a third of that rate. For most organisations, the cheaper and more reliable path is to apply mature, existing AI well, not to construct bespoke systems from scratch.
05The human layer
Where this lands for security awareness
There's a reason this argument sits naturally with us. The same logic that says "don't hand a deterministic task to a probabilistic model" also explains why fully automated training rarely moves the needle on security behaviour.
A compliance module is a deterministic system: same content, same completion screen, same result for everyone. Genuinely useful for recording that training happened. The wrong tool for the actual problem, which isn't a record. It's a change in how a real person behaves under pressure.
That moment, ambiguity, context, judgement, a person who needs to be understood rather than processed, is precisely where automation reaches its limit and human-led support earns its place. Seat counts and completion rates are the security-awareness equivalent of hollow usage statistics: evidence that something ran, not evidence that anything changed.
Used well, AI has a real and honest place in this work: surfacing the threat patterns that matter, sharpening simulations, freeing human specialists to spend their attention where only human attention works. That's the line we hold: the technology serves the people, never the other way around. If the problem is a record, automate it. If the problem is a person, support the person.
06Before you commit
Seven questions to answer first
If the logic fits on one page, automate the rule. Save AI for ambiguity, language, and unstructured inputs.
Compare the AI solution against the non-AI alternative, RPA, a query, a template, or a human, not against doing nothing.
Estimate token consumption per task, model the variable cost at realistic volume, and design caching and batching in from the start.
Default to the smallest model that meets the quality bar; escalate only where accuracy genuinely demands it.
Most failures trace to brittle, poorly integrated workflows. A pilot that can't connect to real systems, or real people, won't scale.
Given the success-rate gap, prefer mature, proven solutions over bespoke construction unless there's a defensible reason to build.
Define the metric before deployment. Usage statistics, seat counts, hours logged, are not evidence of value.
·Key takeaways
Value comes from precision, not ubiquity
They applied AI less discriminately. The discipline that separates success from failure is unglamorous, not technical.
If you can write the rule on one page, automate the rule. Reserve AI for genuine ambiguity and judgement.
Mature, purchased solutions succeed roughly 67% of the time. Internal builds succeed at about a third of that rate.
A completion record isn't behaviour change. Where the problem is a person, not a record, human-led support still earns its place.


