The smartest AI decision an organisation makes is sometimes the decision not to use it
MIT's Project NANDA, in The GenAI Divide: State of AI in Business 2025, found that despite an estimated $30–40 billion in enterprise spending, roughly 95% of generative-AI pilots delivered no measurable impact on profit and loss. Separately, S&P Global reported that 42% of companies abandoned most of their AI initiatives in 2025 — more than double the prior year.
The failures aren't primarily about weak models. They're about organisations applying AI to the wrong problems, in the wrong way, without integration into real workflows.
The question is rarely "is the AI good enough?" It's "should this problem have been handed to AI at all?" That's a discipline we recognise, because we make the same argument every day about a different category of tool. In security awareness, the reflex is to buy the platform and assume the problem is handled. It rarely is. The lesson transfers cleanly: the tool is not the answer. The judgement about where the tool belongs is the answer.
of generative-AI pilots showed no measurable P&L impact.
MIT NANDA, 2025of companies abandoned most of their AI initiatives in 2025 — more than double the prior year.
S&P Globalsuccess rate for buying from specialist vendors, versus roughly a third of that for internal builds.
MIT NANDA, 2025Simple automation doesn't need artificial intelligence
A large share of "AI projects" are, on inspection, deterministic problems wearing a fashionable label. If a task has fixed rules, predictable inputs, and a single correct output, a language model is usually the most expensive and least reliable way to perform it. The alternatives have existed for years:
The distinction is between deterministic and probabilistic systems. A rules engine returns the same answer for the same input by design. A language model produces a likely answer — a feature for open-ended language tasks, a liability for tasks that demand exact, repeatable outcomes.
A practical heuristic: if you can write down the rule on a single page, automate the rule. Reserve AI for the problems where the rule can't be written down — ambiguous language, unstructured documents, judgement, and genuine variability.
How expensive — and how predictable?
Generative AI introduces a cost structure most organisations have never managed: consumption-based, per-token billing. A model charges for every unit of text it reads and produces, priced per million tokens, with output almost always more expensive than input. As of mid-2026, published rates ranged from budget tiers around $0.10–$0.40 per million tokens to frontier reasoning models charging $25–$180 — a difference of two to three orders of magnitude for the same unit of work.
The most powerful model is usually overkill
Reasoning-grade models cost more per token and consume extra tokens internally as they "think," so real-world costs run several times higher than the headline rate. Using a premium model to classify a support ticket is the computational equivalent of hiring a senior consultant to file paperwork — processing 10,000 tickets has been estimated at roughly $16 on a mid-tier model versus around $0.80 on a budget tier.
Variability is itself a risk. A fixed licence is easy to budget. A per-token bill scales with usage, prompt length, and context resent on every call. Without controls, costs drift unpredictably — which is why finance and engineering increasingly track spend per task, not merely per month. Citi has identified a credit-spread penalty for companies classed as AI "adopters" versus "enablers" — the debt market charging a premium for spending without demonstrated return. Spending on AI without a predictable cost model is now a signal of risk, not ambition.
Match the model to the challenge
If a problem genuinely warrants AI, the next decision is which AI — and the dominant failure mode is reaching for the largest model by default. Premium tiers can cost roughly five times their mid-tier siblings. The disciplined approach matches capability to the task:
High-volume, low-complexity tasks
Classification, extraction, routing, short replies. A budget or balanced model is almost always sufficient — the accuracy gain from a premium tier rarely justifies the cost multiple.
Conversational & customer-facing work
A balanced mid-tier model typically offers the best quality-to-cost ratio for interactive, customer-facing tasks.
Complex reasoning & high-stakes judgement
Reserve the frontier tier, and only where the accuracy gain genuinely justifies the cost — not as the default starting point.
This is where MIT's most actionable finding belongs: buying from specialised vendors and building partnerships succeeded roughly 67% of the time, while internal "build-it-ourselves" efforts succeeded at about a third of that rate. For most organisations, the cheaper and more reliable path is to apply mature, existing AI well — not to construct bespoke systems from scratch.
Where this lands for security awareness
There's a reason this argument sits naturally with us. The same logic that says "don't hand a deterministic task to a probabilistic model" also explains why fully automated training rarely moves the needle on security behaviour.
A compliance module is a deterministic system — same content, same completion screen, same result for everyone. Genuinely useful for recording that training happened. The wrong tool for the actual problem, which isn't a record — it's a change in how a real person behaves under pressure.
That moment — ambiguity, context, judgement, a person who needs to be understood rather than processed — is precisely where automation reaches its limit and human-led support earns its place. Seat counts and completion rates are the security-awareness equivalent of hollow usage statistics: evidence that something ran, not evidence that anything changed.
Used well, AI has a real and honest place in this work — surfacing the threat patterns that matter, sharpening simulations, freeing human specialists to spend their attention where only human attention works. That's the line we hold: the technology serves the people, never the other way around. If the problem is a record, automate it. If the problem is a person, support the person.
Seven questions to answer first
If the logic fits on one page, automate the rule. Save AI for ambiguity, language, and unstructured inputs.
Compare the AI solution against the non-AI alternative — RPA, a query, a template, or a human — not against doing nothing.
Estimate token consumption per task, model the variable cost at realistic volume, and design caching and batching in from the start.
Default to the smallest model that meets the quality bar; escalate only where accuracy genuinely demands it.
Most failures trace to brittle, poorly integrated workflows. A pilot that can't connect to real systems — or real people — won't scale.
Given the success-rate gap, prefer mature, proven solutions over bespoke construction unless there's a defensible reason to build.
Define the metric before deployment. Usage statistics — seat counts, hours logged — are not evidence of value.
Value comes from precision, not ubiquity
The 95% didn't have worse models
They applied AI less discriminately. The discipline that separates success from failure is unglamorous, not technical.
Deterministic problems don't need probabilistic tools
If you can write the rule on one page, automate the rule. Reserve AI for genuine ambiguity and judgement.
Buying beats building, twice as often
Mature, purchased solutions succeed roughly 67% of the time. Internal builds succeed at about a third of that rate.
The same discipline applies to security awareness
A completion record isn't behaviour change. Where the problem is a person, not a record, human-led support still earns its place.
If this raised a question about where AI belongs in your security program
That conversation is worth having. Click or Flick Corporate applies exactly this discipline — AI where it earns its place, human-led support where the problem is a person, not a record.