Your customers are asking for AI. Your competitors are claiming it. Your board or your investors want to know what your AI strategy is.
And the tools have never been easier to use. Most modern platforms now ship with native LLM integration, retrieval over your own documents, and agent frameworks built right in. Your developers can have an AI feature demoing in a couple of weeks... and they're excited to do it.
That's exactly when this conversation matters most.
There's a line from Jurassic Park I think about a lot these days. Ian Malcolm warns that the scientists were so preoccupied with whether they could, nobody stopped to ask whether they should. Swap "scientists" for "developers," and you have the biggest risk in AI right now. Not that we can't build it. That we build it without judgment.
Why do so many AI features fail quietly?
Most AI failures aren't dramatic. Nobody's getting eaten by a dinosaur. But things do get loose, and they come with real costs.
The confident chatbot
It tells a customer something wrong, and it sounds completely sure about it.
The drifting agent
It slowly starts making decisions it was never approved to make.
The surprise bill
A feature that suddenly costs ten times what it did last month.
The feature nobody uses
It clutters your interface and adds to your maintenance budget for years before somebody finally removes it.
We've all seen that last one play out in the software industry long before generative AI came along. The difference now is how fast you can build the thing you shouldn't have built.
So how do you decide whether you should? Before we get to that, there's one concept everything else depends on.
What makes AI different from the software you already run?
Most of the software your business runs on is deterministic. A SQL query, a tax calculation, a validation rule... the same input produces the same output, every time. It's predictable, and you can test it.
Generative AI is probabilistic. Ask it the same question twice and you can get two different answers. The same agent, given the same task, can take a completely different path to get there.
Think about the spam filter in your email. The probabilistic part looks at an incoming message and produces a score, something like "87% likely to be spam." On its own, that score doesn't do anything. A deterministic rule acts on it: if the score is above a certain threshold, move the message to junk. The score is probabilistic. The rule is deterministic. It's the combination of the two that makes it useful.
Generative AI works on the same principle, with a different mechanism. Under the hood, it's a lot like the predictive text on your phone. If you've ever tapped the next suggested word over and over, you know what happens: you get a string of words that look like they belong together but don't actually say anything. A large language model is that same kind of algorithm, trained on an enormous amount of data and wrapped in a lot more context. It's very good at predicting which words belong together. But it doesn't understand any of it, not in a human sense.
The useful part is everything around the model: the prompts you control, the documents you ground it in, the output validation, the guardrails, the audit trail.
What are the five questions?
When you're evaluating an AI use case, there are five things you need to assess:
Value
Is it worth it?
Fit
Is AI the right tool?
Risk
What if it goes wrong?
Operability
Can you run it?
Trust
Will it still work in six months?
If you can answer all five honestly, you have what you need to make a good decision. If you can't answer one of them, it's time to do some research, not start your next sprint.
1VALUE
What measurable outcome does it produce, and what will it really cost?
Value is the cheapest place in the whole lifecycle to kill a bad idea.
A lot of AI business cases are basically fantasy math. There are some words about how great the idea is, an upside based on anecdotes, very little about long-term operating costs, and almost never a comparison against the cost of doing nothing. You need three numbers, with real metrics behind each one.
Business value
Pick an outcome that matters (revenue, retention, cost savings, compliance) and quantify it against today. "It'll save a lot of time" isn't a metric. "It takes our team 43 minutes per case" is.
Cost of doing it
Not just the build. Ongoing maintenance, the human review layer, and token costs, with a plan to watch them as usage grows. AI looks a lot cheaper in a proof of concept than in production.
Cost of not doing it
The one most teams skip. Compare against the status quo, not against zero. The status quo is almost never free.
Here's what that third number looks like with real metrics:
A 43-minute task, done 150 times a week, at a fully burdened $25 an hour, costs about $2,700 a week. That's the number your AI feature needs to beat.
Walk into your CFO's office with all three numbers and you can have an adult conversation about whether to fund it. Walk in with only one and you're going to lose that conversation. And frankly, you should.
2FIT
Is this the right problem for a probabilistic system?
Of the five questions, this is the one most business leaders are least equipped to answer on their own. Generative AI has real limits. It's not the right tool for every use case... but refusing to use it for anything at all is its own kind of mistake. The goal is to see clearly which problems it actually fits.
That comes down to a tension you have to keep in mind every single time: variation is a feature for creative work, and a defect for business automation.
Picture a dial. On one end are things like creative writing, image generation, and brainstorming, where you want a lot of variation. If you use the same prompt twice and get the same output, you've got a bad creative tool.
In the middle are things like code generation, legal document drafting, and highly personalized drafts that depend on a clear understanding of specific human context. Variation is still useful there. The model can write a solid first draft. But with stakes this high, you need a human expert who can think critically about what comes out.
On the other end are document summarization, data extraction, and financial calculations. Here, variation is a defect. You want as little of it as possible, or none at all.
Once you know where a use case sits, the work is building the guardrails that keep it there. Here's a tip: put this dial in front of the people asking for the feature and let them place their own use case on it. It turns "can we have AI?" into a much better conversation about how much variation they can actually live with.
Where it lands tells you whether you're looking at an easy fit, a hard fit, or no fit at all.
Easy fit
Variation is a benefit and the stakes are low. It's an easy pass, and quick approvals here earn you credibility for the harder conversations.
Hard fit
Doable, with a plan to tune it and keep it inside your tolerance: careful prompt engineering, deterministic checks on every output, monitoring that catches drift, and a human review layer for what the checks miss. The answer is "yes, and here's what it takes to do it safely."
No fit
Anywhere "approximately right" isn't acceptable. It's either right or it's wrong, and wrong has consequences. The better answer is usually deterministic code that produces the same output every time.
No fit doesn't always mean no AI. If there's a real reason to use it in the workflow, wrap it in a strong deterministic layer that decides what gets shown, what gets stored, and what gets acted on. Sometimes the best move is to build your rules into a tool or an MCP endpoint. You define what's allowed, and the AI has to follow your protocol, so even a complex system behaves predictably.
While you're thinking about Fit, think about how the feature might break in production. There are two main ways:
- Hallucination: the model confidently says something that just isn't true.
- Interpretation failure: the model gives a correct answer to a different question than the one that was asked. This one is harder to catch, because there's nothing obviously wrong to flag. You have to know what was meant and notice that the answer doesn't match.
3RISK
What's the blast radius if it fails or gets misused?
When people hear "governance," a lot of them hear "barrier to innovation." But it's not a tug-of-war between the people who want speed and the people who want control. It's another set of ditches on either side of the road.
One ditch is no governance. You ship fast and break things, until an incident is big enough that somebody else writes your rules for you. The other ditch is governance so heavy that nothing ever ships, and your competitors eat your lunch.
The teams that ship AI most effectively usually have explicit but concise rules. Nobody reads a 120-page AI policy. A three-page one, plus a simple process for getting a use case reviewed, gets real buy-in. There are three layers to look at.
Data
Where most of your risk lives. Classify what goes in (public, internal, or sensitive like PII, health, or payment data), decide what comes out, how it's protected, which regulations apply, and what your audit trail captures.
Ethics
Does it make determinations that need human judgment? Do inputs and outputs need review? Could biased output cause harm? Can you protect the sensitive data involved?
Vendors
Your feature depends on someone else's pricing, terms, and models. Build for multiple providers, avoid vendor-specific lock-in, and read the contract terms closely.
Log the inputs, model version, outputs, human actions, and timestamps, so you can walk a customer or a regulator through a specific result six months later. And remember that putting a human in the loop isn't enough by itself.
4OPERABILITY
Can you actually run it, monitor it, and pay for it at scale?
Operability is the question almost nobody plans for at proposal time. When you ship an AI feature without an operating model, nobody owns it, nobody's watching it, the bill goes up, and the quality goes down.
AI features decay differently than traditional software, and they can be wrong for a long time before anyone notices. You need a plan to monitor eight things:
- 1Quality drift: an evaluation set with known-good answers, run on a schedule
- 2Model version changes: pin versions and treat every update like a deployment
- 3Cost drift: cost per unit, actually watched
- 4Prompt injection and abuse: logged and reviewed
- 5Human-in-the-loop health: how often reviewers edit or override
- 6Ownership: a named owner, with a backup
- 7User feedback: signals that it's actually helping
- 8Team AI literacy: people who understand how it works
If your plan doesn't cover all eight, you don't have an operating model. You have hope.
5TRUST
How will you know it's still doing what you approved six months from now?
Trust is the hardest of the five, because every answer you've given so far is a snapshot. With probabilistic systems, you can't set it and forget it for a few years until it's convenient to update.
A review date on the calendar
Quarterly, on every major model change, or whatever the blast radius justifies. When it hits, re-ask the first four questions. Is it still saving 43 minutes per case, or only 12 now?
Tuning between reviews
Refine prompts as you learn what users actually ask, grow the evaluation set as edge cases show up, and only adopt new models when they pass your tests, on your schedule, not the vendor's.
How do you turn the answers into a decision?
Once you've worked through all five questions, put the result into one of three categories:
Green: ship
Low blast radius, internal only, no sensitive data. Your team can ship these at its own discretion.
Yellow: ship with conditions
Customer-facing, sensitive data, or a new kind of use case. Spell out exactly what has to be true before it ships.
Red: not yet
Ethical concerns, sensitive data without the right controls, or no named owner.
Treat everything as yellow and nothing ships. Treat everything as green and you'll have incidents. Short, decisive, and explicit is the goal, and revisit the categories as your platform and the regulations change.
What should you do Monday morning?
- Pick your top three candidate AI features.
- Walk each one through Value, Fit, Risk, Operability, and Trust.
- The ones that pass all five are your first wins. The ones that don't need a redesign, or shouldn't ship.
- Then ask one more question: who owns AI governance at your company? If the answer is "no one," that's the first problem to solve. Everything else depends on it.
Get the Five Questions guide
The five questions, the decision categories, and a checklist for any AI feature already in production, in a two-page PDF you can share with your team.
Earning the right to keep delivering it
Every major technology shift brings the same worry: the jobs are going away. But history shows a different pattern. When something that used to be a bottleneck gets automated, the bottleneck doesn't disappear. It moves. With AI, it's moving away from building and toward judgment.
That's what these five questions are for. They aren't a checklist you run once to approve a feature. They're a way of looking after it across its whole life, which is the same idea behind our Software Stewardship Framework.
When you answer these questions, and keep answering them as the ground shifts, you're not just shipping a cool new AI feature. You're earning the right to keep delivering it.
If you'd like help putting this into practice, PALADEM builds AI features into SaaS platforms with these guardrails in place, and we can provide a dedicated development team with the leadership to run it.