AI Product Strategy: Deciding Where LLMs Belong in Your Product

Somewhere in your company right now, someone is being asked for an AI strategy. Usually the request arrives from a board member or a competitor’s announcement, and it is phrased as a deliverable: “What is our AI plan?” The honest answer is that “AI” is not a strategy any more than “software” was one. What you need is a set of decisions about which of your customers’ problems a language model should touch, how much autonomy it should have, what it will cost to run, how you will know it works, and what could go wrong.

I have spent this year close to this question from two sides. At Micro1.ai, I advise on product for LLM training across three projects, designing agentic response processes and scoring the quality of reasoning, and I test and analyze more than 80 business and commercial applications in development through sprint-based user testing. At Spoon Hire AI, I own the roadmap, pricing and go-to-market for an AI-native platform and keep the operation lean by building my own automations in Zapier, n8n and Claude Code. Behind both sits ten years in strategy consulting, market intelligence and product, which taught me to separate a decision from an enthusiasm. The two experiences point at the same lesson: the teams that do well with AI are not the ones with the most models. They are the ones that chose the right problems.

This post is the framework I use. It starts with the decisions, walks through the tests I apply to each candidate use of an LLM, and ends with the economics, evaluation, moat and risk questions that decide whether a feature survives its first year. For the general method on writing strategy, read how to write a product strategy that actually gets implemented first or alongside this one.

The six decisions of an AI product strategy

When I strip away the vocabulary, an AI product strategy is six decisions made in order. Skipping the early ones is how teams end up with a demo and no product.

1. Problem selection. Which customer problems are worth solving, and which of those can a model help with?
2. Approach selection. For each problem, is the right tool rules, classical analytics or machine learning, an LLM, an agent, or a person?
3. Autonomy. How much should the system do on its own, and when should a human approve?
4. Economics. What does each task cost to run, and does the pricing cover it with margin?
5. Evaluation and trust. How do you define “good”, measure it, and keep it from regressing?
6. Defensibility and risk. What makes the product hard to copy, and what could harm customers or the company?

The rest of this post takes them one at a time.

Decision 1: Start from the problem, not the model

Every failed AI feature I have seen began with the technology looking for a job. The antidote is dull: describe the customer’s problem in terms that do not mention AI, quantify it, and only then ask whether a model changes the answer.

Use the same diagnostic discipline as for any strategy. Gather evidence from customers, from your own usage data, and from competitors, using the methods in consumer research methods. Size what the problem costs in time or money, so you can later judge whether the solution is worth building, and use a bottom-up approach like the one in market sizing for expansion if the opportunity is new.

Then write a one-sentence job statement that a model could plausibly do better, faster or cheaper than the current approach. “Help customers write better emails” is vague. “Draft a first reply to routine support tickets that an agent can approve in under ten seconds” is a job.

Decision 2: Pick the simplest approach that works

Language models are powerful, flexible and expensive relative to simpler tools. They also fail in ways that are hard to predict. That makes them a poor default. Before you commit, run the problem through a ladder from simple to complex.

Rules and plain software. If the task is deterministic and the logic can be written down, write it down. A tax calculation, a validation check or a routing rule needs correctness, not creativity. Rules are cheaper, faster, auditable and never hallucinate.

Classical analytics or machine learning. If the task involves prediction from structured data, such as churn risk, fraud flags or demand forecasting, a purpose-built model often outperforms a general-purpose LLM and costs far less to run.

An LLM. Reach for it when the input or output is language, when the task involves interpretation, summarization, extraction from messy text, or generation, and when exact correctness is less important than usefulness.

An agent. Use an LLM that plans and uses tools only when a task genuinely requires multiple steps, decisions along the way, and access to external systems, and when you can bound what it is allowed to do. I cover the design side in designing agentic products users trust.

A human. Some tasks should stay with people, either because the stakes are high, the context is subtle, or the relationship is the product. Deciding that a task is not for AI is a legitimate strategic choice, and often a wise one.

The rule I follow: use the simplest tool that meets the quality bar. Often the right answer is a hybrid, where rules handle the common deterministic cases and an LLM handles the long tail.

The six-question test for any LLM use case

When a team proposes using an LLM for a task, I run it through six questions. A use case that scores well on all six is a strong candidate. One that scores badly on any of the first two should be redesigned or dropped.

1. What does a wrong answer cost? If an error is cheap and visible, such as a draft that the user edits, the risk is low. If it is expensive or silent, such as an incorrect figure in a financial document, the bar for reliability rises sharply. Match the use case to your tolerance for error, not to the model’s best-case demo.

2. Can the output be verified? The easiest tasks are those where a person, or another system, can check the result quickly. Code that runs, a summary that can be compared to the source, an extraction that can be validated against a schema. Hard-to-verify output needs stronger guardrails or should stay assistive.

3. How often does the task occur, and how much is it worth? Value is frequency times savings. A rare task that saves an hour may not justify engineering effort. A frequent task that saves thirty seconds can be transformative at scale.

4. Does the model have the context it needs? Models are only as good as the information they can see. Does the product hold the customer data, documents and history required to produce a good answer, and can you retrieve it reliably? The difficulty of that retrieval is the subject of RAG in production.

5. What are the latency and cost constraints? Some interactions must feel instant, and some tolerate a wait. A task that runs a hundred thousand times a day has very different economics from one that runs a hundred times.

6. How reversible is it, and how much do users trust it? Actions that cannot be undone, such as sending an email, moving money or deleting data, deserve more caution than ones the user can reverse with a click. Trust is built in small, reversible steps.

Decision 3: Set the autonomy level deliberately

How much should the system do on its own? I find it easiest to think of a ladder with five rungs, and to choose a rung for each task rather than for the product as a whole.

1. Assist. The model answers when asked. The user does the work.
2. Suggest. The model proposes a draft, a recommendation or a next step. The user decides.
3. Act with approval. The model prepares an action and a person confirms it before it happens.
4. Act within guardrails. The model acts alone within defined limits, such as spend caps, allowed actions or confidence thresholds, and escalates outside them.
5. Act autonomously. The model acts and the user reviews afterwards, if at all.

Most products should start near the top of the ladder and climb as evidence accumulates. Moving up a rung is a decision that should be earned with data: acceptance rates, error rates, the cost of the errors that did slip through. A team that launches at rung four on the strength of a good demo is gambling with customers’ trust.

I pair every rung with explicit guardrails: what the system may never do, how it fails gracefully, and how a human can take over. The playbook for these is in shipping AI features safely.

Decision 4: Know the economics before you ship

AI features have a cost structure that traditional software does not. Every task consumes compute, and the bill scales with usage. A strategy that ignores this will discover it on the first invoice.

Work through three numbers.

Cost per task. Estimate the tokens in and out, the number of model calls per task (agents may make many), and any retrieval or tool costs. Model your own numbers with the LLM cost calculator, and compare candidate models with the LLM model selector.

Value per task. What is the task worth to the customer, in time saved or revenue gained? This sets your price ceiling.

Margin at scale. Cost per task times expected volume, against the price you can charge. A feature that is profitable per task can still hurt overall margin if heavy users dominate.

If the numbers do not work, you have levers. Route easy tasks to smaller, cheaper models and reserve the largest for hard ones. Cache repeated work. Shorten prompts and constrain outputs. Batch where latency allows. The detail is in LLM cost optimization and the architecture behind it is in multi-model architecture.

Pricing must reflect the cost structure. Flat per-seat pricing with unlimited AI usage invites a few heavy users to destroy your margin. At Spoon Hire AI, I designed credit plans for both B2B and B2C users, plus a pay-as-you-go tier, so that price scales with consumption. The reasoning and trade-offs are in designing credit plans and pay-as-you-go pricing, and the wider picture is in usage-based pricing. The strategic point is that pricing and architecture are connected: decide what a unit of value is, and make sure your cost and your price both track it.

Decision 5: Treat evaluation as part of the strategy

Most AI teams discover late that they cannot answer a basic question: is the system getting better or worse? Without an answer, you cannot safely change a prompt, swap a model, or climb the autonomy ladder.

Evaluation is the capability that makes everything else possible, so it belongs in the strategy, not the test plan. My experience scoring the quality of reasoning for LLM training at Micro1.ai reinforced a few principles I now apply everywhere.

Define “good” before you build. Write the criteria down in language a reviewer can apply consistently: is the answer correct, grounded in the source, complete, appropriately cautious, in the right tone. Vague standards produce vague results and arguments between reviewers.

Build a golden set. A collection of real, representative cases with known good answers, including awkward edge cases and examples of past failures. Run it whenever anything changes.

Use rubrics and, carefully, models as judges. Structured scoring makes quality measurable. Using a model to grade another model scales well but needs calibration against human judgment, or you will optimize for the grader’s quirks.

Keep humans in the loop for the hard cases. Automated metrics catch regressions. Human review catches the failures nobody anticipated.

Test with users, not only with data. A system that scores well on paper can still fail on the first real workflow, because the failure is about expectations and flow, not accuracy. Sprint-based user testing, repeated across many builds, surfaces that gap. Log what users do, not only what they say.

For the technical practice, see evaluating LLM features. The strategic point is simple: if you cannot measure quality, you cannot promise it, price it or scale it.

Decision 6: Decide what makes it defensible

Here is the uncomfortable fact about building on foundation models: your competitors can rent the same ones. A feature whose entire value is a prompt on a popular model has no moat. When I advise teams, I use Hamilton Helmer’s framework of seven powers as a checklist for where durable advantage might come from, and ask which are plausible for an AI product.

Workflow and switching costs. If your product is embedded in how customers work, with their data, integrations and habits, leaving is costly. This is usually the strongest AI-era moat. The model is a component. The workflow is the product.

Proprietary data and context. Not data for training, necessarily, but unique context a model can use at inference time: customer history, domain knowledge bases, verified outcomes. The better your retrieval over data others cannot get, the better your answers.

Distribution. Owning the channel to the customer matters more when the underlying capability is widely available.

Trust and compliance. In regulated or high-stakes domains, a reputation for reliability, auditability and safety is a real advantage and takes years to build.

Learning loops. A product that improves from usage, with evaluation and feedback feeding back into quality, compounds.

Equally important is what not to rely on: the model itself, a clever prompt, or being first. Hedge against dependence on any single provider. Avoiding LLM vendor lock-in explains how to build provider-agnostic systems, and choosing an LLM for your business covers how to select among them for a particular task.

Build, buy or partner for AI

The build-versus-buy question comes up in every AI roadmap, and the right answer depends on how central the capability is to your advantage.

Buy or rent commodity capabilities: transcription, general summarization, translation, off-the-shelf classifiers. Building them rarely differentiates you and the market moves fast.

Build what sits at the heart of your proposition: the workflow logic, the evaluation, the domain-specific context and the user experience around the model.

Partner where you need capability, data or distribution you cannot build quickly, and where the relationship does not create dangerous dependence.

Two checks help. First, ask whether you would regret the decision in two years, when models are cheaper and better. If the capability will be commoditized, do not invest heavily in building it. Second, run the same scrutiny you would on an acquisition, testing whether the vendor’s claims are real and durable, as in commercial due diligence. Pilot with real data before you commit.

Risk, trust and regulation in Europe

If you serve customers in the European Union, regulation shapes the strategy. I am not a lawyer and this is not legal advice, but a product leader should know the outline, and take proper counsel for specifics.

The EU AI Act takes a risk-based approach, with obligations that rise with the risk of the use case: a small set of practices are prohibited, high-risk uses face requirements around risk management, data quality, documentation, transparency and human oversight, certain systems must disclose that users are interacting with AI, and there are separate rules for providers of general-purpose models. The rules phase in over time, so check current timelines. GDPR applies on top, covering the lawful basis for processing personal data, data minimization, retention and the rights of individuals, which affects what you can send to a model and where it is processed.

Treat these as design inputs, not afterthoughts. Classify each use case early. Decide where data may flow and which providers meet your residency requirements. Build in human oversight, logging and a way to explain outcomes. Customers in regulated sectors will ask, and a clear answer is a sales advantage. A product that can show how it manages risk wins deals that a flashier competitor cannot.

Roadmapping when the outcome is uncertain

AI work has an extra layer of uncertainty: you often do not know whether a capability is feasible until you try. A conventional roadmap, with confident dates for specific features, breaks under that. Adapt it.

Use spikes to buy information. For uncertain items, schedule a short, time-boxed investigation with a stated question: can the model hit this quality bar on our data? The output is a decision, not a feature.

Gate launches on evaluation. Set the quality threshold in advance. If the system does not meet it, it does not ship, whatever the date. This is also the guard against the demo-to-production gap.

Roll out in stages. Internal use first, then a small group of customers, then wider release, with monitoring at each step. Moving up the autonomy ladder follows the same logic.

Plan for change. Models, prices and capabilities shift quickly. Build abstractions so you can swap components, and review the strategy often. The relationship between strategy and roadmap is the subject of product strategy vs roadmap vs vision, and AI work makes the distinction more important, not less.

A useful discipline is to run AI inside your own company first. Using it for internal workflows teaches you about quality, cost and failure modes at low risk. Running a business lean on self-built automations has taught me a lot about where models help and where they quietly fail.

A worked example

Here is an illustrative scenario, not a client. A company sells customer support software to small online retailers and wants to add AI.

Candidate uses and the decision for each:

• Routing tickets to the right queue. Mostly deterministic with a long tail. Decision: rules for the common cases, a small classifier for the rest. Cheap, fast, auditable. Not an LLM.
• Drafting replies to routine questions. High frequency, errors visible to the agent, verifiable against the knowledge base. Decision: LLM with retrieval, at the “suggest” rung. Agent approves each reply. Track acceptance rate and edit distance.
• Summarizing long ticket threads for handover. Language task, easily checked against the source. Decision: LLM, assistive, low risk.
• Issuing refunds. High cost of error, irreversible. Decision: LLM may prepare the refund and the justification, but a person approves. Revisit autonomy only after months of evidence, and then with spend caps.
• Answering customers directly without an agent. Highest value and highest risk. Decision: pilot with a small share of simple intents, strict guardrails, immediate escalation on low confidence, evaluation gate before expansion.

Economics. Cost per drafted reply times volume, against the time an agent saves. Credit-based pricing so heavy users pay for what they consume.

Defensibility. The moat is the integration into the retailer’s order and customer data, and the learning loop from agent edits, not the model.

Risk. Customer data is personal data, so provider choice and data flows are decided early. Chatbot disclosure is built in.

The outcome is a roadmap with five very different items at different autonomy levels, each justified by the same tests. That is what an AI strategy looks like when it is done properly.

Common anti-patterns

The demo trap. A model that works on ten curated examples ships, and fails on the thousands of real ones.

AI as a checkbox. A feature added to say the product has AI, with no problem behind it.

One model for everything. The largest model used where a small one, or a rule, would do, at ten times the cost.

No evaluation. Changes made on feel. Nobody can say whether quality improved.

Unlimited usage at a flat price. The most engaged customers become the least profitable.

Autonomy by default. Launching at the top of the ladder because it looks impressive.

A moat made of prompts. Advantage that disappears when a competitor writes a similar one.

Ignoring the regulator. Discovering obligations after the product is built.

The short version

• “AI strategy” is six decisions in order: problem, approach, autonomy, economics, evaluation and trust, and defensibility with risk.
• Start from a customer problem stated without mentioning AI. Choose the simplest tool that meets the quality bar: rules, analytics, an LLM, an agent, or a person.
• Test each LLM use case on error cost, verifiability, value, available context, latency and cost, and reversibility.
• Set the autonomy level per task on a five-rung ladder, and climb only with evidence.
• Know the cost per task, the value per task and the margin at scale, and price so that revenue tracks consumption.
• Build evaluation into the strategy: define good, keep a golden set, calibrate any model judges, and include humans and user testing.
• Do not rely on the model for defensibility. Workflow, proprietary context, distribution, trust and learning loops last longer.
• Treat EU regulation and data protection as design inputs from the start.
• Plan roadmaps for uncertainty with spikes, evaluation gates and staged rollouts.

Where to go next

If you want the underlying method, read how to write a product strategy that actually gets implemented, which this post builds on. To see how the strategic choices flow into a plan, product strategy vs roadmap vs vision is the natural next read. And if you want help deciding where models belong in your own product, what a product strategy consultant does explains how that work is scoped.

---

I am Deepanshu Grover, a Senior Product Strategy Consultant in Paris. I take ambiguous, high-stakes product and business questions, run the analysis, and see the recommendation implemented. If you are deciding where AI belongs in your product, connect on LinkedIn or get in touch.