Getting a model to produce impressive output is no longer the hard part of AI product work. A capable engineer can wire up an API, write a prompt, and have something demo-worthy in days. That is exactly why so many teams are stuck: the demo was cheap, so everyone assumed the product would be too. Then months pass and there is still no production feature.

I build AI product features daily at Writer, an enterprise generative AI platform, and the pattern is consistent: the model is a component, and rarely the one that blocks launch. What blocks launch is the workflow around the model — what the feature does when retrieval comes back empty, who is allowed to see what, where a human signs off, and whether anyone can explain what the AI did last Tuesday. That layer is not glue code. It is the product.

Why the demo lies to you

A demo is a curated path through a system. The presenter knows what to type. The documents exist and are well-formatted. The connected API responds quickly and the OAuth token hasn’t expired. Every input has been implicitly chosen to make the model look good — which is fine, because that is what demos are for.

Production inverts all of it. Users paste content far larger than the context you budgeted for. The document the answer depends on was deleted last month. A third-party API rate-limits you mid-workflow. Someone asks a question the feature was never designed to answer, and the model answers it anyway, confidently. None of these are edge cases; at any real usage volume they are the daily weather.

The practical consequence: estimating an AI feature from the demo is like estimating a bridge from a sketch. The demo tells you the happy path is possible. The engineering work is almost entirely in the unhappy paths, and those don’t appear until you go looking for them deliberately.

Decide what “grounded” actually means for your product

“Grounded in your data” is a marketing phrase until your team can complete this sentence in writing: an answer is grounded when — and then state the rule. Which sources count as approved? Does the model get to blend retrieved content with its own general knowledge, or is it restricted to what retrieval returned? When you show a citation, what exactly are you promising the user it proves?

These are product decisions with hard engineering consequences. Retrieval runs inside a token budget, which means you are always ranking and truncating sources — and the ranking policy determines what the feature believes. Retrieval scope has to be tied to permission scope, so the feature never quotes a document to a user who couldn’t open it themselves. And the empty case needs a designed answer: when nothing relevant is found, a trustworthy feature says so and shows what it searched, rather than letting the model improvise.

In my experience the teams that struggle most are the ones that treated grounding as a retrieval library choice. The library is the smallest part. The definition is the work.

Permissions and identity are launch blockers, not polish

If you sell to companies of any size, your AI feature will meet a security review before it meets most of its users. I have built SSO, SCIM provisioning, and IdP integration flows, and OAuth-based knowledge connectors, and I can tell you where the uncomfortable questions concentrate: identity. Not model quality — identity.

AI features make permissions harder, because the model becomes a new path to data. A connector that holds OAuth tokens to a customer’s systems has to respect the source system’s permissions on every retrieval, or the feature quietly becomes a privilege-escalation tool: ask the AI, and it summarizes a document you were never allowed to read. Agent workflows sharpen the question further — when an agent acts, whose permissions is it acting under, and what happens to its scheduled runs and stored tokens when SCIM deprovisions that user?

Every one of those questions will be asked eventually. Answering them in a security questionnaire after the architecture is fixed is expensive and sometimes fatal to the deal. Answering them in the design doc is close to free. This is the least glamorous work in AI product engineering and the most reliably decisive.

Approval and fallback states are features, not error handling

There is a mindset shift that separates AI features users trust from ones they quietly stop using: human approval steps and fallback states are designed product surfaces, not apologies bolted on when something fails.

A human belongs in the loop at specific, chosen points: before an irreversible action, on the first runs of a newly configured workflow, wherever the cost of a wrong output exceeds the cost of a review. That placement is a design decision you should be able to defend, and the approval surface deserves the same care as any other screen — the reviewer needs to see what the AI is about to do and why, not just an accept button.

The same goes for failure. When a tool call fails mid-workflow, the difference between a resumable state and a start-over is the difference between a product and a toy. A structured, interruptible workflow whose state a user can inspect will beat a more autonomous one that fails opaquely — because in an enterprise, “the AI did something unexpected” is not an acceptable incident report. Legible behavior is what earns automation its autonomy.

You cannot operate what you cannot see

Conventional monitoring tells you the request returned 200 in acceptable time. It tells you nothing about whether the answer was wrong, ungrounded, or subtly off-policy. AI features need their own observability layer, and it has to exist before launch, because its absence is invisible right up until the first serious complaint.

What that means concretely: structured events for every step of a workflow — what was retrieved and from where, which tools were called with what arguments, what was ultimately shown to the user. The ability to pull up a specific past run and walk through it. User feedback captured against the trace it refers to, not floating free in a spreadsheet. And evaluation sets built from real production failures, so every incident permanently raises the bar instead of just getting patched.

The test is simple. When someone says “the AI did something weird yesterday,” can you answer with a trace, or only with a shrug? Until it is the trace, you are not operating a production feature — you are hosting a prototype with production traffic.

A pragmatic sequence

Teams often ask where to start when the prototype exists and the pressure to ship is real. The order below is the one I would defend, because each step is dramatically cheaper before the next one than after it.

  • Write the workflow down as explicit states before writing more code — including every failure state. If a transition is not on the diagram, the model will find it in production.
  • Define grounding in one sentence, tie retrieval scope to permission scope, and design the empty-retrieval state on purpose.
  • Map identity early: SSO, roles, and permission scopes for every data source the feature touches, plus what deprovisioning does to tokens and scheduled work.
  • Build the approval and fallback surfaces in the first real iteration, not as a post-launch hardening phase.
  • Instrument from day one: per-step traces, replayable runs, feedback tied to traces.
  • Ship narrow — one workflow, a real cohort — and only broaden scope when the traces have become boring.

Where I can help

None of this requires exotic engineering. It requires having been through it, knowing which decisions are expensive to reverse, and refusing to let the demo set the definition of done. Building exactly this layer — agent workflows, knowledge connectors, enterprise identity — has been my daily work at Writer since 2023, inside a platform where enterprise security review is the gate every feature passes through.

If your AI feature is stuck between demo and production, the fastest way forward is usually a focused conversation about where it is stuck: grounding, identity, workflow design, or visibility. Describe the situation and I will tell you honestly whether I can shorten the path — and where the real risk in your current plan sits.