I have shipped two products where a language model does the central work. Recuvix generates SEO-structured blog drafts. NovaEdge News ingests Hindi RSS feeds, rewrites each item and publishes on a schedule with nobody operating it.
In both, getting a good result out of the model took a few days. Getting a system that keeps producing good results without me watching it took most of the project. If you are estimating an AI feature from the prompt outwards, the estimate will be wrong by a large multiple.
Deduplication comes before generation
RSS feeds repeat themselves. They resend items, they syndicate the same story from three sources, and they change a headline slightly and send it again. Generating an article you already published is two failures at once: a duplicate on the site and a bill for producing it.
So every item is fingerprinted on a normalised title and source URL before any model call happens. That check is unglamorous and it is the highest-value code in the pipeline.
Every external call is going to fail
Feeds go down. They return malformed XML. Providers time out, rate-limit, or come back with something unusable. A pipeline written for the path where everything works stops publishing in its first week.
Failures are caught per item, not per run. One dead source degrades that source and the run still finishes. Scheduled runs are idempotent, so a retry after a partial failure picks up where it stopped instead of duplicating what already went out.
A demo needs the model to work once. A product needs the system to keep working on the day the model does not.
Cost belongs in the architecture
Model spend scales with volume, and volume is the thing you were hoping for. That makes cost an architectural concern rather than something to check at the end of the month.
Concretely: deduplicate before you generate, cache anything deterministic, keep prompts short enough that you are not paying to resend context you could have looked up, and reach for the cheaper model on the passes that do not need the expensive one.
Be honest in the interface
The hardest part of Recuvix was not the generation. It was the editor. My first layout made generation the hero and editing a side panel, which had it backwards, because nobody publishes a first pass.
The editor now owns the main column and generation feeds into it. Untouched model output carries a faint left rule that clears once you edit that paragraph. It puts a small amount of pressure on review, which is the behaviour the entire product depends on.
That is an interface decision doing the work of a disclaimer. Telling a user the output might be wrong changes nothing. Showing them which paragraphs they have not looked at yet changes what they do.
What I would tell someone starting
Budget the prompt at a tenth of the work. Spend the rest on deduplication, failure handling, cost controls, prompt versioning so a quality drop is traceable, and a review surface that treats the output as a draft. That list is the product. The model call is one line in it.
Frequently asked questions
- What is the biggest challenge in shipping AI products to production?
- Managing edge cases around the LLM call: deduplication, latency, fallback routing, rate limiting, and building human-in-the-loop review interfaces.
- How do you reduce API costs when building LLM features?
- Cache identical queries, pre-validate inputs, use smaller quantized or open-weight models for extraction tasks, and reserve frontier models for complex generation.
Written by Amit Kumar Raikwar, full-stack engineer & product designer in Indore, India. If you want something built, start here.
Working on something?
Let's build it together.
I take projects from an empty Figma file to a live product. Fixed scope, weekly demos, code you own.