An existing workflow, a small AI proposal, and the questions that made me leave it on the drawing board.
I was recently thinking through an AI feature for a product that already guides people through a structured questionnaire. It collects the relevant answers, runs an existing calculation, and shows the result on a summary screen.
The obvious proposal was a chatbot. There was useful domain content to retrieve, structured information to work with, and plenty of questions a user might ask. Technically, it was easy to imagine a convincing demo.
I wasn't convinced about the product.
The questionnaire already did something valuable: it helped people move forward without knowing which questions to ask. Replacing that guidance with an empty text box would put more work on the user. Adding a chat button beside it would still need a reason to exist.
The discussion eventually narrowed to something smaller: a short explanation of an already-calculated result. That seemed worth considering. Then came the question of sending personal information to another service, and I paused.
This is a proposal I am still thinking about. There is no launch story or conversion lift at the end of this article. The useful part is the decision process, because the same questions apply to many products considering their first AI feature.
Start with the uncertainty the user actually has
A blank chat interface assumes that the user can describe what they need. Sometimes that is exactly right. Someone researching an unfamiliar topic may want an open conversation with room to change direction.
A guided product has a different responsibility. It has already made decisions about what to ask, what to explain, and which next steps are relevant. Those decisions can be a substantial part of its value.
Before adding AI, I would look for a specific moment where that guidance breaks down. Does the user misunderstand a question? Enter the same information twice? Reach the result and still wonder why it looks that way?
Each problem suggests a different intervention. Better wording might fix the first. An integration might fix the second. A contextual explanation might help with the third.
“Users can ask anything” does not tell us which of those problems we are solving. It also makes evaluation awkward. We could measure conversations, but more conversations might mean the product became more confusing.
For the idea I was considering, the narrow question became: would an explanation help someone understand the result well enough to take the next step?
Respect the work the existing product already does
In this case, the product already had the answers and the calculation behind the result. It did not need a model to rediscover the business logic.
That changes the architecture. The calculation should remain authoritative. The model could receive a deliberately prepared explanation context and turn supported facts into readable language. It would have no reason to change the result, choose new rules, or fill in missing answers.
Consider an illustrative product that calculates a service estimate. Its engine knows the selected options, the quantity, an applicable adjustment, and the final amount. A useful explanation could identify which selected options account for the estimate and point to an answer the user can review.
An unhelpful explanation would simply restate every line item in a paragraph. A dangerous one would invent a reason for an adjustment because the real explanation was missing from the input.
There is a subtle issue here: the largest input is not necessarily the largest contributor to a result. If the product wants to describe the “main driver,” the calculation layer needs to supply a defensible contribution or explanation. The model should not infer causality by looking at whichever number happens to be biggest.
The same applies to comparisons. Explaining why this result differs from a previous one requires comparable inputs and calculation versions. Without those, a confident story about the change would be speculation.
Design the smallest useful interaction
I would keep the existing result prominent. The explanation would sit beside it, with a clear purpose and a short default length.
Something like “What contributed to this result?” gives the user a much better starting point than “Ask our AI.” It also gives the implementation a bounded job.
A second interaction could point to a relevant input: “Review the quantity you entered.” That should open the existing editing flow. If the user changes an answer, the normal engine recalculates the result, and any old explanation becomes invalid.
This matters because an explanation can be correct when generated and misleading a minute later. The relationship between the text and the underlying calculation must survive ordinary product behavior: edits, reloads, saved drafts, and returning to an old session.
I would attach every explanation to an immutable calculation version or identifier. The interface should only display it beside the matching result. Generating new prose should never be a prerequisite for seeing the calculation itself.
If generation fails, the original summary remains useful. A timeout should not turn a functioning product into a broken one.
Give the model a deliberate input
The simplest prototype would pass the complete questionnaire into a prompt. I would avoid that shortcut.
A context builder should select the facts needed for this particular explanation. A small conceptual contract could include:
- A calculation identifier and version
- The result already produced by the engine
- Supported explanation facts, each with a stable identifier
- Approved labels and definitions for those facts
- Relevant uncertainty or missing information
- The requested language and length
This is a proposed contract, not a description of an implementation I have shipped.
The output could reference the supplied fact identifiers alongside its text. Those references would make review easier, but they would not prove that a sentence is true. A sentence can cite the right fact and still misrepresent it.
For important amounts, I would prefer rendering the original values directly in the interface rather than asking the model to reproduce them. If the text contains numbers, they need checks against the source. Unsupported claims should cause the explanation to be withheld or replaced with a simpler fallback.
There is also a useful boundary around follow-up questions. A suggestion such as “What if I change this answer?” can sound harmless while inviting the model to improvise a new calculation. Any actual scenario comparison should go back through the product's calculation engine.
The explanation service gets to describe the result. It does not acquire authority over the result by being conversational.
Compare it with a version that uses no model
Before choosing a provider, I would write several explanations by hand using invented examples. Then I would try producing the same explanations with templates.
If the product has a small set of common outcomes, a template may be enough. It can insert verified values, explain the relevant rule, and offer the next step without introducing a new data recipient or variable wording.
A model becomes more interesting when the useful explanation depends on a combination of facts that is cumbersome to express through templates. Even then, the extra flexibility has to help the reader.
A practical comparison would show people the existing summary, a template-based explanation, and the proposed generated explanation. Ask them to explain what happened and what they would do next. Watch for confident misunderstandings as carefully as hesitation.
I would look at comprehension, successful completion of the next step, and the kinds of help people still request. Latency and operating cost belong in that comparison too. These would be measurements for an experiment, not claims about benefits already achieved.
If a clearer label performs just as well, the product decision gets easier.
Privacy belongs in the product decision
This was the part that made me pause. The feature would be explaining information personal to a user. Sending it elsewhere needs a justification that is stronger than “the model could write something nice.”
Removing names and contact details is a start, but a record can still reveal personal information through its contents. A smaller input helps reduce exposure; it does not automatically make the input anonymous.
I would work through the actual data flow before moving beyond synthetic examples. Which fields leave the application? Which service receives them? What ends up in request logs, error reports, traces, caches, and support tools? Who can inspect those systems, and how long is the information retained?
Those questions need answers for the chosen configuration. They cannot be resolved by a general statement about a vendor, or by assuming that an existing infrastructure relationship covers a new use of data.
The product question remains part of that review. If the explanation adds little value, there is little reason to introduce the extra processing. If it materially helps users, there is a clearer case for doing the necessary privacy and security work.
Pausing here is a valid outcome. Technical feasibility does not create an obligation to launch.
What would make the proposal worth revisiting
I would return to this idea with a small set of representative, synthetic cases and three versions of the result screen: current, templated, and generated.
The prototype should include awkward cases from the beginning. Missing explanation facts. Contradictory inputs. An old calculation version. A changed answer while generation is running. A result that cannot support a meaningful comparison.
For each case, the product needs an acceptable response. Sometimes that will be a short explanation. Sometimes it will be a precise statement about what is unknown. Sometimes the right response is to show the existing summary without extra prose.
Only after that comparison would I decide whether a model earns its place. Provider selection, orchestration, and production infrastructure come after the case for the feature, even if those are the more tempting engineering questions.
The original questionnaire was doing useful work. Any enhancement needs to respect that, make a particular moment easier, and justify the additional complexity it introduces.
That is the bar I would use for a chatbot too.