I built an AI support agent and spent most of my time stopping it from making things up. Here’s what worked.
I’m a solo developer from Accra, Ghana, and I’ve been building a customer support agent platform. The hardest part wasn’t getting the AI to answer questions. It was stopping it from confidently answering questions it shouldn’t.
A few things I learned:
1. Some conversations shouldn’t be AI at all. Refunds, order changes, quote requests: businesses don’t want creativity there. They want the same steps every time. So I built “flow agents” that follow a fixed path the business designs word for word, and only use AI where open-ended answers make sense. Most real setups end up as a hybrid.
2. “I don’t know” has to be a designed outcome. The default behaviour of an LLM is to be helpful, which in support means inventing a policy. The agent needs an explicit path for “I can’t answer that, let me get a person”, and that handoff has to be easy for staff to pick up.
3. Ban whole categories, not just bad answers. Telling the agent never to quote prices or promise delivery dates removed a whole class of risky answers. Businesses cared about this more than anything.
4. Only answer from the business’s own material. Grounding answers in their uploaded docs, and nothing else, made owners far more comfortable switching it on.
The stack is Python/FastAPI on the backend and Next.js with TypeScript. I’m happy to go deeper on any of this.
If anyone’s curious, it’s called Flexaura (flexaura.ai) and there’s a free plan. Mostly I’d love to hear how others are handling hallucination risk in customer-facing AI.