AI for repetitive customer support questions, grounded self-service, and real-time answer retrieval

I've noticed something changing in customer support lately, and I suspect you might as well.

The chat button that used to connect you with a person now opens a bot. Even my mechanic's phone line has become an AI booking system. The shift has happened quickly. In 2020, only about 5% of support teams were using chatbots. By 2025, that number had climbed to more than 80%.

I understand why. AI scales, it's available 24/7, and it can handle a huge volume of repetitive questions. But when something actually goes wrong, most people still want another person. Not because AI is inherently bad at support, but because when you're frustrated you want to know that someone understands what you're experiencing and cares that your experience hasn't been ideal.

Working in customer support has taught me that some of the most valuable product insights don't come from dashboards or bug reports. They come from conversations.

A customer mentions an odd workaround they've been using for months. Three different people describe the same issue using completely different words. Someone casually says, "It's not a bug, but..." and suddenly you've uncovered the real problem.

AI can absolutely help support teams work more efficiently. But I don't think we should automate away the conversations that teach us the most about the people using our products.

That's something I've been thinking about recently, and I wrote more about it here: https://shorturl.at/MrzzE

With the General Availability of Service Agent and MCP tools in Microsoft 365 Copilot, service organizations continue to get the benefits of Copilot in customer service, now powered by the rich business context and grounded intelligence of both Dynamics 365 and Microsoft 365. Whether they’re new in career or experienced customer service representatives, your employees get tools from Microsoft to help deliver a better support experience for your customers. With this release, we have added rich, interactive app-in-chat experiences, ensuring that customer service representatives do not need to solely rely on text-based conversational UX. Employees can investigate issues, navigate complex processes, and complete tasks without leaving the conversation. The result is a more connected, action-oriented experience that helps service professionals move from understanding to resolution faster in Dynamics 365 Customer Service and across Microsoft 365 apps.

61 percent of your users will leave for a competitor after exactly one bad experince.

I was biting my tongue today during a hallway conversation so I would not scream this number out loud. A colleague asked why we even need human Customer Support anymore. "Why not just automate everything with AI and close the tickets?"

I love tech people. I really do. But sometimes they think human frustration can be resolved with a version update.

Do not get me wrong, AI is a fantastic sidekick. I use it to answer simple tickets. To summarize past chats, analyze sentiment, and pull up user history. But AI has absolutely zero judgment.

Take a classic edge case. A user buys a time-based package, but real life happens and they cannot use it. The database shows the purchase was delivered successfully. If an AI handles this, it looks at the policy, sees a successful delivery, and politely tells the user to go away.

A human looks at the exact same ticket, realizes we want this person to actually like our brand, bends the rules, and restores the package.

Trap a frustrated user with a bot that makes a mistake, give them no way to reach a real human, and trust is instantly broken. People just want to know there is an actual person behind the system.

Use AI to read the data. Use humans to read the room.

My career has been all over the place honestly. I started in customer support, moved into training the support team, then into learning and development, then implementation, then chose to go back to support because honestly it's where I do my best work, except this time as a team of one at a startup where the job also includes knowledge management and content creation on any given day.

So I've seen this stuff from a lot of angles.

Over the last 18 months my current role has shifted more than it did in the previous five years combined. AI handles a big chunk of what used to fill my day, and the stuff that still comes my way is genuinely different now, messier, more emotionally charged, the situations where someone just needs a real person.

That shift has made me think a lot about how people talk about support experience on resumes, because most of it sounds identical. "Handled customer inquiries." "Resolved tickets." Cool, so did everyone else.

What actually made me hireable across really different roles wasn't any of that. It was the call I talked someone down on. The pattern I noticed before it became a real problem. The moment I went off script because the script would have made things worse. Creating a training program from scratch because no one else had the bandwidth. Sitting in product and engineering meetings as the person who actually knew what customers were saying, and translating that into something the team could act on.

If I was actively job searching right now I'd be writing about those moments specifically, not the volume of tickets I closed.

And honestly, not hiding that you work with AI tools daily is worth mentioning now. A year ago it felt like a weird thing to bring up. Now people who pretend they don't use them just come across as out of touch.

Anyone else who's bounced around roles like this, curious how you're framing that experience right now. Does it feel like an asset or does it still read as unfocused to hiring managers?

A new automation potential report analyzes your customer conversations and identifies requests that can be automated with AI agents. This report provides brand-specific insights and sample ticket data, showing you exactly how an AI agent would respond to customer inquiries.

(Advanced) Agentic AI for advanced email AI agents is now generally available. This enables AI agents to understand emails, answer questions, automate procedures, and escalate when needed, reducing back-and-forth with customers.

Copilot now includes macro content suggestions and trust and safety recommendations. These new recommendation types help you improve agent productivity and account security without spending time going through complex settings.

Generative search in help center now supports a follow-up question for customers using the Web Widget. This enhancement creates a smoother path from self-service search to a conversational experience with an AI agent, reducing repetition and keeping context from the original search.

Hot take: AI doesn't replace great support teams. It reveals them.
I've spent years taking struggling operations and dragging them into something better than I found them. And the biggest unlock in the last few years hasn't been a new hire or a reorg. It's been getting my team out of survival mode so they can actually do the work they're good at.
Here's what that looked like in practice:
We introduced an AI agent to our student support team. Her first month, she handled 25% of all inbound volumn solo. Escalation rate? Around 5%. Meanwhile, our old chat platform was managing 200 conversations a month.


She did nearly 4,000 without missing a single kids baseball game. Guess who got to go to those? The team.
Did people panic? Yes, but I reminded my CS team that the invention of the calculator didn't eliminate the need for mathematician, did it? Let's be MASTERS of this.
What I told them was this: the team members who learn to work alongside AI, who master the tools instead of fearing them, who let the AI catch the routine stuff while they focus on the complex, emotional, high-stakes moments, those people become indispensable.
And here's the thing nobody talks about enough: when AI absorbs the backlog, humans stop drowning. My team went from perpetually behind to actually present for the students who needed them most. The work got better. The humans got better.
One of our highest-rated agents in feedback? Was the AI. Students praised her by name, not knowing she wasn't human. Not because we were hiding something, but because she was genuinely helpful.
That's the bar. That's what good AI deployment looks like.
AI in support isn't a threat. It's a force multiplier, but only if you deploy it with intention, build trust with your team through the transition, and stay honest about what you know and what you don't.
The leaders who figure that out are going to build teams that can do more than they ever could before.
That's the unlock.

TL;DR:

Built directly from agent feedback, Helpdesk 2.0 fixes real workflow pain points. The redesign focuses on reducing friction and helping agents handle more context-heavy tickets.

A chat-style interface replaces the old email layout. Conversations are easier to follow and resolve in one view.

Customer context is shown beside the conversation in a right-side panel. Agents can view history, orders, and details without leaving the ticket.

AI handoffs come with clear summaries. Agents instantly see what happened, what was tried, and what to do next.

Navigation is simpler and faster across teams. Clean menus, structured queues, and multi-store access keep agents moving efficiently.

I recently created a short walkthrough on setting up a customer support AI agent for a website, and wanted to share the basic workflow here.

The setup process I followed was:

Create the AI agent

Configure the basic settings

Train the agent using website pages

Add specific webpages manually if needed

Use advanced crawling settings for better control

Add files or direct text content for extra knowledge

Customize the widget tabs

Preview the widget before publishing

One thing I noticed is that the quality of the agent depends a lot on how clean and specific the training content is.

If the website content is too generic, the agent gives generic answers. But if the content is structured well, the responses become much more useful.

For customer support use cases, I think the most important parts are:

- Clear FAQ content

- Product/service details

- Pricing or plan information

- Contact/support escalation rules

- Lead capture questions

- A proper fallback message when the agent does not know the answer

I also feel that businesses should not treat AI agents as just chat widgets. The real value comes when the agent is trained properly and connected to business outcomes like support, lead capture, booking, or qualification.

I recorded the setup process here in case it helps anyone:

https://youtu.be/eakbdcI6a0I?si=OtbsGFba46YjmJi_

Would love to know how others here are training AI agents for customer support. Are you mostly using website content, documents, API integrations, or a combination?

I see a lot of people automating everything they can and then wondering why their product feels soulless. Automation is incredible but knowing what NOT to automate is the real skill.

I run two products solo and I've automated about 15 hours of weekly work. But there are things I refuse to automate even though I technically could.

The stuff I automated and never looked back. Customer support for repetitive questions. Same 10 questions every day, AI handles them now on chat and phone, I only step in for real problems. Content repurposing. I was spending 6 hours a week cutting clips manually, now AI does it in 20 minutes and I just pick the ones I want. Transactional emails. Welcome messages, payment confirmations, all event-driven now.

The stuff I keep manual on purpose. Every Reddit comment and LinkedIn post is me typing. Not scheduled, not templated, not AI generated. This is where my reputation lives and if people ever feel like they're talking to a bot I lose everything I've built. Product decisions stay fully human too. What to build, what to skip, how to price it. No AI can understand the weird mix of user feedback, gut instinct, and market timing that goes into those calls.

The rule I follow is simple. If the same input always needs the same output, automate it. If it needs judgment, context, or a human touch, don't. Customer asks "what's your pricing?" Same answer every time. Automate. Customer asks "should I use your product for my specific situation?" That needs real understanding. Keep it human.

The founders who automate everything including the human parts end up with a product that feels like nobody's home. The ones who automate nothing burn out in 6 months. The sweet spot is somewhere in the middle.

What have you automated that you wish you hadn't? Or what are you still doing manually that you know you should automate?

So I have been deep in this space for about a year now across our support queue and honestly the conversations I keep seeing online still feel too clean compared to what actually happens in production.

Here is what I have actually learned from running this:

Intercom Fin - strong at deflecting repetitive volume but the setup to get it talking properly about your specific product is more work than they make it sound

Zendesk AI - powerful if you are already deep in the ecosystem, felt clunky to configure outside of it

Ada - serious automation muscle but when it misses it misses confidently which is the worst version of wrong

Chatbase - been on this one the longest, about a year now. The Zendesk integration is what kept us on it. When the agent cannot resolve something the full conversation history transfers with the ticket automatically so agents never pick up cold. 71% resolution rate, CSAT held.

Freshdesk Freddy - fine for getting started, hit its ceiling faster than expected

The thing nobody talks about enough is the maintenance side. Every single one of these tools is only as good as what you feed it and how often you update it. The ones that fell apart on us fell apart because we treated them like infrastructure instead of something that needs a weekly 15 minute review.

The bar has shifted from can it reply to can it actually close the ticket. But I would add a third question now: can it stay accurate six months after you deployed it without someone actively maintaining it. That is where most of them quietly fail.

What are you all running? And genuinely curious if anyone else has had something work great in month one and then slowly fall apart.