Every CX leader has seen the same story play out. A new bot gets switched on, the knowledge base is uploaded, and expectations are high. Then, almost immediately, things slide: customers complain about vague or wrong answers, CSAT starts to dip, and your best agents end up spending their time rewriting context instead of solving problems.
The technology isn’t broken. The AI is doing exactly what it was told to do. The problem is that no one is managing it.
That gap, between plugging in a bot and running it as part of a CX operation, is why most AI support investments underperform. A recent MIT study showed that 95% of enterprise AI projects are unprofitable. Not because the technology can’t deliver, but because companies expect “set and forget” to work in customer service.
Content isn’t management
The default advice in CX right now is: “just connect your knowledge base.” The assumption is that if agents can use the docs, so can a bot.
But knowledge bases are written for people, not machines.
-
They’re long and descriptive, with too much fluff.
-
They use shorthand and jargon your team understands but customers don’t.
-
They change slowly, while products and policies move fast.
What AI needs isn’t a dump of knowledge base articles. It needs atomic facts: short, precise statements that leave no room for confusion.
Take a simple example. A human-facing knowledge base might say: “Our hats are typically in stock online, but availability varies depending on demand.” An AI that’s asked about stock doesn’t need the nuance. It needs: “Hats: currently out of stock.”
Without that level of re-engineering, content becomes noise. The bot starts hallucinating, contradicting itself, or worst of all, giving customers the wrong answer.
The missing role in CX
Think about how you bring on a new human agent. You don’t just hand them the wiki and walk away. You train them, check their answers, and coach them through tricky conversations until they’re ready to handle customers solo.
AI needs the same structure. That’s where the AI Agent Manager comes in.
Their job is to:
-
Re-engineer knowledge base content into atomic facts.
-
Define clear rules for when the bot should answer and when it should escalate.
-
Monitor transcripts weekly to catch errors and gaps.
-
Test changes in a safe environment before customers see them.
This is the difference between an AI that quietly undermines your brand and one that becomes a reliable member of the support team.
We’ve even seen clients who didn’t realise they were already doing this job. Once the role had a label, it clicked — this is a new function inside CX, just like QA or workforce planning.
What changes when AI is managed
The risk of unmanaged AI isn’t that it won’t answer. It’s that it will answer badly. One wrong interaction is enough to make a customer swear off your bot for good.
With management, the experience looks very different:
-
Useful first responses. The metric that really matters isn’t “tickets touched by AI” but “was the first response useful?” Either it solves the problem outright, or it gathers the right details so the handover is smooth.
-
Fewer handovers. The AI takes care of repetitive questions, so agents focus on empathy and complex issues.
-
Faster iteration. When a product changes, the AI is updated in days instead of months.
-
Clear visibility. Leaders get weekly reports showing resolution rates, CSAT, and where improvements are happening.
Left unmanaged, AI just adds to the workload. With management, it becomes a force multiplier.
What it looks like in practice
When AI is managed week to week, the impact shows up quickly.
At one company, first responses handled by AI rose from 37% to 52% in just weeks. Customers got more useful answers, and CSAT held steady at 4.5+. Without management, those extra tickets would have gone back to human agents and added pressure to the team.
At another company, re-engineering their knowledge into atomic facts tripled the share of queries the bot could handle — from 9% to 33%. The AI wasn’t the problem. The way it was being fed information was.
And in one case, when a client launched a new returns system, the bot was updated the same day. Customers never saw a gap, even though human agents hadn’t been trained yet. That kind of speed only happens when someone is actively watching the system and making changes in real time.
These aren’t one-off stories. They’re what happens when AI is treated as part of the team instead of a static tool.
From experiment to discipline
Most brands still treat AI as a project — something you set up once and leave to run. But CX doesn’t work like that. Your human team has managers, QA, and training because customer service is dynamic. AI needs the same oversight.
Right now, too many bots are being left on autopilot. The result is predictable: declining CSAT, wasted investment, and customers who actively avoid the channel you built for them.
What’s emerging is a new discipline: AI Agent Management. Today, it’s a competitive advantage. Tomorrow, it will be as standard as workforce planning or QA.
The question for CX leaders is whether you want to be ahead of that curve or behind it.
Takeaway
AI in support doesn’t fail because the technology is flawed. It fails because no one is managing it. Content matters, but without management it doesn’t add up to a reliable customer experience.
If you want AI to work in support, treat it like part of the team — not a one-time project.
Get started with Influx
Influx has been helping brands deliver world-class support since 2013, partnering with more than 750 companies globally.
With AI Agent Management, we make sure your bots don’t just run — they improve every week. From monitoring and context updates to prompt tuning and reporting, our AI Agent Managers keep customer trust high and your team focused on complex work.
Make your support operations faster, smarter and more reliable with a managed AI layer.
See how AI Agent Management works.
