What an AI Customer Support Agent Actually Needs to Work

A customer writes to you at 2am. Their order has not arrived, or their booking is wrong, or they cannot find the answer to a simple question that is sitting on your website. Nobody is awake. They wait until morning, and in the meantime they have decided how they feel about your business.

An AI support agent is the obvious answer, and a lot of businesses have tried one. A good share of those attempts went badly. The bot invented a refund policy, answered a question it should have passed on, or gave the same unhelpful reply three times until the customer left. The failure is rarely the model. It is usually everything built, or not built, around it.

I build and run these agents alongside the admin work I do as a virtual assistant, including 24/7 support and follow-up agents for client businesses. This post covers what a support agent needs to function properly, what it should leave to a person, how to test one before customers see it, and what it costs compared with staffing the night shift.

A chatbot answers, an agent acts

The word agent gets used loosely, so here is the distinction I use.

A chatbot answers questions from a script or a document. It talks. An agent does that and can also take actions in your systems: look up an order, check a calendar, create a ticket, update a record, book an appointment. That ability is what makes it useful, and it is also what makes it risky, because an agent that can act can act wrongly.

Voice agents add more layers: speech recognition, a text-to-speech voice, and the fact that people interrupt, mumble, and call from noisy places. The same core rules apply, with the added requirement that conversations need to feel natural and handle a bad connection gracefully.

Most small businesses should start with text, on one channel, before considering voice.

The six parts of a support agent that works

1. Knowledge. The agent is given your actual content: product details, policies, pricing, opening hours, shipping times, the answers to your top fifty questions. It is told to answer only from this material. If the answer is not in there, it says it does not know and offers a human. A model answering from general knowledge will sound confident and be wrong about your business.

2. Guardrails. A short list of things it must never do. Quote a price you have not published. Promise a refund. Give legal or medical advice. Discuss another customer’s account. Argue with someone. Guardrails are written as plain instructions and tested by trying to break them.

3. Permissions. What it is allowed to touch. A support agent might read order status but not edit it, create a ticket but not issue a refund. I give an agent the least access that lets it do its job, the same way you would for a new hire in their first week.

4. Memory and context. Within a conversation it should remember what was said. Across conversations it should know who the customer is and what has happened before, so nobody has to explain the same problem twice. This part is easy to underbuild.

5. Escalation. Clear conditions that trigger a handover to a person: an angry tone, a complaint, a legal or safety question, a request outside its scope, or two failed attempts to resolve something. The handover includes a summary, so the human starts with context instead of a blank page.

6. Logging and review. Every conversation is saved and searchable. Somebody reads a sample each week. This is how you find the question it keeps misunderstanding, and it is the part that separates an agent that improves from one that slowly drifts.

Skip any one of these and the agent has a hole. Skip the last two and you will not know about it.

What it handles well, and what it hands over

An agent is strong at questions with a clear answer in your documentation, such as opening times, delivery windows, how to reset a password, what a plan includes, or where an order is. It is strong at gathering information before a human steps in: name, order number, the nature of the problem. It is strong at booking and rescheduling when the calendar is clear. And it never gets tired at 2am.

It should hand over anything involving judgment, money exceptions, emotion, or reputation. A customer who is upset needs a person. A refund outside policy needs a person. A question that touches a legal matter needs a person. A thoughtful, long-standing customer who writes in with something unusual deserves a human reply, and an agent that recognises when to step back is worth more than one that tries to handle everything.

The aim is not to remove people from support. The aim is to make sure that when a person does reply, they spend their time on the conversations that actually need them.

Two conversations, side by side

Here is a simplified example from a booking business. A customer writes at 2:10am to say their appointment is tomorrow and they need to move it. The agent asks for the booking reference, looks up the appointment, checks the calendar for open slots, and offers two. The customer picks one. The agent updates the booking, sends a confirmation, and logs the change.

Now a different message at 2:40am. A customer says they were charged twice and wants a refund immediately. The agent does not touch the money. It apologises once, collects the booking reference and the amounts, tells the customer that a person will review it first thing in the morning and when to expect a reply, and creates a ticket with everything attached. At nine, a person opens the ticket, sees the full story, and resolves it in two minutes instead of fifteen.

Both conversations went well, for opposite reasons. In the first, the agent acted. In the second, it knew when to stop.

Testing before customers see it

I do not put an agent in front of customers until it has passed four checks.

Fifty real questions. Pull fifty actual messages from your inbox and run them through. Read every answer. You will find gaps in the knowledge base within the first ten.

Adversarial tests. Try to make it fail. Ask for a discount it should not give. Ask it to ignore its instructions. Be rude. Ask something unrelated. Paste a long confusing message. A guardrail you have not tested is a guess.

Shadow mode. For one to two weeks, let the agent draft replies while humans send them, or run it silently alongside real conversations and compare what it would have said. This is the safest way to see real traffic without real consequences.

One channel live. Go live on a single channel with a visible way to reach a person. Widen only after a few weeks of clean logs.

Keeping it accurate over time

Launch day is the easy part. Your prices change, a product is discontinued, a delivery partner is replaced, and the agent keeps quoting the old facts unless someone updates its knowledge. I treat the knowledge base as a living document with a named owner and a monthly review date.

Each week I read a sample of conversations and mark any answer that was wrong or unhelpful. Then I fix the cause rather than the symptom. When the agent gives a wrong answer, the question is whether the source material was missing, unclear, or contradicted by something else. Most errors trace back to one of those three. Fixing them at the source means the same mistake does not return next month in a different form.

How I build them

I build agents with Claude as the model, and workflow tools such as Make and n8n, with LangChain where more structure is needed. The tools are the smaller part of the job. The larger part is writing the knowledge base, the guardrails, and the escalation rules, which takes real knowledge of how support actually runs.

That is where my admin background helps. I have spent years answering customers, managing inboxes, and handling the awkward exceptions, so I know what the exceptions look like before I write rules for them. A developer who has never run a support inbox tends to build for the easy ninety percent and be surprised by the rest.

I also set the agent up with a standard operating procedure, the same way I would for a human, so anyone on your team can read what it does and why.

What it costs compared with staffing nights

Take a business that wants coverage for eight overnight hours, seven days a week. That is 56 hours of coverage a week, which is more than a single full-time employee works, so in practice it means at least two people, or one person on shifts plus overtime. Whatever your local wages are, you are paying for hours when most messages are quiet.

An agent changes the shape of that cost. There is a one-off build, scoped after I look at your volume, channels, and systems. There are ongoing running costs for the model and the tools, which scale with usage. And there is a few hours a month of human review. The running costs are not zero, and anyone who tells you the agent is free is selling something.

For many small businesses the agent costs a fraction of overnight staffing, particularly when most night-time messages are routine. For a business with a high proportion of complex or emotional conversations, the savings shrink and the agent should be treated as a first responder rather than a replacement. I would rather tell you that up front than have you find out after the build.

Questions to ask any builder

If you are comparing people to build an agent, these questions tell you a lot.

What happens when it does not know the answer? The right reply involves a handover, not a guess.

Which actions can it take, and how are they limited? You want specifics, not “it can do anything.”

How will I see what it said to customers? If the answer is not “full logs,” keep looking.

How do you test before launch? Listen for shadow mode and real examples.

Who maintains it afterwards? An agent needs updating when your products and policies change.

What did it get wrong in your last project? A builder who says “nothing” has not built many.

Where agents fall short

They struggle with ambiguous requests, strong emotion, and situations your documentation never anticipated. They can misread a sarcastic message. They need maintenance whenever your offer changes, because an out-of-date knowledge base is a source of confident errors. And they are only as good as the process underneath them, so a support operation that is chaotic today will not be fixed by adding a bot.

If your answers are inconsistent between team members, start by writing them down. That documentation is the foundation of any agent, and it is useful even if you never build one.

Plan for these limits rather than hoping they will not appear. A support agent with a clear route to a person is a safety net. One without it is a liability.

If you want support that does not sleep

I am Souptik, a virtual assistant with a technical background and eight years of running operations for founders. I run the admin side and build the agents and automations around it, so the support system matches how your business actually works.

You can read how I work at souptik.site/process, see the record at souptik.site/va-portfolio, or check pricing at souptik.site/pricing. If you would like an honest view on whether an agent suits your support volume, message me on LinkedIn with a description of the questions customers ask most, and I will tell you what is worth automating and what is not.