AI in customer service: what to automate, and what not to

Customer service is one of the areas most often overestimated and, at the same time, most often underestimated for AI automation. Overestimated when companies believe a chatbot can replace the entire support team. Underestimated when nothing is automated out of caution, even though a large share of requests are repetitive and clearly structured.

The practical path lies in between: a clear dividing line between cases a system can safely handle and cases that need a person, plus a clean handover point between the two.

What can be automated well

1. Recurring status questions

"Where is my order?", "How long does shipping take?", "How do I send something back?" These questions have a clear answer that can be derived from system data. An assistant with access to order and shipping data can answer them reliably, around the clock, without involving an employee.

2. Product and usage questions with a documented answer

Questions about materials, sizes, compatibility or care instructions can be answered if the underlying information is maintained and up to date. The assistant should point to the source so answers stay verifiable and outdated content gets noticed.

3. Pre-sorting and prioritization

Even if a request ends up with a person, AI can sort it in advance by topic, urgency and required information. This shortens handling time because staff don't have to reconstruct the context first.

4. Drafts for individual replies

For more complex but non-critical requests, a model can prepare a draft reply based on order data and the content of the request. An employee reviews it, adjusts it and sends it. This speeds up handling without giving up decision-making authority.

What should stay with people

1. Refunds, goodwill and financial commitments

As soon as a reply involves money, discounts or exceptions to policy, a person should decide. Automated goodwill without limits quickly leads to inconsistent or abused commitments.

2. Complaints with legal or health implications

Product safety, health questions, warranty and legal claims need human responsibility and, where necessary, legal review. A language model may prepare here, but not give the final answer.

3. Emotionally charged or escalated cases

When a customer is already upset or an escalation looms, it takes empathy, context and room for decision-making that a set of rules can't capture. An automated, impersonal reply pattern often makes such situations worse.

4. New or unclear case patterns

If a type of request occurs for the first time or doesn't fit any documented pattern, it should go to a person, also so the team can derive a new, vetted reply template from it instead of a model guessing under uncertainty.

The handover point determines quality

The biggest weakness of many support automations is not the AI itself but a poor handover to a person. If a customer has already explained everything to a chatbot and has to start over when switching to an employee, it causes more frustration than no automation at all.

• Pass on the context: The complete history so far must reach the employee, not just "customer wants to talk".
• Name the reason for escalation: Why was the case handed over: uncertainty, rule violation, an explicit request for a person?
• Set expectations: The customer should know they are being transferred and what to expect next.

Trust comes from transparency, not perfection

Customers rarely react allergically to a system answering, as long as the answer is correct and it stays clear how to reach a person. It becomes critical when an assistant feigns certainty it doesn't have, or when there is no visible way out to an employee.

Two simple principles help: the assistant identifies itself as such, and an escalation path is visible and usable at all times.

A realistic start

1. Categorize the requests from the last three months and determine the share of clearly structured, recurring cases.
2. Start with the largest low-risk cluster, usually status or shipping questions.
3. Define escalation rules before go-live, not only when the first critical case occurs.
4. Check answer quality with spot checks and document error cases, instead of only measuring handling time.
5. Expand step by step once quality and escalation logic have proven themselves in operation.

The knowledge base determines answer quality

Automated support is only ever as good as the information it draws on. The most common reason for poor answers is not the model but a knowledge base made up of scattered, contradictory and outdated documents.

What makes a usable knowledge base:

• One source per topic. If the return period is stated differently in three documents, the system will pick one of them, and not necessarily the valid one.
• Answers instead of prose. A manual chapter on shipping methods is less suitable than ten clearly worded question-and-answer pairs on the questions that are actually asked.
• Visible change date. Every entry needs a date and a responsible person. Without that, the knowledge base decays within a few months.
• Deliberate gaps. Topics without an approved answer should be explicitly marked "do not answer automatically", otherwise the system improvises.

The most pragmatic start: export the last 300 to 500 support requests and group them by topic. As a rule, ten to fifteen topics cover between 60 and 80% of the volume. Write clean answers for exactly these topics, and enable automation only for them. Everything else goes straight to a person.

This approach has a pleasant side effect: work on the knowledge base also improves human support, because new employees find the same clear answers. Even if the automation comes later, this step is not lost.

What you should measure, and what tells you nothing

Support automation is often judged by its automation rate: What percentage of requests were resolved without human involvement? This number is attractive because it rises easily, and misleading because it rises when the bot simply closes more cases, whether or not they were resolved satisfactorily.

Four other metrics are more informative:

• Repeat contact rate: How many customers get in touch again about the same issue within 72 hours? If this value rises, the automation only appears to resolve cases.
• Handover rate and handover quality: How often does a person take over, and how often does that person have to make the customer explain everything again?
• Handling time of handed-over cases: Good pre-sorting makes the remaining cases faster, not slower. If handling time rises, the bot has skimmed off the easy cases and handed over the difficult ones less well prepared.
• Satisfaction by channel: Measure automated and human-handled cases separately. An overall score hides exactly the deterioration you want to find.

What matters is the comparison with the situation before. Without baseline values, you can't later prove whether the automation helped or merely shifted work. Collecting these baselines takes a week, and reconstructing them later is impossible.

Data protection and responsibility in support

Customer service is where the most sensitive data accumulates: names, addresses, order histories, occasionally health or payment details. Anyone who automates here automatically makes data protection decisions as well.

Three points should be settled before the start. First, data processing: Which provider processes the requests, where are the servers, and is there a data processing agreement? Second, scope: Does the system really need access to the full customer history, or are the order number and status enough? Data minimization is not just a legal requirement here, it also reduces the damage if something goes wrong. Third, transparency: Customers should be able to tell that they are talking to an automated system.

There is also a practical question that is often overlooked: What happens to the conversation logs? If they are used for training, the legal basis and the retention period must be settled. If they are not used, valuable potential for improvement is lost, and then an anonymized analysis of the most common drop-off points is worthwhile.

These questions are no reason to forgo automation. They are a reason to introduce it with the same care as any other system that processes customer data. For detailed legal questions, expert review is part of the process. growald does not provide legal advice.

A rollout that doesn't gamble with trust

The riskiest moment is going live for real customers. A staged approach over about eight weeks reduces this risk considerably without materially delaying the benefit.

Stage 1: shadow mode, two weeks. The system processes incoming requests, but nothing goes to customers. An employee compares the generated answers with the ones actually sent. This quickly shows, with no consequences, where the knowledge gaps and misjudgments are.

Stage 2: draft mode, two to three weeks. The system writes draft replies, and a person reviews and sends them. This already saves measurable time while keeping full control. Many companies stay at this stage permanently, which is perfectly legitimate, because a large part of the benefit is already realized here.

Stage 3: partial automation, from week six. Only the clearly defined topics with a high hit rate are answered fully automatically, and everything else stays in draft mode. The scope is expanded step by step, each time after reviewing the repeat contact rate.

Two conditions should apply across all stages: every automated answer contains a simple way to reach a person, and there is an emergency off switch that can fully disable the automation within minutes. Anyone who builds this switch only when it's needed builds it too late.

Conclusion: automation as a filter, not a replacement

AI in customer service works best as a filter that catches clear, recurring cases and cleanly hands complex, sensitive or emotional cases over to people. If this separation is taken seriously, the load on the team goes down without putting customer trust at risk.