Case studies

How to take the load off support with AI: what we did in-house

← All articles

We are a small team. And like everyone else, we spend a lot of time on support. Most of the questions are the same ones over and over. We decided to check how many of them an AI agent could handle, and started with ourselves.

In this article, we’ll tell you how we did it, what worked and which pitfalls we hit. And at the end, when you will be able to do the same.

First, we counted what people ask us about

Before building anything, we exported all of our support conversations in Telegram from 2022 to 2026 and sorted them by topic. That came to 1,574 requests.

  • 43% are about money. Almost always it’s “I paid, but my balance wasn’t topped up.” Most often the person simply forgot to enter their user ID in Kaspi QR or the invoice number in the payment reference.
  • 11% are about moderation. Usually it’s “approve it urgently, we’re going live.”
  • Next come SMS and sender names, pricing, and questions about call campaigns.

One more observation. Most unanswered questions come in the evening and at night. After 8 p.m. we are usually away from our computers, and the customer needs an answer right now.

The conclusion was simple. Almost everything can be answered by looking at the help center or the account data. Our staff member has to open the admin panel for that. An AI agent needs access to that data. Only very careful access.

Who is Aya

Aya is our AI agent. She used to only call people who ordered a test call on the website. Now she works in our support:

  • replies on our WhatsApp. If someone sends her a voice message, she replies with a voice message too;
  • answers when people call support by phone or through WhatsApp;
  • knows our help center. That’s 37 articles we put together from real conversations, the website and the dashboard.

Here is what it looks like in a chat:

AyaAutoCall.kz support
Hi, I paid 20,000 through Kaspi an hour ago, but my balance still hasn’t been topped up
Good afternoon! My name is Aya, AutoCall.kz support team. I checked your account: there is no 20,000 ₸ payment yet. Most likely, the user ID wasn’t entered in Kaspi QR. Please send a screenshot of the receipt, and I’ll pass it on to my colleagues.
receipt photo
Thank you, I’ve passed it on. My colleagues will find the payment and credit it manually. Next time, enter your user ID in Kaspi, and the money will arrive on its own.
Sample conversation. Amounts are for illustration.

What she can do on her own

Aya can’t see the database. She only has tools we wrote for specific tasks, and each one returns exactly what is needed for the answer. For example:

  • balance and plan, and why the plan is what it is;
  • whether a payment has arrived and which invoices have been issued;
  • the status of a call campaign or broadcast, and the moderator’s comment if it was rejected;
  • how much was spent over the last few days.

She can also take a few actions. For example, speed up moderation if the person really asks, or stop a call campaign. But only after the person clearly says “yes.”

And here is what she never does:

  • doesn’t credit or refund money;
  • doesn’t create call campaigns or broadcasts;
  • doesn’t show API keys or passwords;
  • doesn’t change the account’s phone number. She verifies the old and new numbers with codes, and one of our staff makes the change.

How she knows who she’s talking to

This is the most important part. If an AI agent shows the balance and payments to a stranger, that would be very bad. So our server decides who she is talking to. Aya only asks the questions.

  • Messages on WhatsApp from the account’s number. This is simple: WhatsApp itself has already verified the number.
  • Phone calls from the account’s number. The caller’s number was most likely entered at sign-up, but it’s better to be safe. So Aya also asks for the name on the account or the company name.
  • Messages or calls from another number. Then Aya sends a code to the account’s number via WhatsApp or SMS. A repeat code to the same number goes out only after 24 hours.
  • Just in case, Aya asks extra questions about what the company does, its brand and so on.

One more thing. Aya can’t “look at someone else’s account,” even if you ask her very nicely. Her tools simply have no field where another number could be entered. The server picks the account for them after verification.

When AI can’t handle it

Sometimes a question is complex, or the person just wants to talk to a human. That’s fine. Then Aya hands the conversation over to us:

  • on a call, she connects the caller to a staff member. If no one picks up, she leaves a request with a short summary, and we call back;
  • on WhatsApp, she hands over the conversation with its full history. If a staff member replies directly, Aya stays silent in that chat for two hours so as not to get in the way.

We see all of Aya’s conversations in our team chat: what was asked, what she answered and which data she looked at. That shows right away where she made a mistake, and the help center can be fixed the same day.

The pitfalls we hit

The interesting part started when Aya met real people.

  • Auto-replies. Many people use WhatsApp Business with an auto-reply. We send someone an OTP code, their auto-reply sends something like “Thank you for contacting us,” and Aya politely answers the auto-reply. Now she skips the first message that comes right after our code or notification. If the person writes again, it’s really them.
  • Bot meets bot. If there is also a bot on the other side, they can keep chatting forever. Aya notices when the replies come too fast or sound too “AI-like,” and goes quiet.
  • Messages in pieces. People often write several messages in a row. Replying to each one makes a mess. So Aya waits a little for the person to finish, and meanwhile they see “typing” in the chat.
  • Endless goodbyes. “Thank you,” “You’re welcome,” “No, thank you”… Now, after saying goodbye, she no longer replies to a short “ok” or “thanks.”
  • Refunds. At first, Aya agreed too easily to pass refund requests to our accountant. Now she first explains the rule from the agreement and hands it to a person only if a refund is actually possible.
  • Trolls. Of course there are trolls. Aya simply doesn’t respond to nonsense or rudeness.

What you’ll be able to do

To be honest, right now only our own support works this way. But we are building the same thing for you, and here is how it will happen, step by step.

  1. Within the next week, we’ll open inbound call handling by AI agents. That means someone calls you, and your AI agent picks up.
  2. Then we’ll teach the assistant that sets up agents to build the same kind of support we have. You tell it about your pricing, rules and common questions, and it builds an agent that answers based on them. There’s a lot of work here, so this will come a bit later.
  3. Then you look at the results: call transcripts, what people ask about, where the agent fell short. And you keep improving it until it handles most questions on its own.

And if the AI can’t handle it, the call goes to your operators in the call center. This already works today: the agent can transfer to a number, including through your PBX. Later we’ll do the same for WhatsApp calls.

When it’s ready, we’ll post about it on the blog and in our Telegram channel. Subscribe so you don’t miss it.

Frequently asked questions

Can I build an agent like this right now?

You can create a voice AI agent for outbound calls in your dashboard right now; it usually takes about 15 minutes. Here are the instructions. Inbound call handling by the agent opens within the next week.

What if a customer wants to talk to a real person?

The agent will connect them to your operator. This already works today, including through your own PBX.

What if the AI gives a wrong answer?

It happens. That’s why the agent needs clear rules, and you should keep an eye on it through call transcripts. Spot a mistake, fix the rules, and next time the agent will answer correctly.

Create your own AI agent

No contract and no monthly fee.
A test balance right after sign-up.

Sign up free →