1. Insights
  2. Digital Experience
  3. Article
  • Share on X
  • Share on Facebook
  • Share via email

What KPIs matter most when measuring chatbot customer service?

Posted May 2, 2018
Smartphone with graph

Chatbots are fast becoming a preferred customer service tool for companies of all kinds. With the ability to address non-complex customer queries and route more nuanced calls to the appropriate agent, chatbot technology built on artificial intelligence (AI) creates a more convenient and satisfying customer experience.

“One of the best uses of AI is when you have a bot working in concert with the customer service rep,” says John Asher, customer service expert and CEO of growth strategy consulting firm Asher Strategies. “If that chatbot can automatically send out personalized messages, answer standard questions, and recognize when it needs to turn the call over to an agent…it’s a great boon for customer service.”

Evaluating the success of your chatbot-customer interactions requires a number of different metrics. Here are five Key Performance Indicators (KPIs) for assessing your chatbot’s value to your overall customer service strategy.

1. Activation Rate

Before you can measure the overall effectiveness of your chatbot, you first have to make sure that people are using it. Activation rate allows companies to accurately determine the extent to which the chatbot option is being selected by customers.

This KPI encompasses multiple metrics, including the total number of users, how many users opened a message they received from a chatbot and the number of users who engaged with the bot by sending back a message. “With activation rate, you want to look at new, active, and engaged users separately and on a monthly basis,” Asher explains.

Comparing these monthly data sets will enable brands to substantiate the merit of their investment or find better ways to use the technology in a more engaging fashion.

2. Volunteer Users

Measuring the number of users who interact with a chatbot of their own accord – without waiting for the bot to initiate a conversation – can reveal useful information about your customers’ preferences. “This is typically a quasi-indication of your marketing and branding efforts,” Asher says. “Volunteer users are what you want. These people come to your site because they’ve heard about (your chatbot) and recognize its value. That usually means they’re coming in with a real purpose, and will be more engaged.”

3. Retention Rate

Confirming that your customers continue to use your automated channels ensures that they find value in the automated interaction. A retention rate will indicate the percentage of users who revisit your bot within a given time frame, whether monthly, quarterly, or annually. This KPI can indicate whether your existing level of investment in the technology is sustainable and whether aspects of the technology need to be tweaked or refined to improve the experience.

Keep in mind that while multiple return visits to your bot might appear to suggest that a customer’s issue wasn’t resolved, it could just as easily mean your customers prefer using this platform to others. “Sentiment analysis and natural language processing like modulation of voice can tell you a lot about why they’re coming back,” says Asher. These supplementary AI technologies can also help you reduce areas of friction and the accompanying customer frustration.

4. Response Time

A recent survey conducted by media and trend analysis company PSFK found that 74 percent of customers favor chatbots when seeking rapid responses to their questions. John Di Lemme, a strategic business consultant coach and marketing consultant, agrees that the ultimate metric for chatbots is response time. “You want to put yourself in your customers’ shoes to see the mechanics of the chatbot in terms of how it works and especially how quickly it can respond,” he says.

A chatbot should be able to process queries quickly and provide a near-instantaneous response. However, an immediate response means nothing to the customer if it’s not correct or doesn’t fully address their issue.

5. Customer Satisfaction

Above all else, companies using customer service chatbots must measure the impact of this tool on customer satisfaction. The self-serve aspect of messaging technology means your contact center has limited access to the end-user’s experience. It isn’t always clear what the customer thinks and feels about their chatbot interactions, making the evaluation of customer satisfaction a must.

One way to accomplish this is by tracking chatbot errors and confusion triggers — which can indicate problems with the experience — alongside metrics like Net Promoter Score (NPS), a customer loyalty metric that measures the likelihood your customers will recommend your brand to others. In many cases, the higher your customer satisfaction scores resulting from a chatbot interaction, the higher your NPS will be.

Di Lemme also recommends looking for common threads among customer complaints logged by your chatbot. “If a question is consistently being asked, the company should take note and focus on resolving the root cause of it. Your ultimate goal should always be to fully leverage all tools and technologies in your arsenal to enhance the customer service experience,” he says.

The appetite to implement chatbots in the customer journey is undeniable, but their implementation doesn’t mark the end of the line for brands looking to deliver exceptional customer service. Selecting the appropriate KPIs to monitor chatbot performance, and actioning the results where applicable, are critical steps for ongoing innovation and improvement.


Check out our solutions

Leverage our proven process for adding bots to your customer service operations.

Learn more