Technology Partners

Our Team

Our Awards

Social Impact

Newsroom

Locations

Careers

About Us

Contact

Do Not Sell my Personal Information

Cookie Preferences

Culture Value Chain

iLabs Center for Innovation

Industries

Insights

Investor Relations

TELUS International is a subsidiary of: 

Privacy Shield Framework

Privacy Policy

Solutions

Subscribe to Newsletter

Website User Terms

logo-x

Icon facebook

Icon youtube

Icon linkedin

Technology

Financial Services & Fintech

Automotive

Healthcare

Communications & Media

Games

Retail & Ecommerce

Travel & Hospitality

infobrief

Guide

Resource Type Article

Article

case-studies thumb

Case Study

Preventing fraud for a global tech company in the travel and hospitality space image with fern in background

TI resource case 900x510

Video

whitepapers thumb

Research & Reports

checklists thumb

Checklist

brochures thumb

Brochure

ebooks thumb

E-book

Podcast Type Image

Podcast

The recent proliferation of generative AI (GenAI) models has sparked a lot of interest and excitement. These advanced large language models (LLMs) can do many extraordinary things — from crafting email responses to recommending a fitness routine to generating computer code; the list is seemingly endless.

However, these models aren't infallible. In fact, their propensity to sometimes hallucinate — a phenomenon where they provide responses that incorporate fabricated data that appears authentic — could significantly impact user trust. A recent TELUS International survey found that 61% of respondents were concerned about generative AI increasing the spread of inaccurate information online. 

Understanding what hallucinations are, why they happen and how to prevent them is critical to maintain users' trust in GenAI applications.

Introduction to generative AI hallucinations

A hallucination describes a model output that is either nonsensical or outright false. An example is asking a generative AI application for five examples of bicycle models that will fit in the back of your specific make of sport utility vehicle. If only three models exist, the GenAI application may still provide five — two of which are entirely fabricated. While this is a relatively low-stakes example of a hallucination, some could have far greater consequences. 

Recently a New York attorney representing a client's injury claim relied on a conversational chatbot to conduct his legal research. The federal judge overseeing the suit noted that six of the precedents quoted in his brief were bogus. It turns out that not only did the chatbot make them up, it even stipulated they were available in major legal databases.

While guardrails to prevent GenAI hallucinations need to be put in place, the first step is understanding how, or rather why, these inaccuracies happen.

Why do generative AI hallucinations occur?

While leading AI experts aren't entirely sure what causes hallucinations, there are several factors that are often cited as triggers.

First, hallucinations can occur if the training data used to develop the model is insufficient or includes large gaps leading to edge cases that are unfamiliar to the model. For example, in drafting a contract, a generative model that was fine-tuned using contracts from the financial services industry may not have sufficient exposure to terms and concepts in healthcare law to draft contracts for that industry. Most of the time, generative AI's primary goal is to produce an output in response to a prompt. Even if it doesn't "understand" the prompt, it might craft a response based on insufficient training data, thereby leading to a faulty result.

Similarly, some machine learning (ML) models suffer from overfitting, where the output is accurate for its training data, but inaccurate for new data. The goal of ML model training is to generalize from training instances so that the model can correctly operate on instances it hasn't seen before. Overfitting occurs when a model is trained to the point where it effectively memorizes the inputs and appropriate outputs for the training set, leaving it unable to generalize effectively to new data. For example, a model used to approve loan applications may be considered to be 90% accurate in predicting the likelihood of loan default. However, if it has been subject to overfitting, it likely has an accuracy rate closer to 70%. Therefore, applying the model to future loan decisions will result in a decrease in business profits and an increase in dissatisfied customers.

Another model development challenge is ensuring training texts and prompts are properly encoded. LLMs map terms to a set of numbers — a process known as a vector encoding — and these encodings have some key advantages over working with words directly. For example, words with multiple meanings can have one encoding per meaning, reducing the chance of ambiguity. So, the word "bank" would use one vector representation when referring to a financial institution and another when referring to the shore of a river. Vector representations also allow mapping semantic operations — like finding a similar word — into mathematical operations. Problems with encoding and decoding between text and representations can lead to hallucinations.

Best practices to prevent generative AI hallucinations

While generative AI hallucinations are a concerning issue, there are ways to reduce their frequency and intensity. Consider the following best practices.

The most obvious protection from hallucinations is high-quality input data. It's critical that your GenAI model is trained on a diverse and representative dataset that covers a wide range of real-world examples. Using sufficient, unbiased data is key to training an accurate, resilient model.

Another crucial part of the training process is incorporating human feedback, also known as reinforcement learning from human feedback (RLHF). Because AI models lack a nuanced understanding of language, they can produce responses that are out of context or irrelevant. RLHF provides models with regular corrections via feedback from people, so that an understanding of the world that only a human can have is incorporated into the training process.

Transparency in AI ensures that it's possible to understand a model's inner workings and how it makes decisions. The issue is coming to the forefront as LLMs evolve, particularly because more advanced models are harder to understand.

Users and machine learning professionals should be able to evaluate how a model was trained and review for potential weaknesses in the learning process. Transparency also makes it easier to identify and correct errors that can lead to hallucinations.

A malicious user of a generative AI system could experiment and probe a model looking for ways to make it hallucinate. It's necessary to incorporate counter measures to help monitor and control the types of inputs allowed in generative AI systems. These include using adversarial examples in the model's training to improve its ability to respond appropriately and to audit the model's anomaly detection capabilities. A further means is by data sanitization, which involves checking that no malicious inputs are being fed into the model. And finally, security updates and patches can help block external threats to your AI model.

While generative AI hallucinations are a frustrating hurdle that can significantly impact user trust, they aren't insurmountable. By training models using high-quality data, ensuring model transparency and enacting effective quality control, developers can improve their model's resiliency to deliver accurate predictions.

Hallucinations are an obstacle to building user trust in generative AI applications. Learn about the phenomenon, including best practices for prevention.

Single line drawing of the side profile of a head and the term "AI" imposed on a wavy background connoting hallucination

ai hallucination

Check out our solutions

Generative AI

Generative AI hallucinations: Why they occur and how to prevent them

6-steps-for-Conversational-Bot-success-A-primer 

Make the most of your conversational bot investment with our easy-to-follow guide featuring best practices that can be applied to your digital transformation journey.

A conversational bot floating above a mobile phone

(Hero) 6-steps-for-Conversational-Bot-success-A-primer hero

An illustration of a bot floating above a mobile phone

(Featured) 6-steps-for-Conversational-Bot-success-A-primer

Six steps for conversational bot success: A helpful primer for chatbot implementation

A copy of the Digital customer experience priorities in 2023 report on top of a green leafy plant.

Digital Customer Experience Priorities 2023 Preview Image

TELUS International, in partnership with Statista, surveyed 250 enterprise leaders to explore where they plan to invest in digital customer experience (CX) technology in 2023. Uncover the results.

An illustrated representation of customer profiles and leaving a five-star review online.

Digital Customer Experience Priorities 2023 Hero

Survey results: Digital customer experience priorities for leaders in 2023

IDC Info Snapshot - Cloud thumbnail

Uncover the advantages of partnering with a trusted advisor to create and deliver your cloud vision in this IDC Info Snapshot.

IDC Info Snapshot - Cloud hero

IDC info snapshot cloud feature image

Partnering with a managed service provider can help companies achieve agility, scale, and business value with cloud

As the world’s biggest work from home revolution continues, it’s time to take a long-term look at your data security measures.

Graphic representations of different cybersecurity techniques like passwords, encryption, firewalls and more.

remote data security

Data security considerations for the future of remote work

Discover five key privacy and security traits to consider when selecting a customer experience delivery partner.

Person using tablet with an overlay of icons that convey cybersecurity

Cybersecurity-Partner 1540867048313

Cybersecurity: What traits to look for in an outsourcing partner

Deploying automation to assist agents at a consumer robotics company image with fern in background

Preview image: Deploying automation to assist agents at a consumer robotics company

Learn about how we helped a consumer robotics company improve their digital customer experience with agent-facing automation.

Illustration of consumer robotics including a butler, a chef and cat riding a robot vacuum.

Deploying automation to assist agents at a consumer robotics company

In this virtual chat with TELUS International computer vision experts Ajinkya Malasane and Kaushik PS, we’ll dive deep into end-to-end sensor fusion data collection and annotation solutions for autonomous vehicle training needs.

Three cars on road with labeling boxes around them

3d data annotation

Leveraging automation for labeling large 3D pipelines

Cybersecurity threats are beginning to redefine excellence in customer service. This article explores how to rise to the challenge, even strengthening customer relationships and loyalty - along the way, and after the fact.

series of locks with on unlocked on a digital number background

cyberlock 1540867048316

Cybersecurity: How to excel at customer service in a cybersecurity crisis

The cover of The future of content moderation: Strategies and tools for 2023 and beyond e-book with a fern in the background.

The future of content moderation: Strategies and tools for 2023 and beyond

Organizations have a responsibility to create and maintain digital experiences that are truthful, welcoming and safe. Learn how brands can keep pace with the growing levels of user-generated content (UGC) to build trust with their customers and protect their reputation.

An illustration of various people creating user generated content on their phones.

Preview image of the Forrester report "How To Set CX Metrics Goals" and a plant

TI Resource ForresterReport

Gain an understanding of the challenges faced by customer experience (CX) leaders in setting goals for their metrics — and effective techniques to persevere.

Illustration of a person and various graphs and scales symbolizing metrics, measurement and KPIs

Hero image for Forrester - How to set CX metrics goals - 1620x550

Forrester - How to set CX metrics goals - card image

How to set CX metrics goals

Uncover how travel and hospitality brands can combat fake reviews, scams and more with a robust content moderation strategy.

Airplane leaving a trail of five stars, meant to symbolize positive customer experiences in the travel and hospitality industry

travel como3

Four ways content moderation can build trust for travel and hospitality brands

Preview image of the case study "Creating annotation work opportunities through impact sourcing" with plant in background. 

resource annotation impact sourcing

Discover how TELUS International and the SETU Foundation spearheaded an initiative to provide occupational and learning opportunities to young women in India while delivering high-quality results for the client.

Two women sit in front of computer monitors doing annotation work.

annotation impact sourcing

Creating annotation work opportunities through impact sourcing

On this episode, we discuss the connection between cybersecurity and digital customer experience — and how rethinking data security can help maintain customer trust.

Episode cover image for Questions for now, a TELUS International podcast, featuring an illustrated character holding a phone and giving a verified business profile a five-star review.

Questions for now data security image

How can brands rethink data security to maintain customer trust?

Preview image of the report "CX outsourcing in 2023" accompanied by a plant

TI Resource WhitePaper 2023

Delve into new insights about customer experience outsourcing from a survey of 712 enterprise executives with strategic decision-making authority.

Illustration of customer support agents surrounded by related iconography like charts, star ratings and text boxes.

CX Outsourcing2023 Hero Image

Cover image for the dynamic report "CX outsourcing in 2023" featuring the title and illustrations of customer support agents and related iconography 

CX Outsourcing in 2023

Executive survey results: CX outsourcing in 2023

TI resource eBook 800x450 The essential guide to training data 

Discover best practices for the sourcing, labeling and analyzing of training data from TELUS International, a leading provider of AI data solutions.

guide to AI training data - website hero image 

The essential guide to AI training data

Robert Zirk: There's no question that automation is changing the way customers and brands interact with each other.

There's customer-facing chatbots, agent-facing assist bots and robotic process automation or RPA. And to give you an idea of just how rapidly this sector is growing, Grand View Research estimates the market for RPA, which as of 2022 sits at around $2.3 billion, will reach more than $30 billion by 2030.

Now more than ever, businesses are looking to implement automation to reduce costs, improve accuracy and scale how they deliver their customer experience.

And with change comes uncertainty. There are perceptions that automation will render jobs obsolete, that brands implementing automation can set and forget their deployments, or that automation is only an option for big brands with big budgets.

But that's not the case, according to the automation experts we spoke with. So today on Questions for now, we'll ask: What do people get wrong about automation?

Welcome to Questions for now, a podcast from TELUS International where we ask today's big questions in digital customer experience. I'm Robert Zirk.

To start out, we need to define what automation is. According to Nigel Devaraj, senior product manager for hyperautomation solutions at TELUS International, automation means completing repeatable tasks and reducing the need to interact with multiple systems in what are often referred to as swivel chair processes. As an example, think of the idea of entering data manually into one system, then swiveling your chair around to enter that same data into a different system.

Nigel Devaraj: The automation could be triggered based on an event or it could be triggered by a human being based on an activity they're performing, meaning "go validate this task" or "pull information for me." That is what automation is. And if you look at customer service, the world of CX, automation is really about streamlining day-to-day activities. We're talking about reducing swivel chair processes or eliminating swivel chair processes and simplifying the work or the activities involved in servicing our customer.

Robert Zirk: While the roots of automation date as far back to around 270 BC, it was the Industrial Revolution that marked the invention of the very first automated industrial process - that was the automatic flour mill in 1785.

That was well before we even had the term "automation" to describe it, which only started to become prominent in the mid to late 1940s.

But let's fast forward to one of the types of automation that's most applicable to customer experience today: robotic process automation or RPA.

Chandrakant Binwani: In terms of leap from using technology in customer service itself, the biggest thing that has happened in the past decade or so, the technology that most of you may be familiar with, is RPA.

Robert Zirk: That's Chandrakant Binwani, former director of intelligent automation solutions at Automation Anywhere, which is one of the leading software developers in the robotic process automation space.

Chandrakant Binwani: It's essentially a software robot doing what a human does today with mouse and keyboard sitting in front of a screen. So what has happened is now when you call in, instead of that CSR...

Robert Zirk: ...a customer service representative...

Chandrakant Binwani: ...having to do all those things manually, and remember they have to do this many times, they get so many calls day in and day out, so they have to do this repeatedly. Now they actually can use automation, RPA.

Robert Zirk: Some potentially automateable tasks can include gathering customer information, searching the knowledge base and completing after call work, like summarizing the outcome of the call and completing follow-ups with the customer.

Chandrakant Binwani: From an agent perspective, it helps them with their productivity, with their experience. They can focus on talking to you, connecting with you, understanding your problems, simplifying your experience, improving your experience as well as simplifying and improving the experience of the CSR.

Robert Zirk: The technologies that shaped what we know today as RPA became more widely used in tandem toward the late 1990s. These include screen scraping, which pulls data to be used for other applications; workflow automation, which could be anything from performing calculations to repeating functions through software macros; and some forms of artificial intelligence, like image or text recognition.

It wasn't until 2012 that the term to describe the confluence and advancement of these technologies, "robotic process automation", was created. Since then, RPA has increasingly been adopted by businesses, with a 2020 Deloitte global survey of executives showing that 78% say that they're already implementing RPA.

Chandrakant shared some of the benefits of RPA from a customer perspective.

Chandrakant Binwani: You are now getting answers faster. You're getting quality service. There is no incorrect information from a quality perspective that's been serviced. Bots will do their job perfectly every single time. That's one big change that I've seen.

The other change that has happened is even before you hit the customer service, which is you interact with chatbots. Now chatbots are automation, essentially. They respond to you in natural language and in plain simple English. But, behind the scenes, they actually are taking a lot of work off the agent now. They can collect a lot of information even before you get to an agent, which is very important, critical. It avoids, again, those wait times when you call a customer service agent, they having to ask you and collect all that information.

Robert Zirk: It's not only about speed, but accuracy. Think of how frustrating it can be to a consumer when they find out they have to be transferred only to have to be transferred yet again or even to end up getting disconnected altogether. So automation can help ensure the person the customer is speaking with is the person who's able to help the most effectively with their concern or issue. Or even bypass that process entirely, if it's more efficient and something that the customer prefers to do.

Chandrakant Binwani: Imagine what has happened in the past decade or so. Earlier there was only one channel available. Users or customers used to call to get their queries answered.

Now there is self-service. You can actually log into the website and you can do a lot of things that you had to call for earlier yourself. That's also powered by automation. And now if you're unable to solve yourself using those self-service channels, there is, you know, a pretty good option of chat available.

Now you can chat, either with a customer service agent or again with automation, chatbots powered by this automation that we are talking of. They can answer your queries, they can help you without having to call someone again. And then, again, if even that is unable to solve your queries, that's when you call, and even when you call, your first experience is with automation. There are voice bots collecting the required information so they're able to route to the right agent to be able to help you. So again the automation, the experience that you have, automation powers a lot of that without you realizing.

Robert Zirk: Automation can even enhance the customer experience in more proactive ways, as Nigel mentions.

Nigel Devaraj: I think we are getting more comfortable using automation for not just that repetitive task, but part of that is data validation and really how automation's transforming customer experience is using automation to deliver a personalized service to the customer. So using automation to gather data, cross-reference it, and really provide a personalized recommendation to a customer. Like the concept of a concierge service, that is really what's making a big splash in terms of leveraging automation for customer experience. This is what's happening in the world of CX and you leveraging automation to really transform how you deliver services.

Robert Zirk: Beyond personalization and efficiency, automation helps ensure a consistent interaction that underpins a positive customer experience.

Nigel shared a case study from TELUS International's work with a customer that offers an array of home appliances that have a wide range of error messages and codes.

Nigel Devaraj: We've used automation where we cross-reference the customer calling in with the type of models registered, possible error messages, using that to actually coach our team members to actually reduce the number of questions we ask the customer and spend less time looking for information so they can actually deliver the customer a better service. So this has drastically reduced things like average handle time, dramatically improved CSAT scores for customers.

And average handle time was not just "okay, I could get the information faster and provide the most accurate information in the fastest way," but also getting to the point where we're using automation to actually close the ticket. There's a whole summary process, there's an email that needs to go out to the customer. Leveraging automation to actually take all those activities, things our team members didn't enjoy doing, but it's very important.

Robert Zirk: From the customer perspective, automation that's properly implemented can greatly improve the overall experience.

Nigel Devaraj: CSAT is one of the big ones in terms of customer satisfaction because they're getting more effective responses, faster responses and our agents can actually spend more time providing that empathetic experience. And it could also mean that, at the end of the day, our customers...

Robert Zirk: ...the brands TELUS International supports...

Nigel Devaraj: ...on the backend that we might not directly see, they're getting better customer reviews on their website.

Robert Zirk: And automation can also help create a positive experience from the agent perspective as well. Nigel points out that employees are more apt to embrace automation as it reduces workloads and stress, and they often contribute to improving automation processes.

In fact, a TELUS International survey conducted by Pollfish showed that eight out of 10 employed Americans said bots have enhanced their work environment since early 2020.

Nigel Devaraj: If you think about where we actually use automations to enable our agents, we've done that as a program to support our customers - across multiple customers, actually. We've reduced, we call it nesting time. Basically, we reduce the amount of time a new agent needs to spend learning that account before they can actually be on their own supporting that customer, because automation actually takes some of the fear away, takes some of the learning curve away, and really helps them to become more confident in doing that.

Robert Zirk: Nigel explained automation can also help increase capacity for team supervisors, allowing them to focus on other tasks. He noted one case where automation reduced the amount of time supervisors needed to provide direct support to their team members by 90%.

Nigel Devaraj: We also find automation helps our team and it reduces their stress. They're happier because they're not having to worry about these little mundane activities because we leverage automation to actually take care of those mundane activities for them and they can really focus on the end customer service. We talked about call wrap ups, right? No agent that I've met really enjoys that activity. We've got to fill these and we've got to classify it, categorize it appropriately. Using automation to do some of that takes some of that stress, the day-to-day stress of doing the job, away and they have a much more enjoyable experience.

Robert Zirk: According to a study commissioned by UiPath and conducted by Forrester, 86% of organizations that used RPA saw greater efficiency. Two thirds said that it increased the insights they had on their customers. 57% noted improved customer service. And that same percentage of respondents reported improved employee engagement.

But despite the benefits that automation can provide, Chandrakant knows that there are still quite a few misconceptions about automation in the workplace, with perhaps the largest one being that automation will completely replace humans in the workforce.

Chandrakant Binwani: It is, I would say, unlikely to replace all human jobs.

Human interaction is important. It is important for building trust. It is important for addressing complex issues and even providing that personalized service.

Cybersecurity

How can brands rethink data security to maintain customer trust?

How can brands rethink data security to maintain customer trust?

Data security considerations for the future of remote work

Data security considerations for the future of remote work

Cybersecurity: Identifying your weak points before they get hacked

Cybersecurity: Identifying your weak points before they get hacked

Cybersecurity: What traits to look for in an outsourcing partner

Cybersecurity: What traits to look for in an outsourcing partner

Cybersecurity: How to excel at customer service in a cybersecurity crisis

Cybersecurity: How to excel at customer service in a cybersecurity crisis

Be the first to know