1. Insights
  2. AI Data
  3. Article
  • Share on X
  • Share on Facebook
  • Share via email

How do recommender systems facilitate great CX?

Posted October 21, 2021
Cartoon of a banner that reads "Recommended" and a person hand and arm giving a "thumbs up" gesture

Artificial intelligence is increasingly helping customers decide everything from what to buy to what to stream. In fact, predictive algorithms that use machine learning to recommend products or services based on data collected from past experience have become a mainstay for eCommerce and online entertainment companies.

A recommender system uses machine learning to predict the likelihood that a user will prefer a particular item or service. There are three widely used methodologies for recommender systems — collaborative, content-based and hybrid — that learn from data sources such as previous user behavior and product or service details.

Collaborative filtering minimizes complexity

A collaborative filtering system aggregates data from a group of users and then uses that information to recommend options to new users. Movie recommendations from online streaming services incorporate this type of filtering to make suggestions. A simplified example of the underlying dataset might include columns for a user ID, a movie ID and a rating. If a user streams the movie Citizen Kane, the system might, for example, recommend similar titles that are popular among other users who also watched that film.

One of the primary benefits of collaborative filtering systems is that they don’t require extensive datasets to train the model. These systems can work by identifying similarities between users. On the other hand, collaborative filtering doesn’t factor in potentially important granular details, such as specific features of products or services.

Content-based systems use weighted data

Content-based systems make recommendations based on aggregated user profile data. This type of system can learn from user preferences and will make recommendations based on either actual keywords in a profile or on similar tags that are inferred from a customized dataset. Different iterations might include variables, such as age, gender and location — along with further details about specific products and services.

A common foundation for this type of system is what is known as a tf-idf vectorizer. This is a mathematical model that ranks the importance of a term based on, for example, the number of times it is mentioned by a user. The term frequency weights a data point, like a movie title, based on how often it appears.

The idf, or inverse document frequency, part of the equation provides a way to reduce the weight of superfluous terms. For example, terms such as “the” or “and” are not helpful keywords for weighting and as a result, idf reduces the weight of frequently occurring, but less meaningful terms. Various weighting factors can be used to further refine the analysis to adjust characteristics like document or query term weight.

One of the significant advantages of content-based recommender systems is that they solve what is known as the “cold-start” problem. What this means is that a machine learning model can’t make predictions or inferences on products or services for which it has no data. A simple example is when a new movie is added to Netflix. Until users begin rating or accessing that movie, a collaborative system has no basis for suggesting it. But when the dataset includes product or service details, it can begin to make recommendations immediately.

One weakness of content-based systems is that they tend to be weighted in favor of the same types of products or services. A movie recommendation system might, for instance, be able to recommend The Many Saints of Newark — a new movie — because other users watched The Godfather, another movie. But it might not recommend The Sopranos, which is thematically similar but a new data type (a television show).

Hybrid systems combine the best of both worlds

Because collaborative and content-based systems have different benefits, most enterprise-level recommender algorithms include both types of functionality. Incremental updates flowing into hybrid data tables enable these systems to dynamically update themselves, learning as more user data becomes available.

Netflix is a great example of a hybrid recommender system in action. The Netflix app makes recommendations by combining data from similar users — collaborative filtering — with data about the shared characteristics of “movie” or “television show” — content-based filtering.

Recommender systems are widely used by media and retail companies. But back in 2006, when Netflix was transitioning from DVD rentals, machine learning was still in its infancy. While Netflix knew it needed a better online system, it did not have the in-house expertise to develop its own recommendation algorithms. To address this, it created the Netflix Prize, offering a million dollars in an open competition for the development of a collaborative filtering system based on user ratings. The catch was that the system had to produce better results than the CineMatch algorithm Netflix was using.

It took until 2009 before anyone won the prize. From there, the competition ultimately got complicated and was terminated — but not before Netflix emerged as a streaming media powerhouse.

The Netflix initiative helped fuel the development of hybrid recommender systems. These days, companies from Best Buy and Sephora, to YouTube and Amazon are using recommendation algorithms to improve customer experiences and boost sales.

The upside potential in terms of sales underscores just how beneficial AI can be. In fact, Amazon, which has built the gold standard hybrid recommendation system, attributes 35% of its sales to machine learning functionality.

Today’s businesses have to be data-driven. Better quality data translates into a more effective recommendation engine. Companies in every industry sector can use machine learning to help close sales by making better, more relevant recommendations to online shoppers and potential customers. A recommender system can also help with post-sales fulfillment by ensuring users know about important product or service upgrades.

The modern Internet is a cacophony of noise and options. Implementing a recommender system helps filter out the noise while aligning multi-channel communications with targeted recommendations that resonate with customers at just the right time and place on their buying journey. Make it easier for customers to make purchasing decisions and the profits will follow.

Learn more about TELUS International’s data validation and relevance services.

Check out our solutions

Ensure that your model performs at the highest level of accuracy and precision against real world data.

Learn more