Recommender systems are lifesavers in the infinite seething sea of e-commerce, improving customer experience. Recommender engines are eliminating the tyranny of choice, smoothing the way for decision-making, and boosting online sales. Moreover, ubiquitous AI technologies are sneaking into e-commerce, too, not only solving the problems of irrelevant recommendations but predicting the customer’s next steps.
In light of the pandemic’s impact, online sales are projected to grow twice in the next three years. Under those circumstances, businesses should ensure a perfect customer experience with precise recommendations to stand out. Read on to find out how recommender systems work, which algorithms stand behind them, and apply deep learning algorithms.
Traditional recommender systems
Not surprising that media, retail, job listings, education, real estate, and travel companies are already using deep learning. It brings a killer feature for predicting ratings, defining the following items in the basket, and providing a personalized customer experience. For instance, 80% of content watched on Netflix, and 60% of videos on YouTube came from recommendations,
First, learn how traditional recommendation systems work before diving into the complex deep learning-based ones.
Traditional recommender systems (RSs) include content-based and collaborative filtering (CF) systems grounding their recommendations on historical interactions and user/item attributes. Content-based recommendations are mainly drawn on the user’s item and profile features, and CF seeks a similar audience’s preferences.
Furthermore, the CF approach is divided into memory-based (using the nearest neighbor classification method) and model-based (includes machine learning and data mining techniques). Also, the hybrid approach reaps the harvest of both content-based and collaborative filtering methods.
At the same time, these systems have their constraints. For example, the cold start problem is an issue of irrelevant recommendations for a new user who still has performed few system interactions. Also, one can face a data sparsity problem. Think of millions of items on Amazon and a small amount of actual item-user interactions per typical user.
We have explained all the magic and constraints of traditional RSs before. Check out “Inside Recommendations: How a Recommender System Recommends“ for details.
Deep learning-based recommender systems: How does it actually work?
In general, deep learning (DL) is the subfield of ML learning from multiple levels of data representations and abstractions. Some tech companies are already using DL systems drawn on different neural networks (NNs) to make the customer experience better. For instance, YouTube, eBay, Yahoo, and Twitter choose deep neural networks (DNNs), while Spotify prefers convolutional neural networks (CNNs).
Meanwhile, DNNs and CNNs are only a few types of networks applied, as we can continue this list of deep learning algorithms. Why do we need a range of them? The answer is related to the business domain, particular task, or recommender scenario. Owing to the use case, one might leverage different NNs or even a hybrid approach to DL-based recommendation systems.
The principal feature that differs DL-based recommender systems from traditional ones is coping with complex interaction patterns and precisely reflecting the user’s preferences. Content-based and collaborative filtering models are relatively linear systems that cannot deliver such deep user insights.
Deep learning for recommendations: Glossary
To explain how each particular RS operates and what kind of benefits it brings, we have developed an overview of the most popular techniques.
Convolutional neural networks based recommender systems
Typical CNN architecture
Convolutional Neural Networks (CNNs) are a good fit for unstructured multimedia data processing given effective feature extraction. They are processing the data like image, text, audio, and video. CNNs help to eliminate the cold start problem or empower traditional systems like collaborative filtering. This feature is critical for e-commerce, as most customers conclude their decisions by assessing goods’ visuals.
CNNs is also an option for non-Euclidean data (non-ordinal or hierarchical data) like social networks, protein-interaction networks, and knowledge graph. For instance, this kind of system could apply to Pinterest recommendations.
Recurrent neural networks based recommender systems
Unfolded basic recurrent neural network
Recurrent Neural Networks (RNNs) could become a killer feature for sequential data processing, defining temporal dynamics of interactions and sequential user behavior patterns. For example, YouTube recommends content for a particular time of the day or predicts the next piece of content drawn on the already watched one.
The majority of websites today do not require a user to log in for navigation. In other words, it means no access to customers’ long-term interests or consumption habits. Meanwhile, a cookie mechanism (i.e., session mechanism) can solve this task. RNNs can help build session-based recommendations without user identification information or even predict what users can buy next based on their click history.
Restricted Boltzmann machine based recommender systems
Restricted Boltzmann Machine (RBM) belongs to the most old-fashioned RSs rooted from 2007 but is still in play. Significantly, RBM, combined with collaborative filtering, won the Netflix Prize for better recommendations on the streaming platform in 2009. Moreover, RBM-based techniques are still scalable to large data sets and producing high-quality recommendations of items per particular user.
Autoencoder based recommender systems
In essence, an autoencoder is a neural network that reconstructs its input data in the output layer. It has an internal hidden layer that describes a code used to represent the input. The autoencoder consists of two main parts. They are an encoder that maps the information into the code and a decoder that maps the code to reconstruct the input. The beauty of autoencoder is in its agility in data dimensionality reduction, data reconstruction, and feature extraction.
Neural attention based recommender systems
Attention mechanism derives from computer vision and natural language processing domains. Simply put, it is a vector of importance weights that predicts the next item. The attention mechanism is based on correlation with other elements (e.g., a pixel in the image or the next word in a sentence). In essence, human visual attention stands as a source of inspiration for this technique. The system can “focus” on a particular element to make its following recommendation.
Illustration of the self-attention module. The input is the embedding matrix of the latest interacted L items, and the output is the self-attentive representations.
Applying an attention mechanism to the recommender system can help filter out uninformative content and choose the most representative items. It provides good interpretability at the same time. It is also possible to integrate neural attention models with DNNs or CNNs.
Deep learning recommender systems: Pros and cons
When it goes about complexity or numerous training instances (an object that an ML model learns from), deep learning is justified for recommendations. While neural network models show higher results, it is also possible to tune up conventional RSs with neural architecture to be on par. However, it needs some extra manipulations. To define when you need to opt for DL based RS, keep in mind the following advantages and drawbacks:
Are there any drawbacks of applying a complex technique like DL for recommender systems? The answer is yes. Depending on the way of using, some potential limitations could enter the picture:
Bottom line
Deep learning-based recommender systems outperform traditional ones due to their capability to process non-linear data. Non-linear transformation, representation learning, sequence modeling, and flexibility are the principal benefits of applying DL for recommendations.
Moreover, DL techniques could be tailored for specific tasks. For example, CNNs are a good fit for non-Euclidean data, and DNNs are for sequential data processing. Autoencoder helps to secure data dimensionality reduction, and neural attention-based systems are suitable for filtering needed data and choosing the most representative items.
Posted initially on SciForce blog