Cutting-Edge AI Research Techniques for Personalizing Your Customer Experience

This research summary is part of our AI for Marketing series which covers the latest AI & machine learning approaches to 5 aspects of marketing automation:

• Attribution
• Optimization
• Personalization
• Analytics
• Content Generation: Images
• Content Generation: Videos
• Content Generation: Text

In this piece, we cover how AI can help personalize the customer experience, leading to higher satisfaction rates and greater revenue growth.

Customers are used to getting a personalized experience from each company they interact with. You can personalize the experience of your customers by building effective recommender systems. These are the systems that personalize product placement and search results for each consumer. When you recommend products or content that customers are more likely to purchase, this gives the customer a better sales experience while driving more revenue for businesses through cross-selling and up-selling.

Recommender systems can be based on collaborative filtering when it’s assumed that people with similar characteristics and interests are likely to prefer the same items, clustering algorithms that group together users who have similar interests, or deep neural networks that rank the recommendations according to a user’s history and contextual conditions.

We have summarized for you the key research advances introduced during the last few years for building state-of-the-art recommender systems.

If these accessible AI research analyses & summaries are useful for you, you can subscribe to receive our regular industry updates below.

If you’d like to skip around, here are the papers we featured:

1. Sequence-Aware Recommender Systems
2. Variational Autoencoders for Collaborative Filtering
3. Explainable Recommendation via Multi-Task Learning in Opinionated Text Data
4. Modeling customer online behaviours with neural networks: applications to conversion prediction and advertising retargeting
5. Stabilizing Reinforcement Learning in Dynamic Environment with Application to Online Recommendation
6. Deep Learning Recommendation Model for Personalization and Recommendation Systems
7. Personalized Purchase Prediction of Market Baskets with Wasserstein-Based Sequence Matching
8. Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches

Important Personalization Research Papers

1. Sequence-Aware Recommender Systems by Massimo Quadrana, Paolo Cremonesi, Dietmar Jannach

Original Abstract

Recommender systems are one of the most successful applications of data mining and machine learning technology in practice. Academic research in the field is historically often based on the matrix completion problem formulation, where for each user-item-pair only one interaction (e.g., a rating) is considered. In many application domains, however, multiple user-item interactions of different types can be recorded over time. And, a number of recent works have shown that this information can be used to build richer individual user models and to discover additional behavioral patterns that can be leveraged in the recommendation process. In this work, we review existing works that consider information from such sequentially-ordered user- item interaction logs in the recommendation process. Based on this review, we propose a categorization of the corresponding recommendation tasks and goals, summarize existing algorithmic solutions, discuss methodological approaches when benchmarking what we call sequence-aware recommender systems, and outline open challenges in the area.

Our Summary

In this paper, the researchers provide a review of the available sequence-aware recommender systems. They categorize various scenarios of sequence-aware recommendations approaches, review the existing algorithms that were proposed to extract and leverage patterns from interaction logs, and discuss specific issues when benchmarking different recommendation methods. Finally, the paper outlines open challenges in the area. The main goal of this research is to lay the path for more standardized and better reproducible research in this area.

What’s the core idea of this paper?

• Depending on the application scenario, sequence-aware recommender systems can help in achieving the following goals:
  • context adaptation;
  • trend detection;
  • repeated recommendation;
  • consideration of order constraints and sequential patterns.
• There are three main classes of algorithms used for the extraction of patterns from the sequential log of user actions:
  • sequence-learning:
    • frequent pattern mining;
    • sequence modeling using Markov models, reinforcement learning and recurrent neural networks;
    • distributed item representations;
    • supervised learning with sliding windows;
  • sequence-aware matrix factorization;
  • hybrids that combine the flexibility of sequence learning methods with the robustness to data sparsity of factorization-based matrix-completion techniques.

What’s the key achievement?

• Providing a comprehensive overview of the existing sequence-aware recommender systems.
• Outlining future research areas.

What does the AI community think?

• The ACM Conference on Recommender Systems (ACM RecSys 2018) hosted a tutorial session based on this survey research paper.

What are future research areas?

• The paper identifies a number of open research questions, including:
  • intent detection;
  • combining short-term and long-term profiles;
  • leveraging additional data and general trends;
  • developing standardized and more comprehensive evaluations.

What are possible business applications?

• This overview of the existing sequence-aware recommender systems can help companies in:
  • understanding the weaknesses of their recommender systems and possibilities for improvement;
  • choosing a solution that is the best fit for particular business needs.

Where can you get implementation code?

• The authors provide the code for the hands-on session of the tutorial on Sequence-Aware Recommenders at ACM RecSys 2018.

2. Variational Autoencoders for Collaborative Filtering by Dawen Liang, Rahul G. Krishnan, Matthew D. Hoffman, Tony Jebara

Original Abstract

We extend variational autoencoders (VAEs) to collaborative filtering for implicit feedback. This non-linear probabilistic model enables us to go beyond the limited modeling capacity of linear factor models which still largely dominate collaborative filtering research. We introduce a generative model with multinomial likelihood and use Bayesian inference for parameter estimation. Despite widespread use in language modeling and economics, the multinomial likelihood receives less attention in the recommender systems literature. We introduce a different regularization parameter for the learning objective, which proves to be crucial for achieving competitive performance. Remarkably, there is an efficient way to tune the parameter using annealing. The resulting model and learning algorithm have information-theoretic connections to maximum entropy discrimination and the information bottleneck principle. Empirically, we show that the proposed approach significantly outperforms several state-of-the-art baselines, including two recently-proposed neural network approaches, on several real-world datasets. We also provide extended experiments comparing the multinomial likelihood with other commonly used likelihood functions in the latent factor collaborative filtering literature and show favorable results. Finally, we identify the pros and cons of employing a principled Bayesian inference approach and characterize settings where it provides the most significant improvements.

Our Summary

The joint group of researchers from Netflix, Google AI, and MIT introduces a novel approach to collaborative filtering with variational autoencoders (VAEs). They claim that in contrast to the traditional approaches to collaborative filtering, VAEs enable exploration of non-linear features, and thus significantly boost the recommendation performance. However, VAEs need two adjustments to be applied to recommender systems: (1) using a multinomial likelihood for data distribution; (2) adjusting the regularization parameter for the learning objective. Empirical results demonstrate that with these adjustments VAEs become a practical solution to collaborative filtering. Moreover, they significantly outperform several state-of-the-art baselines.

What’s the core idea of this paper?

• Introducing a variant of VAE for collaborative filtering on implicit feedback data:
  • using multinomial likelihood function for modeling user-item implicit feedback data;
  • applying Bayesian inference for parameter estimation;
  • introducing a different regularization parameter to only partially regularize a VAE.

What’s the key achievement?

• Achieving state-of-the-art results on three real-world datasets when compared to various strong baselines including the neural-network-based approaches.
• Showing that multinomial likelihood outperforms more common Gaussian and logistic likelihoods.
• Comparing a principled Bayesian inference approach with a point estimate approach and listing the pros and cons of both approaches. In particular:
  • the principled Bayesian approach is more robust regardless of the scarcity of the data, while
  • the point estimate approach requires fewer parameters in the bottleneck layer.

What does the AI community think?

• The paper was presented at the 2018 Web Conference (WWW 2018).

What are future research areas?

• Investigating the tradeoff introduced by the additional regularization parameter.
• Getting a better understanding of why the suggested approach works so well.
• Extending the introduced model by conditioning on side information.

What are possible business applications?

• Variational autoencoders with the suggested adjustments can be a good practical solution for building a well-performing and robust recommender system.

Where can you get implementation code?

• The authors provide the source code to reproduce the experimental results.

3. Explainable Recommendation via Multi-Task Learning in Opinionated Text Data by Nan Wang, Hongning Wang, Yiling Jia, Yue Yin

Original Abstract

Explaining automatically generated recommendations allows users to make more informed and accurate decisions about which results to utilize, and therefore improves their satisfaction. In this work, we develop a multi-task learning solution for explainable recommendation. Two companion learning tasks of user preference modeling for recommendation and opinionated content modeling for explanation are integrated via a joint tensor factorization. As a result, the algorithm predicts not only a user’s preference over a list of items, i.e., recommendation, but also how the user would appreciate a particular item at the feature level, i.e., opinionated textual explanation. Extensive experiments on two large collections of Amazon and Yelp reviews confirmed the effectiveness of our solution in both recommendation and explanation tasks, compared with several existing recommendation algorithms. And our extensive user study clearly demonstrates the practical value of the explainable recommendations generated by our algorithm.

Our Summary

This research paper stresses the importance of explaining the recommendations to the users. The system that not only makes good recommendations but also explains to the customers why they may like the suggested product leads to more accurate and informed decisions, and thus, higher customer satisfaction. The researchers take a multi-task learning approach to integrate two complementary tasks: 1) providing a recommendation by modeling user preference and 2) explaining this recommendation by modeling opinionated content. Extensive experiments demonstrate the effectiveness of the proposed solution with regards to both item recommendation and explanation generation tasks.

What’s the core idea of this paper?

• The idea of the paper is to build a recommender system that will be able to produce recommendations in the form: “We recommend this phone to you because of its high-resolution screen.”
• To provide such explainable recommendations, the researchers suggest exploiting the opinionated review text data that users give in addition to their overall assessments of the recommended items.
• The suggested approach integrates two companion tasks:
  • user preference modeling to provide item recommendation;
  • opinionated content modeling to provide an explanation of the recommendation.
• The task of item recommendation:
  • is modeled by a three-way tensor over user, item, and feature;
  • and describes users’  preferences on each feature of the item.
• The task of opinionated content analysis:
  • is modeled by two three-way tensors – one over user, feature, opinionated phrase, and another over item, feature, opinionated phrase;
  • both tensors are constructed from the user-generated review content.

What’s the key achievement?

• The suggested approach significantly outperforms several strong baselines on both Amazon and Yelp datasets. The experiments demonstrate that this method:
  • identifies users’ true feature-level preference;
  • is able to predict the detailed reasons that a user might endorse the item.
• Extensive qualitative analysis also demonstrates the practical value of the explainable recommendations:
  • users in the simulation-based user study provided positive feedback on the generated recommendations and explanations.

What does the AI community think?

• The paper was presented at ACM SIGIR 2018, an important conference on research and development in Information Retrieval.

What are future research areas?

• Incorporating external resources for more accurate modeling of the dependency among different entities.
• Using more advanced text synthesis techniques to generate more complete and natural sentences for explanations.
• Conducting user study with the real-world deployment of the suggested approach to evaluate its utility with real user populations.

What are possible business applications?

• The recommender system based on the suggested approach can be of high practical value in the business setting since:
  • this method not only improves the quality of the generated recommendations but also provides valuable explanations of these recommendations;
  • good recommendations supplemented with accurate explanations benefit both sales and customer satisfaction.

Where can you get implementation code?

• The researchers provide the implementation of the suggested approach on GitHub.

4. Modeling customer online behaviours with neural networks: applications to conversion prediction and advertising retargeting by Yanwei Cui, Rogatien Tobossi, Olivia Vigouroux

Original Abstract

In this paper, we apply neural networks into digital marketing world for the purpose of better targeting the potential customers. To do so, we model the customer online behaviours using dedicated neural network architectures. Starting from user searched keywords in a search engine to the landing page and different following pages, until the user left the site, we model the whole visited journey with a Recurrent Neural Network (RNN), together with Convolution Neural Networks (CNN) that can take into account of the semantic meaning of user searched keywords and different visited page names. With such model, we use Monte Carlo simulation to estimate the conversion rates of each potential customer in the future visiting. We believe our concept and the preliminary promising results in this paper enable the use of largely available customer online behaviours data for advanced digital marketing analysis.

Our Summary

The researchers from the French insurance company suggest a way to take advantage of the largely available customer online behaviors data and estimate the conversion rate of each potential customer by modeling his/her behaviors with dedicated neural networks. In particular, they use Convolutional Neural Networks (CNNs) to embed user searched keywords and the visited page names, and then fed the embedded feature vectors into Recurrent Neural Networks (RNNs) to model the customer visited journey.  The probability of online conversion is estimated using the Monte Carlo simulation. The experiments show that the model generates realistic scenarios of consumer behavior.

What’s the core idea of this paper?

• The paper introduces a novel approach for modeling customer online journey with neural networks. The method relies on three key components:
  • a Convolutional Neural Network for embedding the searched keywords and page names, and thus, capturing critical information about the visitor’s motivation;
  • a Recurrent Neural Network, specifically a multi-layer LSTM, for modeling the long term dependencies among different pages, and thus, reflecting the in-session customer visited journey;
  • Monte Carlo process for simulating the customer journey in order to estimate the probabilities of online conversion for each predefined objective.

What’s the key achievement?

• The experiments demonstrate that:
  • the model is able to mimic the actual visits behavior;
  • the simulated situations are realistic, according to the expert opinions.

What are future research areas?

• Incorporating the attention mechanism into the model.
• Integrating the proposed methods as utility estimation module in online display advertising within the reinforcement learning framework for programmatic marketing.

What are possible business applications?

• The suggested approach generates conversion prediction for each customer for each objective, enabling the precise online advertising retargeting.

5. Stabilizing Reinforcement Learning in Dynamic Environment with Application to Online Recommendation by Shi-Yong Chen, Yang Yu, Qing Da, Jun Tan, Hai-Kuan Huang, Hai-Hong Tang

Original Abstract

Deep reinforcement learning has shown great potential in improving system performance autonomously, by learning from iterations with the environment. However, traditional reinforcement learning approaches are designed to work in static environments. In many real-world problems, the environments are commonly dynamic, in which the performance of reinforcement learning approaches can degrade drastically. A direct cause of the performance degradation is the high-variance and biased estimation of the reward, due to the distribution shifting in dynamic environments. In this paper, we propose two techniques to alleviate the unstable reward estimation problem in dynamic environments, the stratified sampling replay strategy and the approximate regretted reward, which address the problem from the sample aspect and the reward aspect, respectively. Integrating the two techniques with Double DQN, we propose the Robust DQN method. We apply Robust DQN in the tip recommendation system in Taobao online retail trading platform. We firstly disclose the highly dynamic property of the recommendation application. We then carried out online A/B test to examine Robust DQN. The results show that Robust DQN can effectively stabilize the value estimation and, therefore, improves the performance in this real-world dynamic environment.

Our Summary

Instability of reinforcement learning methods in dynamic environments is one of the key obstacles to using these approaches in a real-world setting. In this paper, the researchers from Nanjing University and Alibaba Group introduce the Robust DQN method that combines Double DQN with two techniques directed at stabilizing the agent’s performance. In particular, they propose to address the problem from both the sample aspect and the reward aspect by incorporating the stratified sampling replay strategy and the approximate regretted reward, respectively. The experiments demonstrate that Robust DQN is effective at stabilizing the value estimation, and thus, leads to better performance in the real-world setting.

What’s the core idea of this paper?

• Dynamic environments, like online recommendation systems, bring the direct challenge for the reinforcement learning methods caused by a large variance in the reward estimation. In particular, when the agent receives an increase of the reward, it cannot distinguish if this increase is the result of the previous action, or is just due to the environment change.
• To improve the reward estimation in dynamic environments, the researchers propose two strategies:
  • the stratified sampling replay to introduce a prior customer distribution for addressing the variance caused by the customer distribution changes;
  • the approximate regretted reward for addressing the reward bias caused by environment changes.
• The suggested strategies are integrated into a classical Q-learning algorithm, i.e., Double DQN, resulting in the Robust DQN algorithm.

What’s the key achievement?

• Online A/B test that compared Robust DQN with the original Double DQN, demonstrates that:
  • both the stratified sampling replay and the approximate regretted reward strategies improve the reward estimation;
  • when the strategies are used together the performance improves even further leading to more effective recommendations.

What does the AI community think?

• The paper was presented at the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD 2018).

What are future research areas?

• Exploring the ways to further reduce the variance of the reward.

What are possible business applications?

• The stabilizing techniques introduced in this research paper make reinforcement learning approaches more suitable for real-world recommendation systems.

6. Deep Learning Recommendation Model for Personalization and Recommendation Systems, by Maxim Naumov, Dheevatsa Mudigere, Hao-Jun Michael Shi, Jianyu Huang, Narayanan Sundaraman, Jongsoo Park, Xiaodong Wang, Udit Gupta, Carole-Jean Wu, Alisson G. Azzolini, Dmytro Dzhulgakov, Andrey Mallevich, Ilia Cherniavskii, Yinghai Lu, Raghuraman Krishnamoorthi, Ansha Yu, Volodymyr Kondratenko, Stephanie Pereira, Xianjie Chen, Wenlin Chen, Vijay Rao, Bill Jia, Liang Xiong, Misha Smelyanskiy

Original Abstract

With the advent of deep learning, neural network-based recommendation models have emerged as an important tool for tackling personalization and recommendation tasks. These networks differ significantly from other deep learning networks due to their need to handle categorical features and are not well studied or understood. In this paper, we develop a state-of-the-art deep learning recommendation model (DLRM) and provide its implementation in both PyTorch and Caffe2 frameworks. In addition, we design a specialized parallelization scheme utilizing model parallelism on the embedding tables to mitigate memory constraints while exploiting data parallelism to scale-out compute from the fully-connected layers. We compare DLRM against existing recommendation models and characterize its performance on the Big Basin AI platform, demonstrating its usefulness as a benchmark for future algorithmic experimentation and system co-design.

Our Summary

The Facebook research team addresses personalization by combining perspectives from recommendation systems and predictive analytics. Specifically, they introduce a Deep Learning Recommendation Model (DLRM) that uses embeddings to process sparse features and a multilayer perceptron (MLP) to process dense features. Then, the model combines these features explicitly and defines the event probability using another MLP. The experiments demonstrate the effectiveness of the suggested approach in building a recommender system.

What’s the core idea of this paper?

• To tackle personalization with neural networks, the Facebook team introduces a Deep Learning Recommendation Model (DLRM):
  • All categorical features are represented by an embedding vector.
  • The continuous features are transformed by a multilayer perceptron (MLP) providing a dense representation of the same length as the embedding vectors.
  • Then, the second-order interaction of different features is computed explicitly following the approach of handling sparse data provided in factorization machines. Namely, the dot product between all pairs of embedding vectors and processed dense features is computed.
  • In the next step, the dot products are combined with the original dense features and post-processed using another MLP.
  • Finally, the output of the MLP is fed into a sigmoid function to give a probability.

What’s the key achievement?

• Introducing a novel deep learning-based recommendation model that exploits categorical data and demonstrates good performance compared to the existing approaches.
• Open-sourcing implementation of the introduced model.

What are future research areas?

• Further improving the model’s performance by tuning hyperparameters and experimenting with model design.

What are possible business applications?

• Companies can use the open-sourced implementation of the suggested deep learning-based model to enhance their recommender systems with neural networks.

Where can you get implementation code?

• The authors provide the implementation of DLRM in PyTorch and Caffe2 on GitHub.

7. Personalized Purchase Prediction of Market Baskets with Wasserstein-Based Sequence Matching, by Mathias Kraus and Stefan Feuerriegel

Original Abstract

Personalization in marketing aims at improving the shopping experience of customers by tailoring services to individuals. In order to achieve this, businesses must be able to make personalized predictions regarding the next purchase. That is, one must forecast the exact list of items that will comprise the next purchase, i.e., the so-called market basket. Despite its relevance to firm operations, this problem has received surprisingly little attention in prior research, largely due to its inherent complexity. In fact, state-of-the-art approaches are limited to intuitive decision rules for pattern extraction. However, the simplicity of the pre-coded rules impedes performance, since decision rules operate in an autoregressive fashion: the rules can only make inferences from past purchases of a single customer without taking into account the knowledge transfer that takes place between customers. In contrast, our research overcomes the limitations of pre-set rules by contributing a novel predictor of market baskets from sequential purchase histories: our predictions are based on similarity matching in order to identify similar purchase habits among the complete shopping histories of all customers. Our contributions are as follows: (1) We propose similarity matching based on subsequential dynamic time warping (SDTW) as a novel predictor of market baskets. Thereby, we can effectively identify cross-customer patterns. (2) We leverage the Wasserstein distance for measuring the similarity among embedded purchase histories. (3) We develop a fast approximation algorithm for computing a lower bound of the Wasserstein distance in our setting. An extensive series of computational experiments demonstrates the effectiveness of our approach. The accuracy of identifying the exact market baskets based on state-of-the-art decision rules from the literature is outperformed by a factor of 4.0.

Our Summary

The research team from ETH Zurich addresses market basket prediction by considering the complete shopping histories of all available customers. Specifically, their algorithm learns to identify co-occurrences between shopping histories from different customers. The sequence-based similarity matching is computed according to the Wasserstein distance. Thereby, market baskets are interpreted as probability distributions of products from an assortment. The experiments on three real-world datasets demonstrate that the suggested approach significantly outperforms association rules and naïve heuristics with respect to the accuracy of market basket prediction.

What’s the core idea of this paper?

• The proposed algorithm for prediction of market baskets includes four steps:
  • Building item embeddings with each item represented as a multi-dimensional vector and similar items being closer together.
  • Utilizing the Wasserstein distance to compute distances between market baskets as the minimum cost of turning one probability distribution of products into the other.
  • Building a k-nearest neighbor sequence matching with a subsequence dynamic time warping (i.e., kNN-SDTW).
  • Making a prediction of the next market basket by choosing the most similar shopping histories.

What’s the key achievement?

• Introducing a novel market basket prediction algorithm that:
  • can learn hidden structure among products and leverage cross-customer knowledge for improved predictions;
  • achieves scalability by deriving a fast variant of subsequence matching;
  • outperforms baseline models by 2.54% on a multi-category dataset;
  • improves the ratio of correct predictions by a factor of 4.0 for a dataset covering food, office supplies, and furniture.

What does the AI community think?

• The paper was presented at KDD 2019, the leading conference in knowledge discovery and data mining.

What are future research areas?

• Exploring the ways to capture complex substitution effects driven by spontaneous purchases or price promotions.

What are possible business applications?

• The introduced algorithm can help companies achieve higher accuracy in predicting the future purchases of their customers, leading to an improved shopping experience for these customers and increased sales.

Where can you get implementation code?

• The authors provide the implementation of the presented approach on GitHub.

8. Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches, by Maurizio Ferrari Dacrema, Paolo Cremonesi, Dietmar Jannach

Original Abstract

Deep learning techniques have become the method of choice for researchers working on algorithmic aspects of recommender systems. With the strongly increased interest in machine learning in general, it has, as a result, become difficult to keep track of what represents the state-of-the-art at the moment, e.g., for top-n recommendation tasks. At the same time, several recent publications point out problems in today’s research practice in applied machine learning, e.g., in terms of the reproducibility of the results or the choice of the baselines when proposing new models.

In this work, we report the results of a systematic analysis of algorithmic proposals for top-n recommendation tasks. Specifically, we considered 18 algorithms that were presented at top-level research conferences in the last years. Only 7 of them could be reproduced with reasonable effort. For these methods, it however turned out that 6 of them can often be outperformed with comparably simple heuristic methods, e.g., based on nearest-neighbor or graph-based techniques. The remaining one clearly outperformed the baselines but did not consistently outperform a well-tuned non-neural linear ranking method. Overall, our work sheds light on a number of potential problems in today’s machine learning scholarship and calls for improved scientific practices in this area.

Our Summary

The researchers question the progress that deep learning techniques bring into the recommender system area. They conduct a systematic analysis of 18 research papers that have introduced new algorithms for proposing top-n recommendations and have been presented at top conferences during the last few years. The authors identify two major issues with this research: (1) lack of reproducibility, with only 7 out of 18 papers providing sufficient information for reproducing their research; (2) lack of progress, with 6 out of 7 reproduced models being outperformed using simple heuristic methods. Thus, the researchers call for more rigorous research practices with respect to the evaluation of new contributions in this area.

What’s the core idea of this paper?

• Despite neural networks becoming a popular tool for building recommender systems, the progress that these methods introduce compared to simple heuristic practices is questionable.
• The factors that contribute to that phenomena include:
  • weak baselines that researchers use when comparing their novel approaches;
  • difficulties with reproducing results across papers as source code is often not shared;
  • using different types of datasets, evaluation protocols, performance measures, and data preprocessing steps, which makes it more difficult to reproduce and compare the introduced approaches.

What’s the key achievement?

• Demonstrating that new deep learning-based approaches for top-n recommendation tasks are not making much progress compared to methods based on nearest-neighbor or graph-based techniques:
  • Only 7 out of 18 research papers selected for analysis could be reproduced.
  • 6 out of 7 models were outperformed by comparably simple heuristic methods.
  • One model (Mult-VAE) clearly outperformed the baselines but did not consistently perform better than a well-tuned non-neural linear ranking method.

What does the AI community think?

• The paper was presented at RecSys 2019, the 13th ACM Conference on Recommender Systems.

What are future research areas?

• Extending the analysis to other publication outlets beyond conferences and other types of recommendation problems.
• Considering more traditional algorithms as baselines (e.g., matrix factorization).

Where can you get implementation code?

• The authors provide the implementation of their evaluation on GitHub.

Enjoy this article? Sign up for more AI for marketing research updates.

We’ll let you know when we release more summary articles like this one.