AI Approaches For Text Generation In Marketing & Advertising Use Cases

This research summary is part of our AI for Marketing series which covers the latest AI & machine learning approaches to 5 aspects of marketing automation:

• Attribution
• Optimization
• Personalization
• Analytics
• Content Generation: Images
• Content Generation: Videos
• Content Generation: Text

Can AI help you write high converting copy for your advertising and marketing campaigns?

Alibaba introduced its AI copywriter over a year ago. This is a copywriting tool that generates product listings after learning from millions of top-quality existing samples. It can produce 20,000 lines of copy in a second, and lots of brands and advertisers already take advantage of this tool. All you need to do is to insert a link to any product page, and click the “Produce Smart Copy” button to get multiple copy ideas you can choose from.

Are you excited about the text generation capabilities of AI that can be leveraged in marketing and advertising? To help you stay aware of the latest research advances in this area, we have summarized for you key research papers that introduce novel AI-powered approaches to optimizing product listing and generating advertising keywords and puns.

If these accessible AI research analyses & summaries are useful for you, you can subscribe to receive our regular industry updates below.

If you’d like to skip around, here are the papers we featured:

1. Using General Adversarial Networks for Marketing: A Case Study of Airbnb
2. Domain-Constrained Advertising Keyword Generation
3. Pun Generation with Surprise
4. Towards Controllable and Personalized Review Generation

Important Text Generation Research Papers

1. Using General Adversarial Networks for Marketing: A Case Study of Airbnb, by Richard Diehl Martinez and John Kaleialoha Kamalu

Original Abstract

In this paper, we examine the use case of general adversarial networks (GANs) in the field of marketing. In particular, we analyze how GAN models can replicate text patterns from successful product listings on Airbnb, a peer-to-peer online market for short-term apartment rentals. To do so, we define the Diehl-Martinez-Kamalu (DMK) loss function as a new class of functions that forces the model’s generated output to include a set of user-defined keywords. This allows the general adversarial network to recommend a way of rewording the phrasing of a listing description to increase the likelihood that it is booked. Although we tailor our analysis to Airbnb data, we believe this framework establishes a more general model for how generative algorithms can be used to produce text samples for the purposes of marketing.

Our Summary

In the peer-to-peer markets like Airbnb and eBay where sellers and buyers need to find one another in a decentralized manner, sellers are responsible for presenting their articles and listing in the most appealing way. However, they often lack marketing expertise to attract potential buyers. Thus, the researchers from Stanford University introduce a solution for the automatic generation of product listings that are likely to increase conversions. Specifically, they suggest combining Generative Adversarial Networks (GANs) with the Diehl-Martinez-Kamalu (DMK) loss function that forces the output to include specific keywords. The experiments show that the introduced approach establishes quite a promising attempt at using GANs models to enhance the marketing efforts of users on peer-to-peer marketplaces.

GAN Framework

What’s the core idea of this paper?

• As the use of online peer-to-peer platforms is becoming widespread, there is a need for the development of self-branding and marketing tools that will help sellers enhance their marketability.
• To that end, the research team suggests a model that can replicate the writing style of high-popularity listing descriptions. In particular, they:
  • introduce Generative Adversarial Network with both generator and discriminator using a feed-forward structure with three layers of depth;
  • extend the basic GAN framework with the Diehl-Martinez-Kamalu (DMK) loss that forces the generated output to contain user-defined keywords.

What’s the key achievement?

• The experiments show that the network trained on DMK loss demonstrates great promise. However, the results are not as good as expected because apparently other factors including location, amenities, and home type play a larger role in the consumer’s decision than listings description.

What are future research areas?

• Extending analysis with recent developments in unsupervised generative models that can significantly improve the quality of generated texts (e.g., variational autoencoders, OpenAI GPT-2).
• Experimenting with various forms of word embeddings, such as improved word vectors (IWV), which are capable of encoding additional information about every word, and so, increasing the accuracy of the marketing data analysis.

What are possible business applications?

• Although the research was based on peer-to-peer platforms, the introduced framework can be also applied to generate text samples that increase marketing conversions in various marketplaces.

2. Domain-Constrained Advertising Keyword Generation by Hao Zhou, Minlie Huang, Yishun Mao, Changlei Zhu, Peng Shu, Xiaoyan Zhu

Original Abstract

Advertising (ad for short) keyword suggestion is important for sponsored search to improve online advertising and increase search revenue. There are two common challenges in this task. First, the keyword bidding problem: hot ad keywords are very expensive for most of the advertisers because more advertisers are bidding on more popular keywords, while unpopular keywords are difficult to discover. As a result, most ads have few chances to be presented to the users. Second, the inefficient ad impression issue: a large proportion of search queries, which are unpopular yet relevant to many ad keywords, have no ads presented on their search result pages. Existing retrieval-based or matching-based methods either deteriorate the bidding competition or are unable to suggest novel keywords to cover more queries, which leads to inefficient ad impressions. To address the above issues, this work investigates to use generative neural networks for keyword generation in sponsored search. Given a purchased keyword (a word sequence) as input, our model can generate a set of keywords that are not only relevant to the input but also satisfy the domain constraint which enforces that the domain category of a generated keyword is as expected. Furthermore, a reinforcement learning algorithm is proposed to adaptively utilize domain-specific information in keyword generation. Offline evaluation shows that the proposed model can generate keywords that are diverse, novel, relevant to the source keyword, and accordant with the domain constraint. Online evaluation shows that generative models can improve coverage (COV), click-through rate (CTR), and revenue per mille (RPM) substantially in sponsored search.

Our Summary

The paper examines the use of generative neural networks for the generation of advertising keywords in sponsored search results. If a purchased keyword is provided as input, the proposed model can suggest a set of keywords based on the semantics of the input keyword. Thanks to including a domain constraint network, the generated keywords belong to the domain of the source keyword or several appropriate domains. Moreover, the researchers use reinforcement learning to adaptively utilize domain-specific information and further improve domain consistency and keyword quality. Extensive results of offline and online experiments demonstrate that the proposed model can generate diverse, completely novel, and domain-specific keywords, and improves the performance of a sponsored search.

What’s the core idea of this paper?

• There are two common challenges in ad keyword suggestion in sponsored search:
  • the keyword bidding problem, where popular keywords are too expensive for most of the advertisers;
  • the inefficient ad impression issue, where a substantial proportion of search queries that are relevant for many keywords, have no ads presented on their search result pages.
• To address these issues, the researchers introduce the Domain-Constrained Keyword Generator (DCKG) to generate diversified keywords with domain constraints using three mechanisms:
  • a latent variable sampled from a multivariate Gaussian distribution to generate diversified keywords;
  • a domain constraint network to facilitate generating domain-consistent keywords;
  • a reinforcement learning algorithm to further optimize the decoder to adjust the word generation distribution.

What’s the key achievement?

• The experiments demonstrate that the proposed approach outperforms other baselines by:
  • obtaining the best performance in domain accuracy and novelty;
  • generating relevant and grammatical keywords;
  • achieving the best recall, indicating that DCKG covers more potential user queries;
  • leading to more user clicks due to generated keywords being of higher quality and more relevant.

What does the AI community think?

• The paper was presented at the 2019 World Wide Web Conference (WWW’19).

What are future research areas?

• Since users are also key in the sponsored search game, additional research needs to be carried out to assess their experience. This will give a comprehensive assessment of all the factors involved in advertising keyword generation.

What are possible business applications?

• DCKG can be a basis for AI-powered tools capable of producing diverse, novel, relevant, and domain-centric keywords for enhancing the performance of sponsored search advertising.

3. Pun Generation with Surprise by He He, Nanyun Peng, Percy Liang

Original Abstract

We tackle the problem of generating a pun sentence given a pair of homophones (e.g., “died” and “dyed”). Supervised text generation is inappropriate due to the lack of a large corpus of puns, and even if such a corpus existed, mimicry is at odds with generating novel content. In this paper, we propose an unsupervised approach to pun generation using a corpus of unhumorous text and what we call the local-global surprisal principle: we posit that in a pun sentence, there is a strong association between the pun word (e.g., “dyed”) and the distant context, as well as a strong association between the alternative word (e.g., “died”) and the immediate context. This contrast creates surprise and thus humor. We instantiate this principle for pun generation in two ways: (i) as a measure based on the ratio of probabilities under a language model, and (ii) a retrieve-and-edit approach based on words suggested by a skip-gram model. Human evaluation shows that our retrieve-and-edit approach generates puns successfully 31% of the time, tripling the success rate of a neural generation baseline.

Our Summary

This paper introduces a novel approach to generating pun sentences based on the so-called local-global surprisal principle. In particular, to generate puns, the researchers create a humorous contrast between the local and global contexts. The local-global surprisal principle is instantiated by (1) developing a quantitative metric for surprise based on the ratio of probabilities of the pun word and the alternative word given local and global contexts under a language model; (2) developing an unsupervised approach to generating puns based on a retrieve-and-edit framework. The human evaluations demonstrate that the proposed approach outperforms other baselines in terms of a success rate as well as funniness and grammaticality scores.

Overview of the pun generation approach

What’s the core idea of this paper?

• The researchers propose a general principle for puns which they call the local-global surprisal principle. This principle posits that the pun word is much more surprising in the local context, while the alternative word is more surprising in the global context.
• To instantiate the local-global surprisal prinicple, the researchers:
  • develop a quantitative metric for surprise following the conditional probabilities of the pun word and the alternative word given local and global contexts under a neural language model;
  • introduce a novel unsupervised approach to generating puns based on a retrieve-and-edit framework.

What’s the key achievement?

• Human evaluation of the generated puns demostrates that:
  • the introduced approach generates puns successfully 31% of the time and triples the success rate of a neural generation baseline with improved funniness and grammaticality scores;
  • when compared with the expert-written puns, the generated puns were rated funnier around 10% of the time.

What does the AI community think?

• The research has created a buzz in the AI community. For example, the Wired published an article, which talked about the breakthrough that He and her team had in generating puns with AI.
• The paper was accepted for oral presentation at NAACL 2019, one of the leading conferences in computational linguistics.

What are future research areas?

• Developing methods that are nuanced enough to recognize the difference between creative texts (novel and well-formed content) and texts without much sense (ill-formed content).

What are possible business applications?

• The introduced approach to pun generation can be used to create tools that automatically generate funny, pun-intended texts in marketing and advertising.

Where can you get implementation code?

• The entire code, data, and experiments for this research paper can be found on CodaLab.

4. Towards Controllable and Personalized Review Generation, by Pan Li and Alexander Tuzhilin

Original Abstract

In this paper, we propose a novel model RevGAN that automatically generates controllable and personalized user reviews based on the arbitrarily given sentimental and stylistic information. RevGAN utilizes the combination of three novel components, including self-attentive recursive autoencoders, conditional discriminators, and personalized decoders. We test its performance on the several real-world datasets, where our model significantly outperforms state-of-the-art generation models in terms of sentence quality, coherence, personalization and human evaluations. We also empirically show that the generated reviews could not be easily distinguished from the organically produced reviews and that they follow the same statistical linguistics laws.

Our Summary

Only a small fraction of users have time to write reviews. To encourage users to provide feedback, the researchers from New York University suggest a model for the generation of controllable and personalized customer reviews. This model, called RevGAN, generates high-quality user reviews based on the product descriptions, sentiment labels, and previous reviews. The model has three components, including a Self-Attentive Recursive Autoencoder for capturing the hierarchical structure and semantic meaning of user reviews, a Conditional Discriminator for generating controllable user reviews in terms of quality and accuracy, and a Personalized Decoder for personalizing the writing style of the users. The experiments on different datasets show that the RevGAN model significantly outperforms strong baselines and generates reviews that are hard to distinguish from the original ones.

Self-Attentive Recursive AutoEncoder

What’s the core idea of this paper?

• The paper seeks to provide an additional tool for encouraging users to provide feedback after getting a product or service via online marketplace websites.
• To this end, the authors propose a novel model RevGAN for automated generation of high-quality and personalized user reviews. The model consists of three main components:
  • A Self-Attentive Recursive Autoencoder that maps users’ reviews and product descriptions into continuous embeddings to capture the hierarchical structure and semantic meaning of textual information;
  • A Conditional Discriminator that controls the sentiment of generated reviews by conditioning sentiment on the discriminator;
  • A Personalized Decoder that decodes the generated review embeddings by taking into account the personalized writing style of the user as captured from the user’s historical reviews.

What’s the key achievement?

• Introducing a novel model for controllable and personalized review generation that:
  • statistically and empirically outperforms state-of-the-art baselines with respect to sentence quality and coherency;
  • automatically generates reviews that are very similar to the organically-generated ones in terms of style and content.

What does the AI community think?

• The paper was presented at EMNLP 2019, one of the leading conferences in natural language processing.

What are future research areas?

• Exploring the methods for generating reviews based on several keywords provided by a user.
• Developing novel methods for distinguishing automatically generated reviews from the organic ones.

What are possible business applications?

• The introduced approach to automated review generation may help companies encourage their customers to provide feedback even if they don’t have time or don’t want to write a review: the clients can simply edit or approve the automatically generated review instead of writing it from scratch.

Enjoy this article? Sign up for more AI for marketing research updates.

We’ll let you know when we release more summary articles like this one.