EDITOR'S NOTE: Generalized Language Models is an extensive four-part series by Lillian Weng of OpenAI. Part 1: CoVe, ELMo & Cross-View TrainingPart 2: ULMFiT & OpenAI GPTPart 3: BERT & OpenAI GPT-2Part 4: Common Tasks & Datasets Do you find this in-depth technical education about language models and NLP applications to be useful? Subscribe below to … [Read more...] about Generalized Language Models: BERT & OpenAI GPT-2
Research
Generalized Language Models: ULMFiT & OpenAI GPT
EDITOR'S NOTE: Generalized Language Models is an extensive four-part series by Lillian Weng of OpenAI. Part 1: CoVe, ELMo & Cross-View TrainingPart 2: ULMFiT & OpenAI GPTPart 3: BERT & OpenAI GPT-2Part 4: Common Tasks & Datasets Do you find this in-depth technical education about language models and NLP applications to be useful? Subscribe below to … [Read more...] about Generalized Language Models: ULMFiT & OpenAI GPT
Generalized Language Models: CoVe, ELMo & Cross-View Training
EDITOR'S NOTE: Generalized Language Models is an extensive four-part series by Lillian Weng of OpenAI. Part 1: CoVe, ELMo & Cross-View TrainingPart 2: ULMFiT & OpenAI GPTPart 3: BERT & OpenAI GPT-2Part 4: Common Tasks & Datasets Do you find this in-depth technical education about language models and NLP applications to be useful? Subscribe below to … [Read more...] about Generalized Language Models: CoVe, ELMo & Cross-View Training
Beyond DQN/A3C: A Survey in Advanced Reinforcement Learning
One of my favorite things about deep reinforcement learning is that, unlike supervised learning, it really, really doesn’t want to work. Throwing a neural net at a computer vision problem might get you 80% of the way there. Throwing a neural net at an RL problem will probably blow something up in front of your face — and it will blow up in a different way … [Read more...] about Beyond DQN/A3C: A Survey in Advanced Reinforcement Learning
Topic Modeling with LSA, PLSA, LDA & lda2Vec
In natural language understanding (NLU) tasks, there is a hierarchy of lenses through which we can extract meaning — from words to sentences to paragraphs to documents. At the document level, one of the most useful ways to understand text is by analyzing its topics. The process of learning, recognizing, and extracting these topics across a collection of documents is called … [Read more...] about Topic Modeling with LSA, PLSA, LDA & lda2Vec
OpenAI GPT-2: Understanding Language Generation through Visualization
Are you interested in receiving more in-depth technical education about language models and NLP applications? Subscribe below to receive relevant updates. In the eyes of most NLP researchers, 2018 was a year of great technological advancement, with new pre-trained NLP models shattering records on tasks ranging from sentiment analysis to … [Read more...] about OpenAI GPT-2: Understanding Language Generation through Visualization
Deconstructing BERT, Part 2: Visualizing the Inner Workings of Attention
This is the second part of a two-part series on deconstructing BERT. In part 1, Deconstructing BERT: Distilling 6 Patterns from 100 Million Parameters, I described how BERT’s attention mechanism can take on many different forms. For example, one attention head focused nearly all of the attention on the next word in the sequence; another focused on the previous … [Read more...] about Deconstructing BERT, Part 2: Visualizing the Inner Workings of Attention
Deconstructing BERT: Distilling 6 Patterns from 100 Million Parameters
The year 2018 marked a turning point for the field of Natural Language Processing, with a series of deep-learning models achieving state-of-the-art results on NLP tasks ranging from question answering to sentiment classification. Most recently, Google’s BERT algorithm has emerged as a sort of “one model to rule them all,” based on its superior performance over a wide variety of … [Read more...] about Deconstructing BERT: Distilling 6 Patterns from 100 Million Parameters
Novel Methods For Text Generation Using Adversarial Learning & Autoencoders
Just two years ago, text generation models were so unreliable that you needed to generate hundreds of samples in hopes of finding even one plausible sentence. Nowadays, OpenAI's pre-trained language model can generate relatively coherent news articles given only two sentence of context. Other approaches like Generative Adversarial Networks (GANs) and Variational … [Read more...] about Novel Methods For Text Generation Using Adversarial Learning & Autoencoders
5 New Generative Adversarial Network (GAN) Architectures For Image Synthesis
AI image synthesis has made impressive progress since Generative Adversarial Networks (GANs) were introduced in 2014. GANs were originally only capable of generating small, blurry, black-and-white pictures, but now we can generate high-resolution, realistic and colorful pictures that you can hardly distinguish from real photographs. Here we have summarized for you 5 … [Read more...] about 5 New Generative Adversarial Network (GAN) Architectures For Image Synthesis