The AI industry is moving so quickly that it’s often hard to follow the latest research breakthroughs and achievements. To help you stay well prepared for 2020, we have summarized the latest trends across different research areas, including natural language processing, conversational AI, computer vision, and reinforcement learning.
We also suggest key research papers in different areas that we think are representative of the latest advancements.
Subscribe to our AI Research mailing list at the bottom of this article to be alerted when we release new research articles.
Major AI & Machine Learning Trends
Natural Language Processing
In 2018, pretrained language models pushed the limits of natural language understanding and generation. These also dominated NLP progress last year.
Teams from top research institutions and tech companies explored ways to make state-of-the-art language models even more sophisticated. Many improvements were driven by massive boosts in computing capacities, but many research groups also found ingenious ways to lighten models while maintaining high performance.
Trending topics
- The new NLP paradigm is “pretraining + finetuning”. Transfer learning has dominated NLP research over the last two years. ULMFiT, CoVe, ELMo, OpenAI GPT, BERT, OpenAI GPT-2, XLNet, RoBERTa, ALBERT – this is a non-exhaustive list of important pretrained language models introduced recently. Even though transfer learning has definitely pushed NLP to the next level, it is often criticized for requiring huge computational costs and big annotated datasets.
- Linguistics and knowledge are likely to advance the performance of NLP models.The experts believe that linguistics can boost deep learning by improving the interpretability of the data-driven approach. Leveraging the context and human knowledge can further improve the performance of NLP systems.
- Neural machine translation is demonstrating visible progress. Simultaneous machine translation is already performing at the level where it can be applied in the real world. The recent research breakthroughs seek to further improve the quality of translation by optimizing neural network architectures, leveraging visual context, and introducing novel approaches to unsupervised and semi-supervised machine translation.
Breakthrough research papers
- Language Models Are Unsupervised Multitask Learners (OpenAI GPT-2)
- Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks
- ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Conversational AI
Conversational AI is becoming an integral part of business practice across industries. More companies are adopting the advantages chatbots bring to customer service, sales, and marketing.
Even though chatbots are becoming a “must-have” asset for leading businesses, their performance is still very far from human. Researchers from major research institutions and tech leaders have explored ways to boost the performance of dialog systems.
Trending topics
- Dialog systems are improving at tracking long-term aspects of a conversation. The goal of many research papers presented over the last year was to improve the system’s ability to understand complex relationships introduced during the conversation by better leveraging the conversation history and context.
- Many research teams are addressing the diversity of machine-generated responses Currently, real-world chatbots mostly generate boring and repetitive responses. Last year, several good research papers were introduced aiming at generating diverse and yet relevant responses.
- Emotion recognition is seen as an important feature for open-domain chatbots. Therefore, researchers are investigating the best ways to incorporate empathy into dialog systems. The achievements in this research area are still modest but considerable progress in emotion recognition can significantly boost the performance and popularity of social bots and also increase the use of chatbots in psychotherapy.
Breakthrough research papers
- Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems
- Do Neural Dialog Systems Use the Conversation History Effectively? An Empirical Study.
- Jointly Optimizing Diversity and Relevance in Neural Response Generation
Computer Vision
During the last few years, we can observe how computer vision (CV) systems are revolutionizing whole industries and business functions with successful applications in healthcare, security, transportation, retail, banking, agriculture, and more.
Recently introduced architectures and approaches like EfficientNet and SinGAN further improve the perceptive and generative capacities of visual systems.
Trending topics
- 3D is currently one of the leading research areas in CV. This year, we saw several interesting research papers aiming at reconstructing our 3D world from its 2D projections. The Google Research team introduced a novel approach to generating depth maps of entire natural scenes. The Facebook AI team suggested an interesting solution for 3D object detection in point clouds.
- The popularity of unsupervised learning methods is growing. For example, a research team from Stanford University introduced a promising Local Aggregation approach to object detection and recognition with unsupervised learning. In another great paper, nominated for the ICCV 2019 Best Paper Award, unsupervised learning was used to compute correspondences across 3D shapes.
- Computer vision research is being successfully combined with NLP. The latest research advances enable robust change captioning between two images in natural language, vision-language navigation in 3D environments, and learning hierarchical vision-language representation for better image caption retrieval and visual grounding.
Breakthrough research papers
- A theory of Fermat Paths for Non-Line-of-Sight Shape Reconstruction
- Local Aggregation for Unsupervised Learning of Visual Embeddings
- Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation
Reinforcement Learning
Reinforcement learning (RL) continues to be less valuable for business applications than supervised learning, and even unsupervised learning. It is successfully applied only in areas where huge amounts of simulated data can be generated, like robotics and games.
However, many experts recognize RL as a promising path towards Artificial General Intelligence (AGI), or true intelligence. Thus, research teams from top institutions and tech leaders are seeking ways to make RL algorithms more sample-efficient and stable.
Trending topics
- Multi-agent reinforcement learning (MARL) is rapidly advancing. The OpenAI team has recently demonstrated how the agents in a simulated hide-and-seek environment were able to build strategies that researchers did not know their environment supported. Another great paper received an Honorable Mention at ICML 2019 for investigating how multiple agents influence each other if provided with the corresponding motivation.
- Off-policy evaluation and off-policy learning are recognized as very important for future RL applications. The recent breakthroughs in this research area include new solutions for batch policy learning under multiple constraints, combining parametric and non-parametric models, and introducing a novel class of off-policy algorithms to force an agent towards acting close to on-policy.
- Exploration is an area where serious progress can be achieved. The papers presented at ICML 2019 introduced new efficient exploration methods with distributional RL, maximum entropy exploration, and a security condition to deal with the bridge effect in reinforcement learning.
Breakthrough research papers
- Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning
- Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables
- Emergent Tool Use From Multi-Agent Autocurricula
Further Reading
To get a more in-depth understanding of the latest trends in AI, check out our curated lists of top research papers:
- Top AI & Machine Learning Research Papers From 2019
- What Are Major NLP Achievements & Papers From 2019?
- 10 Important Research Papers in Conversational AI From 2019
- 10 Cutting-Edge Research Papers In Computer Vision From 2019
- Top 12 AI Ethics Research Papers Introduced In 2019
- Breakthrough Research In Reinforcement Learning From 2019
Enjoy this article? Sign up for more AI research updates.
We’ll let you know when we release more articles like this one.
Thanks for the article!
I think one of the trends are Solving Financial Fraud Detection with Machine Learning Methods. ML algorithms can process millions of data objects quickly and link instances from seemingly unrelated datasets to detect suspicious patterns. They’re one of the only tools left that can help banks and FinTechs keep up with new defrauding schemes, which are growing increasingly sophisticated.
Thanks for sharing this post