This notebook is used to pretrain transformers models using Hugging Face on your own custom dataset. What do I mean by pretrain transformers? The definition of pretraining is to train in advance. That is exactly what I mean! Train a transformer model to use it as a pretrained transformers model which can be used to fine-tune it on a specific … [Read more...] about Pretrain Transformers Models in PyTorch Using Hugging Face Transformers
Top
Top Applications of Graph Neural Networks 2021
At the beginning of the year, I have a feeling that Graph Neural Nets (GNNs) became a buzzword. As a researcher in this field, I feel a little bit proud (at least not ashamed) to say that I work on this. It was not always the case: three years ago when I was talking to my peers, who got busy working on GANs and Transformers, the general impression that they got on me was that I … [Read more...] about Top Applications of Graph Neural Networks 2021
Leading AI & Machine Learning Research Trends 2021
To help you stay well prepared for 2021, we have summarized the latest trends across different research areas, including natural language processing, conversational AI, computer vision, and reinforcement learning. We also suggest key research papers in different areas that we think are representative of the latest advances. Subscribe to our AI Research mailing list at the … [Read more...] about Leading AI & Machine Learning Research Trends 2021
TOPBOTS Guide to NeurIPS 2020
This year’s Annual Conference on Neural Information Processing (NeurIPS 2020) is held 100% virtually from December 6th to 12th, 2020. Historically NeurIPS sells out quickly and can be a challenge to register for. But, this year’s virtual format welcomes everyone with an affordable ticket price of $100. 2020 is our third year of TOPBOTS covering the conference. To … [Read more...] about TOPBOTS Guide to NeurIPS 2020
2020’s Top AI & Machine Learning Research Papers
Despite the challenges of 2020, the AI research community produced a number of meaningful technical breakthroughs. GPT-3 by OpenAI may be the most famous, but there are definitely many other research papers worth your attention. For example, teams from Google introduced a revolutionary chatbot, Meena, and EfficientDet object detectors in image recognition. Researchers … [Read more...] about 2020’s Top AI & Machine Learning Research Papers
GPT-3 & Beyond: 10 NLP Research Papers You Should Read
NLP research advances in 2020 are still dominated by large pre-trained language models, and specifically transformers. There were many interesting updates introduced this year that have made transformer architecture more efficient and applicable to long documents. Another hot topic relates to the evaluation of NLP models in different applications. We still lack evaluation … [Read more...] about GPT-3 & Beyond: 10 NLP Research Papers You Should Read
Novel Computer Vision Research Papers From 2020
Will transformers revolutionize computer vision like they did with natural language processing? That’s one of the major research questions investigated by computer vision scientists in 2020. The first results indicate that transformers achieve very promising results on image recognition tasks. Beyond transformers in vision applications, we also noticed a continuous … [Read more...] about Novel Computer Vision Research Papers From 2020
Graph Convolutional Networks (GCN)
In this post, we’re gonna take a close look at one of the well-known graph neural networks named Graph Convolutional Network (GCN). First, we’ll get the intuition to see how it works, then we’ll go deeper into the maths behind it. Why Graphs? Many problems are graphs in true nature. In our world, we see many data are graphs, such as molecules, social … [Read more...] about Graph Convolutional Networks (GCN)
Evaluation Metrics For Dialog Systems
Evaluation is a crucial part of the dialog system development process. Human judgment is considered a gold standard for the evaluation of dialog agents. However, this is also a very expensive and time-intensive approach. Thus, researchers mostly rely on automatic metrics when developing dialog systems. In this article, we’ll introduce a brief overview of the key evaluation … [Read more...] about Evaluation Metrics For Dialog Systems
Key Dialog Datasets: Overview and Critique
To build a state-of-the-art dialog system, you need challenging tasks for model training and evaluation. There are numerous dialog datasets that assist researchers in building task-oriented and chit-chat dialog agents. In particular, the Facebook Research team has introduced a framework, called ParlAI (pronounced par-lay), where they’ve gathered together 80+ popular dialog … [Read more...] about Key Dialog Datasets: Overview and Critique