3 NLP Interpretability Tools For Debugging Language Models

With constant advances and unprecedented performance on many NLP tasks, language models have gotten really complex and hard to debug. Researchers and engineers often can’t easily answer questions like: 

Why did your model make that prediction?  

Does your model have any algorithmic biases?

What kind of data samples does your model perform poorly on?

NLP interpretability tools help researchers and practitioners make more effective fine-tuning decisions on language models while saving time and resources. With the right toolkit, the researchers can spend less time on experiments with different techniques and input data and end up with a better understanding of model behavior, strengths, and limitations.

In this article, we feature three interpretability tools that can assist NLP researchers and engineers in debugging their models.

Language Interpretability Tool (LIT) 

The Google Research team, headed by Ian Tenney and James Wexler, recently introduced a new tool for visualization and analyses of NLP models. Their Language Interpretability Tool (LIT) has a browser-based user interface, making it accessible to practitioners of different skill levels. 

With its beginner-friendly interface that requires only a few lines of code to add models and data, LIT provides users with the ability to:

• Explore the dataset interactively using different criteria and projections.
• Find and explore interesting data points, like for example, off-diagonal groups from a confusion matrix, outliers, etc.
• Explain local behavior by deep-diving into model behavior on individual data points.
• Generate new data points on the fly either manually via edits or automatically using a range of counterfactual generators (e.g., back translations, adversarial attacks).
• Compare side-by-side two or more models on the same data or the same model on two data points.
• Compute metrics for the whole dataset, current selection, or some specific slices.

The tool is in active development and will likely extend its capabilities after getting some feedback from the NLP research community.

Research paper. Google’s research paper The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models will be presented at EMNLP 2020.

Code. Implementation code and full documentation are available on GitHub.

AllenNLP Interpret

The researchers from the Allen Institute for Artificial Intelligence and the University of California have introduced AllenNLP Interpret, a framework for interpreting NLP models. The tool focuses on instance-level interpretations of two types: gradient-based saliency maps and adversarial attacks. Besides a suite of interpretation techniques implemented for broad classes of models, the AllenNLP Interpret includes model- and task-agnostic APIs for developing new interpretation methods and reusable front-end components to visualize the interpretations interactively.

The toolkit has numerous use cases including uncovering model biases, finding decision rules, diagnosing errors.

The available demos show how AllenNLP Interpret can be leveraged for explaining BERT’s mask predictions and interpreting the performance of different ELMO-based models on textual entailment, sentiment analysis, reading comprehension, and named entity recognition.

Research paper. The researchers from the Allen Institute for Artificial Intelligence presented their research paper AllenNLP Interpret: A Framework for Explaining Predictions of NLP Models at EMNLP 2019.

Code. The authors released the implementation code for AllenNLP Interpret on GitHub.

Demo. The toolkit’s flexibility and utility are demonstrated with live demos for five interpretation methods on a variety of models and tasks.

TensorFlow Model Analysis

TensorFlow Model Analysis (TFMA) is an open-source library that assists with computing and visualizing evaluation metrics. In contrast to the aforementioned interpretation tools, TFMA doesn’t allow for instance-level interpretations and provides only aggregate and sliced metrics. However, it’s not limited to NLP models and can be leveraged for evaluating any TensorFlow models.

The use cases for this tool include:

• evaluating model quality against thresholds, 
• analyzing model behavior on different slices of data, 
• identifying where a model fails.

The metrics can be also visualized in Jupyter notebook for a more convenient evaluation of the results.

Code. TFMA library is available on GitHub.

NLP Models Debugged

Recent Neural NLP models like GPT-3 demonstrate impressive performance at many different NLP tasks. However, they can still make dumb mistakes and break in counterintuitive ways, leaving not only end-users but even their developers puzzled at the poor performance.

Model interpretability tools enable practitioners to better understand the strengths and weaknesses of language models, their propensity to making socially biased predictions, and their applicability to specific business problems.

Enjoy this article? Sign up for more AI research updates.

We’ll let you know when we release more summary articles like this one.