Welcome!

I am a research scientist at Salesforce Research working on Natural Language Processing and Explainable AI. My research focuses on making AI models more interpretable through quantitative methods such as causal mediation analysis and visualization tools including ProVis and BertViz. I am also interested in applying interpretability to scientific discovery, as in my research connecting BERTology and Biology. Recently, I have been exploring strategies for evaluating and strengthening NLP models through my work on Robustness Gym. I am also an avid blogger, and enjoy developing user-facing applications such as 100 Years Ago. Prior to my current role, I worked as a research scientist at Palo Alto Research Center. I received my PhD in Computer Science at the University of Minnesota under the supervision of John Riedl.

Recent Publications

BERTology Meets Biology: Interpreting Attention in Protein Language Models
Jesse Vig, Ali Madani, Lav R. Varshney, Caiming Xiong, Richard Socher, Nazneen Fatema Rajani
ICLR 2021
[Paper] [Blog] [Demo Video]
Investigating Gender Bias in Language Models Using Causal Mediation Analysis
Jesse Vig*, Sebastian Gehrmann*, Yonatan Belinkov*, Sharon Qian, Daniel Nevo, Yaron Singer, Stuart Shieber (*equal contribution)
NeurIPS 2020, Spotlight Presentation
[Paper] [Video] [Poster]
Robustness Gym: Unifying the NLP Evaluation Landscape
Karan Goel*, Nazneen Rajani*, Jesse Vig, Samson Tan, Jason Wu, Stephan Zheng, Caiming Xiong, Mohit Bansal , Christopher Ré (*equal contribution)
[arXiv]
A Multiscale Visualization of Attention in the Transformer Model
Jesse Vig
ACL System Demonstrations 2019
[Paper] [Blog] [Poster]
Analyzing the Structure of Attention in a Transformer Language Model
Jesse Vig, Yonatan Belinkov
ACL BlackboxNLP Workshop 2019
[Paper] [Poster]

Blog Posts

(Re)Discovering Protein Structure and Function Through Language Modeling
Einstein.ai blog, 2020
Deconstructing BERT Part 2: Visualizing the Inner Workings of Attention
IEEE Vis Workshop on Visualization for AI Explainability, 2019
GPT-2: Understanding Language Generation through Visualization
Towards Data Science, 2019
Deconstructing BERT: Distilling 6 Patterns from 100 Million Parameters
Towards Data Science, 2018
A.I. Plays Mad Libs and the Results are Terrifying
Towards Data Science, 2018

Other Projects

100 Years Ago
Winner of Actions on Google Developer Challenge 2017