NLP4Vis: Natural Language Processing for Information Visualization

Half-day tutorial at IEEE Vis Conference 2023.

Overview

This tutorial will provide an introduction to natural language processing (NLP) to interested researchers in the visualization (Vis) community. It will first motivate why NLP4Vis is an important area of research and provide an overview of research topics on combining NLP and Vis techniques. Then an overview of deep learning models for NLP will be covered. A particular focus will be provided on highlighting the recent progress on large language models such as ChatGPT and how such models can be leveraged to solve various NLP tasks for visualizations. In the final part, we will focus on various application tasks at the intersection of NLP and Vis. We will conclude with an interactive discussion of future challenges for NLP+Vis applications. The audience will include researchers interested in applying NLP for visualizations as well as others who focus more generally at the intersection of AI and visualization.

Materials

Tutorial Overview

Introduction [15 mins]

  • What is NLP?
  • What is Vis?
  • Why NLP + Vis?
  • An overview of research topics on combining NLP and Vis techniques
  • An overview of the tutorial

Coffee Break  

Deep Learning for NLP [60 mins]

  • Language modeling
  • Model Architectures
    • Transformer Architecture
    • Encoder, decoder, encoder-decoder
    • Pre-training and Fine-tuning
  • Large language models (LLMs)
    • Scaling LMs to LLMs
    • Prompt Engineering
    • In context Learning
    • Instruction Tuning

NLP + Vis Applications [50 mins]

  • Visual text analytics
  • Natural language interfaces for visualizations
  • ChartNLP (e.g., Chart question answering, Text2Chart)
  • Natural language generation for visualization (e.g., Chart-to-text)
  • Automated data-driven storytelling
  • NLP for chart accessibility and inclusions
  • Live demos

Future Challenges [25 mins]

  • Building benchmarks for training and evaluation
  • Data annotation challenges
  • Addressing concerns of NLP models (e.g., bias, factual errors, hallucinations, explainability)
  • Emerging applications

Slides

Part 1: Introduction

Part 2: Deep Learning for NLP

Part 3: NLP + Vis Applications, Future Challenges

Organizer

Enamul Hoque
Enamul Hoque

Associate Professor, York University