Finspector: A Human-Centered Visual Inspection Tool for Exploring and Comparing Biases among Foundation Models

Abstract

Transformer-based pre-trained large language models are getting increasingly popular and widely used in both academic and industrial settings because of their outstanding performance in many academic benchmarks. Nevertheless, there are still concerns about the hidden biases in these models which can have adverse effects on certain groups of people such as discriminatory outcomes or reinforcing harmful stereotypes. One promising way to inspect and uncover such biases is through visual inspection with human-in-the-loop. In this paper, we present Finspector, a human-centered visual inspection tool for exploring and comparing biases among foundation models. The goal of the tool is to make it easier to identify potential bias in different bias categories easily through a set of intuitive visual analytics using log-likelihood scores generated by language models.