John Thickstun

I work on machine learning and generative models. I’m interested in methods that control the behavior of models, both from the perspective of a user who hopes to use a model to accomplish concrete tasks, and from the perspective of a model provider or policymaker who hopes to broadly regulate the outputs of a model. I am also interested in applications of generative models that push beyond the standard text and image modalities, including music technologies.

Previously I was a Postdoctoral Scholar at Stanford University, advised by Percy Liang. I completed my PhD in the Allen School of Computer Science & Engineering at the University of Washington, where I was co-advised by Sham Kakade and Zaid Harchaoui. I studied Applied Mathematics as an undergraduate at Brown University, advised by Eugene Charniak and Björn Sandstede.

Prospective students: I am recruiting students to join my group starting Fall 2025! Please apply to the Cornell CS PhD program. I plan to admit multiple PhD students this year interested in generative models, music technologies, or both.

The MusicNet dataset has moved to permanent hosting at Zenodo.

news

Jul 1, 2024	I am joining Cornell University as an Assistant Professor of Computer Science, starting Fall 2024!
Jun 17, 2024	Hooktheory released Aria: an AI co-creator for chords and melodies powered by the Anticipatory Music Transformer. Read about it on the AudioCipher Blog.
Mar 18, 2024	We released a new, 780M parameter Anticipatory Music Transformer. Additional discussion here.
Dec 7, 2023	Stanford HAI featured my recent work on the Anticipatory Music Transformer!
Oct 11, 2023	Megha and I released human-LM interaction data that we collected last year for HALIE. We wrote a blog post that documents the data release, and highlights some qualitative trends in the data that we found interesting

selected publications

TMLR

Robust distortion-free watermarks for language models

Kuditipudi, Rohith, Thickstun, John, Hashimoto, Tatsunori, and Liang, Percy

Transactions on Machine Learning Research 2024

Abs arXiv Code Website Talk

We propose a methodology for planting watermarks in text from an autoregressive language model that are robust to perturbations without changing the distribution over text up to a certain maximum generation budget. We generate watermarked text by mapping a sequence of random numbers – which we compute using a randomized watermark key – to a sample from the language model. To detect watermarked text, any party who knows the key can align the text to the random number sequence. We instantiate our watermark methodology with two sampling schemes: inverse transform sampling and exponential minimum sampling. We apply these watermarks to three language models – OPT-1.3B, LLaMA-7B and Alpaca-7B – to experimentally validate their statistical power and robustness to various paraphrasing attacks. Notably, for both the OPT-1.3B and LLaMA-7B models, we find we can reliably detect watermarked text (p≤0.01) from 35 tokens even after corrupting between 40-50% of the tokens via random edits (i.e., substitutions, insertions or deletions). For the Alpaca-7B model, we conduct a case study on the feasibility of watermarking responses to typical user instructions. Due to the lower entropy of the responses, detection is more difficult: around 25% of the responses – whose median length is around 100 tokens – are detectable with p≤0.01, and the watermark is also less robust to certain automated paraphrasing attacks we implement.
TMLR

Anticipatory music transformer

Thickstun, John, Hall, David, Donahue, Chris, and Liang, Percy

Transactions on Machine Learning Research 2024

Abs arXiv Code Website Media Talk

We introduce anticipation: a method for constructing a controllable generative model of a temporal point process (the event process) conditioned asynchronously on realizations of a second, correlated process (the control process). We achieve this by interleaving sequences of events and controls, such that controls appear following stopping times in the event sequence. This work is motivated by problems arising in the control of symbolic music generation. We focus on infilling control tasks, whereby the controls are a subset of the events themselves, and conditional generation completes a sequence of events given the fixed control events. We train anticipatory infilling models using the large and diverse Lakh MIDI music dataset. These models match the performance of autoregressive models for prompted music generation, with the additional capability to perform infilling control tasks, including accompaniment. Human evaluators report that an anticipatory model produces accompaniments with similar musicality to even music composed by humans over a 20-second clip.
JMLR

MAUVE Scores for Generative Models: Theory and Practice

Pillutla, Krishna, Liu, Lang, Thickstun, John, Welleck, Sean, Swayamdipta, Swabha, Zellers, Rowan, Oh, Sewoong, Choi, Yejin, and Harchaoui, Zaid

Journal of Machine Learning Research 2023

Abs arXiv Code

Generative AI has matured to a point where large-scale models can generate text that seems indistinguishable from human-written text and remarkably photorealistic images. Automatically measuring how close the distribution of generated data is to the target real data distribution is a key step in diagnosing existing models and developing better models. We present MAUVE, a family of comparison measures between pairs of distributions such as those encountered in the generative modeling of text or images. These scores are statistical summaries of divergence frontiers capturing two types of errors in generative modeling. We explore four approaches to statistically estimate these scores: vector quantization, non-parametric estimation, classifier-based estimation, and parametric Gaussian approximations. We provide statistical bounds for the vector quantization approach. Empirically, we find that the proposed scores paired with a range of f-divergences and statistical estimation methods can quantify the gaps between the distributions of human-written text and those of modern neural language models by correlating with human judgments and identifying known properties of the generated texts. We conclude the paper by demonstrating its applications to other AI domains and discussing practical recommendations.
ACL Outstanding Paper

Backpack language models

Hewitt, John, Thickstun, John, Manning, Christopher D., and Liang, Percy

In Proceedings of the Association for Computational Linguistics 2023

Abs arXiv Code Website

We present Backpacks: a new neural architecture that marries strong modeling performance with an interface for interpretability and control. Backpacks learn multiple non-contextual sense vectors for each word in a vocabulary, and represent a word in a sequence as a context-dependent, non-negative linear combination of sense vectors in this sequence. We find that, after training, sense vectors specialize, each encoding a different aspect of a word. We can interpret a sense vector by inspecting its (non-contextual, linear) projection onto the output space, and intervene on these interpretable hooks to change the model’s behavior in predictable ways. We train a 170M-parameter Backpack language model on OpenWebText, matching the loss of a GPT-2 small (124M parameter) Transformer. On lexical similarity evaluations, we find that Backpack sense vectors outperform even a 6B-parameter Transformer LM’s word embeddings. Finally, we present simple algorithms that intervene on sense vectors to perform controllable text generation and debiasing. For example, we can edit the sense vocabulary to tend more towards a topic, or localize a source of gender bias to a sense vector and globally suppress that sense.
Neurips Oral Presentation

Diffusion-LM improves controllable text generation

Li, Xiang Lisa, Thickstun, John, Gulrajani, Ishaan, Liang, Percy, and Hashimoto, Tatsunori B.

In Advances in Neural Information Processing Systems 2022

Abs arXiv Code Slides

Controlling the behavior of language models (LMs) without re-training is a major open problem in natural language generation. While recent works have demonstrated successes on controlling simple sentence attributes (e.g., sentiment), there has been little progress on complex, fine-grained controls (e.g., syntactic structure). To address this challenge, we develop a new non-autoregressive language model based on continuous diffusions that we call Diffusion-LM. Building upon the recent successes of diffusion models in continuous domains, Diffusion-LM iteratively denoises a sequence of Gaussian vectors into word vectors, yielding a sequence of intermediate latent variables. The continuous, hierarchical nature of these intermediate variables enables a simple gradient-based algorithm to perform complex, controllable generation tasks. We demonstrate successful control of Diffusion-LM for six challenging fine-grained control tasks, significantly outperforming prior work.
Neurips Outstanding Paper

MAUVE: measuring the gap between neural text and human text using divergence frontiers

Pillutla, Krishna, Swayamdipta, Swabha, Zellers, Rowan, Thickstun, John, Welleck, Sean, Choi, Yejin, and Harchaoui, Zaid

In Advances in Neural Information Processing Systems 2021

Abs arXiv Code Slides

As major progress is made in open-ended text generation, measuring how close machine-generated text is to human language remains a critical open problem. We introduce MAUVE, a comparison measure for open-ended text generation, which directly compares the learnt distribution from a text generation model to the distribution of human-written text using divergence frontiers. MAUVE scales up to modern text generation models by computing information divergences in a quantized embedding space. Through an extensive empirical study on three open-ended generation tasks, we find that MAUVE identifies known properties of generated text, scales naturally with model size, and correlates with human judgments, with fewer restrictions than existing distributional evaluation metrics.
ICML

Parallel and flexible sampling from autoregressive models via Langevin dynamics

Jayaram, Vivek, and Thickstun, John

In International Conference on Machine Learning 2021

Abs arXiv Code Poster Website

This paper introduces an alternative approach to sampling from autoregressive models. Autoregressive models are typically sampled sequentially, according to the transition dynamics defined by the model. Instead, we propose a sampling procedure that initializes a sequence with white noise and follows a Markov chain defined by Langevin dynamics on the global log-likelihood of the sequence. This approach parallelizes the sampling process and generalizes to conditional sampling. Using an autoregressive model as a Bayesian prior, we can steer the output of a generative model using a conditional likelihood or constraints. We apply these techniques to autoregressive models in the visual and audio domains, with competitive results for audio source separation, super-resolution, and inpainting.
ICML

Source separation with deep generative priors

Jayaram, Vivek, and Thickstun, John

In International Conference on Machine Learning 2020

Abs arXiv Code Poster

Despite substantial progress in signal source separation, results for richly structured data continue to contain perceptible artifacts. In contrast, recent deep generative models can produce authentic samples in a variety of domains that are indistinguishable from samples of the data distribution. This paper introduces a Bayesian approach to source separation that uses generative models as priors over the components of a mixture of sources, and noise-annealed Langevin dynamics to sample from the posterior distribution of sources given a mixture. This decouples the source separation problem from generative modeling, enabling us to directly use cutting-edge generative models as priors. The method achieves state-of-the-art performance for MNIST digit separation. We introduce new methodology for evaluating separation quality on richer datasets, providing quantitative evaluation of separation results on CIFAR-10. We also provide qualitative results on LSUN.
ICASSP Oral Presentation

Invariances and data augmentation for supervised music transcription

Thickstun, John, Harchaoui, Zaid, Foster, Dean P, and Kakade, Sham M

In International Conference on Acoustics, Speech and Signal Processing 2018

Abs arXiv Code

This paper explores a variety of models for frame-based music transcription, with an emphasis on the methods needed to reach state-of-the-art on human recordings. The translation-invariant network discussed in this paper, which combines a traditional filterbank with a convolutional neural network, was the top-performing model in the 2017 MIREX Multiple Fundamental Frequency Estimation evaluation. This class of models shares parameters in the log-frequency domain, which exploits the frequency invariance of music to reduce the number of model parameters and avoid overfitting to the training data. All models in this paper were trained with supervision by labeled data from the MusicNet dataset, augmented by random label-preserving pitch-shift transformations.
ICLR

Learning features of music from scratch

Thickstun, John, Harchaoui, Zaid, and Kakade, Sham M

In International Conference on Learning Representations 2017

Abs arXiv Code Poster Website

This paper introduces a new large-scale music dataset, MusicNet, to serve as a source of supervision and evaluation of machine learning methods for music research. MusicNet consists of hundreds of freely-licensed classical music recordings by 10 composers, written for 11 instruments, together with instrument/note annotations resulting in over 1 million temporal labels on 34 hours of chamber music performances under various studio and microphone conditions. The paper defines a multi-label classification task to predict notes in musical recordings, along with an evaluation protocol, and benchmarks several machine learning architectures for this task: i) learning from spectrogram features; ii) end to-end learning with a neural net; iii) end-to-end learning with a convolutional neural net. These experiments show that end-to-end models trained for note prediction learn frequency selective filters as a low-level representation of audio.