Papers by Christopher Clark
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions (N19-1)
Copied to clipboard
| Challenge: | In this paper we build a reading comprehension dataset of yes/no questions that are naturally occurring . they often query for complex, non-factoid information, and require difficult entailment-like inference to solve. |
| Approach: | They build a reading comprehension dataset of yes/no questions that are naturally occurring . they find they are unexpectedly challenging and require difficult inferences to solve . |
| Outcome: | The proposed method achieves 80.4% accuracy compared to 90% accuracy of human annotators and 62% majority-baseline. |
Deep Contextualized Word Representations (N18-1)
Copied to clipboard
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, Luke Zettlemoyer
| Challenge: | a new type of deep contextualized word representation is proposed for language understanding problems . word vectors are learned functions of the internal states of a deep bidirectional language model . |
| Approach: | They propose a new type of deep contextualized word representation that models complex features of word use and how they vary across linguistic contexts. |
| Outcome: | The proposed representations improve the state of the art across six challenging NLP problems. |
Learning to Model and Ignore Dataset Bias with Mixed Capacity Ensembles (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work has shown that datasets contain incidental correlations created by idiosyncrasies in the data collection process. |
| Approach: | They propose a method that detects and ignores dataset-specific correlations by introducing a new method that makes them conditionally independent. |
| Outcome: | The proposed method detects and ignores these kinds of dataset-specific correlations, and does not require the bias to be known in advance. |
Iconary: A Pictionary-Based Game for Testing Multimodal Communication with Drawings and Text (2021.emnlp-main)
Copied to clipboard
Christopher Clark, Jordi Salvador, Dustin Schwenk, Derrick Bonafilia, Mark Yatskar, Eric Kolve, Alvaro Herrasti, Jonghyun Choi, Sachin Mehta, Sam Skjonsberg, Carissa Schoenick, Aaron Sarnat, Hannaneh Hajishirzi, Aniruddha Kembhavi, Oren Etzioni, Ali Farhadi
| Challenge: | Communicating with humans is challenging for AIs because of its complexity and multimodality. |
| Approach: | They propose to use a game of drawing and guessing based on Pictionary to test AIs' understanding of the world and multi-modal gestures. |
| Outcome: | The proposed game is a test for mixing language and visual/symbolic communication in AI. |
Don’t Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases (D19-1)
Copied to clipboard
| Challenge: | Recent advances in neural models exploit dataset-specific patterns that do not generalize well to out-of-domain or adversarial settings. |
| Approach: | They propose to train a model to be more robust to domain shift if it has prior knowledge of dataset biases. |
| Outcome: | The proposed model can be more robust to domain shift if it has prior knowledge of dataset biases. |
Simple and Effective Multi-Paragraph Reading Comprehension (P18-1)
Copied to clipboard
| Challenge: | Existing question answering models cannot scale beyond short paragraphs, so adapting a model to document-level input is difficult. |
| Approach: | They propose a method of adapting neural paragraph-level question answering models to document input. |
| Outcome: | The proposed method achieves state-of-the-art on TriviaQA and SQuAD and a 10 point gain on SQuADA. |
Discovering Language Model Behaviors with Model-Written Evaluations (2023.findings-acl)
Copied to clipboard
Ethan Perez, Sam Ringer, Kamile Lukosiute, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, Andy Jones, Anna Chen, Benjamin Mann, Brian Israel, Bryan Seethor, Cameron McKinnon, Christopher Olah, Da Yan, Daniela Amodei, Dario Amodei, Dawn Drain, Dustin Li, Eli Tran-Johnson, Guro Khundadze, Jackson Kernion, James Landis, Jamie Kerr, Jared Mueller, Jeeyoon Hyun, Joshua Landau, Kamal Ndousse, Landon Goldberg, Liane Lovitt, Martin Lucas, Michael Sellitto, Miranda Zhang, Neerav Kingsland, Nelson Elhage, Nicholas Joseph, Noemi Mercado, Nova DasSarma, Oliver Rausch, Robin Larson, Sam McCandlish, Scott Johnston, Shauna Kravec, Sheer El Showk, Tamera Lanham, Timothy Telleen-Lawton, Tom Brown, Tom Henighan, Tristan Hume, Yuntao Bai, Zac Hatfield-Dodds, Jack Clark, Samuel R. Bowman, Amanda Askell, Roger Grosse, Danny Hernandez, Deep Ganguli, Evan Hubinger, Nicholas Schiefer, Jared Kaplan
| Challenge: | Prior work creates evaluations with crowdwork or existing data sources, which are not always available. |
| Approach: | They generate evaluations automatically with language models (LMs) using crowdwork or existing data sources to find out how they behave . |
| Outcome: | The results show that large LMs repeat back a dialog user’s preferred answer and express greater desire to pursue concerning goals like resource acquisition and goal preservation. |
Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generation (2025.acl-long)
Copied to clipboard
Yue Yang, Ajay Patel, Matt Deitke, Tanmay Gupta, Luca Weihs, Andrew Head, Mark Yatskar, Chris Callison-Burch, Ranjay Krishna, Aniruddha Kembhavi, Christopher Clark
| Challenge: | Vision-language models struggle to understand text-rich images due to the scarcity of diverse text-only large language data. |
| Approach: | They propose a framework that leverages the coding capabilities of text-only large language models to create synthetic text-rich multimodal data. |
| Outcome: | The proposed framework can generate high-quality instruction-tuning data using Python, HTML, LaTeX and other languages. |