Papers by Noah Goodman
Investigating Transferability in Pretrained Language Models (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work on deep NLP models has centered on probing, a method that involves training classifiers for different tasks on model representations. |
| Approach: | They propose a method for determining the impact of each pretrained layer on transfer task performance by ablation. |
| Outcome: | The proposed method shows that pretraining models improve performance on downstream tasks . the results highlight the limitations of methods that operate on frozen models or single data samples. |
pyvene: A Library for Understanding and Improving PyTorch Models via Interventions (2024.naacl-demo)
Copied to clipboard
Zhengxuan Wu, Atticus Geiger, Aryaman Arora, Jing Huang, Zheng Wang, Noah Goodman, Christopher Manning, Christopher Potts
| Challenge: | Existing libraries are often project-based, but pyvene provides a unified and extensible framework for performing interventions on neural models and sharing the intervened upon models with others. |
| Approach: | They propose an open-source Python library that supports customizable interventions on a range of different PyTorch modules. |
| Outcome: | The proposed framework provides a unified and extensible framework for performing interventions on neural models and sharing the intervened upon models with others. |
Question Generation for Adaptive Education (2021.acl-short)
Copied to clipboard
| Challenge: | Existing systems depend on a pool of hand-made questions, limiting how fine-grained and open-ended they can be in adapting to individual students. |
| Approach: | They propose to fine-tune pre-trained language models for deep knowledge tracing to generate reversetranslation questions conditioned on the student and target difficulty. |
| Outcome: | The proposed model can generate well-calibrated language translation questions for second language learners from a real online education platform. |
DisSent: Learning Sentence Representations from Explicit Discourse Relations (P19-1)
Copied to clipboard
| Challenge: | Existing models train on vast amounts of text or require costly, manually curated datasets. |
| Approach: | They propose to leverage the discourse relations between sentences to curate a high quality sentence relation task by leveraging explicit discourse relations. |
| Outcome: | The proposed model can be used to learn the meaning of two sentences in a bidirectional LSTM sentence encoder. |
Open-domain clarification question generation without question examples (2021.emnlp-main)
Copied to clipboard
| Challenge: | Currently, natural language inputs are unclear or ambiguous, causing uncertainty in dialogues. |
| Approach: | They propose a framework for building a visually grounded question-asking model capable of producing polar (yes-no) clarification questions to resolve misunderstandings in dialogue. |
| Outcome: | The proposed model can produce polar (yes-no) clarification questions to resolve misunderstandings in a goal-oriented 20 questions game with synthetic and human answerers. |
Backtracing: Retrieving the Cause of the Query (2024.findings-eacl)
Copied to clipboard
| Challenge: | a number of online content portals allow users to ask questions to supplement their understanding. |
| Approach: | They propose a task of backtracing to retrieve the text segment that most likely caused a user query. |
| Outcome: | The proposed method improves on the backtracing task in three domains . the results show that there is room for improvement and new retrieval approaches . |
Is Child-Directed Speech Effective Training Data for Language Models? (2024.emnlp-main)
Copied to clipboard
| Challenge: | High-performing language models are typically trained on hundreds of billions of words, but human learners use language fluently after far less training data. |
| Approach: | They train GPT-2 and RoBERTa models on 29M words of English child-directed speech and a new matched, synthetic dataset. |
| Outcome: | The proposed models show that child language input is not valuable for training language models. |
Mixed-effects transformers for hierarchical adaptation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Language differs dramatically from context to context, but prompting can be ineffective when contexts are sparse, out-of-sample, or extra-textual. |
| Approach: | They propose a mixed-effects transformer approach for learning hierarchically-structured prefixes to account for structured variation in language use. |
| Outcome: | The proposed approach can be extended to transformer-based architectures while generalizing well to unseen contexts. |
Lost in Machine Translation: A Method to Reduce Meaning Loss (N19-1)
Copied to clipboard
| Challenge: | state-of-the-art translation systems often fail in preserving meaning . ambiguity between source and target languages can cause translation problems . |
| Approach: | They propose to use a pre-trained neural sequence-to-sequence model to define a less ambiguous translation system. |
| Outcome: | The proposed system preserves meaning in two languages without compromising translation quality. |
Value Profiles for Encoding Human Variation (2025.emnlp-main)
Copied to clipboard
Taylor Sorensen, Pushkar Mishra, Roma Patel, Michael Henry Tessler, Michiel A. Bakker, Georgina Evans, Iason Gabriel, Noah Goodman, Verena Rieser
| Challenge: | Using value profiles and a steerable decoder model to estimate ratings is crucial for personalization, pluralistic model alignment, and computational social science. |
| Approach: | They propose to represent individuals using value profiles and a steerable decoder model to estimate ratings conditioned on a value profile or other rater information. |
| Outcome: | The proposed model interpretably changes ratings according to semantic profile differences and is well-calibrated. |
Causal Distillation for Language Models (2022.naacl-main)
Copied to clipboard
Zhengxuan Wu, Atticus Geiger, Joshua Rozner, Elisa Kreiss, Hanson Lu, Thomas Icard, Christopher Potts, Noah Goodman
| Challenge: | Distillation efforts have led to language models that are more compact and efficient without serious drops in performance. |
| Approach: | They propose to augment distillation with a third objective that encourages the student model to imitate the causal dynamics of the teacher through a distillation interchange intervention training objective (DIITO). |
| Outcome: | The proposed method lowers perplexity on the WikiText-103M corpus and improves on the GLUE benchmark, SQuAD, and CoNLL-2003. |
Calibrate your listeners! Robust communication-based training for pragmatic speakers (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work has investigated training NLP systems with communication-based objectives . prior work has focused on supervised learning, but is expensive to collect . |
| Approach: | They propose a method that uses a population of neural listeners to regularize speaker training. |
| Outcome: | The proposed method improves on ensemble- and dropout-based listening populations on reference games and generalizes to new games and listeners. |
Learning from Omission (P19-1)
Copied to clipboard
| Challenge: | a recent study shows that pragmatic reasoning improves language understanding systems . end-to-end training produces more accurate utterance interpretation models . |
| Approach: | They show that pragmatic reasoning can improve the quality of learned meanings . they draw pragmatic inferences from listening to what a speaker says and not to what they do . |
| Outcome: | The proposed model improves the quality of language understanding models when sparse data is used. |
Pragmatically Informative Image Captioning with Character-Level Inference (N18-2)
Copied to clipboard
| Challenge: | a neural image captioner and a Rational Speech Acts (RSA) model are pragmatically informative . previous attempts to combine RSA with neural image-captioning require an inference which normalizes over the entire set of possible utterances. |
| Approach: | They propose a neural image captioner with a Rational Speech Acts model to make it pragmatically informative. |
| Outcome: | The proposed system outperforms a non-pragmatic baseline and word-level RSA captioner on a word-based model. |
Shaping Visual Representations with Language for Few-Shot Classification (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models use natural language descriptions to classify images, but no model uses it for new tasks. |
| Approach: | They propose a model that regularizes visual representations to predict language in a few-shot setting . they propose to use language to improve few- shot visual classification . |
| Outcome: | The proposed model outperforms baseline models in two challenging few-shot domains. |
Concadia: Towards Image-Based Text Generation with a Purpose (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing models fail to generate fluent, truthful text, despite excellent results on benchmark datasets . current systems fail to produce texts that are useful in practice, authors argue . |
| Approach: | They propose to distinguish descriptions from captions based on their communicative roles . descriptions focus on visual features and are meant to replace an image . authors characterize commonalities and differences between descriptions and captions in a Wikipedia corpus . |
| Outcome: | The proposed model can generate fluent, truthful texts in a wide range of scenarios . it can also generate captions that are used to make an image accessible to users who can't see them . |