Papers by Gabriel Ilharco
High Performance Natural Language Processing (2020.emnlp-tutorials)
Copied to clipboard
| Challenge: | a tutorial on scaling natural language processing will recapitulate the state-of-the-art in the field . |
| Approach: | This cutting-edge tutorial recapitulates the state-of-the-art in natural language processing with scale in perspective. |
| Outcome: | This cutting-edge tutorial recapitulates the state-of-the-art in natural language processing with scale in perspective. |
Probing Contextual Language Models for Common Ground with Visual Representations (2021.naacl-main)
Copied to clipboard
| Challenge: | Contextual language models have attracted great interest in probing what is encoded in their representations. |
| Approach: | They propose a probing model that evaluates how effective are text-only representations in distinguishing between matching and non-matching visual representations. |
| Outcome: | The proposed model outperforms text-only language models in instance retrieval, but underperform humans. |
Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus (2021.emnlp-main)
Copied to clipboard
Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, Matt Gardner
| Challenge: | Large text corpora are often introduced with minimal documentation . documenting collection process, composition, intended uses, and other are key for structured, task-specific datasets. |
| Approach: | They propose to document a dataset created by applying filters to a single snapshot of Common Crawl. |
| Outcome: | The proposed dataset shows that blocklist filtering removes text from minority individuals and patents. |
Recognizing Multimodal Entailment (2021.acl-tutorials)
Copied to clipboard
Cesar Ilharco, Afsaneh Shirazi, Arjun Gopalan, Arsha Nagrani, Blaz Bratanic, Chris Bregler, Christina Funk, Felipe Ferreira, Gabriel Barcik, Gabriel Ilharco, Georg Osang, Jannis Bulian, Jared Frank, Lucas Smaira, Qin Cao, Ricardo Marino, Roma Patel, Thomas Leung, Vaiva Imbrasaite
| Challenge: | This tutorial introduces the multimodal entailment task for detecting semantic alignments . the task requires fine-grained understanding of visual and linguistic semantics questions . |
| Approach: | This tutorial introduces the multimodal entailment task to machine learning . it introduces a dataset for recognizing multimodal alignments . |
| Outcome: | This tutorial introduces the multimodal entailment task . it can be useful for detecting semantic alignments when a single modality alone is not enough . |
Exploring The Landscape of Distributional Robustness for Question Answering Models (2022.findings-emnlp)
Copied to clipboard
Anas Awadalla, Mitchell Wortsman, Gabriel Ilharco, Sewon Min, Ian Magnusson, Hannaneh Hajishirzi, Ludwig Schmidt
| Challenge: | Existing methods for predicting distributional robustness fail to generalize reliably in a variety of test conditions. |
| Approach: | They conduct a large empirical evaluation to investigate the landscape of distributional robustness in question answering. |
| Outcome: | The proposed methods are more robust to distribution shifts than fully fine-tuned models, and few-shot prompt models exhibit better robustness than few- shot prompt models. |
TaskWeb: Selecting Better Source Tasks for Multi-task NLP (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent work in NLP has shown that knowing task relationships via pairwise task transfer improves choosing one or more source tasks that help to learn a new target task. |
| Approach: | They propose a method to quantify task relationships via pairwise task transfer and build smaller training sets that improve zero-shot performances across 11 different target tasks. |
| Outcome: | The proposed method improves overall rankings and top-k precision of source tasks by 10% and 38%, respectively. |
Finetuning Pretrained Transformers into RNNs (2021.emnlp-main)
Copied to clipboard
Jungo Kasai, Hao Peng, Yizhe Zhang, Dani Yogatama, Gabriel Ilharco, Nikolaos Pappas, Yi Mao, Weizhu Chen, Noah A. Smith
| Challenge: | Efficient transformers outperform recurrent neural networks in natural language generation, but this comes with significant computational cost and memory footprint during generation. |
| Approach: | They propose to convert a pretrained transformer into its efficient recurrent counterpart, improving efficiency while maintaining accuracy. |
| Outcome: | The proposed transformers outperform recurrent neural networks in natural language generation but come with significant computational and memory footprint during generation. |
Evaluating Models’ Local Decision Boundaries via Contrast Sets (2020.findings-emnlp)
Copied to clipboard
Matt Gardner, Yoav Artzi, Victoria Basmov, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hannaneh Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noah A. Smith, Sanjay Subramanian, Reut Tsarfaty, Eric Wallace, Ally Zhang, Ben Zhou
| Challenge: | Standard test sets for supervised learning evaluate in-distribution generalization but are misleading when a dataset has systematic gaps. |
| Approach: | They propose a more rigorous annotation paradigm for NLP that helps to close systematic gaps in the test data. |
| Outcome: | The proposed model performs significantly lower on contrast sets than on the original test sets—up to 25% in some cases. |