Papers by Adam Trischler
The KnowRef Coreference Corpus: Removing Gender and Number Cues for Difficult Pronominal Anaphora Resolution (P19-1)
Copied to clipboard
| Challenge: | Existing methods for coreference resolution exploit the number and gender of antecedents or have been handcrafted and do not reflect the diversity of naturally occurring text. |
| Approach: | They propose a trick to improve resolution by antecedent switching to target common-sense understanding and world knowledge. |
| Outcome: | The proposed method achieves state-of-the-art results on the GAP coreference task. |
Deconstructing NLG Evaluation: Evaluation Practices, Assumptions, and Their Implications (2022.naacl-main)
Copied to clipboard
| Challenge: | Evaluating natural language generation systems is difficult, as there are many ways to express similar things in text. |
| Approach: | They combine interviews with NLG practitioners to examine ethical considerations and their implications for NLG evaluation. |
| Outcome: | The findings of the study surface goals, community practices, assumptions, and constraints that shape NLG evaluations, and examine their implications and how they embody ethical considerations. |
A Knowledge Hunting Framework for Common Sense Reasoning (D18-1)
Copied to clipboard
| Challenge: | a new system that uses common sense to solve a common sense problem is developed . a winograd schema challenge and a choice of plausible alternatives are popular tests . |
| Approach: | They propose an automatic system that achieves state-of-the-art results on the Winograd Schema Challenge . they use a knowledge hunting module to gather web text for problem resolutions . |
| Outcome: | The proposed system achieves state-of-the-art on the Winograd Schema Challenge . it improves F1 performance on the full WSC by 0.21 over the previous best . |
The KITMUS Test: Evaluating Knowledge Integration from Multiple Sources (2023.acl-long)
Copied to clipboard
Akshatha Arodi, Martin Pömsl, Kaheer Suleman, Adam Trischler, Alexandra Olteanu, Jackie Chi Kit Cheung
| Challenge: | Existing models that make inferences using information from multiple sources are largely understudied . |
| Approach: | They propose a test suite of coreference resolution subtasks that require reasoning over multiple facts and introduce subtask where knowledge is present only at inference time using fictional knowledge. |
| Outcome: | The proposed subtasks differ in terms of which knowledge sources contain the relevant facts and where knowledge is present only at inference time using fictional knowledge. |
Interactive Language Learning by Question Answering (D19-1)
Copied to clipboard
| Challenge: | Existing machine reading comprehension tasks lack interactive information-seeking component of comprehension. |
| Approach: | They propose a question-asking task that asks questions in a text-based environment . they propose QAit, which uses a game generator to build models that include deep reinforcement learning agents. |
| Outcome: | The proposed task poses questions about existence, location, and attributes of objects found in environment. |
An Empirical Study on Neural Keyphrase Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent years have seen a flourishing of neural keyphrase generation (KPG) works, including the release of several large-scale datasets and a host of new models to tackle them. |
| Approach: | They propose to compare the generalizability of KPG models with other models by analyzing the most crucial factors that may affect their generalizarability. |
| Outcome: | The proposed model can be used to predict keyphrases from a set of input sequences, and it can be compared with existing models. |
Interactive Machine Comprehension with Information Seeking Agents (2020.acl-main)
Copied to clipboard
| Challenge: | Existing machine reading comprehension (MRC) models do not scale effectively to real-world applications like web-level information retrieval and question answering (QA). |
| Approach: | They propose a method that reframes existing machine reading comprehension (MRC) datasets as interactive, partially observable environments. |
| Outcome: | The proposed method "occludes" the majority of a document’s text and adds context-sensitive commands that reveal "glimpses" of the hidden text to a model. |
A Generalized Knowledge Hunting Framework for the Winograd Schema Challenge (N18-4)
Copied to clipboard
| Challenge: | a new system that performs well on common-sense reasoning tasks is developed . the Winograd Schema Challenge (WSC) is a popular alternative to the Turing test . |
| Approach: | They propose an automatic system that performs well on two common-sense reasoning tasks. |
| Outcome: | The proposed system improves performance on the Winograd Schema Challenge and COPA by 0.16 over the previous best. |
How Reasonable are Common-Sense Reasoning Tasks: A Case-Study on the Winograd Schema Challenge and SWAG (D19-1)
Copied to clipboard
| Challenge: | a recent study has improved the state-of-the-art on common-sense reasoning benchmarks . a san francisco-based approach to common-ense reasoning is challenging . |
| Approach: | They propose to use common-sense reasoning benchmarks to test machine learning's common-sentence inference task SWAG to test common-mind systems. |
| Outcome: | a new study shows that improved performance on common-sense reasoning benchmarks is genuine . the proposed task is more difficult than the current one, but it is more efficient than the previous one. |
Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for integrating knowledge graphs into pre-trained language models have been poorly implemented. |
| Approach: | They propose a self-supervised entity masking scheme that exploits relational knowledge underlying the text. |
| Outcome: | The proposed model achieves improved performance on five benchmarks, including question answering and knowledge base completion. |
On the Systematicity of Probing Contextualized Word Representations: The Case of Hypernymy in BERT (2020.starsem-1)
Copied to clipboard
| Challenge: | Existing studies have found that BERT can correctly retrieve noun hypernyms in cloze tasks, but this does not correspond to systematic knowledge in BERT. |
| Approach: | They propose to use BERT to probe for hypernymy knowledge encoded in representations for cloze tasks to find out whether it is systematic or not . |
| Outcome: | The proposed model can retrieve hypernyms in cloze tasks, but not systematic knowledge in BERT. |
Modeling Event Plausibility with Consistent Conceptual Abstraction (2021.naacl-main)
Copied to clipboard
| Challenge: | Understanding natural language requires common sense, one aspect of which is the ability to discern the plausibility of events. |
| Approach: | They propose a method of forcing model consistency that improves correlation with human plausibility judgements. |
| Outcome: | The proposed method improves correlation with human plausibility judgements. |
An Analysis of Dataset Overlap on Winograd-Style Tasks (2020.coling-main)
Copied to clipboard
| Challenge: | a large number of test instances overlap considerably with pretraining corpora, a study finds . for a number of years, models struggled to exceed chance-level performance . |
| Approach: | They analyze the effects of varying degrees of overlaps that occur between pretraining corpora and test instances in WSC-style tasks. |
| Outcome: | The WSC-Web dataset is the largest to date and has lower overlaps with current pretraining corpora. |
Responsible AI Considerations in Text Summarization Research: A Review of Current Practices (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study examines research and reporting practices for text summarization tasks . text summaries are often overlooked by the responsible AI community . |
| Approach: | They examine research and reporting practices in the context of text summarization . they find that relatively few papers engage with possible stakeholders . |
| Outcome: | The findings highlight current research practices and provide recommendations on research directions. |
Challenges to Evaluating the Generalization of Coreference Resolution Models: A Measurement Modeling Perspective (2024.findings-acl)
Copied to clipboard
| Challenge: | a recent study shows that evaluations of CR models on multiple datasets conflate different factors concerning what is being measured. |
| Approach: | They propose to view evaluations through the lens of measurement modeling . they show that evaluations risk conflating different factors concerning what is being measured . |
| Outcome: | The evaluations on seven datasets show that models that reflect coreference generalization are often correlated with differences in how coreference is defined and operationalized. |
ADEPT: An Adjective-Dependent Plausibility Task (2021.acl-long)
Copied to clipboard
| Challenge: | ADEPT is a large-scale semantic plausibility task that requires a significant degree of world knowledge and common-sense reasoning. |
| Approach: | They propose a large-scale semantic plausibility task that pairs 16 thousand sentences with slightly modified versions obtained by adding an adjective to a noun. |
| Outcome: | The proposed task is easier for humans (85% accuracy), but more difficult for transformer-based models (71% accuracy). |
One Size Does Not Fit All: Generating and Evaluating Variable Number of Keyphrases (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models for keyphrase generation do not provide a desideratum for the number of keyphrases in texts. |
| Approach: | They propose a recurrent generative model that generates multiple keyphrases as delimiter-separated sequences. |
| Outcome: | The proposed model outperforms baseline models on all datasets. |
Exploring and Predicting Transferability across NLP Tasks (2020.emnlp-main)
Copied to clipboard
Tu Vu, Tong Wang, Tsendsuren Munkhdalai, Alessandro Sordoni, Adam Trischler, Andrew Mattarella-Micke, Subhransu Maji, Mohit Iyyer
| Challenge: | Recent advances in NLP demonstrate the effectiveness of training large-scale language models and transferring them to downstream tasks. |
| Approach: | They conduct an extensive study of the transferability between 33 NLP tasks across three broad classes of problems. |
| Outcome: | The proposed model can improve performance even with low-data source tasks that differ substantially from the target task. |