Papers by Brendan O’Connor
Corpus-Level Evaluation for Event QA: The IndiaPoliceEvents Corpus Covering the 2002 Gujarat Violence (2021.findings-acl)
Copied to clipboard
| Challenge: | a new corpus-level evaluation approach for event extraction is needed in social science applications . human annotations are often required to extract the actions of political actors and actors . a novel corpus evaluation approach can guide creation of similar social science-oriented resources . |
| Approach: | They propose a corpus-based approach to event extraction that integrates corpus evaluation with real-world social science . they use human annotations to read and label every document for mentions of police activity events . |
| Outcome: | The proposed method can guide creation of similar social-science-oriented resources in the future. |
Relational Summarization for Corpus Analysis (N18-1)
Copied to clipboard
| Challenge: | Existing methods for summarizing textual content are often ignored . relationshipal questions are ubiquitous and varied. |
| Approach: | They propose a method which generates a natural language summary of the relationship between two lexical items in a corpus without reference to a knowledge base. |
| Outcome: | The proposed method generates a natural language summary of the relationship between two lexical items in a corpus without reference to a knowledge base. |
Query-focused Sentence Compression in Linear Time (D19-1)
Copied to clipboard
| Challenge: | Existing techniques for constrained compression are slow and require third-party solvers. |
| Approach: | They propose a query-focused sentence compression technique which constructs length and lexically constrained compressions in linear time by growing a subgraph in the dependency parse of a sentence. |
| Outcome: | The proposed technique achieves an 11x empirical speedup over baseline methods while improving query-focused applications. |
Text and Causal Inference: A Review of Using Text to Remove Confounding from Causal Estimates (2020.acl-main)
Copied to clipboard
| Challenge: | Unmeasured or latent confounders can bias causal estimates and this has motivated interest in measuring potential confounder from observed text. |
| Approach: | They propose to use text to measure potential confounders in a way that allows for a rich measurement of multiple confounder variables. |
| Outcome: | The proposed method is based on an individual’s entire history of social media posts or the content of a news article. |
ezCoref: Towards Unifying Annotation Guidelines for Coreference Resolution (2023.findings-eacl)
Copied to clipboard
Ankita Gupta, Marzena Karpinska, Wenlong Zhao, Kalpesh Krishna, Jack Merullo, Luke Yeh, Mohit Iyyer, Brendan O’Connor
| Challenge: | Existing datasets vary in definition of coreferences and are curated for linguistic experts. |
| Approach: | They propose to use ezCoref to create a crowdsourcing-friendly coreference annotation methodology that teaches annotators only cases that are treated similarly across existing datasets. |
| Outcome: | The proposed method reannotates 240 passages from seven existing english coreference datasets while teaching annotators only cases that are treated similarly across them. |
The State of Relation Extraction Data Quality: Is Bigger Always Better? (2024.findings-acl)
Copied to clipboard
| Challenge: | Relation extraction (RE) methods extract tuples of relationships from text . many datasets with frequent label errors have been used . |
| Approach: | They review recent surveys and a sample of recent RE methods papers . they find that real-time evaluations of RE methods are possible . |
| Outcome: | a sample of 38 datasets currently being used shows that many have frequent label errors . a small number of relations in specific domains can more realistically evaluate methods . |
Investigating Sports Commentator Bias within a Large Corpus of American Football Broadcasts (D19-1)
Copied to clipboard
| Challenge: | a recent study shows that sports broadcasters build drama into play-by-play commentary by building team and player narratives through subjective analyses and anecdotes. |
| Approach: | They use FOOTBALL to examine racial bias in sports commentary . they identify major confounding factors for researchers examining rraecial bias . |
| Outcome: | The proposed dataset supports previous social science studies on commentator bias . it contains 1,455 broadcast football transcripts annotated with 250K player mentions and racial metadata . |
Harnessing Toulmin’s theory for zero-shot argument explication (2024.acl-long)
Copied to clipboard
| Challenge: | To better analyze informal arguments on public forums, we propose the task of argument explication, which makes explicit a text’s argumentative structure and implicit reasoning by outputting triples of propositions claim, reason warrant. |
| Approach: | They propose to prompt generative large language models to output explicit argument components proposed by Toulmin by prompting with the theory name. |
| Outcome: | The proposed method evaluates the outputs’ coverage and validity through a human study and automatic evaluation based on prior argumentation datasets and performs robustness checks over alternative LMs, prompts, and argumentation theories. |
Evaluating Zero-Shot Event Structures: Recommendations for Automatic Content Extraction (ACE) Annotations (2023.acl-short)
Copied to clipboard
| Challenge: | Zero-shot event extraction (EE) methods infer richly structured event records from unstructured text data, based on a user-supplied natural language specification and no training examples. |
| Approach: | They propose recommendations for future evaluations so the research community can better utilize ACE as an event evaluation resource. |
| Outcome: | The proposed methods can be used to evaluate zero-shot and other low-supervision EE methods, considering up to 32% of correctly identified arguments and 25% of correctly ignored event mentions as false negatives. |
Summarizing Relationships for Interactive Concept Map Browsers (D19-54)
Copied to clipboard
| Challenge: | Concept maps are visual summaries, structured as directed graphs . initial attempts to generate static summary models focused on static summarization . however, in interactive settings, users will need to dynamically query relationships . |
| Approach: | They propose a model which returns a labeled summary of a query concept for display in a visual interface. |
| Outcome: | The proposed model can summarize relationships between two query concepts in a visual network . it is based on a new dataset, and is trained on the dataset . |
-Stance: A Large-Scale Real World Dataset of Stances in Legal Argumentation (2025.acl-long)
Copied to clipboard
| Challenge: | Current tools for legal argument reasoning do not support this task. |
| Approach: | They propose to use a large-scale dataset to facilitate work on the legal argument stance classification task by evaluating whether a case summary strengthens or weakens a legal argument. |
| Outcome: | The proposed dataset is used to facilitate work on the legal argument stance classification task, which involves assessing whether a case summary strengthens or weakens a legal argument (polarity) and to what extent (intensity). |
Automated main concept generation for narrative discourse assessment in aphasia (2025.findings-acl)
Copied to clipboard
| Challenge: | Several advances have been made towards developing theoretical and computational methods for understanding narratives. |
| Approach: | They propose a method that generates MCs from novel stories that experts can edit manually. |
| Outcome: | The proposed method can generate most of the gold standard MCs for stories from an existing narrative summarization dataset. |
Twitter Universal Dependency Parsing for African-American and Mainstream American English (P18-1)
Copied to clipboard
| Challenge: | We analyze the performance disparities between AAE and Mainstream American English (MAE) because of Twitter-specific conventions and dialectal language. |
| Approach: | They develop a dataset of 500 tweets, 250 of which are in AAE, within the Universal Dependencies 2.0 framework and annotate it. |
| Outcome: | The proposed model improves performance for AAE tweets with no or very little in-domain labeled data and assesses its lexical and syntactic features. |
Uncertainty-aware generative models for inferring document class prevalence (D18-1)
Copied to clipboard
| Challenge: | Existing methods for inferring the relative frequency of classes of unlabeled examples are imperfect. |
| Approach: | They propose a generative probabilistic modeling approach to prevalence estimation . they back out an implicit individual-level likelihood function to conduct fast inference . |
| Outcome: | The proposed method provides better confidence interval coverage than an alternative and is significantly more robust to shifts in the class prior between training and testing. |
Monte Carlo Syntax Marginals for Exploring and Using Dependency Parses (N18-1)
Copied to clipboard
| Challenge: | Dependency parsing research focuses on improving accuracy of single-tree predictions . ambiguity is inherent to natural language syntax, and communicating it is important for error analysis . |
| Approach: | They propose a transition sampling algorithm to sample from the full joint distribution of parse trees defined by a model and demonstrate its usefulness. |
| Outcome: | The proposed method can be used to propagate parse uncertainty to two downstream applications. |