Papers by Julian Michael
Crowdsourcing Question-Answer Meaning Representations (N18-2)
Copied to clipboard
| Challenge: | Existing datasets for predicate-argument relationships are lacking highly skilled and trained annotators. |
| Approach: | They propose a crowdsourcing scheme to generate question-answer pairs that represent predicate-argument relationships in sentences as a set of question-announcer pairs. |
| Outcome: | The proposed model covers the vast majority of predicate-argument relationships in existing datasets along with many previously under-resourced ones, including implicit arguments and relations. |
Asking It All: Generating Contextualized Questions for any Semantic Role (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to question generation require conditioning on existing answers in text . previous work required human-curated templates, limiting coverage and question fluency . |
| Approach: | They propose a task of role question generation that produces a prototype and revises it to be contextually appropriate for the passage. |
| Outcome: | The proposed model generates diverse and well-formed questions for a large, broad-coverage ontology of predicates and roles. |
Prompting Contrastive Explanations for Commonsense Reasoning Tasks (2021.findings-acl)
Copied to clipboard
| Challenge: | Large pretrained language models (PLMs) can achieve near-human performance on commonsense reasoning tasks, but provide little human-interpretable evidence of the underlying reasoning they use. |
| Approach: | They propose to use large pretrained language models to generate evidence for commonsense reasoning NLP tasks . they use models to contrast alternative explanations based on key attribute(s) required to justify the correct answer . |
| Outcome: | The proposed model improves performance on two commonsense reasoning benchmarks compared to previous non-contrastive alternatives. |
Inducing Semantic Roles Without Syntax (2021.findings-acl)
Copied to clipboard
| Challenge: | Semantic roles are a key component of linguistic predicate-argument structure, but syntax can be difficult to define, annotate, and predict. |
| Approach: | They propose to use QA-SRL to automatically induce semantic roles from ontologies that use question-answer pairs to represent predicate-argument structure. |
| Outcome: | The proposed method outperforms existing models and a state-of-the-art model over gold syntax. |
What Do NLP Researchers Believe? Results of the NLP Community Metasurvey (2023.acl-long)
Copied to clipboard
Julian Michael, Ari Holtzman, Alicia Parrish, Aaron Mueller, Alex Wang, Angelica Chen, Divyam Madaan, Nikita Nangia, Richard Yuanzhe Pang, Jason Phang, Samuel R. Bowman
| Challenge: | Getting sociological beliefs wrong can slow research and lead to wasted effort, missed opportunities, and needless fights. |
| Approach: | They present the results of the NLP Community Metasurvey, run from May to June 2022. |
| Outcome: | The NLP community metasurvey elicited opinions on controversial issues from May to June 2022. |
Supervised Open Information Extraction (N18-1)
Copied to clipboard
| Challenge: | Existing methods for Open Information Extraction (Open IE) use semisupervised approaches or rule-based algorithms. |
| Approach: | They propose a supervised approach to Open Information Extraction (Open IE) they build on recent deep Semantic Role Labeling models to extract Open IE tuples . |
| Outcome: | The proposed model outperforms state-of-the-art Open IE systems on benchmark datasets. |
Asking without Telling: Exploring Latent Ontologies in Contextual Representations (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on model analysis indicates that they may learn a lot about linguistic structure, including part of speech, syntax, word sense, and more. |
| Approach: | They introduce latent subclass learning, a modification to classifier-based probing that induces a latent categorization (or ontology) of the probe’s inputs. |
| Outcome: | The proposed model induces a latent categorization (or ontology) of the probe’s inputs without access to fine-grained gold labels. |
Nearest Neighbor Zero-Shot Inference (2022.emnlp-main)
Copied to clipboard
| Challenge: | Using non-parametric memory for retrieval-augmented language models yields significant performance boosts over strong zeroshot baselines. |
| Approach: | They propose a retrieval-augmented language model with fuzzy verbalizers that expands the verbalizes that define different end-task class labels. |
| Outcome: | The proposed model outperforms non-retrieval-augmented language models on perplexity-based evaluations but gains transfer marginally . the main challenge is to achieve coverage of the verbalizer tokens that define the different end-task class labels. |
Controlled Crowdsourcing for High-Quality QA-SRL Annotation (2020.acl-main)
Copied to clipboard
Paul Roit, Ayal Klein, Daniela Stepanov, Jonathan Mamou, Julian Michael, Gabriel Stanovsky, Luke Zettlemoyer, Ido Dagan
| Challenge: | Question-answer driven Semantic Role Labeling (QA-SRL) is an open and natural flavour of SRL, potentially attainable from laymen. |
| Approach: | They propose a question-answer driven semantic role labeling approach that uses question-announced questions to label predicate-argument relationships. |
| Outcome: | The proposed method yields high-quality annotation with dramatically higher coverage, enabling future replicable research of natural semantic annotations. |
Selectively Answering Ambiguous Questions (2023.emnlp-main)
Copied to clipboard
| Challenge: | Prior work has focused on the case in which the question is clear and the answer is unambiguous but possibly unknown. |
| Approach: | They propose to use a sampled set of questions to calibrate answers to ambiguous questions with varying model scales. |
| Outcome: | The results show that sampling-based confidence scores help calibrate answers to relatively unambiguous questions, with more dramatic improvements on ambiguous ones. |
Analyzing the Role of Semantic Representations in the Era of Large Language Models (2024.naacl-long)
Copied to clipboard
Zhijing Jin, Yuen Chen, Fernando Gonzalez Adauto, Jiarui Liu, Jiayi Zhang, Julian Michael, Bernhard Schölkopf, Mona Diab
| Challenge: | Existing studies show the benefits of semantic representations in NLP tasks . Existing work using AMR is concerned with trainable models . |
| Approach: | They propose an AMR-driven chain-of-thought prompting method that uses AMR . they propose to use it to predict which input examples AMR may help or hurt on . |
| Outcome: | The proposed method hurts performance more than it helps on five different tasks. |
Large-Scale QA-SRL Parsing (P18-1)
Copied to clipboard
| Challenge: | a crowd-sourced approach to learning semantic parsers to predict predicateargument structures is open to many researchers. |
| Approach: | They propose a large-scale corpus of Question-Answer driven Semantic Role Labeling annotations . they also propose QA-SRL Bank 2.0, a crowd-sourcing scheme that can be used to train high quality parsers . |
| Outcome: | The proposed QA-SRL parser can generate high-quality questions at low cost and is intuitive to non-experts. |
AmbigQA: Answering Ambiguous Open-domain Questions (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing open-domain question answering systems assume questions have a single welldefined answer. |
| Approach: | They propose an open-domain question answering task which involves finding every plausible answer and rewriting the question for each one to resolve the ambiguity. |
| Outcome: | The proposed task is based on a dataset covering 14,042 open-domain questions . it shows that strong models benefit from weakly supervised learning . |
We’re Afraid Language Models Aren’t Modeling Ambiguity (2023.emnlp-main)
Copied to clipboard
Alisa Liu, Zhaofeng Wu, Julian Michael, Alane Suhr, Peter West, Alexander Koller, Swabha Swayamdipta, Noah Smith, Yejin Choi
| Challenge: | Ambiguity is an intrinsic feature of natural language, allowing us to anticipate misunderstandings and revise our interpretations as listeners. |
| Approach: | They use AmbiEnt to capture ambiguity in a sentence and analyze it to evaluate pretrained LMs. |
| Outcome: | The proposed model can flag political claims in the wild that are misleading due to ambiguity. |