Aggregating Crowdsourced and Automatic Judgments to Scale Up a Corpus of Anaphoric Reference for Fiction and Wikipedia Texts (2023.eacl-main)
Copied to clipboard
Juntao Yu, Silviu Paun, Maris Camilleri, Paloma Garcia, Jon Chamberlain, Udo Kruschwitz, Massimo Poesio
| Challenge: | Existing approaches to scale up anaphoric annotation have not overcome these limitations. |
| Approach: | They propose to use a game-with-a-purpose to ‘complete’ markable annotations by using an anaphoric resolver and an aggregation method for anaphorism. |
| Outcome: | The proposed method could be adopted to greatly speed up annotation time in other projects involving games-with-a-purpose. |
Similar Papers
A Crowdsourced Corpus of Multiple Judgments and Disagreement on Anaphoric Interpretation (N19-1)
Copied to clipboard
| Challenge: | a corpus of anaphoric information (coreference) is crowdsourced through a game-with-a-purpose . its main feature is the large number of judgments per markable: 20 on average, and over 2.2M in total. |
| Approach: | They propose to crowdsource anaphoric information corpus by a game-with-a-purpose and to use it to train a coreference resolver. |
| Outcome: | The proposed corpus contains annotations for 108,000 markables and 20 judgments per markable, and 2.2M in total. |
Universal Anaphora: The First Three Years (2024.lrec-main)
Copied to clipboard
Massimo Poesio, Maciej Ogrodniczuk, Vincent Ng, Sameer Pradhan, Juntao Yu, Nafise Sadat Moosavi, Silviu Paun, Amir Zeldes, Anna Nedoluzhko, Michal Novák, Martin Popel, Zdeněk Žabokrtský, Daniel Zeman
| Challenge: | Universal Anaphora initiative aims to push forward the state of the art in anaphora and anaphorism resolution by expanding the aspects of anaphonic interpretation which are or can be reliably annotated in an anagraphic corpora. |
| Approach: | They propose to develop a standard for anaphoric annotations and a method for evaluating models that can carry out this type of interpretation. |
| Outcome: | The Universal Anaphora initiative aims to push forward the state of the art in anaphora and anaphorism resolution by producing unified standards to annotate and encode annotations, delivering datasets encoded according to these standards, and developing methods for evaluating models that carry out this type of interpretation. |
The Universal Anaphora Scorer (2022.lrec-1)
Copied to clipboard
| Challenge: | a new version of the Reference Coreference Scorer is proposed to evaluate anaphoric interpretations . the proposed approach to evaluation of split antecedent anaphorisms is entirely novel . |
| Approach: | They propose an extended version of the Reference Coreference Scorer to evaluate anaphoric interpretations . the UA scorer supports the evaluation of split antecedent anaphorisms and discourse deixis . |
| Outcome: | The proposed method can be used to evaluate anaphoric interpretations in an extended range of anas . it supports evaluations of split antecedent anaphorisms and discourse deixis, for which no tools exist . |
A Probabilistic Annotation Model for Crowdsourcing Coreference (D18-1)
Copied to clipboard
| Challenge: | Existing methods to generate annotated corpora for coreference are expensive and limited. |
| Approach: | They propose a model of annotation for aggregating crowdsourced anaphoric annotations. |
| Outcome: | The proposed model can extract from crowdsourced annotations coreference chains comparable to those obtained with expert annotation. |
ezCoref: Towards Unifying Annotation Guidelines for Coreference Resolution (2023.findings-eacl)
Copied to clipboard
Ankita Gupta, Marzena Karpinska, Wenlong Zhao, Kalpesh Krishna, Jack Merullo, Luke Yeh, Mohit Iyyer, Brendan O’Connor
| Challenge: | Existing datasets vary in definition of coreferences and are curated for linguistic experts. |
| Approach: | They propose to use ezCoref to create a crowdsourcing-friendly coreference annotation methodology that teaches annotators only cases that are treated similarly across existing datasets. |
| Outcome: | The proposed method reannotates 240 passages from seven existing english coreference datasets while teaching annotators only cases that are treated similarly across them. |
Free the Plural: Unrestricted Split-Antecedent Anaphora Resolution (2020.coling-main)
Copied to clipboard
| Challenge: | a limitation of coreference resolution models is the focus on single-antecedent anaphors. |
| Approach: | They propose a model for unrestricted resolution of split-antecedent anaphors with multiple antecedents . they use auxiliary corpora where split-antcedent ananaphor was annotated by crowd . |
| Outcome: | The proposed model significantly improves on a baseline enhanced by BERT embeddings on anaphoric reference corpus. |
CLAUSE-ATLAS: A Corpus of Narrative Information to Scale up Computational Literary Analysis (2024.lrec-main)
Copied to clipboard
| Challenge: | XIX and XX century English novels annotated automatically contain 41,715 labeled clauses . a new approach to analyze novels based on clauses captures structural patterns within books, as well as qualitative differences between them. |
| Approach: | They propose to use a corpus of XIX and XX century English novels annotated automatically to study stories as sequences of eventive, subjective and contextual information. |
| Outcome: | The proposed method captures structural patterns within books, as well as qualitative differences between them. |
Proceedings of the First Workshop on Aggregating and Analysing Crowdsourced Annotations for NLP (D19-59)
Copied to clipboard
| Challenge: | The first workshop on crowdsourcing for NLP is open to all . |
| Approach: | The first workshop on crowdsourcing annotations for NLP is held at the acl.com . the workshop will focus on methods for aggregating and analysing crowdsourced data for Nl-specific tasks. |
| Outcome: | The first workshop on crowdsourcing for NLP received 16 submissions and accepted 7 . the workshop will focus on ambiguous, subjective or ambiguity analysis of crowdsourced data . |
Crowdsourcing and Aggregating Nested Markable Annotations (P19-1)
Copied to clipboard
| Challenge: | Existing methods for identifying markables for coreference annotation are task and language-independent and can be used for a variety of other annotation tasks. |
| Approach: | They propose a method for identifying markables for coreference annotation that combines automatic markable detectors with checking with a Game-With-A-Purpose (GWAP) and aggregation using a Bayesian annotation model. |
| Outcome: | The proposed method improves mention boundaries on news and other genres by over seven percentage points compared with state-of-the-art, domain-independent automatic mention detectors and almost three points over an in-domain mention detector. |
A Gold Anaphora Annotation Layer on an Eye Movement Corpus (L18-1)
Copied to clipboard
| Challenge: | Anaphora resolution is a complex process in which multiple linguistic factors play a role. |
| Approach: | They used annotated anaphorical pronouns from newspaper articles read by humans to model reading time of pronounes. |
| Outcome: | The proposed resource allows to study human anaphora resolution on natural data. |