| Challenge: | Existing methods for identifying markables for coreference annotation are task and language-independent and can be used for a variety of other annotation tasks. |
| Approach: | They propose a method for identifying markables for coreference annotation that combines automatic markable detectors with checking with a Game-With-A-Purpose (GWAP) and aggregation using a Bayesian annotation model. |
| Outcome: | The proposed method improves mention boundaries on news and other genres by over seven percentage points compared with state-of-the-art, domain-independent automatic mention detectors and almost three points over an in-domain mention detector. |
Similar Papers
A Neural Model for Aggregating Coreference Annotation in Crowdsourcing (2020.coling-main)
Copied to clipboard
| Challenge: | Existing studies of natural language labelling tasks have shown that crowd-sourced labels can be noisy. |
| Approach: | They split the aggregation into mention classification and coreference chain inference tasks to predict the correct labels. |
| Outcome: | The proposed model predicts the class of each mention using an autoencoder while taking into account the mention’s annotation complexity and annotators’ reliability at different levels. |
A Crowdsourced Corpus of Multiple Judgments and Disagreement on Anaphoric Interpretation (N19-1)
Copied to clipboard
| Challenge: | a corpus of anaphoric information (coreference) is crowdsourced through a game-with-a-purpose . its main feature is the large number of judgments per markable: 20 on average, and over 2.2M in total. |
| Approach: | They propose to crowdsource anaphoric information corpus by a game-with-a-purpose and to use it to train a coreference resolver. |
| Outcome: | The proposed corpus contains annotations for 108,000 markables and 20 judgments per markable, and 2.2M in total. |
Proceedings of the First Workshop on Aggregating and Analysing Crowdsourced Annotations for NLP (D19-59)
Copied to clipboard
| Challenge: | The first workshop on crowdsourcing for NLP is open to all . |
| Approach: | The first workshop on crowdsourcing annotations for NLP is held at the acl.com . the workshop will focus on methods for aggregating and analysing crowdsourced data for Nl-specific tasks. |
| Outcome: | The first workshop on crowdsourcing for NLP received 16 submissions and accepted 7 . the workshop will focus on ambiguous, subjective or ambiguity analysis of crowdsourced data . |
Active Learning for Coreference Resolution using Discrete Annotation (2020.acl-main)
Copied to clipboard
| Challenge: | Exhaustively annotating coreference is expensive as it requires tracking coreference chains across long passages of text. |
| Approach: | They propose a pairwise annotation technique which asks annotators to identify mention antecedents if a presented mention pair is not coreferent. |
| Outcome: | The proposed method is much more efficient when combined with a mention clustering algorithm for selecting which examples to label . future work can use the proposed protocol to develop coreference models for new domains. |
A Probabilistic Annotation Model for Crowdsourcing Coreference (D18-1)
Copied to clipboard
| Challenge: | Existing methods to generate annotated corpora for coreference are expensive and limited. |
| Approach: | They propose a model of annotation for aggregating crowdsourced anaphoric annotations. |
| Outcome: | The proposed model can extract from crowdsourced annotations coreference chains comparable to those obtained with expert annotation. |
Neural Coreference Resolution with Deep Biaffine Attention by Joint Mention Detection and Mention Clustering (P18-2)
Copied to clipboard
| Challenge: | Existing methods for identifying and clustering mentions in text are complex and require heuristics to solve. |
| Approach: | They propose to use a biaffine attention model to get antecedent scores for each possible mention and optimize mention detection and mention clustering accuracy given the mention cluster labels. |
| Outcome: | The proposed model achieves the state-of-the-art performance on the CoNLL-2012 shared task English test set. |
They Exist! Introducing Plural Mentions to Coreference Resolution and Entity Linking (C18-1)
Copied to clipboard
| Challenge: | Unlike singular mentions each of which represents one entity, plural mentions stand for multiple entities. |
| Approach: | They propose a novel coreference resolution algorithm that selectively creates clusters to handle both singular and plural mentions and a deep learning-based entity linking model that jointly handles both types of mentions through multi-task learning. |
| Outcome: | The proposed model outperforms existing models designed for singular mentions and plural mentions. |
Aggregating Crowdsourced and Automatic Judgments to Scale Up a Corpus of Anaphoric Reference for Fiction and Wikipedia Texts (2023.eacl-main)
Copied to clipboard
Juntao Yu, Silviu Paun, Maris Camilleri, Paloma Garcia, Jon Chamberlain, Udo Kruschwitz, Massimo Poesio
| Challenge: | Existing approaches to scale up anaphoric annotation have not overcome these limitations. |
| Approach: | They propose to use a game-with-a-purpose to ‘complete’ markable annotations by using an anaphoric resolver and an aggregation method for anaphorism. |
| Outcome: | The proposed method could be adopted to greatly speed up annotation time in other projects involving games-with-a-purpose. |
Do UD Trees Match Mention Spans in Coreference Annotations? (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to annotate mention spans are based on delimiting token intervals, but there is no syntactic representation of the mention span. |
| Approach: | They propose to integrate coreference annotation with syntactic annotation to make them convergent in the long term. |
| Outcome: | The proposed approach could be advantageous in the long term, the authors argue. |
A Cluster Ranking Model for Full Anaphora Resolution (2020.lrec-1)
Copied to clipboard
| Challenge: | Anaphora resolution systems designed for CONLL 2012 dataset can handle key aspects of the full anaphora task such as the identification of singletons and of certain types of non-referring expressions. |
| Approach: | They propose an architecture to identify non-referring expressions and build coreference chains, including singletons, using system mentions. |
| Outcome: | The proposed model performs better on the CONLL 2012 dataset than the state-of-the-art system. |