Challenges to Evaluating the Generalization of Coreference Resolution Models: A Measurement Modeling Perspective (2024.findings-acl)
Copied to clipboard
| Challenge: | a recent study shows that evaluations of CR models on multiple datasets conflate different factors concerning what is being measured. |
| Approach: | They propose to view evaluations through the lens of measurement modeling . they show that evaluations risk conflating different factors concerning what is being measured . |
| Outcome: | The evaluations on seven datasets show that models that reflect coreference generalization are often correlated with differences in how coreference is defined and operationalized. |
Similar Papers
A Controlled Reevaluation of Coreference Resolution Models (2024.lrec-main)
Copied to clipboard
| Challenge: | a pretrained language model is used in state-of-the-art coreference resolution models. |
| Approach: | They evaluate five coreference resolution models and control for language model used . they find that encoder-based CR models outperform decoder--based models in accuracy . |
| Outcome: | The encoder-based model outperforms the decoder--based models in accuracy and speed . older model generalizes the best to out-of-domain textual genres . |
Conundrums in Entity Coreference Resolution: Making Sense of the State of the Art (2020.emnlp-main)
Copied to clipboard
| Challenge: | despite significant progress on entity coreference resolution, there is a general lack of understanding of what has been improved. |
| Approach: | They present an empirical analysis of entity coreference resolvers to provide an understanding of what has been improved. |
| Outcome: | The proposed model improves the performance of entity coreference resolvers. |
Assessing the Capabilities of Large Language Models in Coreference: An Evaluation (2024.lrec-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are a new approach to coreference resolution, but their performance is not yet fully understood. |
| Approach: | They propose that future efforts should improve scope, data, and evaluation methods of traditional coreference research to adapt to the development of LLMs. |
| Outcome: | The proposed methods improve scope, data, and evaluation methods of traditional coreference research to adapt to the development of LLMs. |
Evaluating the Impact of a Hierarchical Discourse Representation on Entity Coreference Resolution Performance (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work on entity coreference resolution (CR) follows current trends in Deep Learning . traditional approaches do not make use of hierarchical representations of discourse structure . |
| Approach: | They propose to leverage automatically constructed discourse parse trees within a neural approach to generate anaphoric mentions. |
| Outcome: | The proposed model improves on two benchmark entity coreference-resolution datasets. |
Conundrums in Event Coreference Resolution: Making Sense of the State of the Art (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent years have seen the successful application of span-based neural models to entity-based information extraction tasks such as entity coreference resolution (CR) Existing event coreference resolvers focused on feature engineering are few and far between, let alone event corefers. |
| Approach: | They propose to adapt existing span-based event reference systems to event coreference by adapting the models originally developed for entity coreference to event CR. |
| Outcome: | The proposed model improves the representations of entity mentions in entity-based IE tasks compared to non-span models . |
On the Influence of Coreference Resolution on Word Embeddings in Lexical-semantic Evaluation Tasks (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing word embeddings rely on local information delimited by context windows or dependency parents to predict word relations. |
| Approach: | They propose to use coreference resolution to find all spans of a text that refer to the same entity to improve the F1-Scores. |
| Outcome: | The proposed methods do not benefit significantly from pronoun substitution. |
Major Entity Identification: A Generalizable Alternative to Coreference Resolution (2024.emnlp-main)
Copied to clipboard
| Challenge: | Prior work identified annotation differences as one of the main reasons for the limited generalization gap in coreference resolution models. |
| Approach: | They propose an alternative referential task where the target entities are assumed to be specified in the input and the task is limited to the frequent entities. |
| Outcome: | The proposed model generalizes well across domains on multiple datasets with supervised models and LLM-based few-shot prompting. |
Bridging Resolution: A Survey of the State of the Art (2020.coling-main)
Copied to clipboard
| Challenge: | bridging resolution is an anaphora resolution task that is less studied than entity coreference resolution. |
| Approach: | This paper presents a survey of the current state of research on bridging resolution . it identifies and resolves bridling/associative anaphors, which are anamorphic references to non-identical associated antecedents. |
| Outcome: | The proposed task is more difficult than entity coreference resolution because of the lack of annotated corpora and lack of standardized evaluation protocols. |
Coreference Reasoning in Machine Reading Comprehension (2021.acl-long)
Copied to clipboard
| Challenge: | Existing datasets for machine reading comprehension do not reflect the natural distribution and, consequently, the challenges of coreference reasoning. |
| Approach: | They propose to use existing coreference resolution datasets to train machine reading comprehension models to better reflect the natural distribution and, consequently, the challenges of coreference reasoning. |
| Outcome: | The proposed method improves the performance of state-of-the-art models on a set of coreference-related datasets. |
End-to-End Neural Bridging Resolution (2022.coling-1)
Copied to clipboard
| Challenge: | state-of-the-art resolvers for bridging resolution are weaker than entity coreference resolution. |
| Approach: | They evaluate bridging resolvers in an end-to-end setting and strengthen them with better encoders . they also try to gain a better understanding of them through perturbation experiments . |
| Outcome: | bridging resolvers are evaluated in an end-to-end setting and strengthened with better encoders . bribridging resolution is the task of identifying briating anaphors and linking them to their antecedents - a paper by the journal bribing resolution argues . |