Neural Token Representations and Negation and Speculation Scope Detection in Biomedical and General Domain Text (D19-62)
Copied to clipboard
| Challenge: | Existing evidence for improved performance on natural language tasks is unclear to what degree the learned token representations capture and encode highlevel morphological/syntactic knowledge about the usage of a given token in a sentence. |
| Approach: | They propose to use context-aware token representations to capture morphological/syntactic knowledge about the usage of a given word/token in a sentence. |
| Outcome: | The proposed representations capture and encode high-level morphological/syntactic knowledge about the usage of a given token in a sentence. |
Similar Papers
Learning with Structured Representations for Negation Scope Extraction (P18-2)
Copied to clipboard
| Challenge: | Existing approaches to negation scope detection have been criticized for capturing information related to negations, long-distance dependencies and structural information. |
| Approach: | They propose to use conditional random fields, semi-Markov CRF and latent-variable CRF models to capture useful information such as long-distance dependencies and some latent structural information. |
| Outcome: | The proposed approaches can capture useful information such as features related to negation cue, long-distance dependencies and some latent structural information. |
NegBERT: A Transfer Learning Approach for Negation Detection and Scope Resolution (2020.lrec-1)
Copied to clipboard
| Challenge: | Negation is an important characteristic of language, and a major component of information extraction from text. |
| Approach: | They propose to use a popular transfer learning model to solve Negation Detection and Scope Resolution tasks in 3 datasets that have gained popularity over the years. |
| Outcome: | The proposed model outperforms existing systems on the BioScope Corpus, the Sherlock dataset and the SFU Review Corpus in scope resolution. |
Lexicosyntactic Inference in Neural Models (D18-1)
Copied to clipboard
| Challenge: | lexicosyntactic inferences are triggered by surprising aspects of the syntactical context that a word occurs in. |
| Approach: | They build a factuality judgment dataset for English clause-embedding verbs in various syntactic contexts and use it to probe the behavior of current state-of-the-art neural systems. |
| Outcome: | The proposed model makes systematic errors that are visible through the lens of factuality prediction. |
To Learn or Not to Learn: Replaced Token Detection for Learning the Meaning of Negation (2024.lrec-main)
Copied to clipboard
| Challenge: | State-of-the-art language models perform well on a variety of language tasks, but struggle with understanding negation cues in tasks like natural language inference (NLI). |
| Approach: | They propose a new learning strategy for negation building on ELECTRA’s replaced token detection objective. |
| Outcome: | The proposed approach leads to substantial gains on a variant of RTE with additional negation. |
Revisiting subword tokenization: A case study on affixal negation in large language models (2024.naacl-long)
Copied to clipboard
| Challenge: | Negation is central to language understanding but is not properly captured by modern NLP methods. |
| Approach: | They propose to use subword tokenization methods to detect negation in large language models . they find that models can reliably recognize negation, despite mismatches in tokenization accuracy . |
| Outcome: | The proposed models can detect negation in English using subword tokenization methods despite some mismatches in tokenization accuracy and negation detection performance. |
Negation Scope Conversion: Towards a Unified Negation-Annotated Dataset (2024.lrec-main)
Copied to clipboard
| Challenge: | Negation scope resolution models that use pre-trained language models perform worse when fine-tuned on a combined dataset. |
| Approach: | They propose to automatically convert the negation scopes of BioScope and SFU to those of Sherlock and merge them into a unified dataset. |
| Outcome: | The proposed method improves on the unified dataset compared to the simply combined dataset. |
NegPar: A parallel corpus annotated for negation (L18-1)
Copied to clipboard
| Challenge: | NegPar is the first parallel corpus annotated for negation in the narrative domain. |
| Approach: | They present NegPar, a parallel corpus annotated for negation in the narrative domain . they follow the annotation guidelines in the CONANDOYLE-NEG corpus . |
| Outcome: | The proposed corpus is based on the CONANDOYLE-NEG corpus and is reannotated to ensure more consistent and interpretable representations. |
Fishing for Magikarp: Automatically Detecting Under-trained Tokens in Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Disconnect between tokenizer creation and model training in language models allows for specific inputs, such as the infamous SolidGoldMagikarp token, to induce unwanted model behaviour. |
| Approach: | They propose to use tokenizer analysis, model weight-based indicators, and prompting techniques to detect problematic tokens in large language models. |
| Outcome: | The proposed methods show that tokenizers are under-trained across a diverse set of models and provide insights into improving the efficiency and safety of language models. |
Predicting the Focus of Negation: Model and Error Analysis (2020.acl-main)
Copied to clipboard
| Challenge: | Experimental results show that a scope detector can predict the focus of negation . negation is a complex phenomenon present in all human languages . |
| Approach: | They propose to leverage a scope detector to introduce the scope of negation as an additional input to the neural network. |
| Outcome: | The proposed model obtains the best results to date, and analyzes errors depending on scope and context information. |
Trainable, Multiword-aware Linguistic Tokenization Using Modern Neural Networks (2026.eacl-srw)
Copied to clipboard
| Challenge: | Tokenization is a fundamental task in natural language processing that forms the first step of many pipelines. |
| Approach: | They propose to use a standard tokenizer trained without MWE-awareness as a baseline and a character-level SRN+CRF model to train token-level models. |
| Outcome: | The proposed tokenizers are based on a character-level and token-level sequence labeling problem and are consistent with the proposed pipelines. |