Papers with NSP
Handling Ontology Gaps in Semantic Parsing (2024.starsem-1)
Copied to clipboard
| Challenge: | Existing methods to detect hallucinations in closed-ontology models are limited by ontology gaps. |
| Approach: | They propose a framework for stimulating and analyzing NSP model hallucinations . they propose 'hallucination simulation framework' to detect hallucinosities in presence of ontology gaps . |
| Outcome: | The proposed framework improves the F1-Score and the IQ Pro benchmark datasets. |
Improving Text-to-SQL Semantic Parsing with Fine-grained Query Understanding (2022.emnlp-industry)
Copied to clipboard
Jun Wang, Patrick Ng, Alexander Hanbo Li, Jiarong Jiang, Zhiguo Wang, Bing Xiang, Ramesh Nallapati, Sudipta Sengupta
| Challenge: | Recent research on Text-to-SQL semantic parsing relies on parser or heuristic based approach to understand natural language query. |
| Approach: | They propose a general-purpose, modular neural semantic parsing framework that is based on token-level fine-grained query understanding. |
| Outcome: | The proposed framework outperforms the state-of-the-art model by 2.7% on a WikiTableQuestions test set. |
NCPrompt: NSP-Based Prompt Learning and Contrastive Learning for Implicit Discourse Relation Recognition (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent prompt learning methods have demonstrated success in IDRR, but they fail to fully exploit critical semantic features shared among various forms of templates. |
| Approach: | They propose an NSP-based prompt learning and contrastive learning method for IDRR that transforms the IDRR task into a next sentence prediction task. |
| Outcome: | The proposed model can be used to classify the discourse relation sense between argument pairs without an explicit connective. |
A Neural-Symbolic Approach to Natural Language Understanding (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models have enabled deep neural networks to perform natural language understanding tasks, but their performance can drastically deteriorate when logical reasoning is needed. |
| Approach: | They propose a framework for NLU based on analogical reasoning based upon neural processing and logical reasoning using both neural and symbolic processing. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on two NLU tasks, question answering (QA) and natural language inference (NLI). |
Conformal Predictor for Improving Zero-Shot Text Classification Efficiency (2022.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) have been shown effective for zero-shot (0shot) text classification. |
| Approach: | They propose to limit the number of likely labels using a fast base classifier-based conformal predictor calibrated on samples labeled by the 0shot model. |
| Outcome: | The proposed models reduce the average inference time for NLI- and NSP-based models by 25.6% and 22.2% without dropping performance below the predefined error rate of 1%. |
NSP-BERT: A Prompt-based Few-Shot Learner through an Original Pre-training Task —— Next Sentence Prediction (2022.coling-1)
Copied to clipboard
| Challenge: | Recent studies have shown that using prompts to utilize language models to perform downstream tasks is more effective than using token-level methods such as PET. |
| Approach: | They propose to use a BERT original pre-training task abandoned by RoBERTa and other models to construct a sentence-level prompt-based method that does not need to fix the length of the prompt or the position to be predicted. |
| Outcome: | The proposed method performs better than PET and EFL on a BERT pre-training task and is comparable to other prompt-based methods. |
On Losses for Modern Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Devlin et al. ( 2018) released a transformer network (BERT) pre-training over two tasks: masked language modelling (MLM) and next sentence prediction (NSP). |
| Approach: | They clarify NSP's effect on BERT pre-training and explore ways to include multiple tasks into pre-train. |
| Outcome: | The proposed framework outperforms BERTBase on the GLUE benchmark using fewer than a quarter of training tokens. |