Findings of the Association for Computational Linguistics: EMNLP 2022
LogicSolver: Towards Interpretable Math Word Problem Solving with Logical Prompt-enhanced Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in MWP solving are uninterpretable due to shallow heuristics . a new approach to solve automatic word problem solvers requires a solver to predict expression tree and corresponding linguistic logic formulas simultaneously. |
| Approach: | They propose to annotate interpretable logical formulas based on algebraic knowledge as the grounded linguistic logic of each solution equation. |
| Outcome: | The proposed approach improves interpretability of a MWP solver by using logical prompts and interpretation generation. |
Commonsense Knowledge Salience Evaluation with a Benchmark Dataset in E-commerce (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models rank statements solely by confidence scores, and there is no information about which ones are salient from a human perspective. |
| Approach: | They propose a task where a model is required to learn whether a triple is salient . they propose supervised salience evaluation using a new Benchmark dataset . |
| Outcome: | The proposed task is based on a new Benchmark dataset of salience evaluation in e-commerce . it shows that saliency evaluation is hard, where models perform poorly on evaluation set . |
Automatic Rule Induction for Efficient Semi-Supervised Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to generalize from labeled and unlabeled data are difficult to explain and behave unreliably. |
| Approach: | They propose a framework for automatic discovery and integration of symbolic rules into pretrained transformer models by using an attention mechanism. |
| Outcome: | The proposed framework can improve state-of-the-art methods with no manual effort and minimal computational overhead. |
Improving Semantic Matching through Dependency-Enhanced Pre-trained Model with Adaptive Fusion (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on dependency prior structure integration into pre-trained models is still unclear. |
| Approach: | They propose a dependency-based fusion attention paradigm which explicitly introduces dependency prior structure into pre-trained models and adaptively fuses it with semantic information. |
| Outcome: | The proposed model achieves state-of-the-art or competitive performance on 10 public datasets, demonstrating the benefits of adaptively fusing dependency structure in semantic matching task. |
Sparse Mixers: Combining MoE and Mixing to build a more efficient BERT (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Sparse Mixer encoder model outperforms BERT on GLUE and SuperGLUE, trains 65% faster and runs inference 61% faster. |
| Approach: | They combine the capacity of sparsely gated Mixture-of-Experts (MoE) with the speed and stability of linear, mixing transformations to design the Sparse Mixer encoder model. |
| Outcome: | The proposed model outperforms BERT on GLUE and SuperGLUE but trains and runs twice as fast. |
KE-GCL: Knowledge Enhanced Graph Contrastive Learning for Commonsense Question Answering (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for commonsense question answering lack effective representations of knowledge graphs. |
| Approach: | They propose a Knowledge Enhanced Graph Contrastive Learning model by incorporating contextual descriptions into QA pairs and adopting a graph contrastive learning scheme. |
| Outcome: | The proposed model outperforms existing methods consistently on two benchmark datasets. |
Acceptability Judgements via Examining the Topology of Attention Maps (2022.findings-emnlp)
Copied to clipboard
Daniil Cherniavskii, Eduard Tulchinskii, Vladislav Mikhailov, Irina Proskurina, Laida Kushnareva, Ekaterina Artemova, Serguei Barannikov, Irina Piontkovskaya, Dmitri Piontkovski, Evgeny Burnaev
| Challenge: | Acceptability judgments are a key component of generative linguistics, but their ability to judge grammatical acceptability has not been explored. |
| Approach: | They propose to exploit the geometric properties of the attention graph to evaluate the grammatical acceptability of sentences using topological data analysis. |
| Outcome: | The proposed approach outperforms nine statistical and Transformer LM baselines on the BLiMP benchmark and the human-level performance on the same benchmark. |
Clip-Tuning: Towards Derivative-free Prompt Learning with a Mixture of Rewards (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work does not take full advantage of over-parameterized characteristics of large pre-trained language models. |
| Approach: | They propose a method that uses frozen "thinned" networks to obtain a mixture of rewards and advance the derivative-free prompt learning. |
| Outcome: | The proposed method outperforms previous gradient-free prompt learning methods and achieves parity with gradient-based counterparts on seven language understanding benchmarks under few-shot settings. |
Soft-Labeled Contrastive Pre-Training for Function-Level Code Representation (2022.findings-emnlp)
Copied to clipboard
Xiaonan Li, Daya Guo, Yeyun Gong, Yun Lin, Yelong Shen, Xipeng Qiu, Daxin Jiang, Weizhu Chen, Nan Duan
| Challenge: | Existing methods for contrastive pre-training ignore the relevance between codes in large code corpus. |
| Approach: | They propose a Soft-labeled contrastive pre-training framework with positive sample construction methods to learn functional-level code representation. |
| Outcome: | The proposed framework can obtain fine-grained soft-labels through an iterative adversarial manner and use them to learn better code representation. |
Conditioned Masked Language and Image Modeling for Image-Text Dense Retrieval (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Large-scale two-stream pre-trained models like CLIP have achieved tremendous success in image-text retrieval. |
| Approach: | They propose a cross-modal framework for image-text retrieval using two-stream pre-trained models . they embed images and texts into instance representations with two separate encoders . experimental results on MSCOCO and Flickr30k reveal the effectiveness of their framework . |
| Outcome: | The proposed framework improves image-text retrieval performance on two popular cross-modal retrieval benchmarks. |
Does Simultaneous Speech Translation need Simultaneous Models? (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Simultaneous speech translation (SimulST) systems strive for high output quality but also low latency. |
| Approach: | They propose to train SimulST offline without additional training or adaptation . they also show offline training achieves similar or better quality compared to offline training . |
| Outcome: | The proposed model can serve both offline and simultaneous applications without additional training or adaptation. |
Utilizing Language-Image Pretraining for Efficient and Robust Bilingual Word Alignment (2022.findings-emnlp)
Copied to clipboard
Tuan Dinh, Jy-yong Sohn, Shashank Rajput, Timothy Ossowski, Yifei Ming, Junjie Hu, Dimitris Papailiopoulos, Kangwook Lee
| Challenge: | Recent studies show that unsupervised word translation is more accurate and robust without parallel corpora. |
| Approach: | They propose a method for unsupervised word translation that leverages visual observations and pretrained language-image models to align words. |
| Outcome: | The proposed method improves on the state-of-the-art language-image pretraining method for bilingual word alignment. |
Grape: Knowledge Graph Enhanced Passage Reader for Open-domain Question Answering (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Open-domain question answering (QA) models employ a retriever-reader pipeline . however, state-of-the-art readers fail to capture complex relationships between entities . |
| Approach: | They propose a knowledge graph enhanced passage reader that captures entities in questions and retrieved passages. |
| Outcome: | The proposed knowledge graph enhanced passage reader improves on open-domain QA benchmarks by up to 2.2 exact match scores. |
NarraSum: A Large-Scale Dataset for Abstractive Narrative Summarization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies focus on summarizing news documents or structured documents. |
| Approach: | They propose to use a large-scale narrative summarization dataset to encourage research . they find there is a performance gap between humans and the models on NarraSum . |
| Outcome: | The proposed dataset shows that humans and state-of-the-art models perform poorly when summarizing a narrative . it contains 122K narratives collected from synopses of movies and TV episodes with diverse genres . |
NMTScore: A Multilingual Analysis of Translation-based Text Similarity Measures (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) enables several similarity measures to estimate the probability of translations. |
| Approach: | They propose to rank the similarity of short text segments using translation-based similarity measures . they use the NMTScore library to analyze translation-related similarity . |
| Outcome: | The proposed measures show a relatively high correlation to human judgments when used for reference-based evaluation of data-to-text generation in 2 tasks and 17 languages. |
Language Models Understand Us, Poorly (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent large language models have achieved impressive results on benchmark tasks. |
| Approach: | They examine three views of human language understanding: as-mapping, as-reliability and as-representation. |
| Outcome: | The authors argue that language models are inadequate and that they can't understand us . they also argue that as-representation advances a science of understanding . |
Dialogue Meaning Representation for Task-Oriented Dialogue Systems (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on dialogue meaning representations is limited in scalability for complex expressions. |
| Approach: | They propose a pliable and easily extendable representation for task-oriented dialogue . they propose an inheritance hierarchy mechanism focusing on domain extensibility . |
| Outcome: | The proposed representation can be easily extended to a task-oriented dialogue dataset. |
Learning from the Dictionary: Heterogeneous Knowledge Guided Fine-tuning for Chinese Spell Checking (2022.findings-emnlp)
Copied to clipboard
Yinghui Li, Shirong Ma, Qingyu Zhou, Zhongli Li, Li Yangning, Shulin Huang, Ruiyang Liu, Chao Li, Yunbo Cao, Haitao Zheng
| Challenge: | Chinese Spell Checking (CSC) aims to detect and correct Chinese spelling errors. |
| Approach: | They propose a framework which renders Chinese Spell Checking model to learn heterogeneous knowledge from the dictionary in terms of phonetics, vision, and meaning. |
| Outcome: | The proposed framework renders the CSC model to learn heterogeneous knowledge from the dictionary in terms of phonetics, vision, and meaning. |
Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One? (2022.findings-emnlp)
Copied to clipboard
Xilun Chen, Kushal Lakhotia, Barlas Oguz, Anchit Gupta, Patrick Lewis, Stan Peshterliev, Yashar Mehdad, Sonal Gupta, Wen-tau Yih
| Challenge: | Existing sparse retrievers lack the ability to match salient phrases and rare entities in the query. |
| Approach: | They introduce a dense Lexical Model that can be trained to imitate a sparse one. |
| Outcome: | The proposed model outperforms sparse retrievers on a range of tasks including five question answering datasets and the MS MARCO passage retrieval. |
SMARTAVE: Structured Multimodal Transformer for Product Attribute Value Extraction (2022.findings-emnlp)
Copied to clipboard
Qifan Wang, Li Yang, Jingang Wang, Jitin Krishnan, Bo Dai, Sinong Wang, Zenglin Xu, Madian Khabsa, Hao Ma
| Challenge: | Existing methods for product attribute value extraction are noisy and incomplete with missing values for most retailers. |
| Approach: | They propose a Structure Mltimodal trAnsformeR for producT Attribute Value Extraction which jointly encodes the structured product information from multiple modalities. |
| Outcome: | The proposed method outperforms state-of-the-art methods on two multimodal product datasets. |
When Language Model Meets Private Library (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing language models have been pre-trained on large-scale code corpora and generate decent code snippets. |
| Approach: | They propose a framework that can provide pre-trained language models with the ability to generate code using private libraries. |
| Outcome: | The proposed framework can generate code using private libraries using off-the-shelf language models or pre-trained models on code corpus containing API information. |
Cross-Domain Sentiment Classification using Semantic Representation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on cross-domain sentiment classification ignore the semantic relevance between domains. |
| Approach: | They propose to use Abstract Meaning Representation to help with cross-domain sentiment classification by combining sentence-level AMRs with text-graph interaction models. |
| Outcome: | The proposed model is effective over strong baselines and shows its importance over strong models. |
Yes-Yes-Yes: Proactive Data Collection for ACL Rolling Review and Beyond (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to data collection are under-resourced and can be difficult to implement in the peer review domain. |
| Approach: | They propose a donation-based peer review data collection workflow that takes into account ethical, legal and confidentiality-related aspects of data collection into account. |
| Outcome: | The proposed workflows are based on a donation-based peer review platform and show that the datasets are larger than the current workflows. |
AssistSR: Task-oriented Video Segment Retrieval for Personal AI Assistant (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Currently, personal AI assistants on the phone and AR glasses can assist our daily life in addressing our questions like "how to adjust the date for this watch?" |
| Approach: | They propose a task that asks a question about affordance of items in our daily life . they construct a dataset that contains 3.2k multimodal questions on 1.6k video segments . |
| Outcome: | The proposed task outperforms baseline methods while still having room for improvement in the future. |
Dim-Krum: Backdoor-Resistant Federated Learning for NLP with Dimension-wise Krum-Based Aggregation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to reduce backdoor risk of federated learning are validated in the CV field. |
| Approach: | They propose a federated aggregation algorithm that detects errors determined by backdoor strengths for NLP attacks. |
| Outcome: | The proposed method is hard to defend against than CV, and the results validate it. |
Fine-mixing: Mitigating Backdoors in Fine-tuned Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for defending NLP models against backdoors have ignored the clean weights of PLMs. |
| Approach: | They exploit pre-trained weights to mitigate backdoors in fine-tuned NLP models . they use a fine-mixing technique and an Embedding Purification technique to do the same . |
| Outcome: | The proposed method outperforms baseline mitigation methods on three single-sentence sentiment classification tasks and two sentence-pair classification tasks. |
Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completion (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Language models (LMs) have been shown to generate more factual responses by employing modularity in combination with retrieval. |
| Approach: | They extend the recent approach of Adolphs et al. (2021) to include internet search as a module. |
| Outcome: | The proposed method outperforms the state-of-the-art model BlenderBot 2 on open-domain knowledge-grounded conversations for the same number of parameters. |
Stretching Sentence-pair NLI Models to Reason over Long Documents and Clusters (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in modeling and datasets demonstrate promising performance for NLI. |
| Approach: | They explore the direct zero-shot applicability of NLI models to real applications . they analyze the robustness of models to longer and out-of-domain inputs . |
| Outcome: | The proposed models are robust to longer and out-of-domain inputs and can perform on full documents. |
Towards Realistic Low-resource Relation Extraction: A Benchmark with Empirical Baseline Study (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to extract relational facts from text are limited in their ability to learn from limited labeled data. |
| Approach: | They propose to use prompt-based methods with few-shot labeled data to evaluate performance . data augmentation technologies and self-training are also proposed to generate more labeles in-domain data. |
| Outcome: | The proposed methods perform well in low-resource settings with 8 relation extraction datasets. |
CLLE: A Benchmark for Continual Language Learning Evaluation in Multilingual Machine Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing benchmarks for Continual Language Learning (CLL) are limited due to the complexity of the task and the lack of unified benchmarks. |
| Approach: | They propose a Continual Language Learning Evaluation benchmark CLLE in multilingual translation. |
| Outcome: | The proposed method is effective when compared with other strong benchmarks. |
Lexicon-Enhanced Self-Supervised Training for Multilingual Dense Retrieval (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent multilingual pre-trained models perform poorly on multilingual retrieval tasks due to lack of multilingual training data. |
| Approach: | They propose to mine and generate self-supervised training data based on large-scale unlabeled corpus and introduce query generator to generate more queries in target languages for unlabed passages. |
| Outcome: | The proposed method performs better than baselines on a Mr. TYDI dataset and an industrial dataset from a commercial search engine. |
Improve Interpretability of Neural Networks via Sparse Contrastive Coding (2022.findings-emnlp)
Copied to clipboard
| Challenge: | XAI has achieved remarkable advances, but few efforts have been devoted to solving the problem. |
| Approach: | They propose a model-agnostic explanation method termed Sparse Contrastive Coding . they use model-based explanations to explain the black-box in a more model-oriented way . |
| Outcome: | The proposed method outperforms five state-of-the-art methods in interpretability and classification metrics. |
LEMON: Language-Based Environment Manipulation via Execution-Guided Pre-training (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to language-based environment manipulation are difficult to generalize across environments. |
| Approach: | They propose a general framework for language-based environment manipulation tasks that can deal with various environments using the same generative language model. |
| Outcome: | The proposed framework achieves new state-of-the-art results on four of the tasks and the execution-guided pre-training strategy brings remarkable improvements on all experimental tasks. |
CROP: Zero-shot Cross-lingual Named Entity Recognition with Multilingual Labeled Sequence Translation (2022.findings-emnlp)
Copied to clipboard
Jian Yang, Shaohan Huang, Shuming Ma, Yuwei Yin, Li Dong, Dongdong Zhang, Hongcheng Guo, Zhoujun Li, Furu Wei
| Challenge: | Named entity recognition (NER) suffers from the scarcity of annotated training data, especially for low-resource languages without labeled data. |
| Approach: | They propose a cross-lingual entity projection framework to enable zero-shot cross-linguistic NER with the help of a multilingual labeled sequence translation model. |
| Outcome: | The proposed method outperforms the baseline method on two benchmarks by a large margin of +3 7 F1 scores and achieves state-of-the-art performance. |
Handling and Presenting Harmful Text in NLP Research (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Text data can pose a risk of harm, but the risks remain unresolved in the NLP community. |
| Approach: | They propose an analytical framework categorising harms on three axes: harm type, whether harm sought as a feature of research design, whether harmful content is encountered when working on unrelated problems, and who it affects . |
| Outcome: | The proposed framework categorises harms on three axes: harm type, whether harm sought as feature of research design, and whether harmful content is encountered when working on unrelated problems. |
Multimodal Contrastive Learning via Uni-Modal Coding and Cross-Modal Prediction for Multimodal Sentiment Analysis (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work on multimodal representation learning has focused on uni-modality pre-training or cross-modalities integration. |
| Approach: | They propose a framework for multimodal representation learning that uses uni-modal contrastive coding and an efficient unimodal feature augmentation strategy to capture intermodal dynamics. |
| Outcome: | The proposed framework surpasses state-of-the-art methods on two public datasets. |
Towards Unified Prompt Tuning for Few-shot Text Classification (2022.findings-emnlp)
Copied to clipboard
Jianing Wang, Chengyu Wang, Fuli Luo, Chuanqi Tan, Minghui Qiu, Fei Yang, Qiuhui Shi, Songfang Huang, Ming Gao
| Challenge: | Prompt-based fine-tuning has boosted performance of Pre-trained Language Models (PLMs) on few-shot text classification, but PLMs are unfamiliar with prompt-style expressions during pre-training, which limits the few- shot learning performance on downstream tasks. |
| Approach: | They propose a framework for prompt-based fine-tuning that captures prompting semantics from non-target NLP datasets and propose 'Prompt-Options-Verbalizer' for joint prompt learning across different NLP tasks. |
| Outcome: | Experiments show that the proposed framework outperforms state-of-the-art prompt-based fine-tuning frameworks on few-shot text classification tasks. |
Can language models learn from explanations in context? (2022.findings-emnlp)
Copied to clipboard
Andrew Lampinen, Ishita Dasgupta, Stephanie Chan, Kory Mathewson, Mh Tessler, Antonia Creswell, James McClelland, Jane Wang, Felix Hill
| Challenge: | Language Models can adapt to a few in-context examples, but without training. |
| Approach: | They examine how explanations of few-shot examples can help Language Models (LMs) explanations can improve performance even without tuning, they find . |
| Outcome: | The proposed explanations outperform hand-tuned explanations on small validation sets. |
GNN-encoder: Learning a Dual-encoder Architecture via Graph Neural Networks for Dense Passage Retrieval (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to perform large-scale query-passage retrieval are term-based, but they lose interaction between query-pastage pairs. |
| Approach: | They propose to fuse query (passage) information into query representations via graph neural networks that are constructed by queries and their top retrieved passages. |
| Outcome: | The proposed model outperforms existing models on MSMARCO, Natural Questions and TriviaQA datasets and achieves the new state-of-the-art on these datasets. |
Linguistic Rules-Based Corpus Generation for Native Chinese Grammatical Error Correction (2022.findings-emnlp)
Copied to clipboard
Shirong Ma, Yinghui Li, Rongyi Sun, Qingyu Zhou, Shulin Huang, Ding Zhang, Li Yangning, Ruiyang Liu, Zhongli Li, Yunbo Cao, Haitao Zheng, Ying Shen
| Challenge: | Chinese Grammatical Error Correction (CGEC) is a challenging NLP task and a common application in human daily life. |
| Approach: | They propose a linguistic rules-based approach to construct large-scale CGEC training corpora with automatically generated grammatical errors. |
| Outcome: | The proposed method improves performance of existing CGEC models and the benchmark is excellent resource for further development. |
Rethinking the Video Sampling and Reasoning Strategies for Temporal Sentence Grounding (2022.findings-emnlp)
Copied to clipboard
Jiahao Zhu, Daizong Liu, Pan Zhou, Xing Di, Yu Cheng, Song Yang, Wenzheng Xu, Zichuan Xu, Yao Wan, Lichao Sun, Zeyu Xiong
| Challenge: | Existing methods for temporal sentence grounding ignore two crucial issues . 1) Boundary-bias: the video downsampling process may lose these two frames . 2) Reasoning-biases: such incorrect new boundary frames lead to the reasoning bias . |
| Approach: | They propose a siamese sampling mechanism to generate additional contextual frames . they use a reasoning strategy to learn the inter-relationship among these frames a . |
| Outcome: | Extensive experiments demonstrate the effectiveness of a new siamese sampling network on three challenging datasets. |
System 1 + System 2 = Better World: Neural-Symbolic Chain of Logic Reasoning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Current NLP models require more than the ability to learn informative representations from data for logic tasks. |
| Approach: | They propose an architecture that explicitly conducts neural logic reasoning on top of the representation learning models. |
| Outcome: | The proposed architecture improves on the commonsense knowledge graph completion task on a commonsensible task with the two-system architecture. |
Efficient Federated Learning on Knowledge Graphs via Privacy-preserving Relation Embedding Aggregation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing frameworks that share entity embeddings of knowledge graphs (KGs) would incur a severe privacy leakage. |
| Approach: | They propose a new attack method that aims to recover the original embedding information based on the known entity embeddables of FedE. |
| Outcome: | The proposed framework can be used to infer whether a specific relation exists in a private client. |
TextHacker: Learning based Hybrid Local Search Algorithm for Text Hard-label Adversarial Attack (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing textual adversarial attacks use gradient or prediction confidence to generate adversarials, making it hard to be deployed in real-world applications. |
| Approach: | They propose a textual adversarial attack that randomly perturbs lots of words to craft an adversarial example. |
| Outcome: | The proposed attack outperforms existing hard-label attacks in terms of attack performance and adversary quality. |
Visualizing the Obvious: A Concreteness-based Ensemble Model for Noun Property Prediction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Neural language models encode rich knowledge about entities and their relationships but common properties of nouns are difficult to extract because they are rarely explicitly stated in texts. |
| Approach: | They propose to extract perceptual properties from images and use them in an ensemble model to complement the information extracted from language models. |
| Outcome: | The proposed model improves noun property prediction compared to powerful text-based language models. |
It’s Better to Teach Fishing than Giving a Fish: An Auto-Augmented Structure-aware Generative Model for Metaphor Detection (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to identify metaphors use contextual information extracted by transformers for classifications directly. |
| Approach: | They propose to use structure information extraction to transform the classification task into a keywords-extraction task and to use it to expand the limited datasets. |
| Outcome: | The proposed model obtains competitive results compared with state-of-the-art methods . |
Expose Backdoors on the Way: A Feature-Based Efficient Defense against Textual Backdoor Attacks (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing online backdoor defense methods for NLP models focus on anomalies at input or output level, causing fragility to adaptive attacks and high computational cost. |
| Approach: | They propose a feature-based online defense method to detect poisoned samples . they use a distance-based anomaly score to distinguish poisones from clean samples based on feature-level regularization . |
| Outcome: | The proposed method outperforms existing methods in sentiment analysis and offense detection tasks. |
Diving Deep into Modes of Fact Hallucinations in Dialogue Systems (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Knowledge Graph(KG) grounded conversations often use large pre-trained models and suffer from fact hallucination. |
| Approach: | They propose to use a human feedback analysis to identify various modes of hallucination in KG chatbots. |
| Outcome: | The proposed system provides fine-grained signals that control fallacious content while generating responses. |
Representation Learning for Resource-Constrained Keyphrase Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | State-of-the-art keyphrase generation methods depend on large annotated datasets, limiting their performance in domains with limited annotation data. |
| Approach: | They propose a method that first identifies salient information using retrieval-based corpus-level statistics and then learns a task-specific intermediate representation based on a pre-trained language model. |
| Outcome: | The proposed method improves keyphrase generation and zero-shot domain adaptation on multiple keyphrase benchmarks. |
Systematicity in GPT-3’s Interpretation of Novel English Noun Compounds (2022.findings-emnlp)
Copied to clipboard
| Challenge: | e.g., stew skillet, swamp squash) are not fully compositional, but highly predictable based on whether the modifier and head refer to artifacts or natural kinds. |
| Approach: | They propose to compare the interpretations of novel English noun compounds with the large language model GPT-3, which is governed by interpretive principles. |
| Outcome: | The results show that the large language model GPT-3 reasoning only about specific lexical items is consistent with the Levin et al.'s theory. |
CARE: Causality Reasoning for Empathetic Responses by Conditional Graph Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to empathetic response generation only consider causalities between the user’s emotion and the user's experiences and neglect interdependence among causalities and reason them independently. |
| Approach: | They propose to use a conditional variable Graph Auto-Encoder to reason all plausible causalities interdependently and simultaneously given the user’s emotion, dialogue history, and future dialogue content. |
| Outcome: | The proposed method achieves state-of-the-art in a real-world situation. |
TransAdv: A Translation-based Adversarial Learning Framework for Zero-Resource Cross-Lingual Named Entity Recognition (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for named entity recognition are limited by noise in translation . Existing approaches to named entities recognition are mainly based on labeled data . |
| Approach: | They propose a framework to mitigate lexical and syntactic errors of translated data . they propose to use multi-level adversarial learning and multi-model knowledge distillation to mitigate noise . |
| Outcome: | The proposed framework mitigates lexical and syntactic errors of translated data . it achieves competitive performance to state-of-the-art models . |
BARLE: Background-Aware Representation Learning for Background Shift Out-of-Distribution Detection (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for OOD detection focus on identifying semantic-shift OOD samples . background-shift detection is more practical but challenging . |
| Approach: | They propose a background-aware representation learning approach for background-shift OOD detection in NLP. |
| Outcome: | The proposed method improves background-shift OOD detection while maintaining ID classification accuracy. |
What Language Model to Train if You Have One Million GPU Hours? (2022.findings-emnlp)
Copied to clipboard
Teven Le Scao, Thomas Wang, Daniel Hesslow, Stas Bekman, M Saiful Bari, Stella Biderman, Hady Elsahar, Niklas Muennighoff, Jason Phang, Ofir Press, Colin Raffel, Victor Sanh, Sheng Shen, Lintang Sutawika, Jaesung Tae, Zheng Xin Yong, Julien Launay, Iz Beltagy
| Challenge: | Recent years have seen the advent of large language models characterized by emergent capabilities arising from sheer scale alone. |
| Approach: | They propose to use a multilingual model to compare performance to the English-only model by ablation at the billion-parameter scale. |
| Outcome: | The proposed model is based on a multilingual model and its performance against the English-only model. |
Enhancing Out-of-Distribution Detection in Natural Language Understanding via Implicit Layer Ensemble (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Out-of-distribution (OOD) detection aims to discern outliers from the intended data distribution, which is crucial to maintaining high reliability and a good user experience. |
| Approach: | They propose a framework that encourages intermediate features to learn layer-specialized representations and assembles them implicitly into a single representation to absorb rich information in the pre-trained language model. |
| Outcome: | The proposed framework is significantly more effective than previous studies in intent classification and OOD datasets. |
Contrastive Demonstration Tuning for Pre-trained Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies focus on searching discrete or continuous prompts or optimized verbalizers, yet the demonstration examples are crucial for an excellent final performance of prompt-tuning. |
| Approach: | They propose a pluggable, extensible, and efficient approach to prompt tuning that is free of demonstration sampling. |
| Outcome: | The proposed approach can be pluggable, extensible, and efficient on 16 datasets. |
Detect-Localize-Repair: A Unified Framework for Learning to Debug with CodeT5 (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Automated software debugging is crucial for improving productivity of software developers . many neural-based techniques focus only on one or the other, ignoring mutual benefits . |
| Approach: | They propose a framework to adapt a pretrained programming language model to automate debugging . they propose three objectives: bug detection, bug localization, program repair . |
| Outcome: | The proposed framework outperforms baselines from both NLP and software engineering domains on two new datasets. |
Influence Functions for Sequence Tagging Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Named Entity Recognition, Part-of-Speech tagging, and Semantic Role Labeling are standard tasks in NLP, but there has been little work on interpretability methods for sequence taging. |
| Approach: | They propose to extend influence functions to sequence tagging tasks by identifying noisy annotations in NER corpora. |
| Outcome: | The proposed methods are able to identify noisy annotations in NER corpora and are scalable. |
Impact of Pretraining Term Frequencies on Few-Shot Numerical Reasoning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained language models have demonstrated ability to perform numerical reasoning by extrapolating from a few examples in few-shot settings. |
| Approach: | They investigate how well pretrained language models reason with terms less frequent in pretraining data. |
| Outcome: | The models are more accurate on instances whose terms are more prevalent, in some cases above 70% more accurate than the bottom 10%. |
Syntactic and Semantic Uniformity for Semantic Parsing and Task-Oriented Dialogue Systems (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to model natural language use pre-trained language models, but little attention has been paid to the representation of machine-readable formats. |
| Approach: | They propose a data representation framework for semantic parsing and task-oriented dialogue systems . they define a meta grammar for syntactically uniform representations and translate semantically equivalent functions into a uniform vocabulary. |
| Outcome: | The proposed representation improves accuracy and allows for transfer learning across datasets. |
Knowledge-Rich Self-Supervision for Biomedical Entity Linking (2022.findings-emnlp)
Copied to clipboard
Sheng Zhang, Hao Cheng, Shikhar Vashishth, Cliff Wong, Jinfeng Xiao, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, Hoifung Poon
| Challenge: | Entity linking is challenging in high-value domains with myriad entities . standard classification approaches suffer from the annotation bottleneck . |
| Approach: | They propose a self-supervised approach to learn domain knowledge for biomedical entity linking . it generates self-reported mention examples on unlabeled text and trains contextual encoder . |
| Outcome: | The proposed method outperforms existing methods by 20 points in accuracy on biomedical datasets. |
ARTIST: A Transformer-based Chinese Text-to-Image Synthesizer Digesting Linguistic and World Knowledge (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Text-to-Image Synthesis (TIS) is a popular task to convert natural language texts into realistic images. |
| Approach: | They propose a transformer-based Chinese text-to-image synthesizer for high-resolution image generation that incorporates linguistic and relational knowledge facts into the model to ensure better performance without the usage of ultra-large models. |
| Outcome: | The proposed model outperforms existing models in Chinese with linguistic and relational knowledge facts. |
From Spelling to Grammar: A New Framework for Chinese Grammatical Error Correction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Chinese Grammatical Error Correction (CGEC) aims to generate correct sentences from erroneous sequences. |
| Approach: | They propose a zero-shot approach for spelling error correction that is simple but effective . they propose auxiliary task to predict POS sequence of target sentence . |
| Outcome: | The proposed framework achieves 42.11 F-0.5 on the English GEC dataset outperforms the previous state-of-the-art by a wide margin of 1.30 points. |
Language Models Are Poor Learners of Directional Inference (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing datasets fail to test directionality and are infested by artefacts that can be learnt as proxy for entailments, yielding over-optimistic results. |
| Approach: | They propose a benchmark for directional predicate entailments that is extrinsic to existing training sets. |
| Outcome: | The proposed model is incompetent on directional predicate entailments, compared to engorgement graphs, but limited by sparsity. |
Wish I Can Feel What You Feel: A Neural Approach for Empathetic Response Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches focus on a single factor that affects empathy but in practice it is complex and dynamic. |
| Approach: | They propose a method which integrates emotion cause, knowledge graph, and communication mechanism for empathetic response generation. |
| Outcome: | The proposed method generates more informative and empathetic responses on a large-scale dataset. |
Measuring and Improving Semantic Diversity of Dialogue Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evaluation metrics for response diversity do not capture the semantic diversity of generated responses. |
| Approach: | They propose an automatic evaluation metric to measure the semantic diversity of generated responses . they show that it captures human judgments better than existing diversity metrics . |
| Outcome: | The proposed metric captures human judgments on response diversity better than existing lexical diversity metrics. |
Plug-and-Play VQA: Zero-shot VQA by Conjoining Large Pretrained Models with Zero Training (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches require substantial adaptation of pretrained language models for vision-language reasoning tasks. |
| Approach: | They propose to use natural language and network interpretation as an intermediate representation that glues pretrained models together. |
| Outcome: | The proposed framework outperforms the Flamingo model on VQAv2 and GQA by 8.5%. |
TSGP: Two-Stage Generative Prompting for Unsupervised Commonsense Question Answering (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies focus on acquiring relevant knowledge by retrieving external knowledge bases and fine-tuning pre-trained models. |
| Approach: | They propose a two-stage prompt-based unsupervised commonsense question answering framework that leverages implicit knowledge stored in PrLMs to generate knowledge for questions with unlimited types and possible candidate answers independent of specified choices. |
| Outcome: | The proposed framework significantly improves the reasoning ability of language models in unsupervised settings. |
Subword-Delimited Downsampling for Better Character-Level Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Subword-level models are expensive in terms of time and computation, but character-level model with downsampling component can be used for machine translation. |
| Approach: | They propose a character-level downsampling method which is informed by subwords to improve model performance. |
| Outcome: | The proposed method outperforms existing methods and shows that it can be done without sacrificing quality. |
Autoregressive Structured Prediction with Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent years have seen a paradigm shift in NLP towards using pretrained language models for a wide range of tasks. |
| Approach: | They propose to model structures as sequences of actions in autoregressive manner with PLMs . their approach allows in-structure dependencies to be learned without any loss . |
| Outcome: | The proposed approach achieves state-of-the-art on all structured prediction tasks. |
XDoc: Unified Pre-training for Cross-Format Document Understanding (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing pre-trained models target one document format at a time, making it difficult to combine knowledge from multiple document formats. |
| Approach: | They propose a unified pre-trained model which deals with different document formats in a single model. |
| Outcome: | The proposed model achieves comparable or even better performance on a variety of downstream tasks compared with the individual pre-trained models. |
A Few More Examples May Be Worth Billions of Parameters (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work on few-shot learning for natural language tasks explores the dynamics of scaling up either the number of model parameters or labeled examples while controlling for the other variable by setting it to a constant. |
| Approach: | They explore the dynamics of scaling up the number of model parameters versus the number labeled examples across a wide variety of tasks. |
| Outcome: | The results show that scaling parameters yields performance improvements, while adding examples does not. |
MCP: Self-supervised Pre-training for Personalized Chatbots with Multi-level Contrastive Sampling (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies focus on generating implicit user profiles from the user’s dialogue history, thus it suffers from data sparsity and performance degradation. |
| Approach: | They propose a self-supervised learning framework MCP for capturing better representations from users’ dialogue history for personalized chatbots. |
| Outcome: | The proposed model improves on two real-world datasets. |
ExpertPLM: Pre-training Expert Representation for Expert Finding (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to learn expert representations based on historical answered questions are inadequate. |
| Approach: | They propose a pre-training and fine-tuning expert finding framework that integrates expert interest and expertise simultaneously. |
| Outcome: | The proposed framework integrates expert interest and expertise simultaneously and integrates historical answers with the vote score information to model the expert ability. |
You Truly Understand What I Need : Intellectual and Friendly Dialog Agents grounding Persona and Knowledge (2022.findings-emnlp)
Copied to clipboard
Jungwoo Lim, Myunghoon Kang, Yuna Hur, Seung Won Jeong, Jinsung Kim, Yoonna Jang, Dongyub Lee, Hyesung Ji, DongHoon Shin, Seungryong Kim, Heuiseok Lim
| Challenge: | Existing models that ground knowledge and persona at the same time are limited, leading to hallucination and a passive way of using personas. |
| Approach: | They propose a conversational agent that grounds external knowledge and persona simultaneously and a retrieval augmented generation model that generates utterances with lesser hallucination and more engagingness. |
| Outcome: | The proposed agent generates the utterance with lesser hallucination and more engagingness utilizing retrieval augmented generation with knowledge-persona enhanced query. |
Faithful to the Document or to the World? Mitigating Hallucinations via Entity-Linked Knowledge in Abstractive Summarization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing abstractive summarization systems are hampered by content hallucinations in which models generate text that is not directly inferable from the source alone. |
| Approach: | They propose to use external knowledge to latently connect entities and concepts to latences to lend provenance to many of these unfaithful yet factual entities. |
| Outcome: | The proposed model can be used to improve the factuality of summarizations without simply making them more extractive. |
RL with KL penalties is better viewed as Bayesian inference (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Reinforcement learning (RL) is used in fine-tuning large language models to penalize them for undesirable features of generated sequences. |
| Approach: | They analyze challenges associated with treating a language model as an RL policy . they find that RL is equivalent to variational inference: approximating a Bayesian posterior . |
| Outcome: | The proposed approach is flawed because it turns the LM into a degenerate distribution, the authors show . they show that the proposed approach avoids the distribution collapse problem and offers a first-principles derivation for its objective. |
Evaluating Token-Level and Passage-Level Dense Retrieval Models for Math Information Retrieval (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study has shown that dense retrieval methods are suboptimal for capturing contextual similarities in complex data. |
| Approach: | They propose to combine a structure search method and efficient bi-encoder dense retrieval models to capture contextual similarities. |
| Outcome: | The proposed model improves on token-level and passage-level dense retrieval tasks. |
Multi-View Reasoning: Consistent Contrastive Learning for Math Word Problem (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for generating complex semantics and diverse equations are limited by a fixed view. |
| Approach: | They propose a multi-view consistent contrastive learning approach that decouples human reasoning into two independent but consistent views. |
| Outcome: | The proposed approach significantly outperforms existing baselines on complex problems on multiple languages. |
Few-shot initializing of Active Learner via Meta-Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in few-shot and zero-shot learning have limited performance in domain specific applications. |
| Approach: | They propose to initialize an active learner with meta-learned parameters and generate task dependent softmax weights for active learning. |
| Outcome: | The proposed method performs better than the baseline at low budget, the authors show . they show that adding meta-learned learning rates and generating the softmax have negative consequences . |
Bootstrapping meaning through listening: Unsupervised learning of spoken sentence embeddings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a new study addresses the challenge of learning semantic representations from speech signals . speech-based semantic representation can be used for speech mining and spoken language understanding . |
| Approach: | They propose a multimodal sequential autoencoder that converts speech signals into hidden units . they propose s-HuBERT to induce meaning through knowledge distillation . |
| Outcome: | The proposed model achieves a moderate correlation with human judgments without labels or transcriptions. |
Progressive Sentiment Analysis for Code-Switched Text Data (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Multilingual transformer language models are used in cross-lingual transfer learning for many NLP tasks such as text classification and named entity recognition. |
| Approach: | They propose a framework that takes the distinction between resource-rich and low-resource language into account and progressively trains from resource-dominated to low-rsource samples. |
| Outcome: | The proposed model outperforms existing methods on low-resource languages and performs well on medium-resourced to high-res languages. |
Knowledge Stimulated Contrastive Prompting for Low-Resource Stance Detection (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Stance Detection Tasks require background knowledge especially when there is no explicit target mentioned in text. |
| Approach: | They propose a masked language prompt joint contrastive learning approach to stimulate the knowledge inherit from pre-trained models. |
| Outcome: | The proposed model is effective in stance detection on three benchmarks. |
WSpeller: Robust Word Segmentation for Enhancing Chinese Spelling Check (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Chinese spelling check (CSC) detects and corrects spelling errors in Chinese texts. |
| Approach: | They propose a Chinese spelling check model that takes into account word segmentation and a module that can assist the correction module by predicting correct word segmentations from sentences containing spelling errors. |
| Outcome: | The proposed model outperforms baselines on SIGHAN13, SIGHEN14, and SIGHAN15 and maintains equal performance on SSGHAN14. |
Extracting Trigger-sharing Events via an Event Matrix (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to extract multiple events with triggers and arguments are invalid as there may be multiple events. |
| Approach: | They propose a framework for event extraction which models the relations between arguments by an event matrix. |
| Outcome: | The proposed framework beats all the advanced competitors on 3 widely-used datasets. |
TranS: Transition-based Knowledge Graph Embedding with Synthetic Relation Representation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Knowledge graph embedding (KGE) is a computational approach to learn continuous vector representations of relations and entities in knowledge graphs. |
| Approach: | They propose a transition-based method to learn continuous vector representations of relations and entities in knowledge graph (KG) it replaces a single relation vector in the relation part with a synthetic relation representation with entity-relation interactions to solve these problems. |
| Outcome: | The proposed method achieves state-of-the-art on a large knowledge graph dataset. |
Sequential Topic Selection Model with Latent Variable for Topic-Grounded Dialogue (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing topic-grounded dialogue systems focus on the current conversation and corresponding topic sequence to predict the next topic. |
| Approach: | They propose a new approach to exploit topic-guided conversations to better model post-to-response topic-transition and guide the response generation to the current conversation. |
| Outcome: | The proposed model outperforms baselines on prediction and generation tasks. |
Robust Task-Oriented Dialogue Generation with Contrastive Pre-training and Adversarial Filtering (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Task-oriented dialogue models can learn non-transferable generalizations by using shortcuts in the data. |
| Approach: | They propose a contrastive learning framework to encourage models to ignore cues and focus on generalisable patterns. |
| Outcome: | The proposed framework performs exceptionally well on task-oriented dialogue datasets. |
STAR: SQL Guided Pre-Training for Context-dependent Text-to-SQL Parsing (2022.findings-emnlp)
Copied to clipboard
Zefeng Cai, Xiangyu Li, Binyuan Hui, Min Yang, Bowen Li, Binhua Li, Zheng Cao, Weijie Li, Fei Huang, Luo Si, Yongbin Li
| Challenge: | Extensive experiments show that STAR outperforms previous pre-training methods and ranks first on the leaderboard . text-to-SQL parsing aims to translate natural language (NL) questions into executable SQL queries . |
| Approach: | They propose a SQL guided pre-training framework STAR for context-dependent text-to-SQL parsing . they propose two objectives that explore context-dependence of NL utterances and SQL queries . |
| Outcome: | The proposed framework outperforms existing methods on two downstream benchmarks and ranks first on the leaderboard. |
Is MultiWOZ a Solved Task? An Interactive TOD Evaluation Framework with User Simulator (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Task-oriented dialogue systems are drawing more attention in recent studies . current evaluation methods use annotated utterances in multi-turn dialogue sessions . |
| Approach: | They propose an interactive evaluation framework for task-oriented dialogue (TOD) they build a goal-oriented user simulator based on pre-trained models and use it to generate dialogues . |
| Outcome: | The proposed evaluation framework achieves 98% inform and success rates in the multi-woz dataset and the proposed scores measure the response quality besides the inform and successful rates. |
Translating Hanja Historical Documents to Contemporary Korean and English (2022.findings-emnlp)
Copied to clipboard
| Challenge: | The Annals of Joseon Dynasty contain the daily records of the Kings of Joseont, the 500-year kingdom preceding the modern nation of Korea. |
| Approach: | They propose a neural machine translation model that translates historical documents written in Hanja to more easily understandable Korean and to English. |
| Outcome: | The proposed model outperforms baseline models in terms of BLEU scores for both contemporary Korean and English translations. |
Exploring Compositional Image Retrieval with Hybrid Compositional Learning and Heuristic Negative Mining (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing CIR models are pre-trained on uni-modal data, resulting in unimodal data. |
| Approach: | They propose a CIR model HyCoLe-HNM with CLIP as the backbone . they use a gated fusion mechanism from a question answering model to perform compositional learning . |
| Outcome: | The proposed model achieves state-of-the-art performance on three CIR datasets . it borrows a gated fusion mechanism from a question answering model to perform compositional fusion . |
Outlier Dimensions that Disrupt Transformers are Driven by Frequency (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Disabling only 48 out of 110M parameters in BERT-base drops its performance by nearly 30% on MNLI. |
| Approach: | They find that disabling only 48 out of 110M parameters in BERT-base drops its performance by nearly 30% on MNLI. |
| Outcome: | The proposed model outlier phenomenon is associated with the frequency of encoded tokens in pre-training data. |
MiST: a Large-Scale Annotated Resource and Neural Models for Functions of Modal Verbs in English Scientific Text (2022.findings-emnlp)
Copied to clipboard
| Challenge: | modal verbs are used for hedges, but they may also denote abilities and restrictions in scientific texts . modals are often used for hedging, but prior work on this topic has been limited . |
| Approach: | They propose a dataset that contains 3737 modal instances in five scientific domains . they evaluate a set of competitive neural architectures to model the distinctions in MIST . |
| Outcome: | The proposed dataset contains 3737 modal instances in five scientific domains . leveraging non-scientific data is of limited benefit for modeling the distinctions in MIST . |
Late Prompt Tuning: A Late Prompt Could Be Better Than Many Prompts (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Prompt tuning is parameter-efficient but lags behind other state-of-the-art methods. |
| Approach: | They propose a parameter-efficient tuning method that only optimizes a soft prompt to adapt PTMs to downstream tasks. |
| Outcome: | The proposed method is parameter-efficient but lags behind other state-of-the-art methods. |
MICO: A Multi-alternative Contrastive Learning Framework for Commonsense Knowledge Representation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to commonsense reasoning include fine-tuning large pre-trained language models or injecting the entire knowledge base for CKGC. |
| Approach: | They propose to learn commonsense knowledge representation by using a multi-alternative contrastive learning framework on COmmonsense Knowledge graphs. |
| Outcome: | Extensive experiments show that the proposed framework is effective in commonsense reasoning tasks. |
Leveraging Only the Category Name for Aspect Detection through Prompt-based Constrained Clustering (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Aspect category detection (ACD) aims to automatically identify user-concerned aspects from online reviews. |
| Approach: | They propose a method that relies on the category name of each aspect and a pretrained language model to generate constraints for clustering. |
| Outcome: | The proposed framework performs better than existing weakly supervised methods on nine benchmark datasets. |
Controllable Factuality in Document-Grounded Dialog Systems Using a Noisy Channel Model (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent document-grounded dialog systems have seen an increase in popularity. |
| Approach: | They propose a model for document-grounded response generation in dialog that is decomposed into two components according to Bayes’ theorem and propose different approximate decoding schemes. |
| Outcome: | The proposed model is more factual in terms of automatic factuality metrics than the baseline model and can be combined with a recently proposed method to control factuity in grounded dialog, CTRL. |
Transformer Language Models without Positional Encodings Still Learn Positional Information (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Using positional embeddings, Causal transformer language models learn an implicit notion of absolute positions. |
| Approach: | They propose to use positional embeddings to encode positional information in transformer language models. |
| Outcome: | The proposed model learns an implicit notion of absolute positions across datasets, model sizes, and sequence lengths. |
Beyond Model Interpretability: On the Faithfulness and Adversarial Robustness of Contrastive Textual Explanations (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing explanations address the contrastive aspect of explanations but their extension to textual data is under-explored and there is little investigation on their vulnerabilities and limitations. |
| Approach: | They propose a novel evaluation scheme inspired by the faithfulness of explanations by extending the computation of three metrics to textual data and benchmarking POLYJUICE and MiCE on suggested metrics. |
| Outcome: | The proposed methods demonstrate that the connectedness of counterfactuals to their original counterparts is not obvious in both models. |
How Much Does Attention Actually Attend? Questioning the Importance of Attention in Pretrained Transformers (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained language models use the attention mechanism to contextualize input inputs . but, we find that it is not as important as thought for pretrained models . |
| Approach: | They propose a probing method that replaces input-dependent attention matrices with constant ones. |
| Outcome: | The proposed method improves performance of pretrained language models without input-dependent attention. |
What Has Been Enhanced in my Knowledge-Enhanced Language Model? (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge integration methods such as linear probes and prompts have key limitations in answering these questions. |
| Approach: | They propose a new probe model which integrates external knowledge from knowledge graphs into pretrained language models (LMs) ERNIE and K-Adapter are proposed as KI methods . |
| Outcome: | The proposed model interprets two well-known KELMs using graph attention on the corresponding knowledge graph for interpretation. |
Towards Generalized Open Information Extraction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Open Information Extraction (OpenIE) models are evaluated on in-domain test sets aside from the training corpus, which violates the initial task principle of domain-independence. |
| Approach: | They propose to generalize OpenIE over unseen target domains with different data distributions from source training domains. |
| Outcome: | The proposed method beats the previous methods in both in- and out-of-domain settings by 6.0% in F1 score absolutely. |
BioLORD: Learning Ontological Representations from Definitions for Biomedical Concepts and their Textual Descriptions (2022.findings-emnlp)
Copied to clipboard
| Challenge: | BioLORD is a pre-training strategy for producing meaningful representations for clinical sentences and biomedical concepts. |
| Approach: | They propose a pre-training strategy for producing meaningful representations for clinical sentences and biomedical concepts using definitions and ontologies. |
| Outcome: | The proposed model produces more semantic representations that match more closely the hierarchical structure of ontologies. |
Improving the Extraction of Supertags for Constituency Parsing with Linear Context-Free Rewriting Systems (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a new approach to parsing discontinuous constituency structures uses supertags to improve parsability . traditional approaches use grammar formalisms to model hierarchies of noncontiguous phrases . but supertags are still useful for analyzing these grammars and parsers . |
| Approach: | They propose to reformulate and parameterize extraction process for LCFRS supertags to improve parsing quality. |
| Outcome: | The proposed method improves the quality and speed of parsing with supertags over the previous method. |
Mask More and Mask Later: Efficient Pre-training of Masked Language Models by Disentangling the [MASK] Token (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Large-scale pre-trained MLMs can be used to generalize well to a wide range of tasks. |
| Approach: | They propose to append [MASK]s at a later layer to reduce sequence length for earlier layers. |
| Outcome: | The proposed method outperforms RoBERTa for 6 out of 8 GLUE tasks on average by 0.4%. |
SMSMix: Sense-Maintained Sentence Mixup for Word Sense Disambiguation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Word Sense Disambiguation (WSD) is an NLP task aimed at determining the correct sense of a word in a sentence from discrete sense choices. |
| Approach: | They propose a word-level mixup method that maintains the sense of a target word by increasing the frequency of least frequent senses during training to reduce the distributional bias of senses. |
| Outcome: | The proposed method shows high accuracy on the most frequent sense (MFS) and low accuracy on least frequent senses (LFS). |
On the Effectiveness of Automated Metrics for Text Generation Systems (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evaluation methods lack a sound theoretical foundation for evaluation campaigns . imperfect automated metrics and insufficiently sized test sets are some of the factors that cause uncertainty. |
| Approach: | They propose a theoretical framework that incorporates different sources of uncertainty, such as imperfect automated metrics and insufficiently sized test sets. |
| Outcome: | The proposed model can be leveraged to improve evaluation protocols regarding reliability, robustness, and significance of the evaluation outcome. |
Residual Learning of Neural Text Generation with n-gram Language Model (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results show that n-gram models can achieve satisfactory performance on a large proportion of testing cases. |
| Approach: | They propose to learn a neural LM that fits the residual between an n-gram LM and the real-data distribution. |
| Outcome: | The proposed model achieves additional performance gains over popular standalone models on three typical language tasks. |
DiffG-RL: Leveraging Difference between Environment State and Common Sense (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to solving text-based games require background knowledge as the context is important. |
| Approach: | They propose a novel agent that organizes environment states and common sense by interactive objects with a dedicated graph encoder. |
| Outcome: | The proposed agent outperforms baselines in text-based games by 17% of scores. |
Unsupervised Syntactically Controlled Paraphrase Generation with Abstract Meaning Representations (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to syntactically controlled paraphrase generation require annotated paraphrase pairs for training and are costly to extend to new domains. |
| Approach: | They propose to leverage Abstract Meaning Representations (AMR) to improve the performance of unsupervised syntactically controlled paraphrase generation. |
| Outcome: | The proposed model generates more accurate syntactically controlled paraphrases, both quantitatively and qualitatively, compared to the existing unsupervised approaches. |
Can AMR Assist Legal and Logical Reasoning? (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) has been shown to be useful for many downstream tasks. |
| Approach: | They propose neural architectures that utilize linearised AMR graphs in combination with pre-trained language models to capture logical relationships on multiple choice question answering tasks. |
| Outcome: | The proposed models outperform text-only baselines but outperformed text models, suggesting complementary abilities. |
Data Selection Curriculum for Neural Machine Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Neural Machine Translation models are typically trained on heterogeneous data that are concatenated and randomly shuffled. |
| Approach: | They propose a two-stage curriculum training framework where a NMT model is fine-tuned on subsets of data, selected by deterministic scoring and online scoring. |
| Outcome: | The proposed framework improves on six language pairs comprising low- and high-resource languages and shows up to +2.2 BLEU improvement and faster convergence. |
Text Editing as Imitation Game (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Text editing is an important domain of processing tasks to edit the text in a localized fashion, such as text simplification. |
| Approach: | They propose a nonautoregressive decoder for state-to-action demonstrations that parallels the decoding while retaining the dependencies between tokens. |
| Outcome: | The proposed model outperforms the autoregressive baselines on a suite of Arithmetic Equation benchmarks in terms of performance, efficiency, and robustness. |
Seeded Hierarchical Clustering for Expert-Crafted Taxonomies (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Practitioners from many disciplines use expert-crafted taxonomies to make sense of large, unlabeled corpora. |
| Approach: | They propose a weakly supervised algorithm for seeded hierarchical clustering that fits unlabeled data to taxonomies using a small set of labeled examples. |
| Outcome: | The proposed algorithm outperforms baselines on three real-world datasets. |
Knowledge Graph Generation From Text (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for generating text from text are limited due to non-unique graph representation, complex node structure, large output spaces and limited parallel training data. |
| Approach: | They propose a novel end-to-end multi-stage Knowledge Graph generation system from textual inputs that separates the overall process into two stages. |
| Outcome: | The proposed system outperforms existing methods on a WebNLG 2020 Challenge dataset and on TekGen datasets. |
DialogueGAT: A Graph Attention Network for Financial Risk Prediction by Modeling the Dialogues in Earnings Conference Calls (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models focus on extracting useful semantic information from conference call transcripts but ignore subtle yet important information of dialogue structures. |
| Approach: | They propose a graph attention network called DialogueGAT for financial risk prediction by simultaneously modeling the speakers and their utterances in conference calls. |
| Outcome: | The proposed model outperforms baseline models on a dataset of S&P1500 companies. |
Investigating Ensemble Methods for Model Robustness Improvement of Text Classifiers (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to reduce model's reliance on bias features ignore the learnability of these features. |
| Approach: | They propose to reduce models' reliance on bias features by first training models with fixed low-capacity models which ignore the learnability of the bias features. |
| Outcome: | The proposed models can perform better on out-of-distribution datasets than baseline models with a more sophisticated model design. |
Adaptive Ranking-based Sample Selection for Weakly Supervised Class-imbalanced Text Classification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to synthesize training labels with labeling rules ignore data imbalance issue . weak supervision paradigm is often used to reduce human efforts to produce training labels inexpensively. |
| Approach: | They propose a model-agnostic framework to alleviate the data imbalance issue in the weak supervision paradigm by combining labeling rules with a probabilistic margin score. |
| Outcome: | The proposed framework outperforms the state-of-the-art imbalanced learning and WS methods on four text classification datasets with four different imbalance ratios. |
ComFact: A Benchmark for Linking Contextual Commonsense Knowledge (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to retrieve facts from commonsense knowledge graphs are imprecise, requiring heuristics that ignore contexts and ambiguity . a novel benchmark, ComFact, contains 293k in-context relevance annotations for commonsensense triplets . |
| Approach: | They propose a task of commonsense fact linking where models are given contexts and trained to identify situationally-relevant commonsensical knowledge from KGs. |
| Outcome: | The proposed benchmark shows that heuristic fact linking approaches are imprecise . however, the models still significantly underperform humans in the commonsense augmentation task . |
Learning to Perform Complex Tasks through Compositional Fine-Tuning of Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work on how to encode compositional task structure has been limited by semantic parsing and multihop reasoning for the purpose of Q&A. |
| Approach: | They propose an approach to decomposing a target task into component tasks and fine-tuning smaller LMs on a curriculum of such component tasks. |
| Outcome: | The proposed approach outperforms end-to-end learning even with equal data, and gets better as more component tasks are modeled. |
Topic Taxonomy Expansion via Hierarchy-Aware Topic Phrase Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for topic taxonomies focus on frequent terms and local topic-subtopic relations, which leads to limited topic term coverage. |
| Approach: | They propose a framework for topic taxonomy expansion that directly generates topic-related terms belonging to new topics. |
| Outcome: | The proposed framework outperforms baseline methods on two real-world text corpora. |
Language as a fingerprint: Self-supervised learning of user encodings using transformers (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on individual differences and language representations focused on predicting selected attributes from text or conditioning text representations on author attributes. |
| Approach: | They propose a self-supervised approach to learning language-based user encodings using transformers. |
| Outcome: | The proposed model can pick up on complex linguistic signatures of users and infer rich information about them. |
Hyperdecoders: Instance-specific decoders for multi-task NLP (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work in NLP has examined the performance of large pretrained transformer-based models in multi-task settings, where a single model is evaluated on multiple tasks simultaneously. |
| Approach: | They propose a method for multi-tasking using a hypernetwork conditioned on the output of an encoder to generate a unique decoder adaptation for every input instance. |
| Outcome: | The proposed method outperforms previous methods for efficient multi-task fine-tuning and maps from encoder representations to output labels. |
Evaluating the Faithfulness of Importance Measures in NLP by Recursively Masking Allegedly Important Tokens and Retraining (2022.findings-emnlp)
Copied to clipboard
| Challenge: | To explain NLP models, importance measures are often used to inform input tokens are important for making a prediction. |
| Approach: | They propose a faithfulness metric that masks allegedly important tokens and retrains the model. |
| Outcome: | The proposed metric is based on LSTM-attention models and RoBERTa models. |
Towards Explaining Subjective Ground of Individuals on Social Media (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Large-scale language models have been reducing the gap between machines and humans in understanding how the real world works, yet understanding an individual’s perspective and behavior from text is far from being resolved. |
| Approach: | They propose a neural model that learns subjective grounds of individuals and accounts for their judgments on situations posted on social media. |
| Outcome: | The proposed model provides human-readable explanations of an individual’s subjective preference in judging social situations and also learns an individual's orientation towards abstract moral concepts. |
Knowledge Injected Prompt Based Fine-tuning for Multi-label Few-shot ICD Coding (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for ICD coding are limited due to the high-dimensional space of multi-label assignment and the long-tail challenge. |
| Approach: | They propose a prompt-based fine-tuning technique with label semantics to solve this challenge. |
| Outcome: | The proposed method outperforms state-of-the-art methods on a benchmark dataset of code assignment in 14.5% of cases. |
Do Language Models Understand Measurements? (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on numerical reasoning over text (NRoT) tests PLMs to understand numbers in contexts where numbers are an integral part of the context. |
| Approach: | They propose a simple embedding strategy to better distinguish between numbers and units, which leads to a significant improvement in probing tasks. |
| Outcome: | The proposed model distinguishes between numbers and units, which leads to significant improvement in probing tasks. |
Reconciliation of Pre-trained Models and Prototypical Neural Networks in Few-shot Named Entity Recognition (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods do not consider that pre-trained models contain a prominently large amount of information regarding word frequencies, thus biasing prototypical neural networks against learning word entities. |
| Approach: | They propose a one-line-code normalization method to reconcile such a mismatch with empirical and theoretical grounds and propose 'references' for the model enhancement. |
| Outcome: | The proposed method outperforms the state-of-the-art models on nine benchmark datasets and is comparable to the state of the art. |
HCL-TAT: A Hybrid Contrastive Learning Method for Few-shot Event Detection with Task-Adaptive Threshold (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for event detection have failed to address the problem of constantly emerging event types with limited data. |
| Approach: | They propose a novel method for event detection with a task-adaptive threshold . they propose to learn discriminative representations with 'two-view contrastive loss' |
| Outcome: | The proposed method achieves better results than the state-of-the-art methods on a benchmark dataset. |
Doc2Bot: Accessing Heterogeneous Documents via Conversational Bots (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Documents contain various structures that hinder the ability of machines to comprehend . user information needs are often underspecified, and the nature of heterogeneous documents poses challenges. |
| Approach: | They propose a dataset for building machines that help users seek information via conversations . their dataset contains over 100,000 turns based on Chinese documents from five domains . |
| Outcome: | The proposed tasks are challenging and worthy of further research. |
DualNER: A Dual-Teaching framework for Zero-shot Cross-lingual Named Entity Recognition (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to named entity recognition (NER) are limited to high-resource languages like English and Chinese. |
| Approach: | They propose a framework to make full use of annotated source and unlabeled target language text for zero-shot cross-lingual named entity recognition. |
| Outcome: | The proposed framework makes full use of both annotated source and unlabeled target language text for zero-shot cross-lingual named entity recognition (NER). |
Knowledge-augmented Self-training of A Question Rewriter for Conversational Knowledge Base Question Answering (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent rise of conversational applications has promoted the development of conversation KBQA (ConvKBQA). |
| Approach: | They propose a framework to produce a full-fledged rewritten question based on conversation history and then reason the answer by existing single-turn KBQA models. |
| Outcome: | The proposed framework produces a full-fledged rewritten question based on the conversation history and reasoned the answer by existing single-turn KBQA models. |
Extractive Summarization of Legal Decisions using Multi-task Learning and Maximal Marginal Relevance (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Summarizing legal decisions requires the expertise of law practitioners, which is time- and cost-intensive. |
| Approach: | They propose methods for extracting summarized legal decisions using limited expert annotated data. |
| Outcome: | The proposed models achieve ROUGE scores vis-à-vis expert extracted summaries that match inter-annotator comparisons. |
MovieUN: A Dataset for Movie Understanding and Narrating (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Automatic movie narration generation and narration grounding are important to provide a true movie experience for the blind and visually impaired. |
| Approach: | They propose to use movie clips as a benchmark to support automatic movie narration generation and narration grounding tasks. |
| Outcome: | The proposed methods are effective in supporting two movie-based tasks for the blind and visually impaired. |
ASDOT: Any-Shot Data-to-Text Generation with Pretrained Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to data-to-text generation require limited training examples . a data-based approach is based on a set of pre-trained language models with optional finetuning. |
| Approach: | They propose a data-to-text generation task that makes use of any given (or no) examples. |
| Outcome: | The proposed approach improves on baselines on a dataset with zero/few/full-shot settings. |
FCGEC: Fine-Grained Corpus for Chinese Grammatical Error Correction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | grammatical error correction (GEC) is a complex task that requires high-quality data from native speakers. |
| Approach: | They propose a human-annotated corpus to detect, identify and correct grammatical errors in Chinese examinations. |
| Outcome: | The proposed model outperforms other models in low-resource settings, but there is a significant gap between the models and humans that encourages future models to bridge it. |
Audience-Centric Natural Language Generation via Style Infusion (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to text style transfer (TST) with large volumes of parallel or non-parallel data are limiting for two reasons: it is difficult to collect large volumes and some stylistic objectives are hard to define without audience feedback. |
| Approach: | They propose a task of style infusion - infusing stylistic preferences of audiences into pretrained language generation models by leveraging pairwise human judgments to bootstrap a style analysis model and augment a seed set of judgments. |
| Outcome: | The proposed method generates compelling stylized examples with generic text prompts while balancing fluency and style adoption. |
DocFin: Multimodal Financial Prediction and Bias Mitigation using Semi-structured Documents (2022.findings-emnlp)
Copied to clipboard
Puneet Mathur, Mihir Goyal, Ramit Sawhney, Ritik Mathur, Jochen Leidner, Franck Dernoncourt, Dinesh Manocha
| Challenge: | Existing research focuses on textual and audio modalities of financial disclosures but ignores the rich tabular data available in financial reports. |
| Approach: | They propose to combine tabular financial data with text transcripts and audio recordings to improve stock volatility and price movement prediction by 5-12% and reduce gender bias by over 30%. |
| Outcome: | The combined data improves stock volatility and price movement prediction by 5-12% and reduces gender bias caused due to audio-based neural networks by over 30%. |
Not Just Plain Text! Fuel Document-Level Relation Extraction with Explicit Syntax Refinement and Subsentence Modeling (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Document-level relation extraction (DocRE) aims to identify semantic labels among entities within a document. |
| Approach: | They propose a document-level relation extraction framework that captures and exploits instructive information by adding extra syntactic information into text representations. |
| Outcome: | The proposed framework outperforms existing methods on three benchmark datasets. |
Self-supervised Rewiring of Pre-trained Speech Encoders:
Towards Faster Fine-tuning with Less Labels in Speech Processing (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained speech encoders have facilitated great success across various speech processing tasks, but fine-tuning them for downstream tasks requires large training data to converge or to achieve state-of-the-art. |
| Approach: | They propose to rewire pre-trained speech encoders to improve their representation space without task-specific labels by neutrally synthesising audio inputs and frame masking. |
| Outcome: | The proposed model shows consistent improvement in isotropy in the representation space on 6 speech processing tasks. |
RedApt: An Adaptor for wav2vec 2 EncodingFaster and Smaller Speech Translation without Quality Compromise (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained speech Transformers in speech translation systems have facilitated state-of-the-art (SotA) results, but their computational cost is high. |
| Approach: | They propose a Reducer Adaptor block that could be seamlessly integrated within any Transformer-based speech encoding architecture. |
| Outcome: | The proposed Reducer Adaptor block outperforms the existing SotA architecture by an average of 0.68 BLEU score on 8 language pairs from Must-C. |
How sensitive are translation systems to extra contexts? Mitigating gender bias in Neural Machine Translation models through relevant contexts. (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Neural Machine Translation systems are prone to gender biases in their learned representations. |
| Approach: | They propose to use contextual sentences to correct gender bias in Neural Machine Translation models. |
| Outcome: | The proposed method can be used to build better, bias-free translation systems. |
PM2F2N: Patient Multi-view Multi-modal Feature Fusion Networks for Clinical Outcome Prediction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods focused on time series data but ignored clinical notes . fusion of multi-modal features of patients from different views is not feasible due to the time series and clinical notes data being stored as time series. |
| Approach: | They propose to combine time series and clinical notes to fuse multi-modal features of patients from different perspectives using graph neural networks. |
| Outcome: | The proposed method is superior to existing models on MIMIC-III benchmark. |
Long Text and Multi-Table Summarization: Dataset and Method (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing document summarization methods focus on the text and filter out the non-textual content. Existing methods cannot meet the requirements of summarizing long text and multiple tables in each report. |
| Approach: | They propose a dataset for automatic document summarization that uses text and tabular data to produce a concise summary covering the input document's salient information. |
| Outcome: | The proposed method can produce a concise summary covering the input document's salient information. |
MatRank: Text Re-ranking by Latent Preference Matrix (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for text ranking have improved performance, but there are still challenges. |
| Approach: | They propose a method that learns to re-rank the text retrieved for a given query by learning to predict the most relevant passage based on a latent preference matrix. |
| Outcome: | The proposed method outperforms all prior methods on datasets with extensive results. |
Can Language Models Serve as Temporal Knowledge Bases? (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have only considered language models as knowledge bases in a static setting . memorizing conflicting information is still challenging for LMs and hinders memorization of other unrelated one-to-one relationships. |
| Approach: | They propose two requirements for treating language models as temporal knowledge bases . they propose a dataset which is aimed at probing temporally-scoped knowledge . |
| Outcome: | The proposed model can store conflicting information and use stored knowledge for temporal knowledge queries. |
Are Large Pre-Trained Language Models Leaking Your Personal Information? (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) are prone to leaking personal information due to memorization, but the risk of specific personal information being extracted by attackers is low. |
| Approach: | They analyze whether large pre-trained language models are prone to leaking personal information due to memorization. |
| Outcome: | The proposed model is weak at association, so the risk of specific personal information being extracted by attackers is low. |
Self-Distillation with Meta Learning for Knowledge Graph Completion (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge graph completion frameworks for knowledge graphs are far from complete and require missing triples to be added to them. |
| Approach: | They propose a dynamic pruning technique to obtain a pruned model from a large source model, where the pruning mask of the pruned models could be updated adaptively per epoch after the model weights are updated. |
| Outcome: | The proposed framework achieves competitive performance compared to strong baselines, while being 10x smaller than baselines. |
CQR-SQL: Conversational Question Reformulation Enhanced Context-Dependent Text-to-SQL Parsers (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing text-to-SQL methods focus on making full use of history context, but neglect to explicitly comprehend the schema and conversational dependency. |
| Approach: | They propose a CQR-SQL that explicitly exploits schema and conversational dependency for multi-turn SQL parsing. |
| Outcome: | The proposed method exploits schema and contextual dependency for multi-turn SQL parsing. |
Assisting the Human Fact-Checkers: Detecting All Previously Fact-Checked Claims in a Document (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent years have brought us a proliferation of false claims online, which spread fast . fact-checkers have been using automated fact-finding to verify claims . |
| Approach: | They propose a system that can detect claims that can be fact-checked by a given database . they create a manually annotated document dataset and propose evaluation measures . |
| Outcome: | The proposed system achieves sizable performance gains over strong baselines. |
No Word Embedding Model Is Perfect: Evaluating the Representation Accuracy for Social Bias in the Media (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work has relied on word embedding bias measures, such as WEAT, but these methods can be inaccurate due to several representation issues, such low-resource settings and token frequency differences. |
| Approach: | They propose to use WEAT to quantify social bias in US online news articles and embed embedding algorithms to account for the aforementioned issues. |
| Outcome: | The proposed algorithms do not match the literature, but they reduce the gap. |
Scientific and Creative Analogies in Pretrained Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing analogy datasets focus on a limited set of analogical relations with a high similarity of the two domains between which the analogy holds. |
| Approach: | They propose a dataset that encodes analogy in pretrained language models . they use a system that maps attributes and relational structures across dissimilar domains . |
| Outcome: | The proposed dataset shows that state-of-the-art models achieve low performance on analogy tasks . |
Bitext Mining Using Distilled Sentence Representations for Low-Resource Languages (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a new study aims to extend multilingual representation learning beyond the hundred most frequent languages . current work on multilingual sentence representations has focused on training one model which handles all languages of interest . |
| Approach: | They propose a teacher-student approach to extend existing monolingual sentence embedding space to new languages. |
| Outcome: | The proposed model outperforms the original LASER encoder in 44 African languages . the model can be used to train multiple languages and learn new languages if they have the same training data . |
Towards Generalizable and Robust Text-to-SQL Parsing (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Text-to-SQL parsers must be generalizable and robust against input perturbations. |
| Approach: | They propose a novel framework to learn text-to-SQL parsing in stages to improve parser's ability to acquire general SQL knowledge instead of capturing spurious patterns. |
| Outcome: | The proposed framework achieves state-of-the-art performance on the Spider, SParC, and CoSQL datasets. |
EdiT5: Semi-Autoregressive Text Editing with T5 Warm-Start (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained seq2seq models have established strong baselines for text-to-text transduction tasks. |
| Approach: | They propose a semi-autoregressive text-editing approach that combines the strengths of non-auto-regressively text- editing and autoregressive decoding. |
| Outcome: | The proposed model is faster at inference times than conventional models while being capable of modeling flexible input-output transformations. |
A Critical Reflection and Forward Perspective on Empathy and Natural Language Processing (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Empathy recognition and empathetic response generation tasks are well-established research directions, but there is little clarity on what empathy is and how it is being operationalized. |
| Approach: | They argue that current directions will benefit from a clear conceptualization that includes operationalizing cognitive empathy components. |
| Outcome: | The proposed framework will help to define and operationalize empathy in natural language processing. |
A Neural-Symbolic Approach to Natural Language Understanding (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models have enabled deep neural networks to perform natural language understanding tasks, but their performance can drastically deteriorate when logical reasoning is needed. |
| Approach: | They propose a framework for NLU based on analogical reasoning based upon neural processing and logical reasoning using both neural and symbolic processing. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on two NLU tasks, question answering (QA) and natural language inference (NLI). |
Social-aware Sparse Attention Network for Session-based Social Recommendation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for predicting the next item for an anonymous session do not capture user preferences and noisy irrelevant interactions. |
| Approach: | They propose to use social networks and historical sessions to provide personalized recommendations for the current session. |
| Outcome: | The proposed model outperforms existing models on two benchmark datasets. |
SparseAdapter: An Easy Approach for Improving the Parameter-Efficiency of Adapters (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrain-finetuned models are increasingly complex and require more parameters to match the performance of full fine-tuning. |
| Approach: | They propose an efficient Adapter Tuning technique that freezes pretrained language models and fine-tunes a few extra modules. |
| Outcome: | The proposed setting outperforms the standard Adapter Tuning by 80% . the proposed setting is easy to use and has a high sparse ratio . |
Measurement Extraction with Natural Language Processing: A Review (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Information extraction (IE) is a task in natural language processing that extracts information from documents. |
| Approach: | They describe different approaches to measurement extraction and outline challenges posed by this task. |
| Outcome: | The proposed methods are compared with the literature on the extraction of quantitative data from documents. |
Summarizing Procedural Text: Data and Approach (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Procedural text summarization task is a popular task in the NLP field because of its long length and complexity. |
| Approach: | They propose a procedural text summarization task with two granularity . they propose an Entity-State Graph-based Summarizer (ESGS) which aggregates contextual information for each procedure. |
| Outcome: | The proposed model can summarize the entire procedural text or give an overview for each step or both . Experiments on two datasets confirm the proposed model's effectiveness. |
Snapshot-Guided Domain Adaptation for ELECTRA (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing domain-specific knowledge of domain-related tasks is lacking in pre-trained language models. |
| Approach: | They propose a domain-adaptation method which can dynamically select domain-specific tokens and guide the discriminator to emphasize them, without introducing new training parameters. |
| Outcome: | The proposed method can capture domain-specific knowledge of domain-related tasks without introducing new training parameters. |
Exploiting Labeled and Unlabeled Data via Transformer Fine-tuning for Peer-Review Score Prediction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on automatic peer-review aspect score prediction rely on limited data sets. |
| Approach: | They propose a semi-supervised learning method that incorporates the Transformer fine-tuning into the -model to leverage contextual features from unlabeled data. |
| Outcome: | The proposed method outperforms supervised and naive methods in the peer-review dataset. |
HARALD: Augmenting Hate Speech Data Sets with Real Data (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Hate speech detection depends on the availability of variable labeled data. |
| Approach: | They propose a method that uses real unlabelled data from online platforms to augment existing models by harvesting and processing it. |
| Outcome: | The proposed approach improves the classification performance of hate speech classification models. |
Wait-info Policy: Balancing Source and Target at Information Level for Simultaneous Machine Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to balance source and target information at the token level are limited by the number of received source tokens. |
| Approach: | They propose a Wait-info Policy to balance source and target at the information level . they quantify the amount of info contained in each token and compare it with previous outputs . |
| Outcome: | The proposed method outperforms baselines under and achieves better balance . it is based on comparisons between the total info of previous target outputs and received source inputs . |
Turning Fixed to Adaptive: Integrating Post-Evaluation into Simultaneous Machine Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to perform adaptive and fixed translations lack evaluation before taking actions. |
| Approach: | They propose a method to perform adaptive translation policy via post-evaluation into fixed policy . their method evaluates rationality of next action by measuring change in source content . |
| Outcome: | The proposed method exceeds strong baselines under all latency. |
Alleviating Sparsity of Open Knowledge Graphs with Ternary Contrastive Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to learning KG triplets ignore ternary propagation patterns and ignore zero-shot, few-shot and synonymity problems. |
| Approach: | They propose a framework for contrastive learning based on ternary propagation patterns among head, relation and tail. |
| Outcome: | Experiments on benchmarks show that TernaryCL is superior to state-of-the-art models. |
Using Developer Discussions to Guide Fixing Bugs in Software (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work shows that natural language context is useful in guiding bug-fixing models, but requires prompting developers to provide this context. |
| Approach: | They propose to use bug report discussions to prompt developers to provide natural language context for bug-fixing models. |
| Outcome: | The proposed approach reduces the need for additional information from developers. |
AutoCAD: Automatically Generate Counterfactuals for Mitigating Shortcut Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for generating counterfactuals rely on human efforts or task-specific designs. |
| Approach: | They propose to use a fully automatic and task-agnostic CAD generation framework to generate diverse counterfactuals. |
| Outcome: | The proposed framework outperforms human-in-the-loop and task-specific CAD methods on multiple out-of-domain and challenge benchmarks. |
A Multi-Modal Knowledge Graph for Classical Chinese Poetry (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies in classical Chinese poetry area focus on generation and analysis of poetry. |
| Approach: | They propose to integrate the visual information of words in classical Chinese poetry into a multi-modal knowledge graph. |
| Outcome: | The proposed model bridges the semantic gap between two modalities and achieves state-of-the-art performance on the poetry-image retrieval task. |
Assessing Non-autoregressive Alignment in Neural Machine Translation via Word Reordering (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing non-autoregressive neural machine translation models that implicitly model dependencies are sub-optimal in handling word order errors. |
| Approach: | They propose to learn a non-autoregressive language model that can be combined with Viterbi decoding to achieve better reordering performance. |
| Outcome: | The proposed model outperforms state-of-the-art reordering mechanisms under different word permutation settings with a 2-27 BLEU improvement, suggesting high potential for word alignment in NAT. |
Syntax-guided Localized Self-attention by Constituency Syntactic Distance (2022.findings-emnlp)
Copied to clipboard
Shengyuan Hou, Jushi Kai, Haotian Xue, Bingyu Zhu, Bo Yuan, Longtao Huang, Xinbing Wang, Zhouhan Lin
| Challenge: | Recent studies have shown that Transformers is implicitly learning syntactic information from data, albeit is highly dependent on the quality and scale of the training data. |
| Approach: | They propose a syntax-guided localized self-attention model that allows directly incorporating grammar structures from an external constituency parser. |
| Outcome: | The proposed model improves translation performance on a variety of datasets, from small to large datasets and with different source languages. |
CodeExp: Explanatory Code Document Generation (2022.findings-emnlp)
Copied to clipboard
Haotian Cui, Chenglong Wang, Junjie Huang, Jeevana Priya Inala, Todd Mytkowicz, Bo Wang, Jianfeng Gao, Nan Duan
| Challenge: | Existing code-to-text generation models produce only high-level code summaries that do not capture implementation-level choices essential for these scenarios. |
| Approach: | They propose a code explanation generation task that uses code docstrings to refine models. |
| Outcome: | The proposed model can generate well-structured long docstrings comparable to human-written ones. |
PAUQ: Text-to-SQL in Russian (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Semantic parsing is an important task that allows to democratize human-computer interaction. |
| Approach: | They construct and complement a Russian text-to-SQL dataset by integrating a spider query with a RAT-SqL and BRIDGE database. |
| Outcome: | The proposed datasets show that they perform well with monolingual training and improved accuracy in multilingual scenarios. |
Event-Centric Question Answering via Contrastive Learning and Invertible Event Transformation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing QA frameworks that use event-centric reasoning are lacking. |
| Approach: | They propose a novel QA model with contrastive learning and invertible event transformation . they use an invertable transformation matrix to project event vectors into a common event embedding space . |
| Outcome: | The proposed model achieves 8.4% gain in token-level F1 score and 3.0% gain in Exact Match score on the ESTER dataset. |
Label-Driven Denoising Framework for Multi-Label Few-Shot Aspect Category Detection (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for ACD use label information of aspect categories to detect aspect categories . but, they still suffer from noise problems due to lack of supervised data . |
| Approach: | They propose a Label-Driven Denoising Framework to alleviate noise problems for ACD subtask . they use the label information of each aspect to generate a better prototype . |
| Outcome: | The proposed framework improves the performance of the multi-label few-shot Aspect Category Detection task. |
Visual Named Entity Linking: A New Dataset and A Baseline (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing tasks in Visual Entity Linking (VEL) rely on textual data to complement multi-modal linking or only link objects with general entities. |
| Approach: | They propose a task to link regions of images with corresponding entities in Knowledge Bases . they propose three sub-tasks, based on a human-annotated visual person dataset . |
| Outcome: | The proposed task is based on a human-annotated visual person linking dataset . the proposed sub-tasks are validated on the WIKIPerson dataset based upon the proposed methods . |
MAGMA – Multimodal Augmentation of Generative Models through Adapter-based Finetuning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Large-scale pretraining is becoming the norm in Vision-Language (VL) modeling. |
| Approach: | They propose a method for augmenting generative language models with additional modalities using adapter-based finetuning. |
| Outcome: | The proposed method outperforms Frozen on open-ended generative tasks while maintaining the language model weights. |
Towards Tracing Knowledge in Language Models Back to the Training Data (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work on training data attribution (TDA) may offer effective tools for identifying such examples, known as "proponents". |
| Approach: | They propose a benchmark to identify which training examples taught an LM to generate a particular factual assertion. |
| Outcome: | The proposed methods have lower proponent-retrieval precision than baselines that do not have access to the LM. |
ReaRev: Adaptive Reasoning for Question Answering over Knowledge Graphs (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Knowledge Graph Question Answering (KGQA) involves retrieving entities as answers from a Knowledge Flow using natural language queries. |
| Approach: | They propose a method to decode a question into instructions that are dense question representations used to guide the KG traversals. |
| Outcome: | The proposed method improves instruction decoding and execution by using a KG-aware information to update the initial instructions. |
Understanding Social Media Cross-Modality Discourse in Linguistic Space (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on how images are structured with texts to form coherent meanings in human cognition have not addressed the problem. |
| Approach: | They propose a concept of cross-modality discourse which defines how human readers couple image and text understandings. |
| Outcome: | The proposed model shows that trendy encoders based on multi-head attention are unable to understand cross-modality discourse and modeling texts at the output layer helps yield the-state-of-the-art results. |
TAPE: Assessing Few-shot Russian Language Understanding (2022.findings-emnlp)
Copied to clipboard
Ekaterina Taktasheva, Tatiana Shavrina, Alena Fenogenova, Denis Shevelev, Nadezhda Katricheva, Maria Tikhonova, Albina Akhmetgareeva, Oleg Zinkevich, Anastasiia Bashmakova, Svetlana Iordanskaia, Alena Spiridonova, Valentina Kurenshchikova, Ekaterina Artemova, Vladislav Mikhailov
| Challenge: | Recent advances in zero-shot and few-shot learning have shown promise for a scope of research and practical purposes, but lacks standardized evaluation suites for non-English languages. |
| Approach: | They propose a novel benchmark that includes six more complex NLU tasks for Russian, covering multi-hop reasoning, ethical concepts, logic and commonsense knowledge. |
| Outcome: | The proposed benchmark includes six more complex NLU tasks for Russian, covering multi-hop reasoning, ethical concepts, logic and commonsense knowledge. |
A Hierarchical N-Gram Framework for Zero-Shot Link Prediction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to zero-shot link prediction use textual features of relations as auxiliary information to improve the encoded representation. |
| Approach: | They propose a Hierarchical N-gram framework for Zero-Shot Link Prediction that leverages character n-gram information for ZSLP. |
| Outcome: | The proposed method achieves state-of-the-art on two standard ZSLP datasets. |
Quadapter: Adapter for GPT-2 Quantization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Quantization-aware training (QAT) fine-tunes model parameters in response to the numerical error arising from quantization. |
| Approach: | They propose a quantization adapter that scales activations channel-wise to make activations quantization-friendly by scaling them channel- wise. |
| Outcome: | The proposed method prevents overfitting and improves quantization performance. |
BanglaRQA: A Benchmark Dataset for Under-resourced Bangla Language Reading Comprehension-based Question Answering with Diverse Question-Answer Types (2022.findings-emnlp)
Copied to clipboard
Syed Mohammed Sartaj Ekram, Adham Arik Rahman, Md. Sajid Altaf, Mohammed Saidul Islam, Mehrab Mustafy Rahman, Md Mezbaur Rahman, Md Azam Hossain, Abu Raihan Mostofa Kamal
| Challenge: | a lack of diverse and comprehensive question-answering datasets exists in under-resourced languages like Bangla. |
| Approach: | They propose a reading comprehension-based Bangla question-answering dataset . the dataset includes answerable and unanswerable questions covering four categories of questions . |
| Outcome: | The proposed dataset shows that it performs well as a training resource in high-resource languages. |
Chaining Simultaneous Thoughts for Numerical Reasoning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Numerical reasoning over text is an essential skill for AI systems . structure modeling is effective, but structures restrict how a model should grasp the reasoning process . |
| Approach: | They propose a numerical reasoner that models reasoning steps using a directed acyclic graph without pre-defined decoding dependencies. |
| Outcome: | The proposed model produces diverse reasoning steps without pre-defined dependencies and compares relevant ones to reach a solution. |
Inferring Implicit Relations in Complex Questions with Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | A prominent challenge for language understanding systems is the ability to answer implicit reasoning questions where the evidence for answering the question is not mentioned explicitly. |
| Approach: | They propose to decouple inference of reasoning steps from execution by evaluating models of implicit relation inference. |
| Outcome: | The proposed model fails on the implicit reasoning QA task, but infers implicit relations . the proposed model is compared with other models that fail on the same task . |
Eliciting and Understanding Cross-task Skills with Task-level Mixture-of-Experts (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained transformer models are capable of multitasking on diverse NLP tasks, but little is known about how multitaskability and cross-task generalization is achieved. |
| Approach: | They propose to use a transformer-based mixture-of-expert model with a router component to choose among experts dynamically and flexibly. |
| Outcome: | The proposed models improve the average performance gain (ARG) metric by 2.6% when adapting to unseen tasks, and by 5.6% in zero-shot generalization settings. |
On the Curious Case of l2 norm of Sense Embeddings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | l2 norm of sense embeddings encodes information related to frequency of that sense in the training corpus . l2-normal feature is useful for word-in-context (WiC) and word sense disambiguation (WSD) |
| Approach: | They propose to include the l2 norm of a sense embedding as a feature in a classifier to improve word sense learning methods that use static sense embeds. |
| Outcome: | The l2 norm of sense embeddings is a surprisingly effective feature for word sense related tasks such as word-in-context (WiC) and word sense disambiguation (WSD). |
Partially-Random Initialization: A Smoking Gun for Binarization Hypothesis of BERT (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained BERT has been used for natural language processing tasks but its performance is limited by memory and computational complexity. |
| Approach: | They propose to use pre-trained BERT to achieve decent accuracy . they propose to combine binary BERT with a randomly-initialized encoder . |
| Outcome: | The proposed model achieves state-of-the-art on GLUE and SQuAD benchmarks. |
Prompt Consistency for Zero-Shot Task Generalization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work has shown that pre-trained language models can perform zero-shot generalization to new tasks without annotated examples. |
| Approach: | They propose to regularize prompt consistency to encourage consistent predictions over a diverse set of prompts. |
| Outcome: | The proposed approach outperforms the state-of-the-art zero-shot learner, T0, on 9 out of 11 datasets across 4 NLP tasks by 10.6 absolute points in terms of accuracy. |
In-Context Learning for Few-Shot Dialogue State Tracking (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for zero-shot and few-shot learning dialogue state tracking are hard and expensive. |
| Approach: | They propose an in-context learning framework for zero-shot and few-shot learning dialogue state tracking (DST) a large pretrained language model takes a test instance and a few exemplars as input and directly decodes the dialogue state . |
| Outcome: | The proposed framework outperforms state-of-the-art models in few-shot settings . it is flexible and scalable, and requires less data to adapt to new domains and scenarios . |
On Advances in Text Generation from Images Beyond Captioning: A Case Study in Self-Rationalization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Combining visual modality with pretrained language models has been effective for descriptive tasks such as image captioning. |
| Approach: | They ask: do multimodal models combine visual and visual adapted language models? they find that CLIP image representations and scaling of language models do not consistently improve self-rationalization in multimodal tasks. |
| Outcome: | The proposed model types do not consistently improve self-rationalization in multimodal tasks. |
The challenges of temporal alignment on Twitter during crises (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models consider data spanning years to decades, but shorter time spans are critical for crisis data. |
| Approach: | They propose to use domain adaptation techniques to cope with performance degradation by leveraging domain adaptation. |
| Outcome: | The proposed models outperform baseline models under conditions of natural and human-induced disasters while highlighting the limitations of current models. |
Experimental Standards for Deep Learning in Natural Language Processing Research (2022.findings-emnlp)
Copied to clipboard
Dennis Ulmer, Elisa Bassignana, Max Müller-Eberstein, Daniel Varab, Mike Zhang, Rob van der Goot, Christian Hardmeier, Barbara Plank
| Challenge: | a lack of common experimental standards remains an open challenge to the field at large . |
| Approach: | They propose to distill discussions on experimental standards into a single, widely-applicable methodology. |
| Outcome: | Using best practices, we can strengthen experimental evidence, improve reproducibility and enable scientific progress. |
Few-Shot Anaphora Resolution in Scientific Protocols via Mixtures of In-Context Experts (2022.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning has emerged as a promising approach to resolve anaphora, but there are challenges in applying it to scientific protocols. |
| Approach: | They propose a method which combines predictions of hundreds of in-context experts and combines them to yield a 30% increase in F1 over a competitive prompt retrieval baseline. |
| Outcome: | The proposed method yields 30% increase in F1 score over a competitive prompt retrieval baseline. |
Exploring Predictive Uncertainty and Calibration in NLP: A Study on the Impact of Method & Data Scarcity (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Using low-resource languages, we assess the quality of uncertainty estimates from a wide array of approaches, but with more data. |
| Approach: | They train models on sub-sampled datasets in three different languages to assess the confidence of a neural classifier. |
| Outcome: | The proposed models train on sub-sampled datasets in three different languages and show that the quality of uncertainty estimates suffers with more data. |
Conditional Supervised Contrastive Learning for Fair Text Classification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in natural language processing have demonstrated societal bias in existing NLP models. |
| Approach: | They propose to use contrastive learning to learn fair representations for text classification . they conduct experiments on two text datasets to demonstrate their methods are stable . |
| Outcome: | The proposed methods balancing task performance and bias mitigation are stable in different hyperparameter settings. |
SpaBERT: A Pretrained Language Model from Geographic Data for Geo-Entity Representation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Named geographic entities are the building blocks of many geographic datasets. |
| Approach: | They propose a spatial language model that provides a general-purpose geo-entity representation based on neighboring entities in geospatial data. |
| Outcome: | The proposed model improves on two downstream tasks, showing significant performance improvement compared with existing models that do not use spatial context. |
Self-training with Two-phase Self-augmentation for Few-shot Dialogue Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for self-training from meaning representations (MRs) are noisy or uninformative for the model to learn from. |
| Approach: | They propose a two-phase procedure to generate high-quality pseudo-labeled MR-to-Text pairs by aggregating multiple perturbed latent representations from each MR. |
| Outcome: | Empirical results on two benchmark datasets show that the proposed procedure outperforms existing methods on automatic and human evaluations. |
Is NLP Ready for Standardization? (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a number of scientific fields, including telecommunications, networks and multimedia, lack standards in the field of NLP. |
| Approach: | They propose to examine how NLP lacks standards and how that can impact society, industry and regulations. |
| Outcome: | The proposed standards examine the needs of NLP researchers and industry . they argue that the lack of standards can impact the field, society and industry. |
Probing for Incremental Parse States in Autoregressive Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on autoregressive language models shows impressive command of syntax . implicit incremental syntactic inferences underlie next-word predictions . |
| Approach: | They propose a probe for extracting incomplete syntactic structure from autoregressive language models. |
| Outcome: | The proposed probes can predict model preferences on ambiguous sentence prefixes and causally intervene on model representations and steer model behavior. |
Re-Examining Calibration: The Case of Question Answering (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing calibration methods do not provide significant gains in accuracy. |
| Approach: | They propose a new calibration metric that better captures whether the model assigns low confidence to wrong predictions and high confidence to correct predictions. |
| Outcome: | The proposed calibration method better captures whether the model assigns low confidence to wrong predictions and high confidence to correct predictions. |
Accelerating Learned Sparse Indexes Via Term Impact Decomposition (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Novel inverted index-based learned sparse ranking models provide more effective, but less efficient, retrieval performance compared to traditional ranking models. |
| Approach: | They propose a technique that allows for automatic pruning of ranking models by storing metadata about index term importance scores. |
| Outcome: | The proposed technique accelerates top-k retrieval by 9.6X without loss in effectiveness. |
Do Text-to-Text Multi-Task Learners Suffer from Task Conflict? (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing multi-task learning architectures learn a single model across multiple tasks through a shared encoder followed by task-specific decoders. |
| Approach: | They propose to use a shared encoder and language model decoder to learn a single model across multiple tasks. |
| Outcome: | The proposed architecture does surprisingly well across a range of diverse tasks. |
MANTa: Efficient Gradient-Based Tokenization for End-to-End Robust Language Modeling (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Subword tokenization algorithms have been an essential component of language modeling but their static nature results in important flaws that degrade the models’ downstream performance and robustness. |
| Approach: | They propose a module for Adaptive Neural TokenizAtion that is differentiable and trained end-to-end with the language model. |
| Outcome: | The proposed tokenizer improves robustness to character perturbations and out-of-domain data. |
Towards Intelligent Clinically-Informed Language Analyses of People with Bipolar Disorder and Schizophrenia (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on social media data have limited the extent to which they can produce meaningful or generalizable conclusions. |
| Approach: | They propose to use transcribed conversations with people with bipolar disorder and schizophrenia to create a large dataset of transcriptions. |
| Outcome: | The proposed dataset extracts 100+ temporal, sentiment, psycholinguistic, emotion, and lexical features and establishes classification validity. |
Calibrating Trust of Multi-Hop Question Answering Systems with Decompositional Probes (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work in multi-hop QA has shown that performance can be boosted by decomposing questions into simpler, single-hop questions. |
| Approach: | They propose to decompose multi-hop questions into simpler, single-hop ones to create explanations by probing a neural QA model with them. |
| Outcome: | The proposed approach can be used to generate explanations by probing a neural QA model with them. |
CheckHARD: Checking Hard Labels for Adversarial Text Detection, Prediction Correction, and Perturbed Word Suggestion (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work detects adversarial text and corrects the prediction by identifying perturbed words and changing them into their synonyms, but many benign words are also changed. |
| Approach: | They propose to detect adversarial text, correct the prediction, and suggest perturbed words by checking the change in the hard labels from the target’s predictions. |
| Outcome: | The proposed model outperforms existing work on various attacks, models, and datasets. |
Mitigating Covertly Unsafe Text within Natural Language Systems (2022.findings-emnlp)
Copied to clipboard
Alex Mei, Anisha Kabir, Sharon Levy, Melanie Subbiah, Emily Allaway, John Judge, Desmond Patton, Bruce Bimber, Kathleen McKeown, William Yang Wang
| Challenge: | Existing studies on text safety have focused on overtly unsafe, covertly, or indirectly unsafe statements. |
| Approach: | They propose a method to identify physical harm-causing statements as overtly, covertly or indirectly unsafe and a solution to mitigate the generation of such statements. |
| Outcome: | The proposed methods identify the type of unsafe language that can cause physical harm and identify mitigation strategies to inspire future researchers to tackle this challenging problem. |
“I Know Who You Are”: Character-Based Features for Conversational Humor Recognition in Chinese (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study has focused on how to recognize punchlines from dialogues, but has neglected character information. |
| Approach: | They propose a character-fusion conversational humor recognition model that uses character information to recognize punchlines from dialogue. |
| Outcome: | The proposed model improves performance on Chinese sitcoms corpus and punchline identification. |
DebiasGAN: Eliminating Position Bias in News Recommendation with Adversarial Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing news recommendation methods use click behaviors for interest inference and model training, but position biases can be inaccurate in targeting user interest. |
| Approach: | They propose a news recommendation method that eliminates position biases by adversarial learning by a candidate-aware click model and a bias-invariant click model. |
| Outcome: | The proposed method can effectively alleviate position biases on click behaviors and capture unbiased user interest. |
Generating Multiple-Length Summaries via Reinforcement Learning for Unsupervised Sentence Summarization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models to summarize texts without ground-truth summaries are extractive, which remove words from texts and thus are less flexible than abstractive models. |
| Approach: | They propose an unsupervised model that extracts words from texts and makes them mutually enhance each other. |
| Outcome: | The proposed model outperforms both abstractive and extractive models, while generating new words not contained in input texts. |
Multilingual Sentence Transformer as A Multilingual Word Aligner (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Multilingual pretrained language models (mPLMs) have shown their effectiveness in multilingual word alignment induction, but these methods usually start from mBERT or XLM-R. |
| Approach: | They propose to fine tune multilingual sentence Transformer LaBSE for alignment induction using parallel corpus and a parallel corpora model. |
| Outcome: | The proposed model outperforms existing models on seven language pairs and achieves new state-of-the-art on zero-shot language pairs. |
CORE: A Retrieve-then-Edit Framework for Counterfactual Data Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work on counterfactual data augmentation only considered restricted classes of perturbations, limiting their effectiveness. |
| Approach: | They propose a retrieval-augmented framework for creating diverse counterfactual perturbations for CDA. |
| Outcome: | Experiments on natural language inference and sentiment analysis show that the proposed framework can be used to encourage diversity in manually authored perturbations. |
Conversation Disentanglement with Bi-Level Contrastive Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods focus on pairwise utterance relations but pay inadequate attention to utterant-to-context relation modeling. |
| Approach: | They propose a general disentangle model based on bi-level contrastive learning that brings closer utterances in the same session while encouraging each utterrance to be near its clustered session prototypes in representation space. |
| Outcome: | The proposed model achieves state-of-the-art performance on both settings across public datasets. |
You can’t pick your neighbors, or can you? When and How to Rely on Retrieval in the kNN-LM (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have shown that retrieval-enhanced language models can improve perplexity by combining text from large external datastores with a k-nearest neighbors model. |
| Approach: | They propose a retrieval-enhanced language model that interpolates existing LMs with a k-nearest neighbors model and requires no additional training. |
| Outcome: | The proposed model improves on two English language modeling datasets and shows that it is most effective when items have high semantic similarity with the query. |
StuBot: Learning by Teaching a Conversational Agent Through Machine Reading Comprehension (2022.findings-emnlp)
Copied to clipboard
| Challenge: | StuBot provides adaptive feedback for learning by teaching . |
| Approach: | They propose a text-based conversational agent that provides adaptive feedback for learning by teaching. |
| Outcome: | The proposed agent improves learning performance, immersion, and overall experience by providing adaptive feedback to the users who input the explanation text. |
Improved Universal Sentence Embeddings with Prompt-based Contrastive Learning and Energy-based Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing contrastive methods for learning universal sentence embeddings have limitations due to their over-parameterization and poor performance under domain shift settings. |
| Approach: | They propose to integrate an Energy-based Hinge loss to enhance the pairwise discriminative power of contrastive learning for sentence embeddings by combining PLMs with energy-based learning. |
| Outcome: | Empirical results show that the proposed method improves on seven standard semantic textual similarity tasks and a domain-shifted STS task. |
RaP: Redundancy-aware Video-language Pre-training for Text-Video Retrieval (2022.findings-emnlp)
Copied to clipboard
| Challenge: | sparse sampling of videos suffers from inter-modal redundancy and visual redundancies . et al., 2021) proposes to sparsestly sample frames from videos to alleviate temporal redundance . |
| Approach: | They propose to use sparse sampling to alleviate temporal redundancy in videos . they propose to penalize high-redundant video patches and text tokens . |
| Outcome: | The proposed method improves on four benchmark datasets. |
FCGCL: Fine- and Coarse-Granularity Contrastive Learning for Speech Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to perform implicit knowledge transfer from machine translation to ST model are difficult because of the task complexity and data scarcity. |
| Approach: | They recommend a method which conducts explicit knowledge transfer from MT to ST model by fine and coarse granularity contrastive learning. |
| Outcome: | The proposed method improves the performance of the end-to-end speech translation model on all 8 languages. |
InfoCSE: Information-aggregated Contrastive Learning of Sentence Embeddings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on contrastive learning for sentence embeddings are weak . researchers have started to use contrastive training to learn better unsupervised sentences. |
| Approach: | They propose an information-aggregated contrastive learning framework for learning unsupervised sentence embeddings. |
| Outcome: | The proposed framework outperforms SimCSE on several benchmark datasets w.r.t the semantic text similarity task. |
Benchmarking Language Models for Code Syntax Understanding (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models capture the syntactic rules of natural languages without fine-tuning on syntax understanding tasks. |
| Approach: | They propose a benchmarking test to compare pre-trained language models with a large-scale dataset of programs annotated with syntactic relationships in their corresponding abstract syntax trees. |
| Outcome: | The proposed model fails to match baselines based on positional offsets and keywords. |
Learning When and What to Quote: A Quotation Recommender System with Mutual Promotion of Recommendation and Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing quotation recommendation system focuses on what to quote, but ignores whether or when to quote. |
| Approach: | They propose a framework that learns to predict when to quote and what to quote jointly. |
| Outcome: | The proposed framework achieves significantly better performance than baselines on two datasets. |
Think Beyond Words: Exploring Context-Relevant Visual Commonsense for Diverse Dialogue Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to generate intelligent open-domain dialogue agents only consider auxiliary commonsense stored in pure text, ignoring grounding information from the external visual world. |
| Approach: | They propose a VIsual Commonsense enhanced dialogue generaTOR that exploits auxiliary commonsense from images related to context to generate coherent and informative responses. |
| Outcome: | The proposed method outperforms the latest competitive methods in terms of coherence and diversity on two public datasets. |
Gender Bias in Meta-Embeddings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to develop meta-embeddings from source embeddings contain unfair gender-related biases, and how these influence the meta-bedding has not been studied yet. |
| Approach: | They propose to use multiple debiasing methods on a single source embedding to create a gender-based meta-embedding. |
| Outcome: | The proposed method amplifies gender biases compared to input source embeddings. |
Third-Party Aligner for Neural Word Alignments (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work shows that word alignment can be competitive . |
| Approach: | They propose to use word alignments generated by a third-party word aligner to supervise the neural word alignment training. |
| Outcome: | The proposed approach can find more accurate word alignments and delete wrong alignments, leading to better performance than the current best third-party word aligner. |
QaDialMoE: Question-answering Dialogue based Fact Verification with Mixture of Experts (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing research on fact verification focuses on news, tables and Wikipedia passages. |
| Approach: | They propose a question-answering dialogue based fact verification with mixture of experts that exploits questions and evidence effectively in the verification process. |
| Outcome: | The proposed approach outperforms previous approaches on three benchmark datasets and achieves state-of-the-art results. |
Multimodal Knowledge Learning for Named Entity Disambiguation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing attempts to model multimodal information at the knowledge level are lacking multimodal annotation data against the large-scale unlabeled corpus. |
| Approach: | They propose to use multimodal knowledge learning to link ambiguous mentions with textual and visual contexts to a predefined knowledge graph. |
| Outcome: | The proposed method achieves improvements over the state-of-the-art methods on two public MNED datasets. |
Generative Prompt Tuning for Relation Classification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing prompt tuning methods for RC are limited by label spaces and rigid prompt restrictions. |
| Approach: | They propose a generative prompt tuning method to reformulate relation classification as an infilling problem by adding cloze-style phrases to masked language modeling problems. |
| Outcome: | The proposed method exploits rich semantics of entity and relation types and can predict label verbalizations with varying lengths at multiple predicted positions. |
Formulating Few-shot Fine-tuning Towards Language Model Pre-training: A Pilot Study on Named Entity Recognition (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning pre-trained language models are limited . we propose a few-shot fine-uning framework for NER . |
| Approach: | They propose a few-shot fine-tuning framework for named entity recognition (NER) they propose three new types of tokens, "is-entity", "which-type" and "bracket" |
| Outcome: | The proposed framework improves on pre-trained language models on several benchmark datasets. |
Masked Language Models Know Which are Popular: A Simple Ranking Strategy for Commonsense Question Answering (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Empirical results show that pre-trained language models can improve the typical answer generation of GLMs. |
| Approach: | They propose a ranking strategy that exploits WordNet to train a ranker that picks out the most popular answers for commonsense questions. |
| Outcome: | The proposed ranking strategy is tested on a commonsense question answering (QA) dataset and on negative samples from WordNet. |
DialogUSR: Complex Dialogue Utterance Splitting and Reformulation for Multiple Intent Detection (2022.findings-emnlp)
Copied to clipboard
Haoran Meng, Zheng Xin, Tianyu Liu, Zizhen Wang, He Feng, Binghuai Lin, Xuemin Zhao, Yunbo Cao, Zhifang Sui
| Challenge: | DialogUSR is a plug-in and domain-agnostic module that empowers multi-intent detection for chatbots . a single user query triggers inquiries on highspeed train ticket price and weather of destination. |
| Approach: | They propose a dialog utterance splitting and reformulation task that splits multi-intent user query into multiple single-intention sub-queries and recovers all coreferred and omitted information in the sub-questions. |
| Outcome: | The proposed model can be used to split multi-intent user queries into multiple sub-queries . it can be trained in two stages and perform in-depth analyses on the proposed models . |
Low-resource Interactive Active Labeling for Fine-tuning Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing active learning methods for fine-tuning language models are underperforming in low-resource, interactive labeling setting. |
| Approach: | They propose a novel active learning method that employs a hybrid sampling strategy to minimize labeling cost and acquisition latency while providing a framework for adapting to dataset diversity. |
| Outcome: | The proposed method reduces labeling cost and acquisition latency while providing a framework for adapting to dataset diversity via user guidance. |
Getting the Most out of Simile Recognition (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work ignores features other than surface strings and suffers from data hunger issue. |
| Approach: | They propose to use simile sentence classification and simile component extraction to find simile components. |
| Outcome: | The proposed model outperforms current state-of-the-art systems and baselines. |
A Unified Framework for Pun Generation with Humor Principles (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for generating homophonic and homographic puns lack the linguistic attributes of successful puns to resolve the split-up in existing work. |
| Approach: | They propose a framework to generate both homophonic and homographic puns to resolve the split-up in existing works by incorporating three linguistic attributes of puns into the language models: ambiguity, distinctiveness, and surprise. |
| Outcome: | The proposed model over strong baselines shows that it can generate both homophonic and homographic puns. |
Improving English-Arabic Transliteration with Phonemic Memories (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing neural approaches to transliterate names from English to Arabic are limited and focus on leveraging the phonemic association between English and Arabic. |
| Approach: | They propose a model for English-Arabic transliteration using a memory module modeling the phonemic association between English and Arabic to guide the transliterations process. |
| Outcome: | The proposed model improves on EANames corpus, which better represents names in the general public than linked Wikipedia entries that are always names of famous people. |
Mix-and-Match: Scalable Dialog Response Retrieval using Gaussian Mixture Embeddings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches for dialog response retrieval embed the context-response pairs as points in the embedding space. |
| Approach: | They propose a scalable model that can learn complex relationships between context-response pairs . they train the models by optimizing the Kullback-Leibler divergence between the distributions induced by context-responders in the training data. |
| Outcome: | The proposed model performs better than other embedding-based approaches on public conversation data. |
AlphaTuning: Quantization-Aware Parameter-Efficient Adaptation of Large-Scale Pre-Trained Language Models (2022.findings-emnlp)
Copied to clipboard
Se Jung Kwon, Jeonghoon Kim, Jeongin Bae, Kang Min Yoo, Jin-Hwa Kim, Baeseong Park, Byeongwook Kim, Jung-Woo Ha, Nako Sung, Dongsoo Lee
| Challenge: | Existing approaches to improve inference efficiency by accelerating model fine-tuning have not been thoroughly explored. |
| Approach: | They propose to combine parameter-efficient adaptation and model compression to accelerate model . they propose to freeze binary parameters and scale scaling factors for target tasks . |
| Outcome: | The proposed algorithm achieves >10x compression ratio under 4-bit quantization and >1,000x reduction in trainable parameters. |
Learning Invariant Representation Improves Robustness for MRC Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to improve machine reading comprehension models are vulnerable and not robust to adversarial examples. |
| Approach: | They propose to construct positive example pairs which have same answer by augmentation and then introduce stability and contrastive loss to improve invariance of representation. |
| Outcome: | The proposed approach boosts the robustness of QA models across different tasks and attack sets significantly and consistently. |
ER-Test: Evaluating Explanation Regularization Methods for Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Explanation regularization (ER) aims to improve NLM generalization by pushing the NLM’s machine rationales to align with human rationale. |
| Approach: | They propose a framework for evaluating ER models’ OOD generalization along three dimensions: unseen datasets, contrast set tests, and functional tests. |
| Outcome: | The proposed framework evaluates ER models’ OOD generalization across unseen datasets, contrast set tests, and functional tests. |
Learning Cooperative Interactions for Multi-Overlap Aspect Sentiment Triplet Extraction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for aspect sentiment triplet extraction focus on the single interactions between an aspect and an opinion. |
| Approach: | They propose a multi-overlap triplet extraction method which decodes the complex relations between multiple aspects and opinions by learning their cooperative interactions. |
| Outcome: | The proposed method outperforms baselines, especially multi-overlap triplets. |
Different Tunes Played with Equal Skill: Exploring a Unified Optimization Subspace for Parameter-Efficient Tuning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing delta tuning algorithms freeze most of the parameters and only optimize minimal adaptive parameters. |
| Approach: | They propose to decompose DETs into a unified optimization subspace and conduct optimization within the subspace. |
| Outcome: | The proposed DETs achieve comparable performance to the original DET and can be transferred to another DET with non-trivial performance. |
Explainable Slot Type Attentions to Improve Joint Intent Detection and Slot Filling (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods analyze and compute features collectively for all slot types, and have no way to explain slot filling model decisions. |
| Approach: | They propose a method that learns to generate additional slot type specific features to improve accuracy and provides explanations for slot filling decisions for the first time in a joint NLU model. |
| Outcome: | The proposed model improves on two widely used datasets and provides an explanation for slot filling decisions for the first time. |
PseudoReasoner: Leveraging Pseudo Labels for Commonsense Knowledge Base Population (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Commonsense Knowledge Base (CSKB) Population aims at reasoning over unseen entities and assertions on CSKBs, but it requires out-of-domain generalization ability as the source CSMB for training is of a relatively smaller scale (1M) . |
| Approach: | They propose a semi-supervised learning framework that uses a teacher model to provide pseudo labels on the unlabeled candidate dataset for a student model to learn from. |
| Outcome: | The proposed framework can improve the backbone model KG-BERT (RoBERTa-large) by 3.3 points on the overall performance and especially, 5.3 points on out-of-domain performance. |
History-Aware Hierarchical Transformer for Multi-session Open-domain Dialogue System (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing open-domain dialogue systems conduct one-session conversations, but multi-session MSCs are under-investigated. |
| Approach: | They propose a History-Aware Hierarchical Transformer for multi-session open-domain dialogue . they propose to encode history conversations into a history memory and leverage historical information to generate well-informed responses. |
| Outcome: | The proposed model outperforms baseline models on a large-scale MSC dataset. |
Guiding Abstractive Dialogue Summarization with Content Planning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for abstractive dialogue summarization struggle to maintain factual consistency between dialogue and summary. |
| Approach: | They propose a coarse-to-fine model for generating abstractive dialogue summaries and introduce a fact-aware reinforcement learning objective that improves the fact consistency between the dialogue and the generated summary. |
| Outcome: | The proposed model improves the quality of the generated summary, especially in coherence and consistency. |
Truncation Sampling as Language Model Desmoothing (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Long samples of text from neural language models can be of poor quality. |
| Approach: | They propose to think of a neural language model as a mixture of k and a true distribution that avoids infinite perplexity. |
| Outcome: | The proposed methods generate more plausible long documents according to humans and break out of repetition. |
Knowledge-grounded Dialog State Tracking (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Structured knowledge is encoded implicitly into model parameters for downstream tasks, making training inefficient. |
| Approach: | They propose to perform dialog state tracking grounded on knowledge encoded externally. |
| Outcome: | The proposed method outperforms baseline models in the few-shot learning setting. |
Context-aware Information-theoretic Causal De-biasing for Interactive Sequence Labeling (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing deep learning models for sequence labeling are expensive and time-consuming. |
| Approach: | They propose an interactive sequence labeling that allows training directly with the user feedback . they identify context and feedback biases by formulating interactive sequence labels via a Structural Causal Model. |
| Outcome: | The proposed approach can effectively alleviate the biases and can be learnt with the user feedback. |
Simple but Challenging: Natural Language Inference Models Fail on Simple Sentences (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Natural language inference (NLI) tasks are difficult to perform on large datasets . a small number of simple sentences can improve model performance, authors say . |
| Approach: | They propose to use syntactically simple sentences to test the inference ability of NLI models. |
| Outcome: | The proposed set of simple sentences shows that the models fine-tuned on MNLI and SNLI perform poorly on Simple Pair. |
DORE: Document Ordered Relation Extraction based on Generative Framework (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing generative methods do not fit document-level relation extraction tasks where there are multiple entities and relational facts. |
| Approach: | They propose to generate a symbolic and ordered sequence from relation matrix which is easier to learn and introduce several negative sampling strategies to improve the performance with balanced signals. |
| Outcome: | The proposed method can improve the performance of the generative DocRE models on four datasets. |
Explicit Role Interaction Network for Event Argument Extraction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods extract arguments of each role independently, ignoring the relationship between different roles. |
| Approach: | They propose a neural model that captures the correlations between different argument roles within an event. |
| Outcome: | Extensive experiments on the benchmark dataset ACE2005 show the superiority of the proposed model over existing methods. |
Few-Shot Out-of-Domain Transfer Learning of Natural Language Explanations in a Label-Abundant Setup (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to train models to provide natural language explanations (NLEs) require acquisition of task-specific NLEs, which is time- and resource-consuming. |
| Approach: | They propose a few-shot out-of-domain transfer of NLEs from a parent task to a child task . they propose four methods that cover possible fine-tuning combinations of NLESs and labels . |
| Outcome: | The proposed methods cover the possible fine-tuning combinations of labels and NLEs for the parent and child tasks. |
RoChBert: Towards Robust BERT Fine-tuning for Chinese (2022.findings-emnlp)
Copied to clipboard
Zihan Zhang, Jinfeng Li, Ning Shi, Bo Yuan, Xiangyu Liu, Rong Zhang, Hui Xue, Donghong Sun, Chao Zhang
| Challenge: | Pre-trained language models (e.g., BERT) have been proved vulnerable to adversarial texts. |
| Approach: | They propose to fuse Chinese phonetic and glyph features into pre-trained models by using a more comprehensive adversarial graph. |
| Outcome: | The proposed framework outperforms existing methods in significant ways on a wide range of tasks while remaining accurate on benign texts. |
Lexical Entailment with Hierarchy Representations by Deep Metric Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing lexical entailment studies cannot be applied to words that are not included in the training dataset. |
| Approach: | They propose a method that learns a mapping from word embeddings to hierarchical embedds to predict hypernymy relations among words. |
| Outcome: | The proposed method achieves state-of-the-art performance and robustness for unknown words. |
Improving the Sample Efficiency of Prompt Tuning with Domain Adaptation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Prompt tuning is a technique for adapting large-scale pretrained language models for downstream tasks. |
| Approach: | They propose to condition a frozen pretrained language model with soft prompts from data . they propose to use a domain adaptation technique to regularize the decision boundary . |
| Outcome: | The proposed method outperforms full-model tuning in data-scarce settings by a large margin. |
McPhraSy: Multi-Context Phrase Similarity and Clustering (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for estimating phrase similarity use the phrase context only during training, instead relying on the phrase itself. |
| Approach: | They propose a novel algorithm that leverages multiple contexts during inference to estimate the similarity of phrases based on multiple context. |
| Outcome: | The proposed method outperforms existing models on two phrase similarity datasets by 13.3% and a new task that relies on phrase similarities in the product reviews domain. |
CANarEx: Contextually Aware Narrative Extraction for Semantically Rich Text-as-data Applications (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Narrative modelling is a field of active research that conceptualizes narratives as connected entity chains. |
| Approach: | They propose an alternative narrative extraction approach using semantic role labeling to extract tuples from text, then dimensionality reduction to reduce the space of entities and connections separately. |
| Outcome: | The proposed approach improves on a text-as-data task and improves accuracy and recall. |
Narrate Dialogues for Better Summarization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work on dialogue summarization models focuses on generating concise summaries for multi-party dialogues. |
| Approach: | They propose several ways to convert dialogue into a third-person narrative style . they propose to use narration as a valuable annotation for LLMs . |
| Outcome: | Empirical results show that the proposed approach achieves higher scores on ROUGE and a factual correctness metric. |
Towards Identifying Social Bias in Dialog Systems: Framework, Dataset, and Benchmark (2022.findings-emnlp)
Copied to clipboard
Jingyan Zhou, Jiawen Deng, Fei Mi, Yitong Li, Yasheng Wang, Minlie Huang, Xin Jiang, Qun Liu, Helen Meng
| Challenge: | a number of safety concerns hinder the deployment of open-domain dialog systems, such as offensive languages and toxic behaviors, such social bias is difficult to detect. |
| Approach: | They propose a Dial-Bias Framework for analyzing social bias in conversations . they introduce a Chinese social bias dialog dataset and conduct in-depth ablation studies . |
| Outcome: | The proposed framework is the first annotated Chinese social bias dialog dataset . the proposed framework also provides a fine-grained dialog bias measurement benchmark . |
CrossRE: A Cross-Domain Dataset for Relation Extraction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Relation Extraction (RE) evaluation is limited to in-domain setups . despite the drought of research on cross-domain RE, its practical importance remains . |
| Approach: | They propose a cross-domain benchmark for relation extraction which includes multi-label annotations and meta-data to include explanations and flags of difficult instances. |
| Outcome: | The proposed model includes explanations and flags of difficult instances. |
Probing Structural Knowledge from Pre-trained Language Model for Argumentation Relation Classification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Argumentation relation classification (ARC) is the most challenging subtask of argumentation mining. |
| Approach: | They propose a dual prior graph neural network to explore probing knowledge and syntactical information for comprehensively modeling the relationship between AC pairs. |
| Outcome: | The proposed model outperforms the state-of-the-art models on three public datasets. |
LogicNMR: Probing the Non-monotonic Reasoning Ability of Pre-trained Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work examines the non-monotonic reasoning ability of pre-trained language models. |
| Approach: | They construct a non-monotonic reasoning benchmark with explicit default rules and iterative updates. |
| Outcome: | The proposed model achieves a higher accuracy than the benchmark, but performs poorly on the benchmark. |
Cheater’s Bowl: Human vs. Computer Search Strategies for Open-Domain QA (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Open-domain and multi-hop QA is an important problem for both humans and computers. |
| Approach: | They propose a gamified interface where a human answers complex questions with access to traditional and modern search tools. |
| Outcome: | The proposed interface compares human queries to state-of-the-art QA models . human queries can improve the accuracy of existing systems, the authors argue . |
FRSUM: Towards Faithful Abstractive Summarization via Enhancing Factual Robustness (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models of abstractive summarization are able to generate fluent and coherent summaries, but they still suffer from the unfaithful generation problem. |
| Approach: | They propose to improve the faithfulness of existing models by enhancing their factual robustness by using a novel training strategy, namely FRSUM, which teaches the model to defend against both explicit adversarial samples and implicit factual adversarials. |
| Outcome: | The proposed training strategy improves faithfulness of various models, such as T5, BART, and T5 . |
PoeLM: A Meter- and Rhyme-Controllable Language Model for Unsupervised Poetry Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for generating formal verse poetry use existing poems for supervision, which are difficult to obtain for most languages and poetic forms. |
| Approach: | They propose an unsupervised approach to generate formal verse poetry without supervision . they use control codes to describe meter and rhyme scheme constraints, and train a transformer language model . |
| Outcome: | The proposed method generates poems that follow any given meter and rhyme scheme without training . it is comparable to those written by humans and generates comparable quality poems . |
ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work on dataset-generation-based zero-shot learning has shown promising results by training a task-specific model with a dataset synthesized from large pre-trained language models (PLMs). |
| Approach: | They propose a progressive zero-shot dataset generation framework which leverages feedback from the task-specific model to guide the generation of new training data via in-context examples. |
| Outcome: | The proposed framework achieves on-par or superior performance with only 1% synthetic dataset size, when compared to baseline methods without in-context feedback. |
Constructing Highly Inductive Contexts for Dialogue Safety through Controllable Reverse Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to detect toxic generation of pretrained language models rely on templates, data extraction, crowdsourcing workers or automatic generation. |
| Approach: | They propose a method to construct adversarial contexts conditioned on a given response . they augment existing dataset BAD+ and construct a new dataset B AD+ . |
| Outcome: | The proposed method can detect toxic or biased content in large pretrained language models. |
Language Prior Is Not the Only Shortcut: A Benchmark for Shortcut Learning in VQA (2022.findings-emnlp)
Copied to clipboard
Qingyi Si, Fandong Meng, Mingyu Zheng, Zheng Lin, Yuanxin Liu, Peng Fu, Yanan Cao, Weiping Wang, Jie Zhou
| Challenge: | Visual Question Answering (VQA) models are prone to learn the shortcut solution formed by dataset biases rather than the intended solution. |
| Approach: | They propose a dataset that considers varying types of shortcuts by constructing different distribution shifts in multiple OOD test sets. |
| Outcome: | The proposed dataset considers varying types of shortcuts by constructing different distribution shifts in multiple OOD test sets. |
Bridging the Training-Inference Gap for Dense Phrase Retrieval (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for building dense retrievers are often misaligned and do not reflect retrieval scenario at inference time. |
| Approach: | They propose a way to validate dense retrievers using a small subset of the entire corpus. |
| Outcome: | The proposed model improves top-1 phrase retrieval accuracy by 2 3 points and top-20 passage retrieval by 2 4 points for open-domain question answering. |
Beyond Counting Datasets: A Survey of Multilingual Dataset Construction and Necessary Resources (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have examined the quality of labeled data in non-English languages. |
| Approach: | They annotate how datasets are created, input text and label sources, tools used to build them and what they study. |
| Outcome: | The results show that language-proficient NLP researchers' estimated availability correlates with dataset availability. |
ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understanding (2022.findings-emnlp)
Copied to clipboard
Qiming Peng, Yinxu Pan, Wenjin Wang, Bin Luo, Zhenyu Zhang, Zhengjie Huang, Yuhui Cao, Weichong Yin, Yongfeng Chen, Yin Zhang, Shikun Feng, Yu Sun, Hao Tian, Hua Wu, Haifeng Wang
| Challenge: | Existing methods for visually rich document understanding lack layout-centered knowledge . experimental results show that ERNIE-Layout improves layout awareness . |
| Approach: | They propose a document pre-training solution with layout knowledge enhancement in the whole workflow to learn better representations that combine the features from text, layout, and image. |
| Outcome: | The proposed model outperforms existing models on key downstream tasks. |
Do Charge Prediction Models Learn Legal Theory? (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for charge prediction are sensitive, selective, and presumption of innocence . a recent study has shown that deep learning models can predict the charges accurately, but their reliability and interpretability are still underexplored. |
| Approach: | They propose that trustworthy charge prediction models should take legal theories into consideration . they propose three principles for trustworthy models to follow in this task . |
| Outcome: | The proposed framework evaluates whether existing models learn legal theories . it shows that models meet selective and presumption of innocence principles . |
Keep Me Updated! Memory Management in Long-term Conversations (2022.findings-emnlp)
Copied to clipboard
Sanghwan Bae, Donghyun Kwak, Soyoung Kang, Min Young Lee, Sungdong Kim, Yuin Jeong, Hyeri Kim, Sang-Woo Lee, Woomyoung Park, Nako Sung
| Challenge: | Existing studies do not deal with cases where memorized information is outdated, which may cause confusion in later conversations. |
| Approach: | They propose a task where bots keep track of and bring up the latest information about users while conversing through multiple sessions. |
| Outcome: | The proposed method outperforms baselines that leave the stored memory unchanged in terms of engagingness and humanness, and a larger performance gap in the later sessions. |
A Unified Dialogue User Simulator for Few-shot Data Augmentation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to augment large-scale task-oriented dialogues rely on annotated data. |
| Approach: | They propose to build a unified dialogue user simulation model by pre-training on publicly available datasets. |
| Outcome: | The proposed model can be tuned on a target domain with few-shot data. |
An Error-Guided Correction Model for Chinese Spelling Error Correction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing neural network approaches have achieved great progress on Chinese spelling correction, but there is still room for improvement. |
| Approach: | They propose an error-guided correction model that uses pre-trained BERT models to detect errors and integrate the error confusion set into the model. |
| Outcome: | The proposed model outperforms state-of-the-art models on widely used benchmarks and achieves superior performance on both quality and computation speed. |
Describing Sets of Images with Textual-PCA (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a new method to describe images using a common theme is needed to describe the images . a grammatical phrase is not sufficient to describe an image set, since captioning engines are not general enough. |
| Approach: | They propose a method to capture attributes of images and variations within a set . they use a pretrained vision-language model to generate a centroid phrase with the largest average similarity . |
| Outcome: | The proposed method captures the essence of image sets and describes them in a semantically meaningful way . it is easy for humans to identify and describe a common theme, but it is not generic enough . |
Learning to Model Editing Processes (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing sequence generation models produce outputs in one pass, usually left-to-right . current models model only a single edit step, and do not fully model editing . |
| Approach: | They propose to model editing processes, modeling the whole process of iteratively generating sequences. |
| Outcome: | The proposed model improves performance on a variety of axes compared to previous models . iterative refinement and editing are central parts of human creative workflow . |
PALT: Parameter-Lite Transfer of Language Models for Knowledge Graph Completion (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained language models (LMs) are a powerful transfer learning approach for knowledge graph (KG) completion. |
| Approach: | They propose a parameter-lite transfer learning approach for pretrained language models for knowledge graph (KG) completion. |
| Outcome: | The proposed model outperforms the state-of-the-art models on a knowledge graph completion benchmark by tuning 1% of the parameters. |
Prompt-based Connective Prediction Method for Fine-grained Implicit Discourse Relation Recognition (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to aid implicit discourse relation recognition (IDRR) lack explicit connectives and are difficult to implement on fine-grained IDRR. |
| Approach: | They propose a Prompt-based Connective Prediction method that instructs large-scale pre-trained models to use knowledge relevant to discourse relation and utilizes strong correlation between connectives and discourse relation to help the model recognize implicit discourse relations. |
| Outcome: | The proposed method surpasses the state-of-the-art model and achieves significant improvements on those fine-grained few-shot discourse relation classes. |
On Utilizing Constituent Language Resources to Improve Downstream Tasks in Hinglish (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Intuitively, Hindi and English corpora should aid improve task performance on code-switched Hindi-English. |
| Approach: | They propose a meta-learning framework that utilizes the labelled resources of the downstream tasks in the constituent languages to improve task performance. |
| Outcome: | The proposed framework improves the performance on downstream tasks on code-switched Hindi-English. |
SYGMA: A System for Generalizable and Modular Question Answering Over Knowledge Bases (2022.findings-emnlp)
Copied to clipboard
Sumit Neelam, Udit Sharma, Hima Karanam, Shajith Ikbal, Pavan Kapanipathi, Ibrahim Abdelaziz, Nandana Mihindukulasooriya, Young-Suk Lee, Santosh Srivastava, Cezar Pendus, Saswati Dana, Dinesh Garg, Achille Fokoue, G P Shrivatsa Bhargav, Dinesh Khandelwal, Srinivas Ravishankar, Sairam Gurajada, Maria Chang, Rosario Uceda-Sosa, Salim Roukos, Alexander Gray, Guilherme Lima, Ryan Riegel, Francois Luus, L V Subramaniam
| Challenge: | Knowledge Base Question Answering (KBQA) systems have limited generalizability across knowledge bases and multiple reasoning types. |
| Approach: | They propose a modular approach for KBQA that is built on a framework adaptable to multiple knowledge bases and reasoning types. |
| Outcome: | The proposed approach is generalized across multiple knowledge bases and reasoning types. |
Instance-Guided Prompt Learning for Few-Shot Text Matching (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Few-shot text matching is a more practical technique to determine whether two texts are semantically identical. |
| Approach: | They propose a pluggable prompt learning method for few-shot text matching . they use the semantics of instances to regulate the effects of the gate on the prompt tokens . |
| Outcome: | The proposed method outperforms baselines on MRPC and QQP. |
M3: Multi-level dataset for Multi-document summarisation of Medical studies (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing summarisation systems are not up to such complex tasks, yet limited tools exist to determine where and why they are failing. |
| Approach: | They propose to use a dataset to evaluate the quality of summarisation systems in the biomedical domain. |
| Outcome: | The proposed model can be used to evaluate the quality of summarisation systems in the biomedical domain. |
Adapters for Enhanced Modeling of Multilingual Knowledge and Text (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models learn facts from text corpora, but knowledge graphs contain facts in an explicit triple format, restricting their research and application. |
| Approach: | They propose to enhance multilingual language models with knowledge from multilingual knowledge graphs . they propose to use cross-lingual entity alignment and facts from MLKGs to improve performance . |
| Outcome: | The proposed model improves MLLMs with cross-lingual entity alignment and facts from multilingual knowledge graphs for many languages while maintaining performance on other general language tasks. |
SepLL: Separating Latent Class Labels from Weak Supervision Noise (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for learning from weak labels use heuristics and heurism to create weak labels. |
| Approach: | They propose a weakly supervised learning paradigm that uses human intuitions to create weak (noisy) labels. |
| Outcome: | The proposed model is competitive with the state-of-the-art and yields a new best average performance. |
Probing Relational Knowledge in Language Models via Word Analogies (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have focused on probing relational knowledge by filling the blanks in pre-defined prompts such as “The capital of France is —” but these are affected by the co-occurrence of target relation words and entities in the pre-training corpus. |
| Approach: | They extend probing methodologies by using analogical proportions as a proxy to probe relational knowledge in transformer-based PLMs without directly presenting the desired relation. |
| Outcome: | The proposed methods are extremely accurate at (1) and (2), but have room for improvement for (3). |
Semi-Supervised Lifelong Language Learning (2022.findings-emnlp)
Copied to clipboard
Yingxiu Zhao, Yinhe Zheng, Bowen Yu, Zhiliang Tian, Dongkyu Lee, Jian Sun, Yongbin Li, Nevin L. Zhang
| Challenge: | Existing methods to learn languages only focus on supervised learning, and unlabeled data is underexplored. |
| Approach: | They propose a semi-supervised lifelong language learning setting where a model learns sequentially arriving language tasks with both labeled and unlabeled data. |
| Outcome: | The proposed model outperforms baseline models on various language tasks and is effective and superior to existing models. |
Parameter-free Automatically Prompting: A Latent Pseudo Label Mapping Model for Prompt-based Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing manual label mapping methods that require extra parameters and human knowledge are limited in data. |
| Approach: | They propose a Latent Pseudo Label Mapping method that optimizes the label mapping without human knowledge and extra parameters. |
| Outcome: | The proposed method outperforms the standard SOTA method in few-shot learning tasks and significantly outperformed the standard ALM method which requires extra task-specific prior knowledge. |
Exploring Logographic Image for Chinese Aspect-based Sentiment Classification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for aspect-based sentiment classification have focused on English text, but Chinese is a language derived from pictographs and different from other phonetic languages. |
| Approach: | They propose to use a logographic image to capture internal morphological structure from character sequence . they propose to explicitly incorporate a symbolic image with review text for sentiment classification . |
| Outcome: | The proposed method improves over baselines and improves on existing methods. |
On the Role of Bidirectionality in Language Model Pre-Training (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work on language model pre-training explored different architectures and learning objectives, but differences in data, hyperparameters and evaluation make a principled comparison difficult. |
| Approach: | They propose a framework that generalizes prior approaches to pre-training language models by focusing on bidirectionality and controlling each of them separately. |
| Outcome: | The proposed framework generalizes prior approaches including fully unidirectional models like GPT, fully bidirectional models and hybrid models like CM3 and prefix LM. |
You Are What You Talk About: Inducing Evaluative Topics for Personality Analysis (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have focused on the relationship between personality and evaluative language. |
| Approach: | They propose to map evaluative topics to pre-filtered evalative text and link evalueative topics with individual text authors to build their ev emvaluative profiles. |
| Outcome: | The proposed approach is validated by observing correlations consistent with prior research in personality psychology. |
CAT-probing: A Metric-based Approach to Interpret How Pre-trained Models for Programming Language Attend Code Structure (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing code pre-trained models fail to consider inherent characteristics of codes . Existing methods to interpret code pretrained model fail to take into account inherent characteristics . |
| Approach: | They propose a probing method to quantitatively interpret how CodePTMs attend code structure. |
| Outcome: | The proposed method denoises input code sequences and measures commonality between token-level attention scores and pair-wise distances between corresponding AST nodes. |
Learning to Revise References for Faithful Summarization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that noisy reference summaries can be detrimental to model performance. |
| Approach: | They propose to selectively re-write unsupported reference sentences to better reflect source data. |
| Outcome: | The proposed method improves reference quality while retaining all data. |
Towards Intention Understanding in Suicidal Risk Assessment with Natural Language Processing (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Suicide is a global problem, with one suicide case for every 100 deaths worldwide . social networking sites are an essential forum for communication and information sharing . |
| Approach: | This paper compares natural language processing to suicidal ideation detection and risk assessment . it urges better intention understanding for reliable suicide risk assessment with computational methods . |
| Outcome: | This paper compares the performance of natural language processing to suicidal ideation detection and risk assessment tasks. |
On the Impact of Temporal Concept Drift on Model Explanations (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Explanation faithfulness of model predictions is typically evaluated on held-out data from the same temporal distribution as the training data. |
| Approach: | They examine the impact of temporal variation on model explanations extracted by eight feature attribution methods and three select-then-predict models across six text classification tasks. |
| Outcome: | The proposed method shows the most robust faithfulness scores across datasets and in asynchronous settings. |
Text-Only Training for Image Captioning using Noise-Injected CLIP (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a new approach to image captioning requires large datasets of captioned images and is difficult to collect. |
| Approach: | They propose to use a decoder to translate CLIP textual embeddings back into text . they show that this intuition is “almost correct” because of a gap between the embeddable spaces . |
| Outcome: | The proposed approach shows that the intuition is “almost correct” because of a gap between the embedding spaces, and rectifies this via noise injection during training. |
Improving Sharpness-Aware Minimization with Fisher Mask for Better Generalization on Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning pretrained language models suffer from poor generalization . however, they add a perturbation to each model parameter equally, which is sub-optimal . |
| Approach: | They propose a sharpness-aware minimization optimization procedure that introduces a Fisher mask to improve the efficiency of SAM. |
| Outcome: | The proposed method outperforms the vanilla sharpness-aware minimization method on GLUE and SuperGLUE benchmarks. |
TINA: Textual Inference with Negation Augmentation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing transformer-based models perform poorly on textual entailment when examples contain negations. |
| Approach: | They propose a new definition of textual entailment that captures negation and a principled technique for negated data augmentation that can be combined with unlikelihood loss function. |
| Outcome: | The proposed method significantly improves on textual entailment datasets with negations without sacrificing performance on datasets without negation. |
Improving Bilingual Lexicon Induction with Cross-Encoder Reranking (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Current methods for bilingual lexicon induction rely on the induction of cross-lingual word embeddings (CLWEs) such as VecMap or mPLMs are not available for multilingual NLP. |
| Approach: | They propose a semi-supervised post-hoc reranking method which combines cross-lingual lexical knowledge from multilingual pretrained language models with original CLWEs. |
| Outcome: | The proposed method outperforms existing methods on two standard benchmarks spanning a wide spectrum of languages and is robust to different CLWEs. |
Mixed-modality Representation Learning and Pre-training for Joint Table-and-Text Retrieval in OpenQA (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to retrieve evidences from corpus are difficult due to table-text discrepancy and data sparsity problem. |
| Approach: | They propose an optimized OpenQA Table-Text Retriever to retrieve tabular and textual evidences from tabular resources. |
| Outcome: | The proposed OpenQA Table-Text Retriever significantly outperforms existing methods on QA tasks. |
The Effects of Corpus Choice and Morphosyntax on Multilingual Space Induction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work on inductive biases of language models towards natural language has focused on quantifying their ability to build multilingual spaces. |
| Approach: | They propose to use linguistically motivated tasks as a proxy to study inductive biases of language models with respect to natural language phenomena to build multilingual embedding spaces. |
| Outcome: | The proposed model performance is compared with other models using a set of linguistically motivated tasks and a training corpus in 15 languages. |
Modeling Complex Dialogue Mappings via Sentence Semantic Segmentation Guided Conditional Variational Auto-Encoder (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing efforts to identify and avoid CDM to facilitate dialogue learning failed to solve the problem. |
| Approach: | They propose a Sentence Semantic Segmentation guided Conditional Variational Auto-Encoder which can model and take advantage of the CDM data. |
| Outcome: | The proposed method can model and take advantages of the CDM data. |
Graph Embeddings for Argumentation Quality Assessment (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Argumentation is the process by which arguments are constructed, compared, evaluated in several respects and judged in order to establish whether any of them is warranted. |
| Approach: | They propose to annotate 1908 arguments tagged with quality facets from a resource of 402 persuasive essays and to use them to create a neural architecture that takes into account the support and attack relations holding among the arguments. |
| Outcome: | The proposed neural architecture outperforms state-of-the-art and standard arguments on the persuasive essays dataset. |
SMiLE: Schema-augmented Multi-level Contrastive Learning for Knowledge Graph Link Prediction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for linking knowledge graphs lack contextual information in entity neighborhoods, which leads to false prediction results. |
| Approach: | They propose a Schema-augmented Multi-level contrastive LEarning framework to conduct knowledge graph link prediction using a knowledge graph schema. |
| Outcome: | The proposed framework is based on a knowledge graph schema and is compared against state-of-the-art datasets. |
Multilingual Multimodal Learning with Machine Translated Text (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Currently, most vision-and-language pretraining research focuses on English tasks due to the availability of datasets. |
| Approach: | They propose a framework for machine translating English multimodal data to improve training data . they propose two metrics to prevent models from learning from low-quality translated text . |
| Outcome: | The proposed framework can be applied to any multimodal dataset and model. |
Learning From the Source Document: Unsupervised Abstractive Summarization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for abstractive summarization are under supervised training, but obtaining high-quality and large-scale datasets for supervised learning is laboriously difficult. |
| Approach: | They propose an unsupervised method that leverages contrastive learning to generate summaries by rewriting and paraphrasing the source documents to generate good summary. |
| Outcome: | The proposed method outperforms baseline methods on extensive experiments on source documents and fake documents. |
How to Do Things without Words: Modeling Semantic Drift of Emoji (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Emoji have become a significant part of our informal textual communication. |
| Approach: | They propose to model and analyze the semantic drift of emoji and explore the relations between graphical changes and semantic changes. |
| Outcome: | The proposed model and analysis examines the relationship between graphical changes and semantic drift. |
Mind Your Bias: A Critical Review of Bias Detection Methods for Contextual Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for detection of biases in contextual language models are inconsistent and inconclusive. |
| Approach: | They propose to use word embedding association test to detect biases in contextual language models to compare them with other methods. |
| Outcome: | The proposed methods are inconsistent and inconclusive for language models with word embeddings. |
ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that task scaling can be an efficient alternative to model scaling. |
| Approach: | They propose a multitask pretraining approach ZeroPrompt for zero-shot generalization . they focus on task scaling and zero-shooting to improve model performance . |
| Outcome: | The proposed approach improves zero-shot generalization efficiency by 30 times with task scaling. |
Semantic Role Labeling Meets Definition Modeling: Using Natural Language to Describe Predicate-Argument Structures (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to Semantic Role Labeling rely on discrete labels to classify predicate senses and their arguments. |
| Approach: | They propose a generalized formulation of Semantic Role Labeling that leverages Definition Modeling to describe predicate-argument structures using natural language definitions instead of discrete labels. |
| Outcome: | The proposed model can describe predicate-argument structures using natural language definitions instead of discrete labels. |
Is anisotropy really the cause of BERT embeddings not being semantic? (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to train contextual language models for NLP use a lightweight approach called bi-encoder, which takes two sentences as input, but does not perform well with vanilla pre-trained Transformers. |
| Approach: | They conduct a set of experiments to improve our understanding of the lack of semantic isometry in contextualized word representations in BERT. |
| Outcome: | The proposed approach does not perform well with vanilla pre-trained Transformers. |
m^4 Adapter: Multilingual Multi-Domain Adaptation for Machine Translation with a Meta-Adapter (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Multilingual neural machine translation models (MNMT) are effective on transferring knowledge between high-resource languages to low-resourced languages. |
| Approach: | They propose a multilingual multi-domain adapter which combines domain and language knowledge using meta-learning with adapters. |
| Outcome: | The proposed model outperforms other adapter methods in a domain shift and language pair translation task. |
Textual Enhanced Contrastive Learning for Solving Math Word Problems (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that current models rely on shallow heuristics to predict solutions . a textual Enhanced Contrastive Learning framework enforces the models to distinguish semantically similar examples while holding different mathematical logic. |
| Approach: | They propose a textual Enhanced Contrastive Learning framework which enforces models to distinguish semantically similar examples while holding different mathematical logic. |
| Outcome: | The proposed framework improves on benchmark and challenge datasets in English and Chinese. |
What Do Compressed Multilingual Machine Translation Models Forget? (2022.findings-emnlp)
Copied to clipboard
Alireza Mohammadshahi, Vassilina Nikoulina, Alexandre Berard, Caroline Brun, James Henderson, Laurent Besacier
| Challenge: | Recent studies show that pre-trained models achieve state-of-the-art results in NLP tasks but their size makes it more challenging to apply them in resource-constrained environments. |
| Approach: | They assess the impact of compression methods on multilingual Neural Machine Translation models for various language groups, gender, and semantic biases. |
| Outcome: | The proposed compression methods improve models on different benchmarks for language groups, gender, and semantic biases. |
Controllable Dialogue Simulation with In-context Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to generate annotated dialogues require crowdsourcing, which is expensive and time-consuming. |
| Approach: | They propose a dialogue simulation method based on large language model in-context learning that generates new dialogues and annotations in a controllable way. |
| Outcome: | The proposed method can expand a small set of dialogue data with minimum or zero human involvement and parameter update. |
Improving the Factual Correctness of Radiology Report Generation with Semantic Rewards (2022.findings-emnlp)
Copied to clipboard
Jean-Benoit Delbrouck, Pierre Chambon, Christian Bluethgen, Emily Tsai, Omar Almusa, Curtis Langlotz
| Challenge: | Neural image-to-text radiology report generation systems have been successful on NLG metrics, but they are not factually complete or consistent due to inadequate training and evaluation. |
| Approach: | They propose a method to improve the factual completeness and correctness of generated radiology reports by using a dataset containing annotated chest X-ray images. |
| Outcome: | The proposed method significantly improves factual completeness and correctness of generated radiology reports on two open radiology report datasets. |
Recursive Neural Networks with Bottlenecks Diagnose (Non-)Compositionality (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Compositional generalisation is often investigated with artificial languages or highly-structured natural language data. |
| Approach: | They propose to use recursive neural models with bottlenecks to generalise compositionally for artificial languages. |
| Outcome: | The proposed model can generalise compositionally for natural language tasks without limiting the transfer of information between nodes. |
HumSet: Dataset of Multilingual Information Extraction and Classification for Humanitarian Crises Response (2022.findings-emnlp)
Copied to clipboard
Selim Fekih, Nicolo’ Tamagnone, Benjamin Minixhofer, Ranjan Shrestha, Ximena Contla, Ewan Oglethorpe, Navid Rekabsaz
| Challenge: | During humanitarian crises, a quick and accurate analysis of relevant data is critical to a timely and effective response. |
| Approach: | They introduce and release a multilingual dataset of humanitarian response documents annotated by experts in the humanitarian response domain. |
| Outcome: | The proposed dataset provides documents in three languages and covers a variety of humanitarian crises from 2018 to 2021 across the globe. |
Viterbi Decoding of Directed Acyclic Transformer for Non-Autoregressive Machine Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Non-autoregressive models lack the ability to capture sequential dependency . Existing approaches to model sequential dependency have to apply a sequential decision process at inference time . |
| Approach: | They propose a Viterbi decoding framework to capture sequential dependency . they propose to find the optimal translation path under any length constraint . |
| Outcome: | The proposed framework improves the performance of DA-Transformer while maintaining similar speedup. |
Lexical Generalization Improves with Larger Models and Longer Training (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Excessive utilization of lexical overlap heuristics can lead to failure on challenging inputs. |
| Approach: | They analyze the use of lexical overlap heuristics in natural language inference, paraphrase detection, and reading comprehension using a contrastive dataset. |
| Outcome: | The proposed model can be used to improve performance on a wide range of tasks, but it is often wrong. |
Realistic Data Augmentation Framework for Enhancing Tabular Reasoning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to constructing training data for Natural Language Inference (NLI) tasks are expensive and time consuming. |
| Approach: | They propose a semi-automated framework for data augmentation for tabular inference . framework generates hypothesis templates transferable to similar tables . authors say framework could generate human-like tabular examples . |
| Outcome: | The proposed framework generates human-like tabular inference examples . it is based on human-written constraints and premise paraphrasing . |
Inducing Generalizable and Interpretable Lexica (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Lexica are widely used as generalizable language features to predict sentiment, emotions, mental health, and personality. |
| Approach: | They propose to induce lexica using context-oblivious and context-aware approaches and compare their performance using crowd-worker assessment. |
| Outcome: | The proposed models can be induced using context-oblivious and context-aware approaches and evaluate their quality using crowd-worker assessment. |
The Curious Case of Absolute Position Embeddings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | In natural language, it is not absolute position that matters, but relative position . et al., 2017) language models incorporate positional encodings that encode absolute (linear) word order. |
| Approach: | They find that Transformer language models encode word order using positional information . they also find that models that use absolute position embeddings over-rely on positional data . |
| Outcome: | The results raise questions about the efficacy of APEs to model the relativity of position information. |
Goal-oriented Vision-and-Dialog Navigation via Reinforcement Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for vision-and-dialog navigation are limited and do not adapt to domain changes. |
| Approach: | They propose a problem where an agent computes dialog-navigation policies from trial and error. |
| Outcome: | The proposed agent outperforms baselines in success rate in photo-realistic simulations. |
Leveraging Data Recasting to Enhance Tabular Reasoning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to create tabular inference data are limited by human annotation and synthetic generation. |
| Approach: | They propose a framework for semi-automatically recasting tabular data to make use of both approaches. |
| Outcome: | The proposed framework can be used to build tabular NLI instances from five datasets. |
Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Again (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Large pre-trained language models (PLMs) such as GPT-3 have shown strong in-context learning capabilities, which are appealing for domains such as biomedicine that feature high and diverse demands of language technologies but also high data annotation costs. |
| Approach: | They propose to compare the few-shot performance of GPT-3 in-context learning with fine-tuning smaller (i.e., BERT-sized) PLMs on two representative biomedical information extraction tasks: named entity recognition and relation extraction. |
| Outcome: | The proposed model underperforms on two representative biomedical information extraction tasks. |
Attention weights accurately predict language representations in the brain (2022.findings-emnlp)
Copied to clipboard
| Challenge: | In Transformer-based language models, the attention mechanism converts token embeddings into contextual embeddables that incorporate information from neighboring words. |
| Approach: | They analyze fMRI recordings of English language learners and extract attention weights from them to determine how well they can predict brain responses. |
| Outcome: | The resulting hidden state embeddings are more accurate than lexical embeddngs or RNN-based models. |
Improving HowNet-Based Chinese Word Sense Disambiguation with Translations (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work on unsupervised WSD has leveraged lexical knowledge bases, such as WordNet and BabelNet, but these have proven to be less effective for Chinese. |
| Approach: | They propose a system which combines contextual information from a pretrained neural language model with bilingual information obtained via machine translation and sense translation information from HowNet. |
| Outcome: | The proposed system achieves a state-of-the-art for unsupervised Chinese WSD. |
Mask-then-Fill: A Flexible and Effective Data Augmentation Framework for Event Extraction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing data augmentation methods for event extraction are costly and time-consuming. |
| Approach: | They propose a data augmentation framework that randomly masks out an adjunct sentence fragment and infills a variable-length text span with a fine-tuned infilling model. |
| Outcome: | The proposed framework can generate more diverse data while keeping the original structure unchanged . it can replace a fragment of arbitrary length in the text with another fragment of variable length . |
MOBA-E2C: Generating MOBA Game Commentaries via Capturing Highlight Events from the Meta-Data (2022.findings-emnlp)
Copied to clipboard
| Challenge: | e-sports game competitions lack commentators because of the shortage of professional human commentators. |
| Approach: | They propose a data-driven MOBA commentary generation framework for MOBA games . they use a rule-based generator and a generative GPT generator to generate commentaries . |
| Outcome: | The proposed model generates commentaries based on the game meta-data and a rule-based generator and generative GPT generator. |
Enhancing Automatic Readability Assessment with Pre-training and Soft Labels for Ordinal Regression (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models do not exploit ordinal nature of difficulty grades and make little effort for initialization to facilitate fine-tuning. |
| Approach: | They propose a readability assessment task that assigns a difficulty grade to a text . they use ordinal regression and pairwise relative text difficulty to train the model . |
| Outcome: | The proposed model outperforms competitive neural models and statistical classifiers on most datasets. |
Opening up Minds with Argumentative Dialogues (2022.findings-emnlp)
Copied to clipboard
Youmna Farag, Charlotte Brand, Jacopo Amidei, Paul Piwek, Tom Stafford, Svetlana Stoyanchev, Andreas Vlachos
| Challenge: | Recent research on argumentative dialogues has focused on persuading people to take some action, changing their stance on the topic of discussion, or winning debates. |
| Approach: | They present a dataset of 183 argumentative dialogues about veganism, Brexit and COVID-19 vaccination. |
| Outcome: | The proposed model is significantly better on other dialogue properties such as engagement and clarity. |
You Are My Type! Type Embeddings for Pre-trained Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work has shown that Pre-trained language models can encode semantic types, but it is not clear how to use types to steer the output. |
| Approach: | They propose to embed a type by a small set of word examples to promote desired types in a PLM. |
| Outcome: | The proposed model can represent types and steer masking predictions without changes to the prompt text without changes in the prompt. |
Generating Textual Adversaries with Minimal Perturbation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing word-level adversarial approaches for textual data have various limitations due to the large search space consisting of combinations of candidate words. |
| Approach: | They propose a novel attack strategy to find adversarial texts with high similarity to original texts without perturbation. |
| Outcome: | The proposed approach achieves higher success rates and lower perturbation rates in four benchmark datasets compared with state-of-the-art approaches. |
SensePOLAR: Word sense aware interpretability for pre-trained contextual word embeddings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing word embedding models lack interpretability for words . |
| Approach: | They propose to add interpretability to word embeddings by using a POLAR framework that enables wordsense aware interpretations for pre-trained contextual word embeds. |
| Outcome: | The proposed framework achieves comparable performance to existing embeddings across GLUE and SQuAD benchmarks. |
Contextualizing Language Models for Norms Diverging from Social Majority (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies on transformer-based language models have shown that there seems to be a 'moral dimension' to LMs, as they show high accuracy in related downstream tasks such as moral reasoning and action classification. |
| Approach: | They propose a mechanism based on deontic logic to allow for a flexible adaptation of individual norms by de-biasing training data sets and a task-reduction to textual entailment. |
| Outcome: | The proposed mechanism de-biases training data sets and reduces tasks to textual entailment. |
Empathetic Dialogue Generation via Sensitive Emotion Recognition and Sensible Knowledge Selection (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Empathy is a key trait of everyday human conversations. |
| Approach: | They propose a serial encoding and Emotion-Knowledge interaction method for empathetic dialogue generation which is more sensitive to emotion dynamics in conversations. |
| Outcome: | The proposed method outperforms baseline evaluations on the utterance-level annotated EMPATHETICDIALOGUES. |
Joint Multilingual Knowledge Graph Completion and Alignment (2022.findings-emnlp)
Copied to clipboard
Vinh Tong, Dat Quoc Nguyen, Trung Thanh Huynh, Tam Thanh Nguyen, Quoc Viet Hung Nguyen, Mathias Niepert
| Challenge: | Existing work on multilingual KG completion has focused on entity and relation alignments, but understanding of how it can aid multilingual alignments is limited. |
| Approach: | They propose to combine two components that jointly accomplish KG completion and alignment. |
| Outcome: | The proposed model outperforms existing competitive baselines on a public multilingual benchmark and achieves state-of-the-art results. |
A Framework for Automatic Generation of Spoken Question-Answering Data (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing frameworks to automatically generate a spoken question answering dataset are limited by the amount of spoken text documents available. |
| Approach: | They propose to use QG module to generate questions from text documents, TTS module to convert text documents into spoken form and automatic speech recognition module to transcribe spoken content. |
| Outcome: | The proposed framework is efficient for automatically generating spoken QA datasets. |
Readability Controllable Biomedical Document Summarization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing controllable summarization systems for biomedical documents have little attention to readability control, leaving users with incompatible summaries . |
| Approach: | They propose a task of readability controllable summarization for biomedical documents to generate summaries that are incompatible with users' levels of expertise. |
| Outcome: | The proposed model is based on pre-trained language models with prevalent controlling and generation techniques and evaluates the readability discrepancy between lay and technical summaries. |
Beyond Additive Fusion: Learning Non-Additive Multimodal Interactions (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Multimodal fusion addresses the problem of analyzing spoken words in the multimodal context, including visual expressions and prosodic cues. |
| Approach: | They propose to use multimodal fusion to separate unimodal, bimodal, and trimodal interactions in a multimodal model. |
| Outcome: | The proposed model separates unimodal, bimodal, and trimodal interactions while not degrading predictive performance. |
Generalization Differences between End-to-End and Neuro-Symbolic Vision-Language Reasoning Systems (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Widely used multi-modal pretrained models generalize poorly to out-of-distribution data, suggesting shortcomings in the VLE2E pipeline. |
| Approach: | They develop a segment-combine test for multi-image queries and contrast set for cross-benchmark transfer. |
| Outcome: | The proposed method shows that it is possible to train both neural and neuro-symbolic models in the same way. |
Learning to Model Multimodal Semantic Alignment for Story Visualization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Story visualization aims to generate sequence of images to narrate each sentence in a multi-sentence story . current methods face semantic misalignment because of their fixed architecture and diversity of input modalities . |
| Approach: | They propose to use a GAN-based generative model to match semantic levels between text and image representations to solve the semantic misalignment problem. |
| Outcome: | Experiments show that the proposed approach improves image quality and story consistency compared with state-of-the-art methods. |
SciFact-Open: Towards open-domain scientific claim verification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Current scientific claim verification systems can achieve very strong performance on limited contexts, in some cases approaching human agreement. |
| Approach: | They propose to pool and annotate top predictions from four state-of-the-art scientific claim verification models to evaluate their performance against large corpora. |
| Outcome: | The proposed system performs well on a corpus of 500K scientific abstracts. |
COMET-QE and Active Learning for Low-Resource Machine Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Using COMET-QE, we select sentences for low-resource neural machine translation. |
| Approach: | They propose a reference-free evaluation metric to select sentences for low-resource neural machine translation using Swahili, Kinyarwanda and Spanish. |
| Outcome: | The proposed method outperforms two variants of Round Trip Translation Likelihood and random sentence selection by up to 5 BLEU points on a 30k baseline. |
MedicalSum: A Guided Clinical Abstractive Summarization Model for Generating Medical Reports from Patient-Doctor Conversations (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for summarizing medical conversations do not take clinical knowledge into account and are difficult to control. |
| Approach: | They propose a transformer-based sequence-to-sequence architecture for summarizing medical conversations by integrating medical domain knowledge from the Unified Medical Language System (UMLS). |
| Outcome: | The proposed model achieves state-of-the-art ROUGE score improvements of 0.8-2.1 points (including 6.2% error reduction in the PE section) it incorporates medical domain knowledge from the Unified Medical Language System (UMLS). |
Leveraging Training Dynamics and Self-Training for Text Classification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Semi-supervised learning (SSL) is a promising technique for improving deep learning models when training data is scarce. |
| Approach: | They propose a semi-supervised learning approach that leverages training dynamics of unlabeled data. |
| Outcome: | The proposed method achieves an average increase in F1 score of 3.5% over baselines in low resource settings. |
Learning to Infer from Unlabeled Data: A Semi-supervised Learning Approach for Robust Natural Language Inference (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Semi-supervised learning (SSL) is a popular technique for reducing the reliance on human annotations for NLI tasks. |
| Approach: | They propose a way to incorporate unlabeled data into semi-supervised learning (SSL) using a conditional language model, they propose to generate hypotheses for unlabed sentences . |
| Outcome: | The proposed framework significantly improves the performance of four NLI datasets in low-resource settings. |
Unsupervised Text Deidentification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to deidentify textual data are supervised named entity recognition based on 18 identifiers like age, phone number, zip code . prior work ignores this nebulous 18th category, allowing for a supervised method to be used . |
| Approach: | They propose an unsupervised method that masks words that leak personally-identifying information. |
| Outcome: | The proposed method masks words that leak personally-identifying information . it eliminates many identifying aspects that would fall outside of the common named entity based approach . |
Federated Continual Learning for Text Classification via Selective Inter-client Transfer (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Continual Learning (CL) is a privacy-preserving machine learning technique that enables collaborative training of ML models by sharing model parameters across distributed clients. |
| Approach: | They propose a framework which selectively combines model parameters of foreign clients to maximize knowledge transfer while preserving privacy. |
| Outcome: | The proposed framework improves the performance of a text classification task using five datasets from diverse domains while preserving privacy. |
DOROTHIE: Spoken Dialogue for Handling Unexpected Situations in Interactive Autonomous Driving Agents (2022.findings-emnlp)
Copied to clipboard
Ziqiao Ma, Benjamin VanDerPloeg, Cristian-Paul Bara, Yidong Huang, Eui-In Kim, Felix Gervits, Matthew Marge, Joyce Chai
| Challenge: | Empowering autonomous driving agents with the ability to navigate in a continuous and dynamic environment is critical. |
| Approach: | They propose a novel interactive simulation platform that enables the creation of unexpected situations on the fly to support empirical studies on situated communication with autonomous driving agents. |
| Outcome: | The proposed platform enables the creation of unexpected situations on the fly to support empirical studies on situated communication with autonomous driving agents. |
He Said, She Said: Style Transfer for Shifting the Perspective of Dialogues (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on style transfer has focused on controlling formality, authorial style, and sentiment of text. |
| Approach: | They propose a style transfer task that reframes a dialogue from informal first person to formal third person rephrasing . they use a dataset to annotate dialogues from a text summarization corpus . |
| Outcome: | The proposed task improves the performance of extractive models on a dialogue summarization dataset. |
Dynamic Augmentation Data Selection for Few-shot Text Classification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Data augmentation is a popular method for fine-tuning pre-trained language models to increase model robustness and performance. |
| Approach: | They propose a dynamic data selection method to select effective augmentation data from different augmentation sources according to the model’s learning stage by identifying a set of augmentation samples that optimally facilitates the learning process of the most current model. |
| Outcome: | The proposed method outperforms strong baselines on a variety of sentence classification tasks. |
KPDROP: Improving Absent Keyphrase Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to generate absent keyphrases have limited results in generating present keyphrase. |
| Approach: | They propose a model-agnostic approach to generate absent keyphrases by randomly dropping present keyphrase from a document and turning them into artificial absent key phrases during training. |
| Outcome: | The proposed approach improves absent keyphrase generation in both supervised and resource-constrained settings. |
Natural Language Deduction through Search over Statement Compositions (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods focus on the end-to-end discriminative version of this task, but less work has treated the generative version of the task. |
| Approach: | They propose a system that decomposes the task into separate steps coordinated by a search procedure and produces a tree of intermediate conclusions that faithfully reflects the system’s reasoning process. |
| Outcome: | The proposed system proves true statements while rejecting false ones on the EntailmentBank dataset with a 17% absolute higher step validity than the end-to-end T5 model. |
EnDex: Evaluation of Dialogue Engagingness at Scale (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models that measure engagement use expensive human annotas and abstract definitions of the term. |
| Approach: | They propose a human-reaction based model to evaluate dialogue engagingness . they propose combining distant-supervision with a theoretical foundation for engagement . |
| Outcome: | The proposed model is trained on 80k Reddit-based engagement datasets . it uses distant-supervision from human-reaction feedback to evaluate dialogue engagementness . |
LOPS: Learning Order Inspired Pseudo-Label Selection for Weakly Supervised Text Classification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Weakly-supervised text classification methods are noisy due to their heuristic nature . selection of correct pseudo-labels has a huge potential for performance boost . |
| Approach: | They propose a pseudo-label selection method that takes learning order into account . they propose to select samples that are learnt earlier based on their pseudo-labels . |
| Outcome: | The proposed method is ineffective and unstable due to erroneous predictions from poorly calibrated models. |
Train Flat, Then Compress: Sharpness-Aware Minimization Learns More Compressible Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in hardware, modeling, and optimization for deep neural networks have led to improvements in memory and inference efficiency. |
| Approach: | They propose to combine sharpness-aware minimization with various model compression methods to improve model compressibility. |
| Outcome: | Empirically, optimizing for flatter minima leads to greater compressibility of parameters compared to vanilla Adam when fine-tuning BERT models, with little to no loss in accuracy on the GLUE text classification and SQuAD question answering benchmarks. |
Structural Contrastive Representation Learning for Zero-shot Multi-label Text Classification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches for zero-shot multi-label text classification struggle with accuracy and poor training efficiency. |
| Approach: | They propose a structural contrastive representation learning approach that uses randomized text segmentation to generate high-quality contrastive pairs. |
| Outcome: | The proposed approach improves accuracy and speed up training time on publicly available datasets. |
Improving Generalization of Pre-trained Language Models via Stochastic Weight Averaging (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that the flatness of the local minimum correlates well with better generalization. |
| Approach: | They propose to use a method encouraging convergence to a flatter minimum to fine-tune PLMs. |
| Outcome: | The proposed method outperforms state-of-the-art methods on NLP tasks without extra computation cost. |
Learn What Is Possible, Then Choose What Is Best: Disentangling One-To-Many Relations in Language Through Text-based Games (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Language models pre-trained on large self-supervised corpora, followed by task-specific fine-tuning has become the dominant paradigm in NLP. |
| Approach: | They propose to train language models pre-trained on large self-supervised corpora, followed by task-specific fine-tuning on the target domain. |
| Outcome: | The proposed model improves on the previous state-of-the-art model on the Jericho Walkthroughs dataset by 49%. |
Structurally Diverse Sampling for Sample-Efficient Training and Comprehensive Evaluation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to generalize compositionally are inadequate, but there is no evidence for this. |
| Approach: | They propose a model-agnostic algorithm for subsampling instances with diverse structures from a labeled instance pool with structured outputs. |
| Outcome: | The proposed algorithm leads to comparable or better generalization than prior algorithms in 9 out of 10 dataset-split type pairs. |
Unsupervised Multi-Granularity Summarization (2022.findings-emnlp)
Copied to clipboard
Ming Zhong, Yang Liu, Suyu Ge, Yuning Mao, Yizhu Jiao, Xingxing Zhang, Yichong Xu, Chenguang Zhu, Michael Zeng, Jiawei Han
| Challenge: | Experimental results confirm the substantial superiority of GranuSum on multi-granularity summarization over strong baselines. |
| Approach: | They propose to rank events by their salience and annotate a benchmark for GranuSum that contains multiple summaries at different granularities for each document cluster. |
| Outcome: | The proposed framework is capable of producing multi-granular summaries in unsupervised manner over strong baselines. |
HeLo: Learning-Free Lookahead Decoding for Conversation Infilling (2022.findings-emnlp)
Copied to clipboard
| Challenge: | We propose a scalable decoding strategy for conversation infilling . large pretrained language models are effective solutions to many popular natural language generation tasks such as machine translation and conversational dialogue. |
| Approach: | They propose a heuristic guided lookahead decoding strategy for conversation infilling which leverages a greedy lookalike phase before committing to any token. |
| Outcome: | The proposed strategy outperforms baselines when evaluated with automatic and human evaluation metrics, which, we argue, are appropriate for the task. |
Invernet: An Inversion Attack Framework to Infer Fine-Tuning Datasets through Word Embeddings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing word embeddings are data intensive and require large-scale training corpus, sufficient training iterations, and high computational capacity. |
| Approach: | They propose a framework that infers context distributions from a downstream dataset and then uses them to fine-tune the embedding. |
| Outcome: | The proposed framework materializes privacy concern by inferring context distribution in the downstream dataset, which can lead to key information breach. |
LawngNLI: A Long-Premise Benchmark for In-Domain Generalization from Short to Long Contexts and for Implication-Based Retrieval (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a new benchmark for natural language inference is proposed for long premises . lawngNLI can train and test systems for implication-based case retrieval and argumentation. |
| Approach: | They propose a new natural language inference benchmark LawngNLI from U.S. legal opinions with automatic labels with high human-validated accuracy. |
| Outcome: | The proposed benchmark can train and test systems for implication-based case retrieval and argumentation. |
Distillation-Resistant Watermarking for Model Protection in NLP (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing protection methods such as watermarking only work for images but are not applicable to text. |
| Approach: | They propose a technique that injects watermarks into the victim’s prediction probability corresponding to a secret key and is able to detect such a key by probing a suspect model. |
| Outcome: | The proposed technique detects stealing suspects at 100% accuracy on four NLP tasks while the prior method fails on two. |
NeuroCounterfactuals: Beyond Minimal-Edit Counterfactuals for Richer Data Augmentation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to produce counterfactuals rely on small perturbations via minimal edits, resulting in simplistic changes. |
| Approach: | They propose a novel approach to produce counterfactuals that allow for larger edits and linguistic diversity while still bearing similarity to the original document. |
| Outcome: | The proposed approach outperforms existing methods for generalizing natural language models under select settings. |
Don’t Just Clean It, Proxy Clean It: Mitigating Bias by Proxy in Pre-Trained Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Transformer-based pre-trained models can encode societal biases in their contextual representations and in downstream predictions when fine-tuned on task-specific data. |
| Approach: | They propose an approach that selectively eliminates stereotypical associations at fine-tuning, so that the model doesn't learn to excessively rely on those signals. |
| Outcome: | The proposed approach reduces biases from identity words and frequently co-occurring proxies by > 60% in toxicity classification, and also extends to multiple identities. |
The Undesirable Dependence on Frequency of Gender Bias Metrics Based on Word Embeddings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have found word embeddings can capture semantic similarity but may be affected by word frequency. |
| Approach: | They find that word embeddings can capture semantic similarity but may be affected by word frequency . they compare this effect with an alternative metric based on Pointwise Mutual Information . |
| Outcome: | The proposed method does not depend on word frequency, but it does return female bias in low frequency words. |
BioNLI: Generating a Biomedical NLI Dataset Using Lexico-semantic Constraints for Adversarial Examples (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Biomedical research has progressed at a tremendous pace, with PubMed2 indexing well over 1M publications per year in the past eight years. |
| Approach: | They propose a semi-supervised procedure that bootstraps biomedical NLI datasets from positive entailment examples present in biomedically published texts. |
| Outcome: | The proposed procedure bootstraps biomedical NLI datasets from positive entailment examples from biomedically challenging texts. |
Self-supervised Cross-modal Pretraining for Speech Emotion Recognition and Sentiment Analysis (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to multimodal speech emotion recognition and sentiment analysis have not improved results due to their relatively simple fusion mechanisms and lack of proper cross-modal pretraining. |
| Approach: | They propose a deep-fused audio-text bi-modal transformer with carefully designed cross-modal fusion mechanism and stage-wise cross-mod pretraining scheme to facilitate cross-modulation. |
| Outcome: | The proposed method exceeds benchmarks on public IEMOCAP emotion and CMU-MOSEI sentiment datasets by a large margin. |
Multimodal Conversation Modelling for Topic Derailment Detection (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on analysing textual dialogues that derailed into toxic content ignores visual information, such as images and videos. |
| Approach: | They propose a new multimodal conversational architecture that utilises visual and conversational contexts to classify comments for derailment. |
| Outcome: | The proposed approach outperforms existing methods and is more robust to textual noise. |
Active Learning for Abstractive Text Summarization (2022.findings-emnlp)
Copied to clipboard
Akim Tsvigun, Ivan Lysenko, Danila Sedashov, Ivan Lazichny, Eldar Damirov, Vladimir Karlov, Artemy Belousov, Leonid Sanochkin, Maxim Panov, Alexander Panchenko, Mikhail Burtsev, Artem Shelmanov
| Challenge: | Abstractive text summarization (ATS) requires a long document and short summaries. |
| Approach: | They propose a query strategy for AL in abstractive text summarization that uses uncertainty estimation to reduce model performance. |
| Outcome: | The proposed query strategy improves ROUGE and consistency scores for annotated datasets . it also increases the performance of the model, compared to passive annotation. |
Finding Memo: Extractive Memorization in Constrained Sequence Generation Tasks (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Memorization presents a challenge for constrained Natural Language Generation tasks . previous studies focused on counterfactual memorization, linking it to hallucinations . |
| Approach: | They propose an algorithm for extractive memorization in constrained sequence generation tasks . they propose to elicit non-memorized translations of memorized samples from the same model . |
| Outcome: | The proposed algorithm could be leveraged to mitigate memorization in the model through finetuning. |
SALTED: A Framework for SAlient Long-tail Translation Error Detection (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Traditional machine translation metrics are insensitive to the long tail of behavioral problems. |
| Approach: | They propose a specification-based framework for behavioral testing of NMT models . they use high-precision detectors that flag errors between a source sentence and output . |
| Outcome: | The proposed framework provides a reliable view of problems that were previously invisible. |
Discord Questions: A Computational Approach To Diversity Analysis in News Coverage (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Modern news aggregators do the hard work of organizing the news, but choosing which source to read remains challenging. |
| Approach: | They propose a framework to help readers identify source differences and gain an understanding of news coverage diversity by generating questions with a diverse answer pool and reusing existing methods. |
| Outcome: | The proposed framework improves performance from current question generation methods by 5% and achieves 81% balanced accuracy on a realistic test set. |
FocusQA: Open-Domain Question Answering with a Context in Focus (2022.findings-emnlp)
Copied to clipboard
Gianni Barlacchi, Ivano Lauriola, Alessandro Moschitti, Marco Del Tredici, Xiaoyu Shen, Thuy Vu, Bill Byrne, Adrià de Gispert
| Challenge: | a new method for question answering with a context in focus simulates a free interaction with QA systems. |
| Approach: | They introduce question answering with a cotext in focus task that simulates a free interaction with QA systems. |
| Outcome: | The proposed model outperforms state-of-the-art models for question answering with a context in focus up to 21.3% absolute points. |
Challenges and Opportunities in Information Manipulation Detection: An Examination of Wartime Russian Media (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Information manipulation campaigns rely on textbased news and social media content, and NLP can be a valuable tool in combating them. |
| Approach: | They propose to use a dataset to examine the use of NLP in public opinion manipulation campaigns in the 2022 Russia-Ukraine war. |
| Outcome: | The proposed dataset contains 38M+ posts from Russian media outlets on Twitter and VKontakte, as well as public activity and responses, immediately preceding and during the 2022 Russia-Ukraine war. |
Disentangling Task Relations for Few-shot Text Classification via Self-Supervised Hierarchical Task Clustering (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for Few-Shot Text Classification are limited by their global knowledge-shared mechanisms. |
| Approach: | They propose a self-supervised hierarchical task clustering method to address task heterogeneity . they use prior knowledge from historical tasks to leverage prior knowledge . |
| Outcome: | The proposed method can learn a classifier efficiently with few examples . it disentangles the underlying relations between tasks to improve interpretability . |
XRICL: Cross-lingual Retrieval-Augmented In-Context Learning for Cross-lingual Text-to-SQL Semantic Parsing (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work focuses on English datasets, and it is unclear whether large language models can serve as competitive semantic parsers for other languages. |
| Approach: | They propose a framework that learns to retrieve relevant English exemplars for a given query to construct prompts. |
| Outcome: | The proposed framework learns to retrieve relevant English exemplars for a given query to construct prompts. |
Continuation KD: Improved Knowledge Distillation through the Lens of Continuation Optimization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for knowledge distillation (KD) do not mitigate the noise in the teacher’s output: modeling the noisy behaviour of the teacher can distract the student from learning more useful features. |
| Approach: | They propose a method that optimizes the highly non-convex KD objective by starting with the smoothed version of this objective and making it more complex as the training proceeds. |
| Outcome: | The proposed method achieves state-of-the-art performance on NLU and computer vision tasks. |
Detecting Dementia from Long Neuropsychological Interviews (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies suggest examiner's language can influence cognitive impairment classifications. |
| Approach: | They propose a three-stage pipeline to detect dementia from exam recordings to mitigate the influence of the examiner on automatic dementia identification decisions. |
| Outcome: | The proposed pipeline mitigates the influence of the examiner on automatic dementia identification decisions in real-world neuropsychological exams. |
Sarcasm Detection is Way Too Easy! An Empirical Comparison of Human and Machine Sarcasm Detection (2022.findings-emnlp)
Copied to clipboard
| Challenge: | sarcasm detection datasets focus on intended, rather than perceived sarcasm, but there is no comparison between human and machine performance. |
| Approach: | They collect author-annotated sarcasm datasets that focus on intended, rather than perceived sarcasticism . they compare human-level benchmarks to that of state-of-the-art sarkasmatic detection systems . |
| Outcome: | The proposed datasets compare human and machine performance on sarcastic tasks in English and Arabic. |
Cross-lingual Text-to-SQL Semantic Parsing with Representation Mixup (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results show that Rex can benefit from cross-lingual training and improve the effectiveness of semantic parsers. |
| Approach: | They propose a Representation Mixup Framework for effectively exploiting translations in the cross-lingual Text-to-SQL task. |
| Outcome: | The proposed framework can benefit from cross-lingual training and improve the effectiveness of semantic parsers, achieving state-of-the-art performance. |
JamPatoisNLI: A Jamaican Patois Natural Language Inference Dataset (2022.findings-emnlp)
Copied to clipboard
| Challenge: | JamPatoisNLI provides the first dataset for natural language inference in a creole language, Jamaican Patois. |
| Approach: | They propose to use JamPatoisNLI to provide a dataset for natural language inference in a creole language, Jamaican Patois. |
| Outcome: | The proposed dataset shows that the language's relatedness to English significantly boosts the effectiveness of cross-lingual transfer even for the higher order task. |
Are Neural Topic Models Broken? (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evaluation paradigms are often divorced from real-world use . recent results have challenged the validity of the prevailing model evaluation paradigm . |
| Approach: | They show that neural topic models fare worse in both respects compared to an established classical method. |
| Outcome: | The proposed method outperforms the members of the ensemble in both respects. |
Know Thy Strengths: Comprehensive Dialogue State Tracking Diagnostics (2022.findings-emnlp)
Copied to clipboard
Hyundong Cho, Chinnadhurai Sankar, Christopher Lin, Kaushik Sadagopan, Shahin Shayandeh, Asli Celikyilmaz, Jonathan May, Ahmad Beirami
| Challenge: | Recent studies have revealed the vulnerability of dialogue state tracking models to distributional shifts, resulting in poor performance. |
| Approach: | They present a toolkit for standardized and comprehensive dialogue state tracking diagnoses that provides a richer summary of strengths and weaknesses. |
| Outcome: | The proposed toolkit shows that different classes of DST models have clear strengths and weaknesses, while generation models are more promising for handling language variety and span-based classification models are robust to unseen entities. |
Open-domain Question Answering via Chain of Reasoning over Heterogeneous Knowledge (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing open-domain question answering methods rely on the retriever to gather all evidence in isolation, but our approach uses an intermediary module to perform a chain of reasoning over the retrieved set. |
| Approach: | They propose a new open-domain question answering framework that integrates an intermediary module into the current retriever-reader pipeline and integrates it into the model. |
| Outcome: | The proposed framework outperforms the state-of-the-art on two OTT-QA datasets with an exact match score of 47.3 (45% relative gain). |
Detecting Languages Unintelligible to Multilingual Models through Local Structure Probes (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in multilingual pretrained models have proven effective at zero-shot transfer to a wide variety of languages, but this transfer is not universal, with many languages not currently understood by multilingual approaches. |
| Approach: | They propose a general approach that requires only unlabelled text to detect which languages are not well understood by a cross-lingual model. |
| Outcome: | The proposed model can detect which languages are not well understood by a multilingual model on 350 low-resource languages. |
Cards Against AI: Predicting Humor in a Fill-in-the-blank Party Game (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Humor is an inherently social phenomenon, with utterances shaped by what is socially and culturally accepted. |
| Approach: | They propose a dataset of cards Against humanity, including 785K unique jokes, and train machine learning models to predict the winning joke per game. |
| Outcome: | The proposed model performs twice as well as random on the more difficult task of judging novel cards, with the context having little impact. |
Open-Vocabulary Argument Role Prediction For Event Extraction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on event extraction depend on pre-defined argument roles . despite great progress, many studies still rely on hand-crafted ontologies . |
| Approach: | They propose an unsupervised framework for customizing argument roles for event extraction . they propose a human-annotated event extraction dataset with 143 customized argument roles . |
| Outcome: | The proposed framework outperforms existing methods on an event extraction dataset. |
Token-level Sequence Labeling for Spoken Language Understanding using Compositional End-to-End Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | End-to-end spoken language understanding systems model sequence labeling as a sequence prediction task causing a divergence from its well-established token-level tagging formulation. |
| Approach: | They propose to model sequence labeling as a sequence prediction task . their systems explicitly separate the added complexity of recognizing spoken mentions from the NLU task of sequence labelling . |
| Outcome: | The proposed systems outperform both cascaded and direct models on a labeling task of named entity recognition across SLU benchmarks. |
Baked-in State Probing (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work shows language models trained on form can capture aspects of meaning without explicit state supervision. |
| Approach: | They propose to use probing to "bake" state knowledge into language models . they propose to probe for underlying world state knowledge via text prompts . |
| Outcome: | The proposed methods show that language models trained on form can capture the world state without state supervision. |
ClinicalT5: A Generative Language Model for Clinical Text (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent generative language models like BART and T5 are gaining popularity with their competitive performance on text generation and tasks cast as generative problems. |
| Approach: | They propose to build domain-specific PLMs through fine-tuning or pre-training from scratch over domain corpora. |
| Outcome: | The proposed model outperforms existing models on domain-specific tasks and compares favorably with its close baselines. |
Find Someone Who: Visual Commonsense Understanding in Human-Centric Grounding (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Visual scenes often involve multiple people and humans can distinguish between them based on context descriptions about what happened before, their mental/physical states, and intentions. |
| Approach: | They propose a task that tests human-centric commonsense grounding models' ability to distinguish individuals given context descriptions about what happened before and their mental/physical states or intentions. |
| Outcome: | The proposed model outperforms pre-trained and non-pretrained models on 130k commonsense descriptions annotated on 67k images. |
CrisisLTLSum: A Benchmark for Local Crisis Event Timeline Extraction and Summarization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Timeline extraction and abstractive summarization are critical tasks for leveraging large numbers of social media posts about events. |
| Approach: | They propose to build a semi-automated cluster-then-refine algorithm to extract local crisis event timelines from Twitter. |
| Outcome: | The proposed approach performs better than human models on extraction and summarization tasks. |
Prompt-Tuning Can Be Much Better Than Fine-Tuning on Cross-lingual Understanding With Multilingual Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained multilingual language models show significant performance gains for zero-shot cross-lingual model transfer on a wide range of natural language understanding (NLU) tasks. |
| Approach: | They do cross-lingual evaluation using prompt tuning and compare it with fine-tuning . prompt tuning achieves much better cross-linguistic transfer than fine- tuning . |
| Outcome: | The results show that prompt tuning achieves better cross-lingual transfer than fine-tuning across datasets, with only 0.1% to 0.3% tuned parameters. |
BERT Meets CTC: New Formulation of End-to-End Speech Recognition with Pre-trained Masked Language Model (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to connectionist temporal classification (CTC) are based on pre-trained language models (LMs) |
| Approach: | They propose a formulation of connectionist temporal classification that relaxes the conditional independence assumptions used in conventional CTC and incorporates linguistic knowledge through explicit output dependency. |
| Outcome: | The proposed model improves over conventional approaches across variations in speaking styles and languages while maintaining CTC’s training efficiency. |
EtriCA: Event-Triggered Context-Aware Story Generation Augmented by Cross Attention (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for story generation still suffer from problems of relevance and coherence. |
| Approach: | They propose a novel neural generation model which maps contextual and event features to event sequences with a cross-attention mechanism and exploits logical relatedness between events. |
| Outcome: | The proposed model outperforms state-of-the-art models on automatic and human evaluations and shows that it can leverage contextual and event features. |
LADIS: Language Disentanglement for 3D Shape Editing (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for 3D shape editing produce decoupled, local edits to 3D shapes. |
| Approach: | They propose a tool set that learns latent representations that ground language in 3D geometry. |
| Outcome: | The proposed method outperforms existing SOTA methods by 20% in edit locality and up to 6.6% in language reference resolution accuracy. |
Effective Pretraining Objectives for Transformer-based Autoencoders (2022.findings-emnlp)
Copied to clipboard
| Challenge: | ELECTRA is more accurate than BERT, but it is not clear if this is due to its innovative architecture or to the long and extensive training, which highly increases the computation cost for obtaining the final language model. |
| Approach: | They propose to replace BERT’s Masked Language Modeling objective (MLM) with Token Detection (TD) by using a statistical approach to generate light tokens. |
| Outcome: | The proposed method can replace ELECTRA's computationally heavy generators without a significant drop in performance. |
Language Model Detoxification in Dialogue with Contextualized Stance Control (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on Language Model detoxification has focused on reducing the toxicity of the generation itself without consideration of the context. |
| Approach: | They propose a method to do context-dependent detoxification without taking into account the stance of the generated response. |
| Outcome: | The proposed method can learn the context-dependent stance control strategies while keeping a low self-toxicity of the underlying LM. |
Multilingual SubEvent Relation Extraction: A Novel Dataset and Structure Induction Method (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for subevent relation extraction (SRE) focus on sequential order of words in texts to enhance representation learning. |
| Approach: | They propose a method that learns to induce effective graph structures for input texts . they use word alignment frameworks with dependency paths and optimal transport . |
| Outcome: | The proposed method is able to induce effective graph structures for input texts to boost representation learning. |
A Two-Stage Approach towards Generalization in Knowledge Base Question Answering (2022.findings-emnlp)
Copied to clipboard
Srinivas Ravishankar, Dung Thai, Ibrahim Abdelaziz, Nandana Mihindukulasooriya, Tahira Naseem, Pavan Kapanipathi, Gaetano Rossiello, Achille Fokoue
| Challenge: | Existing approaches for Knowledge Base Question Answering focus on a specific knowledge base or evaluating it on underlying knowledge base requires non-trivial changes. |
| Approach: | They propose a framework that separates semantic parsing from knowledge base interaction . they propose KBQA framework that allows generalization across knowledge bases . |
| Outcome: | The proposed framework achieves comparable or state-of-the-art performance on datasets with a different knowledge base. |
Few-Shot (Dis)Agreement Identification in Online Discussions with Regularized and Augmented Meta-Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing annotated datasets do not cover all topics of interest. |
| Approach: | They propose a metric-based meta-learning approach that trains a meta-learner with two key abilities: decoding and generalizing domains. |
| Outcome: | The proposed approach can be quickly applied to analyze opinions for new topics with few labeled instances. |
Data Cartography for Low-Resource Neural Machine Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to improve machine translation (MT) in low-resource settings are limited in the number of languages spoken in the world. |
| Approach: | They apply cartography techniques to characterize the contribution of training samples in two low-resource MT tasks (Swahili-English and Turkish-English) they argue that data augmentation strategies for low-Resource ML would benefit from model-in-the-loop strategies to maximize improvements. |
| Outcome: | The proposed methods show that training samples contribute to model training in low-resource MT tasks, albeit not uniformly throughout the training process. |
Augmenting Multi-Turn Text-to-SQL Datasets with Self-Play (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Numerous architectures and pretraining methods have been proposed for context-dependent text-to-SQL, but the size of the datasets used has been limited due to the high cost of annotating multi-turn dialogue and SQL pairs. |
| Approach: | They propose to augment training datasets using self-play which leverages contextual information to synthesize new interactions to adapt the model to new databases. |
| Outcome: | The proposed model improves accuracy on SParC and CoSQL, two widely used cross-domain text-to-SQl datasets. |
Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction in Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | We explore the idea of compressing the prompts used to condition language models. |
| Approach: | They explore the idea of compressing the prompts used to condition language models . they show that compressed prompts can retain a substantive amount of information about the original prompt . |
| Outcome: | The proposed method can be extended to controllability and toxicity reduction. |
NaturalAdversaries: Can Naturalistic Adversaries Be as Effective as Artificial Adversaries? (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work has explored adversarial example generation for natural language understanding tasks, but these examples are often unrealistic and diverge from the real-world data distributions. |
| Approach: | They propose a framework for adversarial example generation that is effective at fooling a given classifier and a generative model based on the key tokens from the first stage. |
| Outcome: | The proposed framework generalizes across domains and offers insights for future research on improving robustness of neural text classification models. |
Multi-Path Transformer is Better: A Case Study on Neural Machine Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Extensive experiments on 12 WMT tasks show that shallower multi-path models can achieve similar or even better performance than the deeper model. |
| Approach: | They propose to use a parameter-efficient multi-path structure to fuse features extracted from different paths to achieve better performance. |
| Outcome: | The proposed model can achieve better performance with the same number of parameters than the deeper model. |
Unsupervised Learning of Hierarchical Conversation Structure (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Goal-oriented conversations often have sub-dialogue structure, but it can be domain-dependent . Increasingly, language understanding applications involve conversational speech and text . |
| Approach: | They propose an unsupervised approach to learning hierarchical conversation structure . they use turn and sub-dialogue segment labels to decode the structure based on dialogue acts and subtasks . |
| Outcome: | The proposed approach improves neural models for three conversation-level understanding tasks. |
Task Compass: Scaling Multi-task Pre-training with Task Prefix (2022.findings-emnlp)
Copied to clipboard
Zhuosheng Zhang, Shuohang Wang, Yichong Xu, Yuwei Fang, Wenhao Yu, Yang Liu, Hai Zhao, Chenguang Zhu, Michael Zeng
| Challenge: | Existing studies show that multi-task learning with large-scale supervised tasks suffers from negative effects across tasks. |
| Approach: | They propose a task prefix guided multi-task pre-training framework to explore the relationships among tasks. |
| Outcome: | The proposed model can be used as a foundation backbone for a wide range of tasks and as augmentation tool for data augmentation with complementary tasks. |
Sharpness-Aware Minimization with Dynamic Reweighting (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Deep neural networks are often overparameterized and can overfit training data. |
| Approach: | They propose an adversarial weight minimization algorithm that conducts adversarials and finds a common adversaria per-batch. |
| Outcome: | The proposed algorithm finds a common adversarial weight perturbation per-batch. |
Predicting Long-Term Citations from Short-Term Linguistic Influence (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to quantify linguistic influence in timestamped documents are not informative about extent to which a paper affected subsequent publications. |
| Approach: | They propose to quantify linguistic influence in timestamped document collections by estimating a Hawkes process with a low-rank parameter matrix and identify lexical and semantic changes using contextual embeddings and word frequencies. |
| Outcome: | The proposed method is based on an online evaluation with incremental temporal training/test splits, in comparison with a strong baseline that includes predictors for initial citation counts, topics, and lexical features. |
Joint Audio/Text Training for Transformer Rescorer of Streaming Speech Recognition (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have shown that streaming end-to-end speech recognition models suffer from higher word error rates (WER) compared to non-streaming models, streaming endto-ended ASR models are limited to short audio context or not use future context to satisfy low latency constraints. |
| Approach: | They propose a 2nd-pass rescoring model on top of the 1st-pass streaming model to improve recognition accuracy while keeping latency low. |
| Outcome: | The proposed method improves word error rate significantly compared to the existing model without adding any additional parameters or latency. |
TyDiP: A Dataset for Politeness Classification in Nine Typologically Diverse Languages (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Whether politeness phenomena and strategies are universal across languages or not have been controversial among sociologists and linguists. |
| Approach: | They create a dataset containing three-way politeness annotations for 500 examples in each language, totaling 4.5K examples. |
| Outcome: | The proposed model shows a robust zero-shot transfer ability, but falls short of estimated human accuracy significantly. |
Probing Cross-modal Semantics Alignment Capability from the Textual Perspective (2022.findings-emnlp)
Copied to clipboard
| Challenge: | In recent years, vision and language pre-training (VLP) models have advanced the state-of-the-art results in a variety of cross-modal downstream tasks. |
| Approach: | They propose a new probing method that is based on image captioning to first empirically study the cross-modal semantics alignment of VLP models. |
| Outcome: | The proposed method analyzes captions generated by five popular VLP models to reveal how well they align with visual words and how well these align with images. |
Hidden State Variability of Pretrained Language Models Can Guide Computation Reduction for Transfer Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to transfer a pretrained language model include fine-tuning all the parameters in the language model and adapting all its subsets. |
| Approach: | They propose to select layers based on the variability of their hidden states given a task-specific corpus. |
| Outcome: | The proposed model reduces the computational cost of transfer learning methods without sacrificing performance. |
Language Models as Agent Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Language models (LMs) are trained on collections of documents written by individual human agents to achieve specific goals in the outside world. |
| Approach: | a new study shows that language models are models of communicative intentions in a specific, narrow sense . despite recent progress, today's language models still make odd predictions and conspicuous errors . |
| Outcome: | a survey of LMs shows that they can model communicative intentions in a specific, narrow sense . despite recent progress, current models still make odd predictions and conspicuous errors . |
Combinatory Grammar Tells Underlying Relevance among Entities (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches focus on dependencies among words while paying limited attention to other types of syntactic structure. |
| Approach: | They propose an alternative approach that takes advantage of combinatory categorial grammar to detect the relation between entities. |
| Outcome: | The proposed model performs state-of-the-art on two widely used English benchmark datasets. |
Leveraging Open Data and Task Augmentation to Automated Behavioral Coding of Psychotherapy Conversations in Low-Resource Scenarios (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Behavioral coding is a procedure that requires human intervention to be performed manually. |
| Approach: | They propose to use a publicly available conversation-based dataset to transfer knowledge to a low-resource behavioral coding task by meta-learning. |
| Outcome: | The proposed framework predicts target behaviors more accurately than baseline models. |
Learning to Detect Noisy Labels Using Model-Based Features (2022.findings-emnlp)
Copied to clipboard
Zhihao Wang, Zongyu Lin, Junjie Wen, Xianxin Chen, Peiqi Liu, Guidong Zheng, Yujun Chen, Zhilin Yang
| Challenge: | Existing approaches to reduce label noise rely on heuristics and sample losses. |
| Approach: | They propose a method that transfers the noise distribution to a clean set and trains a model to distinguish noisy labels from clean ones using model-based features. |
| Outcome: | Empirically, the proposed approach improves over strong baselines on a wide range of tasks including text classification and speech recognition. |
Keyphrase Generation Beyond the Boundaries of Title and Abstract (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Current approaches to keyphrase generation use only the title and abstract of the articles. |
| Approach: | They propose to integrate full text and semantically similar articles to generate keyphrases from a dataset that includes the full text of the articles along with the title and abstract. |
| Outcome: | The proposed model can generate keyphrases that are present or absent from the text. |
Composition, Attention, or Both? (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work suggests that language models implicitly learn syntactic structures of natural language, even though they do not receive explicit syntatic supervision. |
| Approach: | They propose a novel architecture that recursively compose subtrees with a composition function and selectively attend to previous structural information with sc-attention mechanisms. |
| Outcome: | The proposed architecture can induce human-like syntactic generalization by recursive composition and selective attention to previous structural information. |
CDGP: Automatic Cloze Distractor Generation based on Pre-trained Language Model (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to generate cloze distractors with carefully-designed distractors are limited due to wrong option selection. |
| Approach: | They propose to employ pre-trained language models as an alternative to cloze distractor generation by using pre-designed distractors. |
| Outcome: | The proposed model improves the state-of-the-art cloze test score from 14.94 to 34.17 (NDCG@10) The proposed framework improves clozing distractors by incorporating pre-trained language models. |
G3: Geolocation via Guidebook Grounding (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a new task uses explicit knowledge from human-written guidebooks to improve geolocation accuracy . a state-of-the-art image-only method is unable to predict the location of an image . |
| Approach: | They propose a task that uses streetview images and a guidebook to predict a country for each image . they add clues from the guidebook and supervise attention with country-level pseudo labels . |
| Outcome: | The proposed method outperforms state-of-the-art image-only geolocation methods with 5% improvement in Top-1 accuracy. |
Controlling Bias Exposure for Fair Interpretable Predictions (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to reduce bias in NLP tasks focus on protecting or isolating information related to a sensitive attribute, but they lack control over how much bias is required to be removed. |
| Approach: | They propose a favorable debiasing method that uses sensitive information ‘fairly’, rather than blindly eliminating it. |
| Outcome: | The proposed method achieves a trade-off between debiasing and task performance along with producing debiased rationales as evidence. |
Investigating the Benefits of Free-Form Rationales (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that crowdsourced rationales provide additional background knowledge to models . a qualitative study shows generated rationale is not as useful for humans as crowdsourced ones . |
| Approach: | They investigate whether crowdsourced rationales provide additional background knowledge to models . they find that ECQA rationale provides additional background information to understand a decision . |
| Outcome: | The results show that ECQA rationales provide additional background knowledge to understand a decision . compared to crowdsourced rationale, generated rationale is not as useful for humans . |
Data-Efficient Concept Extraction from Pre-trained Language Models for Commonsense Explanation Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to extract concepts from pre-trained language models are not suitable for commonsense explanation generation. |
| Approach: | They propose a method to extract the key explanation concept from pre-trained language models by fine-tuning it with 20% training data and using a metric to evaluate the retrieved concepts. |
| Outcome: | The proposed method improves evaluation metrics over pre-trained language models and the existing models. |
Unsupervised Domain Adaptation for Joint Information Extraction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Current JIE methods focus on standard supervised learning setting where training and test data come from the same domain. |
| Approach: | They propose a method to induce domain-invariant representations for the tasks in JIE by a generalized version of domain-adversarial learning. |
| Outcome: | The proposed method improves out-of-domain performance for current pipeline approaches for all IE tasks. |
Foiling Training-Time Attacks on Neural Machine Translation Systems (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Neural machine translation systems are vulnerable to backdoor attacks . successful backdoors can cause slander, hate speech, phishing, etc. attacks can target very short trigger phrases, which can be challenging to detect even when included verbatim in poisoned instances. |
| Approach: | They propose a method that exploits asymmetry between source and target sentences to detect outlier tokens. |
| Outcome: | The proposed method reduces the success of attacks by up to 89.0% while not affecting predictive accuracy. |
Learning Action-Effect Dynamics for Hypothetical Vision-Language Reasoning Task (2022.findings-emnlp)
Copied to clipboard
| Challenge: | ‘Actions’ play a vital role in how humans interact with the world and therefore require the capability to perform ‘Reasoning about Actions & Change’ (RAC). |
| Approach: | They propose an encoder-decoder architecture to learn the representation of actions as vectors and combine it with existing modality parsers and a scene graph question answering model to evaluate the proposed system. |
| Outcome: | The proposed system performs better than baselines on the CLEVR_HYP dataset and compares with existing parsers and a scene graph question answering model. |
Named Entity and Relation Extraction with Multi-Modal Retrieval (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to name entity recognition and relation extraction are knowledge-based and may not be highly relevant. |
| Approach: | They propose a multi-modal named entity recognition framework that leverages image information to improve the performance of NER and relation extraction. |
| Outcome: | The proposed framework can achieve state-of-the-art on four multi-modal named entity recognition datasets and one multi-module relation extraction dataset. |
Calibrating Factual Knowledge in Pretrained Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies show that Pretrained Language Models can store factual knowledge, but facts stored in PLMs are not always correct. |
| Approach: | They propose a lightweight method to calibrate factual knowledge in PLMs without re-training from scratch. |
| Outcome: | The proposed method can be used to calibrate factual knowledge in PLMs without re-training from scratch. |
MCPG: A Flexible Multi-Level Controllable Framework for Unsupervised Paraphrase Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on controllable unsupervised paraphrase generation are expensive and require supervised training on large parallel corpora. |
| Approach: | They propose a method for controllable unsupervised paraphrase generation that is flexible to adapt to specific domains without extra training. |
| Outcome: | The proposed method outperforms state-of-the-art unsupervised baselines by a margin. |
WordTies: Measuring Word Associations in Language Models via Constrained Sampling (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Word associations are widely used in psychology to provide insights on how humans perceive and understand concepts. |
| Approach: | They propose an algorithm that allows an asymmetric measurement of associated words, given a cue word as input. |
| Outcome: | The proposed algorithm shares more overlap with human associations and observes the asymmetric property of human associations. |
Exploring The Landscape of Distributional Robustness for Question Answering Models (2022.findings-emnlp)
Copied to clipboard
Anas Awadalla, Mitchell Wortsman, Gabriel Ilharco, Sewon Min, Ian Magnusson, Hannaneh Hajishirzi, Ludwig Schmidt
| Challenge: | Existing methods for predicting distributional robustness fail to generalize reliably in a variety of test conditions. |
| Approach: | They conduct a large empirical evaluation to investigate the landscape of distributional robustness in question answering. |
| Outcome: | The proposed methods are more robust to distribution shifts than fully fine-tuned models, and few-shot prompt models exhibit better robustness than few- shot prompt models. |
Collaborative Reasoning on Multi-Modal Semantic Graphs for Video-Grounded Dialogue Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for video-grounded dialogue generation do not allow information from different modalities to complement each other. |
| Approach: | They propose a video-grounded dialogue generation model that integrates video data into pre-trained language models to allow information from different modalities to complement each other. |
| Outcome: | The proposed model outperforms state-of-the-art models on automatic and human evaluations on two public datasets. |
Partitioned Gradient Matching-based Data Subset Selection for Compute-Efficient Robust ASR Training (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing DSS algorithms for RNN-T have a high cost and performance degradation. |
| Approach: | They propose a distributable DSS algorithm for RNN-T that can be used to train a subset of training data. |
| Outcome: | The proposed algorithm achieves between 3x to 6x speedup with only a small accuracy degradation even in settings where the training data is corrupted with noise. |
Adaptive Graph Convolutional Network for Knowledge Graph Entity Alignment (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Entity alignment (EA) aims to identify equivalent entities from different Knowledge Graphs (KGs) noisy neighbors of entities transfer invalid information, drown out equivalent information, and ultimately reduce the performance of EA. |
| Approach: | They propose a method to deal with neighbor noises to reduce the performance of EA by capturing the differences and complementarities of multiple KGs. |
| Outcome: | The proposed framework outperforms the state-of-the-art methods in supervised and unsupervised settings. |
Towards Robust NLG Bias Evaluation with Syntactically-diverse Prompts (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Past studies have shown biases in natural language generation systems but there has been little work on evaluating the bias evaluation approaches. |
| Approach: | They propose a method for evaluating biases in natural language generation systems by paraphrasing syntactic prompts with different syntaktic structures and paraphrazing them to evaluate demographic bias. |
| Outcome: | The proposed method is more robust and shows that some syntactic structures prompt more toxic content while others could prompt less biased generation. |
PcMSP: A Dataset for Scientific Action Graphs Extraction from Polycrystalline Materials Synthesis Procedure Text (2022.findings-emnlp)
Copied to clipboard
| Challenge: | 305 open access scientific articles are used for synthesis action graphs . lack of annotated data has hindered progress in this field . |
| Approach: | They propose to annotate Polycrystalline Materials Synthesis Procedures PcMSP from 305 open access scientific articles for the construction of synthesis action graphs. |
| Outcome: | The proposed dataset contains the synthesis sentences, entity mentions and intra-sentence relations extracted from the experimental paragraphs. |
Validity Assessment of Legal Will Statements as Natural Language Inference (2022.findings-emnlp)
Copied to clipboard
| Challenge: | This study introduces a dataset that focuses on the validity of statements in legal wills. |
| Approach: | They propose a dataset that focuses on the validity of statements in legal wills. |
| Outcome: | The proposed model achieves 80% macro F1 and accuracy, but group accuracy is in mid 80s at best, suggesting that the models’ understanding of the task remains superficial. |
AdaPrompt: Adaptive Model Training for Prompt-based NLP (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Prompt-based learning can tackle zero-shot and few-shot NLP tasks . authors propose a method that makes use of pre-trained language models . |
| Approach: | They propose to map NLP tasks into natural language prompts, which are then filled by pre-trained language models. |
| Outcome: | The proposed method outperforms standard prompt-based methods in few-shot settings. |
Code Generation From Flowcharts with Texts: A Benchmark Dataset and An Approach (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Currently, researchers focus on generating codes from requirement documents. |
| Approach: | They propose to generate source code from flowcharts with texts instead of directly translating requirements into codes. |
| Outcome: | The proposed model improves on the baselines by transforming flowcharts into pseudo-code . the proposed model is based on 320 flowchartes with their corresponding source codes . |
Focus! Relevant and Sufficient Context Selection for News Image Captioning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work only coarsely leverages the article to extract the necessary context, which makes it difficult for models to identify relevant events and named entities. |
| Approach: | They propose to use a vision and language retrieval model CLIP to localize the visually grounded entities in the news article and then capture the non-visual entities via an open relation extraction model. |
| Outcome: | The proposed model significantly improves on existing models and achieves state-of-the-art on multiple benchmarks. |
Generative Aspect-Based Sentiment Analysis with Contrastive Learning and Expressive Structure (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods struggle with implicit aspects and opinions, such as "it took an hour to be seated". |
| Approach: | They propose two techniques for improved structured generation for ACOS quadruple extraction. |
| Outcome: | The proposed model improves on three ACOS datasets and shows significant improvement on implicit aspect and opinion splits. |
Semantic Dependency Parsing with Edge GNNs (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing semantic dependency parsers use factor graphs to generate a tree structure, but they are ill-suited for a more complex semantic relationship representation. |
| Approach: | They propose a second-order neural CRF parser that uses factor graphs to generate a dependency edge and define neighbors in terms of sibling, co-parent, and grandparent relationships. |
| Outcome: | The proposed model outperforms the first-order biaffine parser on English datasets and shows that it is more efficient than the first order. |
Explore Unsupervised Structures in Pretrained Models for Relation Extraction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Syntactic trees are widely used in relation extraction (RE) but they are not stable on different text domains and a pre-defined grammar may not fit the target relation schema. |
| Approach: | They propose to use unsupervised structures to extract relation extraction models . they also conduct detailed analyses on their abilities of adapting new RE domains . |
| Outcome: | The proposed models obtain competitive (even the best) performance scores on benchmark RE datasets. |
Identifying Human Strategies for Generating Word-Level Adversarial Examples (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that word-level adversarial examples are more natural and grammatical correct than automated attacks. |
| Approach: | They analyze how humans generate word-level adversarial examples against fine-tuned Transformer models that preserve naturalness and grammatical correctness. |
| Outcome: | The authors show that humans generate adversarial examples much more effortlessly than automated attacks. |
Refinement Matters: Textual Description Needs to be Refined for Zero-shot Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Zero-Shot Learning (ZSL) is a new form of learning that uses textual description and attribute to transfer knowledge from seen to unseen classes. |
| Approach: | They propose a non-generative gating-based attribute refinement network for ZSL that uses a circle loss-guided attribute embedder to refine the attributes. |
| Outcome: | The proposed approach outperforms generative methods and most generative ones in all three scenarios. |
SAT: Improving Semi-Supervised Text Classification with Simple Instance-Adaptive Self-Training (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for semi-supervised text classification have shown great performance in few-shot scenarios, where both labeled and unlabeled data are utilized. |
| Approach: | They propose a simple instance-adaptive self-training method for semi-supervised text classification that generates two augmented views for each unlabeled data and trains a meta learner to identify relative strength of augmentations based on the similarity between the original view and the augmented view. |
| Outcome: | The proposed method consistently shows competitive performance with varying sizes of labeled training data compared to existing semi-supervised learning methods. |
Answer Quality Aware Aggregation for Extractive QA Crowdsourcing (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for creating extractive question answering datasets are crowdsourcing, but results are often inconsistent. |
| Approach: | They propose a method for aggregating answers from different crowd workers that takes into account the relations between the answer, question, and context passage. |
| Outcome: | The proposed method outperforms baselines by 16% on precision and effectively conduct answer aggregation for extractive question answering task. |
Search to Pass Messages for Temporal Knowledge Graph Completion (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies on missing facts in temporal knowledge graphs are based on hand-designed architectures and fail to explore the diverse topological and temporal properties of TKGs. |
| Approach: | They propose to use neural architecture search to design a data-specific message passing architecture for TKG completion. |
| Outcome: | The proposed architectures achieve the state-of-the-art performance on three benchmark datasets. |
Code Vulnerability Detection via Nearest Neighbor Mechanism (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to learn code semantics from source code are difficult to identify. |
| Approach: | They propose a method which retrieves multiple neighbor samples and utilizes label information to provide help for model predictions. |
| Outcome: | Extensive experiments show that the proposed method can achieve obvious performance improvements compared to baseline models. |
Robust Question Answering against Distribution Shifts with Test-Time Adaption: An Empirical Study (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on robustness tuning (RT) methods has found that QA models fail when the test data has a distribution shift compared to the training data. |
| Approach: | They propose to use test-time adaptation methods to improve QA models after deployment to evaluate their model against text corruption and changes in language and domain. |
| Outcome: | The proposed method improves TTA to be more robust to variation in hyper-parameters and test distributions over time. |
ParaMac: A General Unsupervised Paraphrase Generation Framework Leveraging Semantic Constraints and Diversifying Mechanisms (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing unsupervised methods for paraphrase generation are weak in semantic equivalence or expression diversity. |
| Approach: | They propose a framework for unsupervised paraphrase generation that employs multi-aspect equivalence constraints and multi-granularity diversifying mechanisms to achieve good semantic equvalence and expressive diversity. |
| Outcome: | The proposed framework achieves 9.1% and 3.3% absolute gains over previous SOTA on Quora and MSCOCO and can improve to 18.0% and 4.6% on GLUE. |
Semi-supervised New Slot Discovery with Incremental Clustering (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to discover new slots rely on unsupervised slot induction or domain adaptation, and are limited in transferring prior knowledge to new slots. |
| Approach: | They propose a Semi-supervised Incremental Clustering method to discover new slots with existing linguistic annotation models and limited known slot data. |
| Outcome: | The proposed method significantly outperforms state-of-the-art models on five public datasets. |
Con-NAT: Contrastive Non-autoregressive Neural Machine Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Neural machine translation models are autoregressive, which means they predict tokens one by one based on source tokens and previously predicted tokens. |
| Approach: | They propose a conditional masked language model which incorporates contrastive learning into the conditional language model. |
| Outcome: | The proposed model improves on WMT’16 Ro-En translation directions with different data sizes. |
Improved Knowledge Distillation for Pre-trained Language Models via Knowledge Selection (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on knowledge distillation have shown that not all knowledge is necessary for learning a good student model. |
| Approach: | They propose an actor-critic approach to selecting appropriate knowledge to transfer during the process of knowledge distillation. |
| Outcome: | The proposed method outperforms several strong knowledge distillation baselines significantly on the GLUE datasets. |
Syntactically Robust Training on Partially-Observed Data for Open Information Extraction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Open Information Extraction models have shown promising results with sufficient supervision, but the syntactic distribution of training data is partially observable in comparison to the real world. |
| Approach: | They propose a syntactically robust training framework that enables models to be trained on a multi-paraphrase distribution based on diverse paraphrase generation. |
| Outcome: | The proposed framework can be applied to other syntactic partial observable domains. |
A Benchmark and Dataset for Post-OCR text correction in Sanskrit (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Sanskrit is a classical language with 30 million manuscripts available for digitisation . however, it is considered to be low-resource when it comes to available digital resources. |
| Approach: | They propose to use a post-OCR text correction dataset to correct errors from OCR predictions from 30 different books in the Indian subcontinent. |
| Outcome: | The proposed model outperforms OCR models on graphemic and lexical levels and shows that it is more accurate than previous models. |
Knowledge-Enhanced Self-Supervised Prototypical Network for Few-Shot Event Detection (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for few-shot event detection are inaccurate and lack a prototype representation module. |
| Approach: | They propose a Knowledge-Enhanced self-supervised prototypical network for few-shot event detection . it adopts hybrid rules which align event types to FrameNet and introduces knowledge to obtain more instances . |
| Outcome: | The proposed network improves few-shot event detection performance on three benchmark datasets. |
VarMAE: Pre-training of Variational Masked Autoencoder for Domain-adaptive Language Understanding (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models have been widely applied to standard benchmarks due to the limited resources available in a domain. |
| Approach: | They propose a Transformer-based language model called VarMAE for domain-adaptive language understanding that encodes the context of a token into a smooth latent distribution. |
| Outcome: | Experiments on science- and finance-domain NLU tasks show that the proposed model can be efficiently adapted to new domains with limited resources. |
Exploring Methods for Building Dialects-Mandarin Code-Mixing Corpora: A Case Study in Taiwanese Hokkien (2022.findings-emnlp)
Copied to clipboard
| Challenge: | CM is a challenging task when mixed languages include dialects. |
| Approach: | They propose to construct a Hokkien-Mandarin CM dataset to overcome the limitation . they propose to use a linguistics-based toolkit to train the model for translation tasks . |
| Outcome: | The proposed model achieves good results on CM data translation while maintaining monolingual translation quality. |
Recurrence Boosts Diversity! Revisiting Recurrent Latent Variable in Transformer-Based Variational AutoEncoder for Diverse Text Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Variational Auto-Encoder (VAE) has been widely adopted in text generation due to its ability to learn flexible representations. |
| Approach: | They propose a Transformer-based recurrent VAE structure that imposes recurrence on segment-wise latent variables with arbitrarily separated text segments and constructs the posterior distribution with residual parameterization. |
| Outcome: | The proposed structure can deduce a non-zero lower bound of the KL term and enhance the entanglement of each segment and preceding latent variables, providing a theoretical guarantee of generation diversity. |
Tweet Based Reach Aware Temporal Attention Network for NFT Valuation (2022.findings-emnlp)
Copied to clipboard
Ramit Sawhney, Megh Thakkar, Ritesh Soun, Atula Neerkaje, Vasu Sharma, Dipanwita Guhathakurta, Sudheer Chava
| Challenge: | Non-Fungible Tokens (NFTs) are a relatively unexplored class of assets due to their extremely volatile nature. |
| Approach: | They propose a reach-aware temporal learning approach to predict future NFT trends from a dataset consisting of over 1.3 million tweets and 180 thousand NFT transactions . |
| Outcome: | The proposed model outperforms state-of-the-art models by an average of 36% on a dataset consisting of over 1.3 million tweets and 180 thousand NFT transactions spanning over 15 NFT collections. |
Entity Embedding Completion for Wide-Coverage Entity Disambiguation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing state-of-the-art ED models do not address out-of vocabulary entities that are absent from training data. |
| Approach: | They propose to extend a state-of-the-art ED model by dynamically computing embeddings of out-ofvocabulary entities by using entity descriptions and mention contexts. |
| Outcome: | The proposed model performs comparable to existing models whose embeddings are trained for all candidate entities as well as embedd-free models. |
Entity-level Interaction via Heterogeneous Graph for Multimodal Named Entity Recognition (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for name-based entity recognition neglect the integrity of entity semantics and conduct cross-modal interaction at token-level. |
| Approach: | They propose a multimodal named entity recognition model that captures visual information and fuses it into tokens to rid non-entity tokens of visual noise. |
| Outcome: | The proposed model captures entity-related visual information and fuses it into tokens . it eliminates visual noise and makes non-entity tokens easily misidentified as entities . |
Status Biases in Deliberation Online: Evidence from a Randomized Experiment on ChangeMyView (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Status is widely used to incentivize user engagement, but visible status indicators could inadvertently bias online deliberation to favor high-status users. |
| Approach: | They propose to quantify status biases in online deliberation using a ChangeMyView platform and to test whether status visibility can inadvertently bias it to favor high-status users. |
| Outcome: | The proposed method increases the persuasion rate of moderate-status users by 84% and lowers the per-su-sion rate for high-statuse users by 41% relative to the control group. |
Empathetic and Emotionally Positive Conversation Systems with an Emotion-specific Query-Response Memory (2022.findings-emnlp)
Copied to clipboard
Zhiliang Tian, Yinliang Wang, Yiping Song, Chi Zhang, Dongkyu Lee, Yingxiu Zhao, Dongsheng Li, Nevin L. Zhang
| Challenge: | Existing emotional conversation systems output responses according to either a given emotion or the user’s emotion reflected in the input queries. |
| Approach: | They propose to generate empathetic responses catering to the user’s emotions while leading the conversation to be emotionally positive by abstracting the conversation corpus and extracting the different responding strategies for different users’ emotions and conversational topics into a memory. |
| Outcome: | The proposed model surpasses the baseline methods in appropriateness, diversity, and generating emotionally positive responses. |
Trial2Vec: Zero-Shot Clinical Trial Document Similarity Search using Self-Supervision (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Clinical trials are expensive and time-consuming to conduct, and lengthy trial documents and lack of labeled data make comparisons difficult. |
| Approach: | They propose a zero-shot clinical trial retrieval method which learns through self-supervision without the need for annotating similar clinical trials. |
| Outcome: | The proposed method improves on baselines on precision/recall and 15% on the downstream trial outcome prediction task. |
From Mimicking to Integrating: Knowledge Integration for Pre-Trained Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for natural language processing (NLP) are fine-tuned and released for research and deployments. |
| Approach: | They propose a PLM reuse paradigm that merges teacher-PLM knowledge into a student model. |
| Outcome: | The proposed paradigm can reduce the computational cost and environmental side-effects of retraining the PLM from scratch. |
Model and Data Transfer for Cross-Lingual Sequence Labelling in Zero-Resource Settings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have proposed data-based cross-lingual transfer as an effective technique for cross-linguistic sequence labelling, but they have failed to perform well. |
| Approach: | They propose to use data-based cross-lingual transfer to train supervised models from a source language to unlabelled target languages. |
| Outcome: | The proposed techniques outperform data-based cross-lingual transfer approaches in a zero-shot setting. |
Early Guessing for Dialect Identification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Current research on dialect identification is model-centric, focusing on performance. |
| Approach: | They propose a data-centric approach to find the shortest input needed to make a plausible guess. |
| Outcome: | The proposed method generalizes across dialects and datasets with two shortening criteria. |
R-AT: Regularized Adversarial Training for Natural Language Understanding (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Currently, adversarial training is a popular and powerful regularization method in the natural language domain. |
| Approach: | They propose to regularize adversarial training via dropout by perturbing word embeddings . they find that R-AT can improve many models by reducing adversariality . |
| Outcome: | The proposed method can reduce the inconsistency between training and testing of models with dropout. |
Multi-View Active Learning for Short Text Classification in User-Generated Data (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Using a microblog, a set of tasks can be tailored for queries. |
| Approach: | They propose a novel active learning model to overcome the lack of labeled data and the informal user language in mining user-generated data. |
| Outcome: | The proposed model outperforms existing models in four publicly available Twitter datasets and in four datasets with different applications. |
Forging Multiple Training Objectives for Pre-trained Language Models via Meta-Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Empirical studies show that learning multiple training objectives in a single model makes the learned language representation barely converge to the desired optimum. |
| Approach: | They propose a meta-learning-based adaptive sampler which learns latent sampling pattern on arbitrary pre-training objectives. |
| Outcome: | Empirical studies show that learning multiple objectives in a single model makes it difficult to achieve the desired optimum. |
ConGen: Unsupervised Control and Generalization Distillation For Sentence Representation (2022.findings-emnlp)
Copied to clipboard
Peerat Limkonchotiwat, Wuttikorn Ponwitayarat, Lalita Lowphansirikul, Can Udomcharoenchaikit, Ekapol Chuangsuwanich, Sarana Nutanong
| Challenge: | Sentence representations are essential in many NLP tasks operating at the sentence level. |
| Approach: | They propose an unsupervised sentence representation method to reduce the supervised-unsupervised performance gap for smaller models. |
| Outcome: | The proposed method outperforms supervised training on STS, text classification, and natural language inference tasks on smaller models. |
Large-Scale Differentially Private BERT (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that scaling up the batch size to millions improves the utility of a DP-SGD step for BERT. |
| Approach: | They propose to use differentially private SGD to pretrain BERT-Large with a batch size of millions to improve the utility of the DP-SGD step. |
| Outcome: | The proposed approach achieves a masked language model accuracy of 60.5% at a batch size of 2M, which is a reasonable privacy setting. |
Improving Zero-Shot Multilingual Translation with Universal Representations and Cross-Mapping (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing model structure and training scheme cannot ensure universal representations and cross-mappings because of lacking explicit constraints. |
| Approach: | They propose a state mover’s distance model to model the difference of the representations output by the encoder and a agreement-based training scheme to minimize the proposed distance to learn universal representations. |
| Outcome: | The proposed model can translate between languages unseen during training, i.e., zero-shot translation. |
Controllable Fake Document Infilling for Cyber Deception (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to deter malicious intrusion generate multiple fake versions of a document that are context-agnostic and produce sub-optimal outputs. |
| Approach: | They propose a context-aware model for creating fake documents that masks important concepts of varied lengths and then infills a realistic but fake alternative considering both the previous and future contexts. |
| Outcome: | The proposed model outperforms baseline models in generating highly believable fakes with moderate modification to protect critical information and deceive adversaries. |
Weakly Supervised Headline Dependency Parsing (2022.findings-emnlp)
Copied to clipboard
| Challenge: | English news headlines have unique syntactic properties documented in linguistics literature since the 1930s. |
| Approach: | They propose to provide the first news headline corpus of annotated syntactic dependency trees to evaluate existing NLP parsers on news headlines. |
| Outcome: | The proposed method improves performance across different news outlets, but is moderated by constructions idiosyncratic to outlet. |
BOOKSUM: A Collection of Datasets for Long-form Narrative Summarization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing text summarization datasets include short-form source documents that lack long-range causal and temporal dependencies and contain strong layout and stylistic biases. |
| Approach: | They propose a dataset for long-form narrative summarization that uses human written summaries on three levels of difficulty. |
| Outcome: | The proposed dataset covers documents from the literature domain, such as novels, plays and stories, and includes highly abstractive, human written summaries on three levels of difficulty. |
Not All Errors are Equal: Learning Text Generation Metrics using Stratified Error Synthesis (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing learning metrics are limited to tasks where large human ratings are available. |
| Approach: | They propose a model-based natural language generation (NLG) evaluation metric that is highly correlated with human judgements without requiring human annotation. |
| Outcome: | The proposed metric outperforms all prior unsupervised metrics on multiple NLG tasks including translation, image captioning, and WebNLG text generation. |
Summarization as Indirect Supervision for Relation Extraction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Relation extraction (RE) models rely on training data with expensive annotations . et al., 2018; Zhao e.t al, 2018) . |
| Approach: | They propose a method that converts RE into a summarization formulation by using constraint decoding techniques. |
| Outcome: | The proposed method improves relation extraction models with high-resource and high-contrast inferences. |
DIGAT: Modeling News Recommendation with Dual-Graph Interaction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing news recommendation methods lack effective news-user feature interaction. |
| Approach: | They propose to use news-graph and user-graph channels to enhance news encodings . they also propose to perform effective feature interaction between news and user graphs based on semantic-augmented graphs. |
| Outcome: | The proposed graph attention networks outperform existing NR methods on the benchmark dataset MIND. |
SMASH: Improving SMAll Language Models’ Few-SHot Ability with Prompt-Based Distillation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Large-scale language models with prompts have shown remarkable performance on few-shot learning. |
| Approach: | They propose an approach to improve SMAll language models’ few-SHot ability by training on intermediate tasks before prompt-based fine-tuning on downstream tasks. |
| Outcome: | The proposed model improves on sentence-pair and sentiment classification tasks by training on intermediate tasks before fine-tuning on downstream tasks. |
Consecutive Question Generation via Dynamic Multitask Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | , . ; ) ()((); ()) .())((2): ""(). |
| Approach: | They propose a sequential sequential question-answer generation task with auxiliary tasks generating other elements to generate logically related question-anchor pairs to understand a passage. |
| Outcome: | The proposed framework improves question generation significantly and benefit multiple related tasks. |
Subword Segmental Language Modelling for Nguni Languages (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Subword segmentation is a standard practice in NLP, but is viewed as a preprocessing step for low-resource languages with complex morphologies. |
| Approach: | They propose a subword segmental language model that learns how to segment words while being trained for autoregressive language modelling. |
| Outcome: | The proposed model outperforms existing models on unsupervised morphological segmentation and outperfies standard subword segmenters on all 4 languages. |
Towards Robust Visual Question Answering: Making the Most of Biased Samples via Contrastive Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have shown that biased samples can be brittle for VQA models . however, the improvements on OOD data severely sacrifice the performance on the in-distribution (ID) data. |
| Approach: | They propose a contrastive learning approach that exploits biased samples for unbiased information that contributes to reasoning. |
| Outcome: | The proposed method achieves competitive performance on the OOD dataset while maintaining robustness on the ID dataset. |
P3LM: Probabilistically Permuted Prophet Language Modeling for Generative Pre-Training (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing autoregressive left-to-right (L2R) models are limited to unidirectional information and constrained on strong local dependencies. |
| Approach: | They propose a probabilistically permuted prophet language model which strengthens the modeling of bidirectional information and long token dependencies for sequence generation. |
| Outcome: | Experiments on GLGE dataset show that P3LM improves on natural language generation tasks. |
Holistic Sentence Embeddings for Better Out-of-Distribution Detection (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained language models have shown superior performance for textual OOD detection, but they estimate sample distance scores in the last-layer CLS embedding space. |
| Approach: | They propose to use token averaging and layer combination to boost OOD detection by deriving more holistic sentence embeddings. |
| Outcome: | The proposed method surpasses the state-of-the-art on a comprehensive suite of benchmarks by a 9.33% FAR95 margin. |
MuGER2: Multi-Granularity Evidence Retrieval and Reasoning for Hybrid Question Answering (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Conventional HQA models retrieve coarse- or fine-grained evidence to reason the answer . however, they neglect a more general scenario requiring reasoning over heterogeneous data to answer a question. |
| Approach: | They propose a multi-granularity evidence retrieval and reasoning approach to answer questions over heterogeneous data using tables and passages linked to table cells. |
| Outcome: | The proposed approach significantly boosts the performance on the HybridQA dataset. |
EntityCS: Improving Zero-Shot Cross-lingual Transfer with Entity-Centric Code Switching (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for CS use dictionaries or parallel sentences with word-alignment to generate CS data by randomly switching words in a sentence. |
| Approach: | They propose a method that focuses on Entity-level Code-Switching to capture fine-grained cross-lingual semantics without corrupting syntax. |
| Outcome: | The proposed method captures fine-grained cross-lingual semantics without corrupting syntax. |
MBTI Personality Prediction for Fictional Characters Using Movie Scripts (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing NLP models cannot predict character's personality types based on text classifications . character comprehension is the cornerstone of understanding stories in psychology and education. |
| Approach: | They propose a benchmark to predict movie character's MBTI or Big 5 personality types based on the narratives of the character. |
| Outcome: | The proposed model outperforms existing models in the task and is more accurate than random guesses. |
A Simple and Strong Baseline for End-to-End Neural RST-style Discourse Parsing (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing discourse parsing methods need a strong baseline for reporting reliable experimental results. |
| Approach: | They integrate existing parsing strategies with transformer-based pre-trained language models to provide a strong baseline for reporting reliable experimental results. |
| Outcome: | The proposed model outperforms the current best model using DeBERTa. |
Probing for Constituency Structure in Neural Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Using standard probing techniques, we examine whether contextual neural language models implicitly learn syntactic structure. |
| Approach: | They investigate to which extent contextual neural language models implicitly learn syntactic structure. |
| Outcome: | The proposed model is able to represent constituents of different categories within the neuron activations of a LM such as RoBERTa with high performance even on manipulated data. |
Table-To-Text generation and pre-training with TabT5 (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) are limited when it comes to structured or semi-structured domains like tables. |
| Approach: | They propose an encoder-decoder model that generates natural language text based on tables and textual inputs. |
| Outcome: | TabT5 achieves 15% increase in sequence accuracy on spreadsheet formula prediction and data-to-text generation domains. |
A POMDP Dialogue Policy with 3-way Grounding and Adaptive Sensing for Learning through Communication (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing POMDP policies for learning from people lack a domain-dependent semantic representation of natural language utterances. |
| Approach: | They propose a POMDP dialogue policy with 3-way grounding of language in the shared physical context, the dialogue context, and persistent knowledge. |
| Outcome: | The proposed policy can learn distinct but related games and continue learning across dialogues for complex games. |
PaCo: Preconditions Attributed to Commonsense Knowledge (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing language models can reason with circumstantial preconditions of commonsense knowledge, but they do not understand the circumstancial precondition. |
| Approach: | They propose to use a dataset to examine the ability of existing language models to understand circumstantial preconditions to improve their reasoning with commonsense knowledge. |
| Outcome: | The proposed task shows that human reasoning with preconditions is an open challenge. |
Improving Few-Shot Domain Transfer for Named Entity Disambiguation with Pattern Exploitation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Named entity disambiguation is a critical subtask of entity linking . a model can be trained on a domain, but it needs to be adapted to the domain . |
| Approach: | They propose to reformulate named entity disambiguation as a masked language modeling problem. |
| Outcome: | The proposed model improves on a mental health news dataset without sacrifices in accuracy. |
Capturing Topic Framing via Masked Language Modeling (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a framework for measuring differential framing of issues is needed to address these issues . issue framers can be expressed explicitly with evaluative language or implicitly . quantitative methods have been used to measure issue framming . |
| Approach: | They propose a framework for modeling the differential framing of issues through masked token prediction using large-scale fine-tuned language models. |
| Outcome: | The proposed framework captures differential framing of issues with high reliability . it can be used to predict tone and word choices in written language . |
WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing datasets are often flooded with repetitive and spurious patterns, leading to a lack of linguistic diversity. |
| Approach: | They propose a method that uses cartography to automatically identify and filter examples that demonstrate challenging reasoning patterns and then automatically compose new ones with similar patterns. |
| Outcome: | The proposed approach improves performance on eight out-of-domain test sets compared to training on the 4x larger MultiNLI dataset. |
Sequentially Controlled Text Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Using GPT-2, long documents can ramble and do not follow human-like writing structure. |
| Approach: | They propose a controlled text generation task that generates documents with structure . they use a news article as a dataset to test different degrees of structural awareness . |
| Outcome: | The proposed task generates documents with a structure that is human-like, but long documents lack structure. |
Revisiting the Roles of “Text” in Text Games (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work has shown that random text hashes could be complementary rather than contrasting in text games. |
| Approach: | They propose a scheme to extract contextual information into an approximate state hash as extra input for an RNN-based text agent. |
| Outcome: | The proposed scheme achieves competitive performance with state-of-the-art text agents using advanced NLU techniques such as knowledge graph and passage retrieval. |
FPT: Improving Prompt Tuning Efficiency via Progressive Training (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent prompt tuning (PT) has gained increasing attention as a parameter-efficient way of tuning pre-trained language models (PLMs). |
| Approach: | They propose a prompt tuning algorithm that uses a small-scale partial PLM and progressively expands its depth and width until the full-model size. |
| Outcome: | The proposed method could save over 30% of training computations while achieving comparable performance. |
Prompt-learning for Fine-grained Entity Typing (2022.findings-emnlp)
Copied to clipboard
Ning Ding, Yulin Chen, Xu Han, Guangwei Xu, Xiaobin Wang, Pengjun Xie, Haitao Zheng, Zhiyuan Liu, Juanzi Li, Hong-Gee Kim
| Challenge: | Extensive experiments on fine-grained entity typing under fully supervised, few-shot, and zero-shot settings show the effectiveness of prompt-learning. |
| Approach: | They propose a prompt-learning pipeline that stimulates versatile knowledge of pre-trained language models (PLMs) by constructing entity-oriented verbalizers and templates and conducting masked language modeling. |
| Outcome: | The proposed approach can be applied to fine-grained entity typing in fully supervised, few-shot, and zero-shot scenarios. |
TransLIST: A Transformer-Based Linguistically Informed Sanskrit Tokenizer (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to SWS fail when encountering out-of-vocabulary tokens . lexicon driven approaches fail when dealing with out- of-vocal tokens, authors say . |
| Approach: | They propose a novel lexicon-driven shallow parser that takes into account sandhi phenomenon . they propose linguistically informed Sanskrit tokenizer module that encodes character input . |
| Outcome: | The proposed system outperforms the current state-of-the-art system by 7.2 points absolute gain on the benchmark datasets. |
Fair NLP Models with Differentially Private Text Encoders (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Encoded text representations often capture sensitive attributes about individuals, raising privacy concerns and making models unfair to certain groups. |
| Approach: | They propose an approach that combines privacy and adversarial training to learn private representations which induces fairer models. |
| Outcome: | The proposed approach improves on four NLP datasets and shows that privacy and fairness can positively reinforce each other. |
Modeling Context With Linear Attention for Scalable Document-Level Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Document-level machine translation models lack quadratic complexity in the sequence length due to their attention layers. |
| Approach: | They evaluate a recent linear attention model with a sentential gate to promote a recency inductive bias and compare it to open-source document translation. |
| Outcome: | The proposed model significantly improves translation quality on IWSLT 2015 and OpenSubtitles 2018 with similar or better BLEU scores. |
What do Large Language Models Learn beyond Language? (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pretraining on text confers models with useful ‘inductive biases’ for non-linguistic reasoning. |
| Approach: | They investigate whether pre-training on text confers these models with helpful ‘inductive biases’ for non-linguistic reasoning. |
| Outcome: | The proposed models outperform non-pretrained models on 19 non-linguistic tasks and show that they retain inductive biases even when training on multi-lingual text and computer code. |
CONSISTENT: Open-Ended Question Generation From News Articles (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work on question generation has largely focused on factoid questions such as who, what, where, when about basic facts. |
| Approach: | They propose an end-to-end system for generating openended questions that are answerable from and faithful to the input text. |
| Outcome: | The proposed model outperforms existing models and can be used in news media organizations. |
Efficient (Soft) Q-Learning for Text Generation with Limited Good Data (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Maximum likelihood estimation (MLE) is the predominant method for training text generation models. |
| Approach: | They propose a new RL formulation for text generation from the soft Q-learning perspective using path consistency learning to combine the best of on-/off-policy updates and learn effectively from sparse reward. |
| Outcome: | The proposed approach outperforms MLE and previous RL methods in a wide range of tasks. |
Lexi: Self-Supervised Learning of the UI Language (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models rely on UI metadata, which is often missing or not accessible. |
| Approach: | They propose a vision and language model that can learn generic visio-linguistic representations of UIs . they use a dataset to train Lexi, which is based on UI metadata . |
| Outcome: | The proposed model can handle unique features of UI screens, including text richness and context sensitivity. |
Inferring the Reader: Guiding Automated Story Generation with Commonsense Reasoning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to automate story generation focus on single-character stories and lack basiccommonsense reasoning. |
| Approach: | They propose a commonsense-inference Augmentedneural StoryTelling framework that introduces commonsensical reasoning into the story generation process. |
| Outcome: | The proposed method produces significantly more coherent, on-topic, enjoyable andfluent stories than existing models in both the single-character and two-character settings. |
How to Stop an Avalanche? JoDeM: Joint Decision Making through Compare and Contrast for Dialog State Tracking (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing state-of-the-art models of dialog state tracking do not address avalanche phenomenon . well-known commercial dialog systems include the Apple Siri, Amazon Alexa, or Microsoft Cortana. |
| Approach: | They propose a dialog state tracking (DST) model which can tackle the avalanche phenomenon . they propose combining a jointly decision making method and a compare and contrast dialogue update technique . |
| Outcome: | The proposed model outperforms existing state-of-the-art methods and proves its validity. |
Contrastive Learning with Prompt-derived Virtual Semantic Prototypes for Unsupervised Sentence Embedding (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies focus on instance-wise contrastive learning, attempting to construct positive pairs with textual data augmentation. |
| Approach: | They propose a novel Contrastive learning method with Prompt-derived Virtual semantic prototypes that constructs virtual semantic prototype to each instance and derives negative prototypes by using the negative form of the prompts. |
| Outcome: | The proposed method performs on semantic textual similarity, transfer, and clustering tasks compared to baselines. |
Weight Perturbation as Defense against Adversarial Word Substitutions (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existence and pervasiveness of textual adversarial examples have raised serious concerns to security-critical applications. |
| Approach: | They propose to perform weight perturbations in the parameter space rather than the input feature space to improve adversarial robustness of NLP models. |
| Outcome: | The proposed method improves adversarial robustness of models by performing weight perturbations in the parameter space rather than the input feature space. |
CORT: A New Baseline for Comparative Opinion Classification by Dual Prompts (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Comparative opinion classification is a common linguistic phenomenon. |
| Approach: | They propose a framework for comparative opinion classification using embedded knowledge in pre-trained language models. |
| Outcome: | The proposed framework delivers state-of-the-art and robust performance on all benchmark datasets. |
APEACH: Attacking Pejorative Expressions with Analysis on Crowd-Generated Hate Speech Evaluation Datasets (2022.findings-emnlp)
Copied to clipboard
| Challenge: | flaming or trolling in online communities is considered hostile behavior . a dataset of hate speech examples can be useful for detecting toxic or pejorative expressions . annotating on existing web text has several limitations that deter the dataset's reliability . |
| Approach: | They propose a dataset that asks users to generate hate speech examples followed by minimal post-labeling. |
| Outcome: | a new approach can collect useful datasets that are less sensitive to overlaps, the authors say . annotating on web text has several limitations that deter the dataset's reliability . |
Guiding Neural Story Generation with Reader Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing systems that generate narratives with neural language models require substantial knowledge engineering of logical constraints, limiting their generality. |
| Approach: | They propose a framework in which a reader model is used to reason about the storyshould progress. |
| Outcome: | The proposed model outperforms baseline models in plot plausibility and staying on topic. |
Reason first, then respond: Modular Generation for Knowledge-infused Dialogue (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models can produce fluent dialogue but often hallucinate factual inaccuracies. |
| Approach: | They propose a modular model for incorporating knowledge into conversational agents that generates a knowledge sequence and then attends to its own generated knowledge sequence. |
| Outcome: | The proposed model hallucinates less in knowledge-grounded dialogue tasks and has advantages in terms of interpretability and modularity. |
Adapting Multilingual Models for Code-Mixed Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work has addressed the lack of gold standard code-mixed to pure language parallel data with data augmentation techniques. |
| Approach: | They propose a back-translation-based training scheme for code-mixed translation which eliminates dependence on external resources. |
| Outcome: | The proposed model beats previous work by up to +3.8 BLEU on code-mixed tasks. |
LPC: A Logits and Parameter Calibration Framework for Continual Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to solve catastrophic forgetting problem are varied . current approaches to learn continuous learning are based on replay-based methods . |
| Approach: | They propose to calibrate parameters and logits so that preserving old parameters and generalized learning on new concepts can be solved simultaneously. |
| Outcome: | The proposed model achieves state-of-the-art performance in all scenarios. |
SlovakBERT: Slovak Masked Language Model (2022.findings-emnlp)
Copied to clipboard
Matúš Pikuliak, Štefan Grivalský, Martin Konôpka, Miroslav Blšták, Martin Tamajka, Viktor Bachratý, Marian Simko, Pavol Balážik, Michal Trnka, Filip Uhlárik
| Challenge: | SlovakBERT is a new masked language model that is based on a Web-crawled corpus. |
| Approach: | They introduce a new Slovak-only transformers-based language model called SlovkBERT . they evaluate the model on several NLP tasks and establish a benchmark for Slovakia . |
| Outcome: | The proposed model achieves state-of-the-art on several NLP tasks and achieves best results . the proposed model could be used by other Slovak researchers or NLP practitioners . |
Efficient Zero-shot Event Extraction with Context-Definition Alignment (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Conventional supervised methods cannot generalize to event types out of the pre-defined ontology. |
| Approach: | They propose to use two separate transformer models to model the definition semantics of an event type name into the same embedding space and then minimize their embeddable distance via contrastive learning. |
| Outcome: | The proposed model outperforms all previous zero-shot EE methods with fast inference speed due to the disjoint design. |
Logical Fallacy Detection (2022.findings-emnlp)
Copied to clipboard
Zhijing Jin, Abhinav Lalwani, Tejas Vaidhya, Xiaoyu Shen, Yiwen Ding, Zhiheng Lyu, Mrinmaya Sachan, Rada Mihalcea, Bernhard Schoelkopf
| Challenge: | Existing language models perform poorly on logical fallacy detection . fallacious arguments can lead to disagreements, conflicts, endless debates, and a lack of consensus . |
| Approach: | They propose a task of logical fallacy detection and propose LogicClimate to detect fallacies in text. |
| Outcome: | The proposed task outperforms the best language model on Logic and LogicClimate . human reasoning is marred by logical fallacies, and some exacerbate misinformation . |
Topic-Aware Response Generation in Task-Oriented Dialogue with Unstructured Knowledge Access (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results indicate that TARG achieves state-of-the-art performance in knowledge selection and response generation, outperforming previous state- of-the art by 3.2, 3.6, and 4.2 points in EM, F1 and BLEU-4 respectively on Doc2Dial. |
| Approach: | They propose to integrate topical information into knowledge-grounded task-oriented dialogue systems by using multiple topic-aware attention mechanisms to derive the importance weighting scheme over dialogue utterances and external knowledge sources. |
| Outcome: | The proposed model outperforms existing models in knowledge selection and response generation. |
Revisiting Transformer-based Models for Long Document Classification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent literature in text classification is biased towards short text sequences . multi-page multi-paragraph documents cannot be efficiently encoded by vanilla transformers based on short text. |
| Approach: | They compare different Transformer-based Long Document Classification approaches to mitigate the computational overhead of vanilla transformers to encode much longer text. |
| Outcome: | The proposed models can process longer text and provide practical advice for long document classification tasks. |
Time-aware Prompting for Text Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a new study investigates the effects of incorporating timestamps into generation systems . textual prompts focus more on non-temporal information and are less sensitive to given timestams . |
| Approach: | They propose a data-to-text generation dataset that includes chronologically ordered revisions of biographical articles from English Wikipedia. |
| Outcome: | The proposed models improve the quality of the data-to-text generation dataset TempWikiBio . the proposed models are more sensitive to time-aware prompts than textual prompts . |
Improving Scheduled Sampling with Elastic Weight Consolidation for Neural Machine Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Autoregressive models trained with maximum likelihood estimation suffer from exposure bias, i.e. the discrepancy between ground-truth prefixes used during training and model-generated prefix at inference time. |
| Approach: | They propose to use Elastic Weight Consolidation to better balance mitigating exposure bias with retaining performance. |
| Outcome: | The proposed method significantly outperforms maximum likelihood estimation and scheduled sampling baselines on four translation datasets. |
Ensemble Transformer for Efficient and Accurate Ranking Tasks: an Application to Question Answering Systems (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Large transformer models are expensive and slow to use in many applications. |
| Approach: | They propose an efficient neural network to distill large transformers into a single smaller model. |
| Outcome: | The proposed model outperforms existing models on English datasets . it outperformed existing models with 2.7 more parameters and 2.5 slower . |
Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysis (2022.findings-emnlp)
Copied to clipboard
Yuxin Xiao, Paul Pu Liang, Umang Bhatt, Willie Neiswanger, Ruslan Salakhutdinov, Louis-Philippe Morency
| Challenge: | Pre-trained language models (PLMs) have gained increasing popularity due to compelling prediction performance in diverse natural language processing tasks. |
| Approach: | They compare three popular options for encoding and Temp Scaling for PLMs . they recommend using Temp Loss as uncertainty quantifier and Focal Loss for fine-tuning . |
| Outcome: | Using pre-trained language models, we compare three options on NLP classification tasks and domain shift. |
How to Represent Context Better? An Empirical Study on Context Modeling for Multi-turn Response Selection (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on building a conversational system for open domain human-machine conversation is attracting more attention . early models concatenate all utterances or independently encode each dialogue turn, which may lead to an inadequate understanding of dialogue status. |
| Approach: | They propose to use a turn-aware context modeling layer to adapt existing models . they propose to model multi-turn contexts from the perspective of sequential relationship, local relationship, and query-alike manner . |
| Outcome: | The proposed method can be adapted to several advanced response selection models. |
CHIA: CHoosing Instances to Annotate for Machine Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Neural machine translation systems perform poorly on low-resource language pairs, for which large-scale parallel data is unavailable. |
| Approach: | They propose a method for selecting instances to annotate for machine translation using existing multi-way parallel datasets. |
| Outcome: | The proposed method outperforms unsupervised methods on 20 languages and a multi-way parallel dataset on high-resource languages. |
Guiding Neural Machine Translation with Semantic Kernels (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Empirical studies show that our approach gains approximately an improvement of 1 BLEU score on most benchmarks over the Transformer baseline. |
| Approach: | They propose to extract several semantic kernels from a source sentence to capture global semantic information. |
| Outcome: | Empirical results show that the proposed approach improves 1 BLEU score on benchmarks . it is also 1.7 times faster than previous works on average at inference time . |
HiSMatch: Historical Structure Matching based Temporal Knowledge Graph Reasoning (2022.findings-emnlp)
Copied to clipboard
Zixuan Li, Zhongni Hou, Saiping Guan, Xiaolong Jin, Weihua Peng, Long Bai, Yajuan Lyu, Wei Li, Jiafeng Guo, Xueqi Cheng
| Challenge: | Temporal Knowledge Graphs (TKGs) store facts as triples in the form of subject, relation, object, timestamps. |
| Approach: | They propose a Temporal Knowledge Graph (TKG) model that extends each triple with a timestamp to describe dynamic facts. |
| Outcome: | The proposed model improves on six benchmark datasets with up to 5.6% performance improvement compared to the state-of-the-art models. |
Dependency Parsing via Sequence Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for dependency parsing are transition-based, graph-based and sequence-to-sequence method. |
| Approach: | They propose to achieve dependency parsing (DP) via Sequence Generation (SG) by utilizing only the pre-trained language model without any auxiliary structures. |
| Outcome: | The proposed method performs well on DP benchmarks including PTB, UD2.2, SDP15 and SemEval16. |
Scaling Laws Under the Microscope: Predicting Transformer Performance from Small Scale Experiments (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Neural scaling laws define a predictable relationship between a model’s parameter count and its performance after training in the form of a power law. |
| Approach: | They perform an empirical investigation of language understanding tasks and evaluate their results to determine whether scaling laws can be used to accelerate model development. |
| Outcome: | The proposed scaling laws can be exploited for debugging convergence when training large models, and can predict the performance of larger models. |
Analyzing the Limits of Self-Supervision in Handling Bias in Language (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that natural language models can perform tasks with little to no in-context supervision . a number of tasks are performed using self-supervised pre-training . |
| Approach: | They define and comprehensively evaluate how well natural language taskprompting captures the semantics of four tasks for bias: diagnosis, identification, extraction and rephrasing. |
| Outcome: | The proposed model performs to wide varying degrees across bias dimensions . the model is largely challenged when prompted to perform these tasks . |
Multiple Instance Learning for Offensive Language Detection (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to detect offensive content are expensive and require massive manual effort. |
| Approach: | They propose an approach capable of utilizing the bag-level labeled data for offensive language detection by an annotation-based model. |
| Outcome: | The proposed model can detect offensive language on both bag-level and sentence level. |
Grounded Keys-to-Text Generation: Towards Factual Open-Ended Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Large pre-trained language models have enabled open-ended generation frameworks to tackle a variety of tasks beyond data-to-text generation. |
| Approach: | They propose a new task to generate a factual description about an entity given guiding keys and grounding passages using a dataset. |
| Outcome: | The proposed model improves factual correctness and recall significantly compared to previous models. |