Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
PiC: A Phrase-in-Context Dataset for Phrase Understanding and Semantic Search (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing benchmarks for phrase-similarity compare phrases alone (without context) and phrases with context (with or without context). |
| Approach: | They propose to use a dataset of 28K noun phrases accompanied by their contextual Wikipedia pages to train machine phrase embeddings. |
| Outcome: | The proposed dataset improves ranking-models’ accuracy and pushes span selection models near human accuracy, which is 95% Exact Match (EM) on semantic search given a query phrase and a passage. |
Enhancing Dialogue Summarization with Topic-Aware Global- and Local- Level Centrality (2023.eacl-main)
Copied to clipboard
| Challenge: | Experimental results show that our model outperforms strong baselines on three public dialogue summarization datasets: CSDS, MC, and SAMSUM. |
| Approach: | They propose a topic-aware global-local centrality model to help select the salient context from all sub-topics. |
| Outcome: | The proposed model outperforms baselines on three public dialogue summarization datasets: CSDS, MC, and SAMSUM. |
Exploiting Summarization Data to Help Text Simplification (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing text simplification datasets are limited to Wikipedia and Newsela, restricting further development of this field. |
| Approach: | They propose an alignment algorithm to extract sentence pairs from summarization datasets and a method to filter suitable pairs. |
| Outcome: | The proposed algorithm can extract sentence pairs from summarization datasets and perform well with real datasets. |
Shironaam: Bengali News Headline Generation using Auxiliary Information (2023.eacl-main)
Copied to clipboard
| Challenge: | Automated headline generation systems have the potential to assist editors in finding interesting headlines to attract visitors or readers. |
| Approach: | They propose to use Bengali news article-headline pairings with auxiliary data to better model headline generation using pre-trained language models. |
| Outcome: | The proposed model improves on a Bengali news headline generation dataset by 3 to 10 percentage points over baselines. |
PCC: Paraphrasing with Bottom-k Sampling and Cyclic Learning for Curriculum Data Augmentation (2023.eacl-main)
Copied to clipboard
| Challenge: | Curriculum Data Augmentation (CDA) presents synthetic data with increasing difficulties to neural models. |
| Approach: | They propose a curriculum-aware paraphrase generation module with bottom-k sampling and cyclic learning strategy that passes through the curriculums multiple times. |
| Outcome: | The proposed framework surpasses competitive baselines on few-shot text classification and dialogue generation. |
A Two-Sided Discussion of Preregistration of NLP Research (2023.eacl-main)
Copied to clipboard
| Challenge: | et al. (2021) suggest NLP research should adopt preregistration to prevent fishing expeditions and promote publication of negative results. |
| Approach: | et al. suggest NLP research should adopt preregistration to prevent fishing expeditions and promote publication of negative results. |
| Outcome: | The proposed approach solves many methodological problems with NLP research. |
WinoDict: Probing language models for in-context word acquisition (2023.eacl-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are unable to reflect the way language changes over time as their training corpus is frozen in time. |
| Approach: | They propose a new in-context learning paradigm to measure Large Language Models' ability to learn novel words during inference. |
| Outcome: | The proposed model improves on Winograd-style co-reference resolution problems by replacing the key concept word with a plausible word that the model must understand to complete the task. |
Sentiment as an Ordinal Latent Variable (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing dictionaries are limited in coverage and sentiment scales vary widely; some are discrete others continuous. |
| Approach: | They propose a Bayesian generative model that learns a composite sentiment dictionary as an interpolation between six existing dictionaries with different scales. |
| Outcome: | The proposed model learns a composite sentiment dictionary as an interpolation between six existing dictionaries with different scales. |
Nationality Bias in Text Generation (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing studies have shown that nationality biases in language models can be a factor in improving the performance of social NLP models. |
| Approach: | They propose to use a text generation model, GPT-2, to analyze how the number of internet users and the country’s economic status affects the sentiment of stories. |
| Outcome: | The proposed model accentuates biases about country-based demonyms and reduces them with the use of adversarial triggering. |
Investigating data partitioning strategies for crosslinguistic low-resource ASR evaluation (2023.eacl-main)
Copied to clipboard
| Challenge: | Automatic speech recognition data sets include a single pre-defined test set consisting of one or more speakers whose speech never appears in the training set. |
| Approach: | They propose to use hold-speaker(s)-out partitioning to partition data for five languages . utterance duration and intensity are more predictive factors of variability . |
| Outcome: | The proposed method can produce results that do not reflect model performance on unseen data or speakers. |
Shortcomings of Question Answering Based Factuality Frameworks for Error Localization (2023.eacl-main)
Copied to clipboard
| Challenge: | Abstractive summarization systems often generate summaries with factual errors . many approaches to detect these errors have been proposed, but this capability has not been evaluated in past research . |
| Approach: | They propose to use question answering-based factuality metrics to detect errors in summaries . they find that QA-based frameworks fail to correctly identify error spans in generated summary . |
| Outcome: | The proposed methods outperform trivial exact match baselines in localizing errors in summaries. |
Socratic Question Generation: A Novel Dataset, Models, and Evaluation (2023.eacl-main)
Copied to clipboard
| Challenge: | Socratic questioning is a form of reflective inquiry often employed in education to encourage critical thinking in students. |
| Approach: | They present a first large dataset of 110K questions, context pairs for Socratic Question Generation. |
| Outcome: | The proposed model produces realistic, type-sensitive, human-like Socratic questions . authors show that the model can be used in counseling and coaching . |
Do we need Label Regularization to Fine-tune Pre-trained Language Models? (2023.eacl-main)
Copied to clipboard
Ivan Kobyzev, Aref Jafari, Mehdi Rezagholizadeh, Tianda Li, Alan Do-Omri, Peng Lu, Pascal Poupart, Ali Ghodsi
| Challenge: | Knowledge Distillation (KD) is a label regularization technique that can be replaced with lighter teacher-free variants such as the label-smoothing technique. |
| Approach: | They propose to use knowledge distillation to train student models by deploying the teacher network during training. |
| Outcome: | The proposed method can be replaced with lighter teacher-free variants on PLMs with more than 600 distinct trials and ran each configuration five times. |
COVID-VTS: Fact Extraction and Verification on Short Video Platforms (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for fact checking multimodal information are limited due to the lack of public data. |
| Approach: | They propose a benchmark for fact-checking multi-modal information involving short-duration videos with COVID19- focused information from both the real world and machine generation. |
| Outcome: | The proposed model detects token-level malicious tampering in different modalities and generates explanations. |
Multimodal Graph Transformer for Multimodal Question Answering (2023.eacl-main)
Copied to clipboard
| Challenge: | a myriad of complex tasks require both prior knowledge and reasoning intelligence. |
| Approach: | They propose a plug-and-play quasi-attention mechanism to integrate multimodal graph information to vanilla self-attention as effective prior. |
| Outcome: | The proposed model is able to perform reasoning across multiple modalities. |
Retrieval Enhanced Data Augmentation for Question Answering on Privacy Policies (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing labeled datasets are heavily imbalanced, limiting the QA performance in this domain. |
| Approach: | They propose a question answering task that captures relevant text segments from unlabeled policy documents and expands the positive examples in the training set. |
| Outcome: | The proposed framework elevates the baseline by a large margin (10% F1) and achieves a new state-of-the-art F1 score of 50%. |
FastKASSIM: A Fast Tree Kernel-Based Syntactic Similarity Metric (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing syntactic similarity metrics are computationally expensive and inconsistent when faced with syntaktically dissimilar documents. |
| Approach: | They propose a metric which pairs and averages the most similar constituency parse trees between a pair of documents based on tree kernels. |
| Outcome: | The proposed metric is more robust to syntactic dissimilarities and runs up to 5.32 times faster than its predecessor over documents in the r/ChangeMyView corpus. |
Friend-training: Learning from Models of Different but Related Tasks (2023.eacl-main)
Copied to clipboard
| Challenge: | Current self-training methods focus on improving model performance on a single task. |
| Approach: | They propose a cross-task self-training framework where models trained to do different tasks are used in iterative training, pseudo-labeling, and retraining processes to help each other for better selection of pseudo-labeled labels. |
| Outcome: | The proposed framework achieves the best performance compared to baselines on two dialogue understanding tasks. |
Understanding Transformer Memorization Recall Through Idioms (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for analyzing memorization use definitions that are based on model performance, which changes between models and often also between training runs. |
| Approach: | They propose idioms as inputs that typically trigger memory recall and propose a set of English idiomas to test their methodological framework for probing and characterizing recall of memorized sequences in transformer LMs. |
| Outcome: | The proposed framework compares model behavior on memorized vs. non-memorized inputs across different model sizes and architectures. |
A Discerning Several Thousand Judgments: GPT-3 Rates the Article + Adjective + Numeral + Noun Construction (2023.eacl-main)
Copied to clipboard
| Challenge: | a number of studies have focused on the use of rare, idiosyncratic constructions. |
| Approach: | They ask GPT-3 to give acceptability judgments on an English-language construction . they validate the prompt and then zero in on the AANN construction based on CoLA corpus . |
| Outcome: | The proposed judgments are similar to human judgments but differ from the literature and from each other. |
Triple-Hybrid Energy-based Model Makes Better Calibrated Natural Language Understanding Models (2023.eacl-main)
Copied to clipboard
| Challenge: | In-distribution (ID) miscalibration and out-of-difference (OOD) detection are main concerns for pre-trained language models. |
| Approach: | They propose a triple-hybrid EBM which combines the benefits of classifier, conditional generative model and marginal generative models altogether. |
| Outcome: | The proposed model outperforms previous methods in terms of ID calibration and OOD detection by a large margin while maintaining competitive accuracy. |
A weakly supervised textual entailment approach to zero-shot text classification (2023.eacl-main)
Copied to clipboard
Marc Pàmies, Joan Llop, Francesco Multari, Nicolau Duran-Silva, César Parra-Rojas, Aitor Gonzalez-Agirre, Francesco Alessandro Massucci, Marta Villegas
| Challenge: | Existing methods to train on weakly supervised datasets are expensive due to the computational cost of pre-training. |
| Approach: | They propose a method that trains on a weakly supervised dataset that is used as a proxy for a textual entailment problem and a target zero-shot text classification task. |
| Outcome: | The proposed model achieves state-of-the-art performance in the scientific domain and competitive results in other areas. |
Fair Enough: Standardizing Evaluation and Model Selection for Fairness Research in NLP (2023.eacl-main)
Copied to clipboard
| Challenge: | Modern NLP systems exhibit a range of biases, which a growing literature on model debiasing attempts to correct. |
| Approach: | They propose to clarify the current situation and plot a course for meaningful progress in fair learning by making clear inter-relations among the current gamut of methods and their relation to fairness theory. |
| Outcome: | The proposed approach addresses the practical problem of model selection, which involves a trade-off between fairness and accuracy and has led to systemic issues in fairness research. |
CHARD: Clinical Health-Aware Reasoning Across Dimensions for Text Generation Models (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing studies show that pretrained language models can act as knowledge bases and reason like humans. |
| Approach: | They propose to use pretrained language models to generate free-flow textual explanations about 52 health conditions across three clinical dimensions. |
| Outcome: | The proposed model can generate concise and readable text, but can be improved on medical accuracy and QA. |
Prompt Tuning with Contradictory Intentions for Sarcasm Recognition (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent advances have shown that Pre-trained Language Models (PLMs) can achieve promising performance in many downstream Natural Language Processing (NLP) tasks. |
| Approach: | They propose to incorporate prior knowledge about contradictory intentions into prompt tuning for sarcasm recognition by mimicking the actual intention by verbalizer engineering. |
| Outcome: | The proposed model mimics the actual intention by prompt construction and indicates whether the actual intent contradicts the literal content by verbalizer engineering. |
COMBO: A Complete Benchmark for Open KG Canonicalization (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing datasets for open KG canonicalization only provide gold entity-level canonization for noun phrases. |
| Approach: | They propose a complete benchmark for open KG canonicalization that provides gold ontology-level canonization for relation phrases and source sentences for extraction. |
| Outcome: | The proposed method improves relation canonicalization and ontology-level canonization of the noun phrase. |
UScore: An Effective Approach to Fully Unsupervised Evaluation Metrics for Machine Translation (2023.eacl-main)
Copied to clipboard
| Challenge: | supervised evaluation metrics are not available for machine translation, despite their wide dissemination. |
| Approach: | They develop fully unsupervised evaluation metrics that leverage parallel data and evaluation metric induction. |
| Outcome: | The proposed metrics beat supervised competitors on 4 out of 5 evaluation datasets. |
Assistive Recipe Editing through Critiquing (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for generating recipes that satisfy dietary restrictions are inconsistent or incoherent and paired datasets are not available at scale. |
| Approach: | They propose to build a hierarchical denoising auto-encoder that edits recipes given ingredient-level critiques by interacting with the predicted ingredients. |
| Outcome: | The proposed model can more effectively edit recipes compared to strong language models and iteratively rewrites recipes to satisfy user feedback. |
DiTTO: A Feature Representation Imitation Approach for Improving Cross-Lingual Transfer (2023.eacl-main)
Copied to clipboard
| Challenge: | Zero-shot cross-lingual transfer has been shown to be sub-optimal across low-resource languages due to the skew in resource distribution in languages. |
| Approach: | They propose to jointly reduce feature incongruity between the source and target language and increase generalization capabilities of pre-trained multilingual transformers. |
| Outcome: | Empirical results show that the proposed approach outperforms the standard zero-shot fine-tuning method on multiple datasets across all languages using only unlabeled instances in the target language. |
“John is 50 years old, can his son be 65?” Evaluating NLP Models’ Understanding of Feasibility (2023.eacl-main)
Copied to clipboard
Himanshu Gupta, Neeraj Varshney, Swaroop Mishra, Kuntal Kumar Pal, Saurabh Arjun Sawant, Kevin Scaria, Siddharth Goyal, Chitta Baral
| Challenge: | Recent work has found that large-scale language models lack commonsense reasoning ability . a dataset evaluating large-level language models is needed to evaluate their understanding of feasibility . |
| Approach: | They propose a question-answering dataset that tests understanding of feasibility . they propose to use commonsense reasoning to reason about when an action is feasible . |
| Outcome: | The proposed dataset shows that state-of-the-art models struggle to answer feasibility questions correctly. |
Efficient Encoders for Streaming Sequence Tagging (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing bidirectional encoders require a restart when a new token is received. |
| Approach: | They propose a Hybrid Encoder with Adaptive Restart that enables asynchronous encoding of a new token in an incremental streaming input. |
| Outcome: | The proposed encoder offers FLOP savings in streaming settings up to 71.1% and outperforms bidirectional encoders for streaming predictions by up to +0% streaming exact match. |
Retrieve-and-Fill for Scenario-based Task-Oriented Semantic Parsing (2023.eacl-main)
Copied to clipboard
Akshat Shrivastava, Shrey Desai, Anchit Gupta, Ali Elkahky, Aleksandr Livshits, Alexander Zotov, Ahmed Aly
| Challenge: | Task-oriented semantic parsing models have achieved strong results in recent years, but they often face obstacles adapting to novel settings with distinct semantics and scarce data. |
| Approach: | They propose a scenario-based semantic parsing model which isolates coarse-grained and fine-grounded aspects of the task and solves them with off-the-shelf neural modules. |
| Outcome: | The proposed model outperforms previous approaches in high-resource, low-resourced, and multilingual settings, and is modular, differentiable, interpretable, and allows extra supervision from scenarios. |
Document Flattening: Beyond Concatenating Context for Document-Level Neural Machine Translation (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing document-level neural machine translation systems concatenate several consecutive sentences to form a pseudo-document, and then learn inter-sentential dependencies. |
| Approach: | They propose a document flattening technique that integrates Flat-Batch Attention (FBA) and Neural Context Gate (NCG) into Transformer model to utilize information beyond the pseudo-document boundaries. |
| Outcome: | The proposed method outperforms baselines on BLEU, COMET and accuracy on the contrastive test set. |
Scaling Back-Translation with Domain Text Generation for Sign Language Gloss Translation (2023.eacl-main)
Copied to clipboard
| Challenge: | Sign language gloss translation aims to translate the sign glosses into spoken language texts, which is challenging due to the scarcity of labeled gloss-text parallel data. |
| Approach: | They propose a back translation technique that generates pseudo-parallel data by translating in-domain spoken language texts into sign glosses. |
| Outcome: | The proposed method outperforms the BT methods on three benchmarks of sign language gloss translation in different languages. |
Realistic Conversational Question Answering with Answer Selection based on Calibrated Confidence and Uncertainty Measurement (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing work uses predicted answers instead of unavailable ground-truth answers as conversation history for inference. |
| Approach: | They propose to filter out inaccurate answers in the conversation history without making any architectural changes to the model. |
| Outcome: | The proposed models outperform baselines on two standard ConvQA datasets. |
PANCETTA: Phoneme Aware Neural Completion to Elicit Tongue Twisters Automatically (2023.eacl-main)
Copied to clipboard
| Challenge: | Phonetic difficulty is hard to characterize and can be expressed in tongue twisters through alliteration and homophony. |
| Approach: | They propose a phoneme-aware neural completion to generate tongue twisters automatically . they leverage phoneme representations to capture phonetic difficulty and train language models . |
| Outcome: | The proposed language model generates novel, phonetically difficult, fluent, and semantically meaningful tongue twisters on two task settings. |
A User-Centered, Interactive, Human-in-the-Loop Topic Modelling System (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent research has demonstrated the value of user feedback, but there are still issues to consider, such as the difficulty in tracking changes and comparing different models. |
| Approach: | They propose a human-in-the-loop topic modeling system that integrates users’ knowledge into the modelling process, enabling them to refine the model iteratively. |
| Outcome: | The proposed system is based on a series of user studies to assess its performance in progressively more realistic applications. |
A Survey of Methods for Addressing Class Imbalance in Deep-Learning Based Natural Language Processing (2023.eacl-main)
Copied to clipboard
| Challenge: | Developing methods to improve model performance in imbalanced data settings has been an active area for decades . |
| Approach: | They propose to use sampling, data augmentation, choice of loss function, staged learning, or model design to address class imbalance in NLP. |
| Outcome: | The proposed approaches are evaluated on a variety of NLP tasks or in the computer vision community. |
Extracting or Guessing? Improving Faithfulness of Event Temporal Relation Extraction (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing models for event temporal relation extraction are based on data-driven machine learning . however, TEMPREL extraction is not accurate under distribution shifts. |
| Approach: | They propose to conduct counterfactual analysis to attenuate the effects of two types of training biases: the event trigger bias and the frequent label bias. |
| Outcome: | The proposed model extracts TempRel and timelines more faithfully compared to SOTA methods . it is based on two perspectives: one is to extract genuinely based upon contextual description . the other is to provide proper uncertainty estimation and abstain from extraction when no relation is described in the text . |
LoFT: Enhancing Faithfulness and Diversity for Table-to-Text Generation via Logic Form Control (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing models for LT2T generation focus on surface-level realizations without much logical inference. |
| Approach: | They propose a model that uses logic forms as fact verifiers and content planners to control LT2T generation. |
| Outcome: | Experimental results show that the proposed model addresses unfaithfulness and diversity issues simultaneously. |
PromptDA: Label-guided Data Augmentation for Prompt-based Few Shot Learners (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing studies on prompt-based few-shot tuning focus on deriving proper label words with a verbalizer or generating prompt templates to elicit semantics from PLMs. |
| Approach: | They propose a framework that leverages label semantics for prompt-based tuning. |
| Outcome: | The proposed framework improves on few-shot text classification tasks by leveraging label semantics and data augmentation. |
Incorporating Question Answering-Based Signals into Abstractive Summarization via Salient Span Selection (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for generating summarizations using QA-based supervision produce higher quality summaries than baseline methods. |
| Approach: | They propose a method for incorporating question-answering signals into a summarization model by automatically marking document NPs as salient based on whether they are answered in the gold summaries. |
| Outcome: | The proposed method generates higher-quality summaries than baseline methods on benchmark summarization datasets. |
Patient Outcome and Zero-shot Diagnosis Prediction with Hypernetwork-guided Multitask Learning (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent advances apply artificial intelligence to predict clinical events or infer the probable diagnosis for clinical decision support. |
| Approach: | They propose a hypernetwork-based approach that generates task-conditioned parameters and coefficients of multitask prediction heads to learn task-specific prediction and balance the multitask learning. |
| Outcome: | Experiments on clinical notes from the real-world MIMIC database show that the proposed model can achieve better performance than baselines and improve zero-shot prediction on unseen diagnoses. |
A Kind Introduction to Lexical and Grammatical Aspect, with a Survey of Computational Approaches (2023.eacl-main)
Copied to clipboard
| Challenge: | Lexical and grammatical aspect plays essential roles in semantic interpretation, but many systems do not address it systematically. |
| Approach: | They propose to model lexical and grammatical aspect using computational approaches . they argue that a good computational understanding of lexic and grammmatical aspects is needed . |
| Outcome: | The proposed models are based on the lexical and grammatical aspect of a situation, the authors argue . they argue that the models need to be able to handle and evaluate the aspect systematically . |
Incorporating Context into Subword Vocabularies (2023.eacl-main)
Copied to clipboard
| Challenge: | Current tokenizers are trained on word frequency statistics over a corpus without considering information about co-occurrence or context. |
| Approach: | They propose a tokenizer that bakes in contextualized signal at the vocabulary creation phase to tailor subwords for their downstream use. |
| Outcome: | The proposed tokenizer is able to keep token contexts cohesive while not incurring a large price in terms of encoding efficiency or domain robustness. |
LoRaLay: A Multilingual and Multimodal Dataset for Long Range and Layout-Aware Summarization (2023.eacl-main)
Copied to clipboard
| Challenge: | Text Summarization is a popular task and a challenge for neural models. |
| Approach: | They propose to exploit visual/layout information to capture long-range dependencies in summarization models by combining layout-aware and long-reaching models. |
| Outcome: | The proposed datasets cover French, Spanish, Portuguese, and Korean languages. |
ViHOS: Hate Speech Spans Detection for Vietnamese (2023.eacl-main)
Copied to clipboard
| Challenge: | Increasing use of social networking sites can cause problems for human moderators to review tagged comments. |
| Approach: | They present a dataset that contains 26k spans on 11k comments and detailed annotation guidelines . they also provide definitions of hateful and offensive spans in Vietnamese comments . |
| Outcome: | The proposed dataset shows that it is difficult to detect specific types of spans in the dataset . the dataset is the first human-annotated corpus containing 26k spans on 11k comments . |
Vote’n’Rank: Revision of Benchmarking with Social Choice Theory (2023.eacl-main)
Copied to clipboard
Mark Rofin, Vladislav Mikhailov, Mikhail Florinsky, Andrey Kravchenko, Tatiana Shavrina, Elena Tutubalina, Daniel Karabekyan, Ekaterina Artemova
| Challenge: | ML benchmarks have been criticized for their construct validity, fragility of the design and task choices. |
| Approach: | They propose a framework for ranking systems in multi-task benchmarks under the principles of the social choice theory and propose 'vote'n'rank' procedures are more robust than the mean average while being able to handle missing performance scores and determine conditions under which the system becomes the winner. |
| Outcome: | The proposed framework can be utilised to draw new insights on benchmarking in several ML sub-fields and identify the best-performing systems in research and development case studies. |
Combining Parameter-efficient Modules for Task-level Generalisation (2023.eacl-main)
Copied to clipboard
| Challenge: | A modular design encourages neural models to disentangle and recombine different facets of knowledge to generalise more systematically to new tasks. |
| Approach: | They propose a modular neural network where a subset of latent skills is associated with a parameter-efficient model adapter. |
| Outcome: | The proposed model improves sample efficiency and few-shot generalisation in supervised learning compared to baselines. |
Self-imitation Learning for Action Generation in Text-based Games (2023.eacl-main)
Copied to clipboard
| Challenge: | Text-based games are situated systems where the game agents observe textual descriptions, and generate textual commands to interact with the environment. |
| Approach: | They propose a confidence-based self-imitation model to generate action candidates for the RL agent by exploiting past valuable trajectories to adapt a pre-trained language model towards a target game. |
| Outcome: | The proposed model performs well in multiple challenging games. |
Investigating the Effect of Relative Positional Embeddings on AMR-to-Text Generation with Structural Adapters (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent approaches to text generation from Abstract Meaning Representation (AMR) have been based on neural-centered encoderdecoder architectures. |
| Approach: | They propose a structure-aware adapter which injects the input graph connectivity within PLMs using Graph Neural Networks. |
| Outcome: | The proposed adapter is robust to a variety of approaches and can be used to generate Graph-to-Text representations. |
On the Intersection of Context-Free and Regular Languages (2023.eacl-main)
Copied to clipboard
| Challenge: | We generalize the Bar-Hillel intersection construction so that the given WFSA may contain -arcs. |
| Approach: | They propose a construction that generalizes the Bar- Hillel in the case the desired automaton has -arcs and generalize the weighted extension so that the given WFSA may contain arcs. |
| Outcome: | The proposed construction can encode the structure of both the input automaton and grammar while retaining the asymptotic size of the original construction. |
Social Influence Dialogue Systems: A Survey of Datasets and Models For Social Influence Tasks (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing research focuses on task-oriented or open-domain dialogue systems with influence skills. |
| Approach: | They propose to define and introduce a category of social influence dialogue systems that influence users’ cognitive and emotional responses. |
| Outcome: | The proposed system is task-oriented or goal-oriented, but it is not open-domain. |
Aggregating Crowdsourced and Automatic Judgments to Scale Up a Corpus of Anaphoric Reference for Fiction and Wikipedia Texts (2023.eacl-main)
Copied to clipboard
Juntao Yu, Silviu Paun, Maris Camilleri, Paloma Garcia, Jon Chamberlain, Udo Kruschwitz, Massimo Poesio
| Challenge: | Existing approaches to scale up anaphoric annotation have not overcome these limitations. |
| Approach: | They propose to use a game-with-a-purpose to ‘complete’ markable annotations by using an anaphoric resolver and an aggregation method for anaphorism. |
| Outcome: | The proposed method could be adopted to greatly speed up annotation time in other projects involving games-with-a-purpose. |
What Makes Sentences Semantically Related? A Textual Relatedness Dataset and Empirical Study (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing work on semantic relatedness has focused on semantic similarity because of a lack of relatedness datasets. |
| Approach: | They propose a dataset for semantic relatedness that has 5,500 English sentence pairs manually annotated using a comparative annotation framework. |
| Outcome: | The proposed dataset has 5,500 English sentence pairs manually annotated using a comparative annotation framework. |
RevUp: Revise and Update Information Bottleneck for Event Representation (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing external (“side”) semantic knowledge has been shown to result in more expressive computational event models. |
| Approach: | They propose a semi-supervised information bottleneck-based discrete latent variable model that reparameterizes discrete variables with auxiliary continuous latent variables and a light-weight hierarchical structure. |
| Outcome: | The proposed model outperforms existing models on multiple datasets. |
NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian Local Languages (2023.eacl-main)
Copied to clipboard
Genta Indra Winata, Alham Fikri Aji, Samuel Cahyawijaya, Rahmad Mahendra, Fajri Koto, Ade Romadhony, Kemal Kurniawan, David Moeljadi, Radityo Eko Prasojo, Pascale Fung, Timothy Baldwin, Jey Han Lau, Rico Sennrich, Sebastian Ruder
| Challenge: | In Indonesia, many languages are endangered and some are even extinct due to the unavailability of data resources and benchmarks. |
| Approach: | They propose a high-quality multilingual parallel corpus that covers 10 local languages from Indonesia. |
| Outcome: | The proposed resource includes sentiment and machine translation datasets, and bilingual lexicons. |
The Functional Relevance of Probed Information: A Case Study (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent studies have shown that transformer models like BERT rely on number information encoded in their representations of sentences’ subjects and head verbs when performing subject-verb agreement. |
| Approach: | They propose to use probing to find out which words contain functionally relevant information encoded in the representations of subject plurality and words that agree with it in number in BERT. |
| Outcome: | The proposed model only uses the subject plurality information encoded in its representations of the subject and words that agree with it in number. |
Do Pretrained Contextual Language Models Distinguish between Hebrew Homograph Analyses? (2023.eacl-main)
Copied to clipboard
| Challenge: | Semitic morphologically-rich languages are characterized by extreme word ambiguity . many of the words are homographs with multiple possible analyses . |
| Approach: | They evaluate existing models for Hebrew homographs using word-piece embeddings . they find they are more effective when the number of word-part splits is limited . |
| Outcome: | The proposed models outperform non-contextualized embeddings on Hebrew homograph challenge sets. |
Parameter-Efficient Tuning with Special Token Adaptation (2023.eacl-main)
Copied to clipboard
| Challenge: | a recent study shows that parameter-efficient tuning is a challenge for multitask deployments. |
| Approach: | They propose a parameter-efficient tuning technique that only updates a small subset of parameters when adapting a pretrained model to downstream tasks. |
| Outcome: | The proposed method achieves comparable performance to fine-tuning in natural language understanding tasks including text classification and NER with only 0.029% of parameters trained. |
Probing Power by Prompting: Harnessing Pre-trained Language Models for Power Connotation Framing (2023.eacl-main)
Copied to clipboard
| Challenge: | Using pre-trained language models, we investigated whether word choices can encode subtle connotative information about power differentials between involved entities. |
| Approach: | They propose a framework to disentangle connotation frames implied by the predicate from its arguments and the sentence structure and to quantify predicates. |
| Outcome: | The proposed framework improves power connotation prediction accuracy by fine-tuning pre-trained language models. |
Zero and Few-Shot Localization of Task-Oriented Dialogue Agents with a Distilled Representation (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing low-cost approaches to build a high-quality functioning dialogue agent are limited to a few widely-spoken languages. |
| Approach: | They propose automatic methods that use ToD training data to build a functioning agent in another language . they compare the method to existing methods that only use a small training set . |
| Outcome: | The proposed method improves the state-of-the-art in Chinese to English transfer using zero-shot data compared to existing full-shot methods . the proposed method achieves 46.7% and 22.0% in task success rate and dialogue success rate, respectively. |
Contextual Semantic Parsing for Multilingual Task-Oriented Dialogues (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for predicting state of a conversation are limited to a few languages . a method that can be applied to other languages will benefit the large population of speakers of many other languages. |
| Approach: | They propose to automatically translate large-scale dialogue data sets in one language to produce an effective semantic parser for other languages using machine translation. |
| Outcome: | The proposed model reduces the compounding effect of translation errors without harming the accuracy in practice. |
Teacher Intervention: Improving Convergence of Quantization Aware Training for Ultra-Low Precision Transformers (2023.eacl-main)
Copied to clipboard
| Challenge: | Quantization-aware training (QAT) is a promising method to lower the implementation cost and energy consumption. |
| Approach: | They propose a method for fast converging QAT of pre-trained Transformers using a layer-wise signal propagation method with the intact signal from the teacher. |
| Outcome: | The proposed method achieves superior accuracy with significantly lower fine-tuning iterations on well-known Transformers of natural language processing as well as computer vision compared to the state-of-the-art methods. |
Generative Replay Inspired by Hippocampal Memory Indexing for Continual Language Learning (2023.eacl-main)
Copied to clipboard
| Challenge: | Continual learning (CL) is a fundamental requirement for human-like general intelligence (Parisi et al., 2019). |
| Approach: | They propose to control sample generation using compressed features of previous training samples by using hippocampal memory indexing to enhance the generative replay. |
| Outcome: | The proposed method outperforms current generative replay methods and generates training samples from previous tasks. |
A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methods (2023.eacl-main)
Copied to clipboard
| Challenge: | Multi-task learning is a popular approach in natural language processing because of its commonalities and differences. |
| Approach: | They propose to summarize recent advances in multi-task learning methods based on their task relatedness into two general multi-step training methods. |
| Outcome: | The proposed methods summarize the tasks and discuss future directions. |
Conclusion-based Counter-Argument Generation (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing work on the automatic generation of natural language counter-arguments does not address the relation to the conclusion, possibly because many arguments leave their conclusion implicit. |
| Approach: | They propose a multitask approach that jointly learns to generate both the conclusion and the counter of an input argument. |
| Outcome: | The proposed approach generates more relevant and stance-adhering counters than strong baselines. |
Question-Answer Sentence Graph for Joint Modeling Answer Selection (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing approaches to automate Question Answering (QA) are graph-based and can target large text databases. |
| Approach: | They propose graph-based approaches for Answer Sentence Selection (AS2) . they train and integrate state-of-the-art (SOTA) models for computing scores . |
| Outcome: | The proposed approach outperforms baseline models on academic benchmarks and a real-world dataset on unseen queries. |
Evaluating and Improving the Coreference Capabilities of Machine Translation Models (2023.eacl-main)
Copied to clipboard
| Challenge: | Currently, end-to-end models learn coreference resolution implicitly by observing aligned sentences in bilingual corpora. |
| Approach: | They develop a method that derives coreference clusters from MT output and evaluates them without requiring annotations in the target language. |
| Outcome: | The proposed model outperforms existing models on three challenging benchmarks. |
Document-Level Planning for Text Simplification (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing work on text simplification is limited to sentence-level inputs . attempts to iteratively apply these approaches fail to preserve discourse structure of document . |
| Approach: | They propose a simplification plan that labels each sentence in the input document while considering both its context and internal structure. |
| Outcome: | The proposed model outperforms baselines on two simplification benchmarks and when used to guide document-level simplification models. |
Efficient Hybrid Generation Framework for Aspect-Based Sentiment Analysis (2023.eacl-main)
Copied to clipboard
| Challenge: | Aspect-based sentiment analysis (ABSA) has attracted broad commercial attention due to its commercial value. |
| Approach: | They propose a framework that generates location and semantic information in parallel and a global hybrid loss function in combination with bipartite matching to achieve end-to-end model training. |
| Outcome: | The proposed framework outperforms state-of-the-art methods in almost all cases and outperfies existing methods in terms of inference efficiency. |
What’s New? Summarizing Contributions in Scientific Literature (2023.eacl-main)
Copied to clipboard
| Challenge: | a growing number of academic articles are shared daily, making it difficult to keep up with the latest findings. |
| Approach: | They propose a task of disentangled paper summarization which generates separate summaries for papers and contexts to make it easier to identify key findings shared in articles. |
| Outcome: | The proposed task is more useful than traditional scientific paper summarization. |
Find Parent then Label Children: A Two-stage Taxonomy Completion Method with Pre-trained Language Model (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing taxonomies focus on adding concepts to the leaf nodes of the existing tree, which does not fully utilize the taxonomy’s knowledge and is unable to update the original taxomy structure. |
| Approach: | They propose a two-stage method called ATTEMPT for taxonomy completion that inserts new concepts into the correct position by finding a parent node and labeling child nodes. |
| Outcome: | The proposed method performs best on taxonomy completion and extension tasks, surpassing existing methods. |
Meta Self-Refinement for Robust Learning with Weak Supervision (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent methods leverage self-training to build noise-resistant models . however, the teacher trained under weak supervision may have fitted a substantial amount of noise and therefore produce incorrect pseudo-labels. |
| Approach: | They propose a framework that encourages teacher to refine its pseudo-labels to effectively combat label noise from weak supervision. |
| Outcome: | The proposed framework outperforms state-of-the-art methods by 11.4% in accuracy and 9.26% in F1 score on eight NLP benchmarks. |
Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translation (2023.eacl-main)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is becoming more accurate, but hallucinations are extremely pathological . previous work focused on artificial settings where the problem is amplified, disregarding some common types of hallucines . |
| Approach: | They propose a method for alleviating hallucinations at test time that significantly reduces the hallucinic rate. |
| Outcome: | The proposed method significantly reduces the hallucinatory rate in a natural setting. |
Investigating UD Treebanks via Dataset Difficulty Measures (2023.eacl-main)
Copied to clipboard
| Challenge: | Treebanks annotated with Universal Dependencies (UD) are currently available for over 100 languages and are only partially reflected in parser evaluations via accuracy metrics like LAS. |
| Approach: | They propose to use dataset cartography, V-information, and minimum description length to analyze UD treebanks using three accuracy-free methods to provide insights about them. |
| Outcome: | The proposed methods provide insights about UD treebanks that would remain undetected if only LAS was considered. |
On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex (2023.eacl-main)
Copied to clipboard
Terry Yue Zhuo, Zhuang Li, Yujin Huang, Fatemeh Shiri, Weiqing Wang, Gholamreza Haffari, Yuan-Fang Li
| Challenge: | Existing techniques for parsing natural-language utterances are vulnerable to adversarial attacks, requiring large amounts of labelled data and expensive human annotation. |
| Approach: | They propose to enhance the adversarial robustness of a prompt-based semantic parser based on a language model trained on code by constructing a set of demonstration examples. |
| Outcome: | The proposed method can be enhanced without significant amounts of labelled data or expensive human annotations on in-domain semantic parsing data. |
Leveraging Task Dependency and Contrastive Learning for Case Outcome Classification on European Court of Human Rights Cases (2023.eacl-main)
Copied to clipboard
| Challenge: | a new method for case outcome classification is being developed for the European Court of Human Rights. |
| Approach: | They propose to use case facts descriptions to classify whether a court finds a violation of conventions. |
| Outcome: | The proposed model improves on single-task and joint models without contrastive loss. |
Semi-supervised Relation Extraction via Data Augmentation and Consistency-training (2023.eacl-main)
Copied to clipboard
| Challenge: | Obtaining high-quality human labelled data is an expensive and noisy process. |
| Approach: | They propose to leverage unlabelled data to improve the sample efficiency of the models. |
| Outcome: | The proposed methods can be used to extract the Cause-Effect relation between a given head entity and tail entity based on context in the input sentence. |
Event Temporal Relation Extraction with Bayesian Translational Model (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods to extract temporal relations between events lack a principled method to incorporate external knowledge. |
| Approach: | They propose a Bayesian-based method that models the temporal relation representations as latent variables and infers their values via Bayessian inference and translational functions. |
| Outcome: | The proposed method outperforms existing methods for event temporal relation extraction on three widely used datasets. |
Persona Expansion with Commonsense Knowledge for Diverse and Consistent Response Generation (2023.eacl-main)
Copied to clipboard
Donghyun Kim, Youbin Ahn, Wongyu Kim, Chanhee Lee, Kyungchan Lee, Kyong-Ho Lee, Jeonguk Kim, Donghoon Shin, Yeonsoo Lee
| Challenge: | Existing researches have focused on generating diverse and consistent responses based on personal traits. |
| Approach: | They propose a consistent persona expansion framework that improves not only the diversity but also the consistency of persona-based responses. |
| Outcome: | The proposed framework improves not only the diversity but also the consistency of persona-based responses on the Persona-Chat dataset. |
UnifEE: Unified Evidence Extraction for Fact Verification (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing models extract evidence in both sentences and table cells from Wikipedia dumps, ignoring potential connections between them. |
| Approach: | They propose a model which uses a mixed evidence graph to extract the evidence in both formats without manually designed conversion rules. |
| Outcome: | The proposed model outperforms existing models and improves the verification step. |
MiniALBERT: Model Distillation via Parameter-Efficient Recursive Transformers (2023.eacl-main)
Copied to clipboard
| Challenge: | Pre-trained Language Models (LMs) are an integral part of natural language processing but their usability is constrained by computational and time complexity and their increasing size. |
| Approach: | They propose a technique for converting knowledge of fully parameterised LMs into a compact recursive student. |
| Outcome: | The proposed models match the performance of bloated models with negligible performance losses. |
Multilingual Normalization of Temporal Expressions with Masked Language Models (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for normalizing temporal expressions are rule-based, which severely limits the applicability in multilingual settings. |
| Approach: | They propose a neural method for normalizing temporal expressions based on masked language modeling and a slot-based prediction scheme for context-independent representations. |
| Outcome: | The proposed method outperforms existing rule-based methods in many languages and in particular, for low-resource languages with performance improvements of up to 33 F1 on average compared to the state of the art. |
K-hop neighbourhood regularization for few-shot learning on graphs: A case study of text classification (2023.eacl-main)
Copied to clipboard
| Challenge: | We show that few-sample word-document graphs can be used for improved learning in low-resource settings. |
| Approach: | They propose a method to utilize word-document graph properties for improved learning in low-resource settings by using a regularizer for heterogeneous graphs. |
| Outcome: | The proposed method outperforms a baseline TextGCN with 17% accuracy over eight languages while performing on par with the state-of-the-art models. |
What Clued the AI Doctor In? On the Influence of Data Source and Quality for Transformer-Based Medical Self-Disclosure Detection (2023.eacl-main)
Copied to clipboard
| Challenge: | Recognizing medical self-disclosure is important in many healthcare contexts, but it has been under-explored by the NLP community. |
| Approach: | They analyze a social media-based task to expand existing medical self-disclosure corpus and compare Transformer-based models to determine their merits. |
| Outcome: | The proposed dataset outperforms the state-of-the-art dataset for this task by 16.73%. |
Improving Visual-Semantic Embedding with Adaptive Pooling and Optimization Objective (2023.eacl-main)
Copied to clipboard
Zijian Zhang, Chang Shu, Ya Xiao, Yuan Shen, Di Zhu, Youxin Chen, Jing Xiao, Jey Han Lau, Qian Zhang, Zheng Lu
| Challenge: | Recent VSE models combine simple pooling methods with hard triplet loss to improve performance. |
| Approach: | They propose an adaptive pooling strategy that allows the model to learn how to aggregate features through a combination of simple pooling methods. |
| Outcome: | The proposed strategy outperforms current state-of-the-art systems on image-to-text and text-toimage retrieval. |
Policy-based Reinforcement Learning for Generalisation in Interactive Text-based Environments (2023.eacl-main)
Copied to clipboard
| Challenge: | Text-based environments allow RL agents to learn to converse and perform interactive tasks through natural language. |
| Approach: | They propose to switch from a value-based update method to a policy-based one within text-based environments and evaluate it on Coin Collector and Question Answering with interactive text (QAit). |
| Outcome: | The proposed policy-based agent is more generalised than value-based methods in two text-based environments designed to test zero-shot performance. |
Logic Against Bias: Textual Entailment Mitigates Stereotypical Sentence Reasoning (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent studies show that textual entailment learning reduces social biases in pretrained sentence encoders. |
| Approach: | They compare pretrained sentence encoders with textual entailment models that learn language logic for downstream language understanding tasks. |
| Outcome: | The proposed models outperform models with lower bias without debiasing processes on stereotype, profession, and emotion bias tests. |
Entity Tracking via Effective Use of Multi-Task Learning Model and Mention-guided Decoding (2023.eacl-main)
Copied to clipboard
| Challenge: | State-of-the-art entity tracking approaches either design complicated model architectures or rely on task-specific pre-training to achieve good results. |
| Approach: | They propose a multi-task learning-enabled entity tracking approach that utilizes knowledge gained from general domain tasks to improve entity tracking. |
| Outcome: | The proposed approach achieves state-of-the-art on two popular entity tracking datasets, even though it does not require any task-specific architecture design or pre-training. |
Conversational Tree Search: A New Hybrid Dialog Task (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing conversational interfaces are limited to FAQs and dialogs, allowing users to search for specific questions. |
| Approach: | They propose a task that bridges the gap between FAQ-style information retrieval and task-oriented dialog. |
| Outcome: | The proposed task bridges the gap between FAQ-style information retrieval and task-oriented dialog. |
A Human Subject Study of Named Entity Recognition in Conversational Music Recommendation Queries (2023.eacl-main)
Copied to clipboard
| Challenge: | NER is a complex task that requires a high degree of precision and a higher level of recall. |
| Approach: | They evaluated the human NER linguistic behaviour on a noisy corpus of conversational music recommendation queries with many irregular and novel named entities. |
| Outcome: | The results show that human NER was hard to perform under a strict evaluation schema and that the model had higher recall because of entity exposure. |
Entity Disambiguation with Entity Definitions (2023.eacl-main)
Copied to clipboard
| Challenge: | Entity Disambiguation (ED) is a crucial problem in Natural Language Processing (NLP). |
| Approach: | They propose to use Wikipedia titles as the textual representation of each candidate to improve the generalization capability over unseen patterns. |
| Outcome: | The proposed model improves on 2 out of 6 benchmarks and is generalized over unseen patterns. |
Exploring Paracrawl for Document-level Neural Machine Translation (2023.eacl-main)
Copied to clipboard
| Challenge: | Document-level neural machine translation (NMT) has outperformed sentence-level NMT on a number of datasets. |
| Approach: | They use Paracrawl to extract parallel paragraphs from Paracral webpages . they also use the extracted parallel paragraph as parallel documents for training . |
| Outcome: | The proposed model outperforms sentence-level NMT on a number of datasets. |
Poor Man’s Quality Estimation: Predicting Reference-Based MT Metrics Without the Reference (2023.eacl-main)
Copied to clipboard
Vilém Zouhar, Shehzaad Dhuliawala, Wangchunshu Zhou, Nico Daheim, Tom Kocmi, Yuchen Eleanor Jiang, Mrinmaya Sachan
| Challenge: | State-of-the-art machine translation quality estimation systems have been achieving remarkable correlations with human judgements yet they require human annotations, which are expensive and computationally heavy. |
| Approach: | They propose a problem where one predicts automated metric scores without the reference. |
| Outcome: | The proposed model can estimate automated metrics at the sentence-level without the reference. |
Integrating Translation Memories into Non-Autoregressive Machine Translation (2023.eacl-main)
Copied to clipboard
| Challenge: | Non-autoregressive machine translation (NAT) has made great progress, but most studies focus on standard translation tasks. |
| Approach: | They propose to train an edit-based NAT model with a Translation Memory (TM) they propose to modify the data presentation and introduce an extra deletion operation to reduce decoding load. |
| Outcome: | The proposed model performs on par with an autoregressive approach while reducing the decoding load. |
Shorten the Long Tail for Rare Entity and Event Extraction (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing approaches for the long-tailed learning problem seek to manipulate the training data by re-balancing, augmentation or introducing extra prior knowledge. |
| Approach: | They propose to transform infrequent candidate mention representation with the average mention representation in the training dataset to handle the generalization challenge. |
| Outcome: | The proposed framework can generalize to rare or unseen expressions of entities or events, especially for rare types without sufficient training examples. |
Do Deep Neural Networks Capture Compositionality in Arithmetic Reasoning? (2023.eacl-main)
Copied to clipboard
Keito Kudo, Yoichi Aoki, Tatsuki Kuribayashi, Ana Brassard, Masashi Yoshikawa, Keisuke Sakaguchi, Kentaro Inui
| Challenge: | Using a pre-trained dataset, we examine how well recent neural models capture compositionality in symbolic reasoning tasks. |
| Approach: | They propose a skill tree on compositionality that defines hierarchical levels of complexity along with three compositionality dimensions: systematicity, productivity, and substitutivity. |
| Outcome: | The proposed model struggled most with systematicity, performing poorly even with relatively simple compositions. |
BLM-AgrF: A New French Benchmark to Investigate Generalization of Agreement in Neural Networks (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing benchmarks for deep learning are based on massive amounts of data, which are effective in hiding some of the shallowness of the learned models. |
| Approach: | They propose to use a French dataset to learn the underlying rules of subject-verb agreement in sentences, inspired by visual IQ tests known as Raven’s Progressive Matrices. |
| Outcome: | The proposed method is based on Raven’s Progressive Matrices, a visual IQ test, and a dataset built using the BLM framework. |
Robustification of Multilingual Language Models to Real-world Noise in Crosslingual Zero-shot Settings with Robust Contrastive Pretraining (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing studies on robustness of pretrained multilingual models are limited to the English language. |
| Approach: | They propose to use data augmentation and contrastive loss term to boost robustness of multilingual models in cross-lingual settings. |
| Outcome: | The proposed model outperforms existing models on clean and noisy data in the cross-lingual setting. |
Unsupervised Anomaly Detection in Multi-Topic Short-Text Corpora (2023.eacl-main)
Copied to clipboard
| Challenge: | Unsupervised anomaly detection is a challenging task when the majority class is heterogeneous. |
| Approach: | They propose to use word embeddings to represent each sample by a dense vector and use a Mixture Model approach to detect which samples deviate the most from the underlying distributions of the corpus. |
| Outcome: | The proposed method is more efficient than state-of-the-art methods on real datasets. |
Metaphor Detection with Effective Context Denoising (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing models focus on semantically relevant information and provide a target-oriented parse tree structure for metaphor detection. |
| Approach: | They propose a new model which introduces a target-oriented parse tree structure for metaphor detection. |
| Outcome: | The proposed model achieves state-of-the-art on several main metaphor datasets and compares with other methods. |
Low-Resource Compositional Semantic Parsing with Concept Pretraining (2023.eacl-main)
Copied to clipboard
| Challenge: | Semantic parsing is a key role in voice assistants by mapping natural language to structured meaning representations. |
| Approach: | They propose an architecture to perform domain adaptation automatically with only a small amount of metadata about the new domain and without any new training data. |
| Outcome: | The proposed architecture outperforms existing models in low-resource settings. |
Made of Steel? Learning Plausible Materials for Components in the Vehicle Repair Domain (2023.eacl-main)
Copied to clipboard
| Challenge: | a novel approach to learn domain-specific plausible materials for components in the vehicle repair domain is proposed . connecting a symptom to an underlying cause is a crucial building block for natural language understanding across domains. |
| Approach: | They propose a method to aggregate salient predictions from a set of cloze task style templates and use a Wikipedia corpus to augment the model. |
| Outcome: | The proposed approach outperforms a traditional pattern-based approach by exploiting the compositionality assumption in a cloze task style setting. |
Self-Adapted Utterance Selection for Suicidal Ideation Detection in Lifeline Conversations (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for identifying suicidal ideation in phone conversations are difficult to use because of their long duration and noisy nature. |
| Approach: | They propose a self-adaptive approach that identifies the most critical utterances that the NLP model can more easily distinguish. |
| Outcome: | The proposed approach outperforms the baseline models in overall performance with an F score of 66.01% and significantly higher F-score in detecting the most dangerous cases. |
Can Pretrained Language Models (Yet) Reason Deductively? (2023.eacl-main)
Copied to clipboard
| Challenge: | Acquiring factual knowledge with Pretrained Language Models (PLMs) has attracted increasing attention, showing promising performance in many knowledge-intensive tasks. |
| Approach: | They conduct a comprehensive evaluation of the learnable deductive reasoning capability of pretrained language models and compare their performance against simple adversarial surface form edits. |
| Outcome: | The models are able to generalise learned logic rules and perform inconsistently against simple adversarial surface form edits, but catastrophically forget the previously learnt knowledge. |
Selective In-Context Data Augmentation for Intent Detection using Pointwise V-Information (2023.eacl-main)
Copied to clipboard
Yen-Ting Lin, Alexandros Papangelis, Seokhwan Kim, Sungjin Lee, Devamanyu Hazarika, Mahdi Namazifar, Di Jin, Yang Liu, Dilek Hakkani-Tur
| Challenge: | Intent detection is a fundamental element in task-oriented dialogue systems, usually occurring within the Natural Language Understanding component. |
| Approach: | They propose an in-context data augmentation approach that fine-tunes a pre-trained language model and synthesizes new datapoints that correspond to given intents. |
| Outcome: | The proposed method produces training data that achieves state-of-the-art on three challenging intent detection datasets and performs on par with the state- of-the art in full-shot settings. |
Multilingual Representation Distillation with Contrastive Learning (2023.eacl-main)
Copied to clipboard
| Challenge: | Contextual representations from large pretrained language models encode semantic information from two or more languages. |
| Approach: | They integrate contrastive learning into multilingual representation distillation and use it for quality estimation of parallel sentences. |
| Outcome: | The proposed model outperforms existing models with similarity searches and filtering tasks across low-resource languages. |
On the inconsistency of separable losses for structured prediction (2023.eacl-main)
Copied to clipboard
| Challenge: | Modern natural language processing (NLP) heavily relies on machine learning, where prediction models are learned by minimizing a loss function over the training data. |
| Approach: | They prove that separable negative log-likelihood losses for structured prediction are not Bayes consistent. |
| Outcome: | The proposed model does not predict the most probable structure in the data distribution for a given input. |
A Systematic Search for Compound Semantics in Pretrained BERT Architectures (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing models for noun compounds have been less successful in predicting compositionality than transformers . authors: suboptimal use of encoded information may be a contributing factor . performance of transformer-based models is poor, authors say . |
| Approach: | They propose to use semantic knowledge derived from pretrained BERT to predict compositionality . they find distinct linguistic roles of heads and modifiers are reflected by differences in BERT representations . |
| Outcome: | The proposed model improves on unsupervised implementations of pretrained BERT . empirical properties such as frequency, productivity, and ambiguity affect performance . |
Efficiently Upgrading Multilingual Machine Translation Models to Support More Languages (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing multilingual machine translation models need to be upgraded as data becomes available in more languages. |
| Approach: | They propose three techniques that speed up the effective learning of new languages and alleviate catastrophic forgetting . |
| Outcome: | The proposed techniques exceed the performance of a same-sized baseline model with 30% computation and recover the performance a larger model trained from scratch with over 50% reduction in computation. |
Summarize and Generate to Back-translate: Unsupervised Translation of Programming Languages (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent developments of multilingual pre-trained sequence-to-sequence models for programming languages have been effective for a broad spectrum of downstream software engineering tasks. |
| Approach: | They propose to combine a source-to-target model with a target-tosource model trained in parallel. |
| Outcome: | The proposed approach performs competitively with state-of-the-art methods. |
The Impacts of Unanswerable Questions on the Robustness of Machine Reading Comprehension Models (2023.eacl-main)
Copied to clipboard
| Challenge: | Pretrained language models have achieved super-human performances on many Machine Reading Comprehension (MRC) benchmarks. |
| Approach: | They propose to fine-tune three state-of-the-art language models on SQuAD 1.1 or SQu AD 2.0 and then evaluate their robustness under adversarial attacks. |
| Outcome: | The proposed model is able to perform better under adversarial attacks than model fine-tuned on SQuAD 1.1 or 2.0. |
FrameBERT: Conceptual Metaphor Detection with Frame Embedding Learning (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing models for concept-level metaphor detection lack explicit knowledge of FrameNet . Metaphor detection is a pervasive linguistic device that is used in cognitive and communicative functions of language. |
| Approach: | They propose a BERT-based model that explicitly learns FrameNet Embeddings for metaphor detection. |
| Outcome: | The proposed model is more explainable and interpretable than existing models. |
Towards More Efficient Insertion Transformer with Fractional Positional Encoding (2023.eacl-main)
Copied to clipboard
| Challenge: | Empirical studies on text generation tasks demonstrate the effectiveness of insertion-based models. |
| Approach: | They propose a reusable positional encoding scheme for insertion transformers that allows reusing representations calculated in previous steps. |
| Outcome: | Empirical studies show that the proposed model reduces the time required to generate a token and improves decoding efficiency. |
SODAPOP: Open-Ended Discovery of Social Biases in Social Commonsense Reasoning Models (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing diagnostic tests for detecting social biases in NLP models only detect stereotypic associations pre-specified by the designer. |
| Approach: | They propose an approach for automatic social bias discovery in social commonsense question-answering by substituting names associated with different demographic groups and generating many distractor answers from a masked language model. |
| Outcome: | The proposed approach uncovers model’s stereotypic associations between demographic groups and an open set of words. |
Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for open-domain question-answering use an open book approach . a recent alternative is to retrieve from a collection of previously-generated question-annwer pairs . |
| Approach: | They propose a new QA system that augments a text-to-text model with a large memory of question-answer pairs and a task for the latent step of question retrieval. |
| Outcome: | The proposed system outperforms closed-book QA and can answer multi-hop questions. |
Gold Doesn’t Always Glitter: Spectral Removal of Linear and Nonlinear Guarded Attribute Information (2023.eacl-main)
Copied to clipboard
| Challenge: | Spectral Attribute removaL is a method to remove private or guarded information from neural representations. |
| Approach: | They propose a method to remove guarded or private information from neural representations by matrix decomposition. |
| Outcome: | The proposed method retains better main task performance after removing guarded information compared to previous work. |
CTC Alignments Improve Autoregressive Translation (2023.eacl-main)
Copied to clipboard
Brian Yan, Siddharth Dalmia, Yosuke Higuchi, Graham Neubig, Florian Metze, Alan W Black, Shinji Watanabe
| Challenge: | Connectionist Temporal Classification (CTC) is widely used for automatic speech recognition (ASR) but lags behind attentional decoder approaches in terms of translation quality. |
| Approach: | They propose to use a CTC/attention framework to validate this hypothesis by modifying the Hybrid CTC-Attention model proposed for automatic speech recognition to support text-to-text translation (MT) and speech-totext translation. |
| Outcome: | The proposed model outperforms pure-attention baselines across six translation tasks. |
Modelling Temporal Document Sequences for Clinical ICD Coding (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing studies on the ICD coding task focus on extracting codes from the discharge summary, but there is potential to automate the task by identifying relevant information from clinical notes. |
| Approach: | They propose a hierarchical transformer architecture that uses text across the entire sequence of clinical notes in each hospital stay for ICD coding. |
| Outcome: | The proposed model exceeds the state-of-the-art when using only discharge summaries as input and achieves performance improvements when all clinical notes are used as input. |
LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization (2023.eacl-main)
Copied to clipboard
| Challenge: | Human evaluation is labor-intensive, expensive to scale, and difficult to design. |
| Approach: | They propose a set of guidelines for human evaluation of faithfulness in long-form summaries that address the following challenges: (1) How can we achieve high inter-annotator agreement on faithfulness scores? (2) How can our annotator minimize workload while maintaining accurate faithfulness? |
| Outcome: | The proposed framework reduces inter-annotator variance in faithfulness scores while minimizing annotator workload while maintaining accuracy. |
Cluster-Guided Label Generation in Extreme Multi-Label Classification (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing classification-based models are poorly per-form for tail labels and ignore semantic relations among labels. |
| Approach: | They propose to guide label generation using label cluster information to hierarchically generate lower-level labels. |
| Outcome: | The proposed model outperforms classification and generation baselines on tail labels and improves in four popular XMC benchmarks. |
Empathy Identification Systems are not Accurately Accounting for Context (2023.eacl-main)
Copied to clipboard
| Challenge: | Empathy is a fundamental phenomenon that allows us to better communicate and relate with others. |
| Approach: | They propose a simple model that checks if an input utterance is similar to a small set of empathetic examples, but does not consider dialogue context. |
| Outcome: | The proposed model outperforms state-of-the-art models on benchmarks and empathetic rationale extraction benchmarks. |
Enhancing Multi-Document Summarization with Cross-Document Graph-based Information Extraction (2023.eacl-main)
Copied to clipboard
| Challenge: | Information extraction (IE) and summarization (summarization) are closely related, but both aims to abstract the most salient information into a generated text summary. |
| Approach: | They propose to use structured IE graphs to enhance the abstractive summarization task by using cross-document IE output to incorporate an alignment loss between IE nodes and their text spans to reduce inconsistencies. |
| Outcome: | The proposed model can generate summaries that are more factual while not losing abstractiveness. |
What happens before and after: Multi-Event Commonsense in Event Coreference Resolution (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing event coreference models cluster event mentions pertaining to the same event, but they fail to leverage commonsense inferences for lexically-divergent mentions. |
| Approach: | They propose a model that extends event mentions with temporal commonsense inferences to generate plausible events that happen before and after the target events. |
| Outcome: | The proposed model generates plausible events that happen before and after the target event, and then after it, such as "he was sentenced". |
Multi-Modal Bias: Introducing a Framework for Stereotypical Bias Assessment beyond Gender and Race in Vision–Language Models (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent advances in self-supervised training have led to a new class of pretrained vision–language models. |
| Approach: | They propose a visual and textual bias benchmark to assess bias in self-supervised multimodal models using 3,800 images and phrases from 14 population subgroups. |
| Outcome: | The proposed model shows that it favors certain groups while maintaining the accuracy of the model. |
CylE: Cylinder Embeddings for Multi-hop Reasoning over Knowledge Graphs (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing geometric-based models cannot handle the logical negation operation . Existing models using cones embeddings are limited to representing queries by two-dimensional shapes . Empirical results show that the performance of multi-hop reasoning task using CylE significantly increases over state-of-the-art geometric- based models for queries without negation. |
| Approach: | They propose a geometric-based model based on three-dimensional shapes with unbounded cylinder embeddings that can handle a complete set of first-order logic operations. |
| Outcome: | Empirical results show that CylE outperforms state-of-the-art models for queries without negation. |
Fiction-Writing Mode: An Effective Control for Human-Machine Collaborative Writing (2023.eacl-main)
Copied to clipboard
| Challenge: | Large-scale pre-trained language models (PLMs) have demonstrated an exceptional aptitude for generating text with an exceptional degree of fluency and structure. |
| Approach: | They propose to integrate writing skills curricula into human-machine collaborative writing scenarios by adding writing modes as a control for text generation models. |
| Outcome: | The proposed model can be used to generate narrative fiction with a high level of accuracy and similarity with the professionally written target story. |
Robustness Challenges in Model Distillation and Pruning for Natural Language Understanding (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent studies have focused on compressing pre-trained language models (PLMs) however, few studies have examined the impact of compression on generalizability and robustness of compressed models for out-of-distribution data. |
| Approach: | They propose to use knowledge distillation and pruning to reduce model generalization and generalization on out-of-distribution data. |
| Outcome: | The proposed compression techniques overfit on shortcut samples and generalize poorly on hard ones. |
Don’t Blame the Annotator: Bias Already Starts in the Annotation Instructions (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent studies have shown that data collected through crowdsourcing often exhibit various biases that lead to overestimation of model performance. |
| Approach: | They propose to model instruction bias in 14 recent NLU benchmarks by analyzing crowdsourcing instructions and analyzing their results. |
| Outcome: | The proposed model can be over-represented in datasets with a large number of examples, and the results are consistent with previous studies. |
Performance Prediction via Bayesian Matrix Factorisation for Multilingual Natural Language Processing Tasks (2023.eacl-main)
Copied to clipboard
| Challenge: | Performance prediction for natural language processing (NLP) is based on a framework of Bayesian matrix factorisation . it avoids hyperparameter tuning and provides uncertainty estimates over predictions. |
| Approach: | They propose to use Bayesian matrix factorisation to predict the performance of language pairs depicted by grey cells. |
| Outcome: | The proposed framework outperforms the state-of-the-art in several NLP benchmarks, including machine translation and cross-lingual entity linking. |
Unified Neural Topic Model via Contrastive Learning and Term Weighting (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent techniques employ pretrained language models to improve topic quality. |
| Approach: | They propose a topic-based model that uses contrastive learning and term weighting to learn from a pretrained language model and discover influential terms from semantically coherent clusters. |
| Outcome: | The proposed model outperforms baselines across multiple topic coherence measures and can be used as an add-on to existing topic models and improves their performance. |
Don’t Mess with Mister-in-Between: Improved Negative Search for Knowledge Graph Completion (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for knowledge graph completion use a dual-encoding framework with a bottleneck that allows for fast approximate search over a vast collection of candidates. |
| Approach: | They propose to use a dual-encoder framework to find more informative negatives by searching for candidates with high lexical overlaps. |
| Outcome: | The proposed methods improve on the large-scale Wikidata5M dataset and combine different kinds of strategies to achieve state-of-the-art performance. |
Semantic Frame Induction with Deep Metric Learning (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent studies have shown the usefulness of contextualized word embeddings in semantic frame induction, but they are not always consistent with human intuitions about semantic frames. |
| Approach: | They propose a model that fine-tunes contextualized embeddings to perform semantic frame induction. |
| Outcome: | The proposed model improves clustering evaluation scores on FrameNet by 8 points or more. |
The Devil is in the Details: On Models and Training Regimes for Few-Shot Intent Classification (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent methods for task-oriented dialog (ToD) intent classification use pretrained language models . but lack of informative ablations prevents identification of factors that drive performance . |
| Approach: | They propose a framework to evaluate components of Few-Shot Intent Classification . they propose to combine cross-encoder architecture and episodic meta-learning . |
| Outcome: | The proposed framework evaluates cross-encoder architecture and episodic meta-learning . it also shows that splitting episodes into support and query sets outperforms non-episodic counterparts. |
Iterative Document-level Information Extraction via Imitation Learning (2023.eacl-main)
Copied to clipboard
| Challenge: | Documents may feature zero or more instances of a template of any given type, and the task of template extraction entails identifying the templates in a document and extracting each template’s slot values. |
| Approach: | They propose to use iterative extraction to extract complex relations, i.e., N-tuples representing a mapping from named slots to spans of text within a document. |
| Outcome: | The proposed model leads to state-of-the-art results on two established benchmarks and a strong baseline on the new BETTER Granular task. |
CLICK: Contrastive Learning for Injecting Contextual Knowledge to Conversational Recommender System (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing CRSs lack capturing comprehensive user preferences . existing systems lack contextual knowledge to capture user preferences from a dialogue context . |
| Approach: | They propose a Contrastive Learning approach for Injecting Contextual Knowledge from Reddit data to a CRS task. |
| Outcome: | The proposed approach captures a user preference from a dialogue context without items . it improves on the existing methods, and the results are published in the journal of cognitive science. |
LEALLA: Learning Lightweight Language-agnostic Sentence Embeddings with Knowledge Distillation (2023.eacl-main)
Copied to clipboard
| Challenge: | Large-scale language-agnostic sentence embedding models suffer from inference speed and computation overhead. |
| Approach: | They propose to train a lightweight sentence embedding model to achieve this by incorporating knowledge from a teacher model. |
| Outcome: | The proposed model can build low-dimensional sentences for 109 languages with a thin-deep encoder. |
Synthesizing Human Gaze Feedback for Improved NLP Performance (2023.eacl-main)
Copied to clipboard
| Challenge: | Prior work on eye tracking and NLP reveals that human scanpaths can aid in understanding and performance of NLP models. |
| Approach: | They propose a model for generating human scanpaths over text that approximates meaningful cognitive signals in human gaze patterns. |
| Outcome: | The proposed model can approximate meaningful cognitive signals in human gaze patterns. |
Memory-efficient Temporal Moment Localization in Long Videos (2023.eacl-main)
Copied to clipboard
| Challenge: | Temporal Moment Localization is a multi-modal task that requires understanding the temporal relationships in the entire input video. |
| Approach: | They propose a stochastic sampling module that can process long videos at a constant memory footprint. |
| Outcome: | The proposed model can process videos as long as 18 minutes at a constant memory footprint and achieves faster and faster results than competing models. |
Extracting Victim Counts from Text (2023.eacl-main)
Copied to clipboard
| Challenge: | Using tagging and regex methods, data on injured, displaced, or abused victims is difficult . data on earthquake injuries and deaths is scarce, subjective, or biased . |
| Approach: | They compare tagging approaches to extract injured, displaced, or abused victims . they discuss calibration and investigate out-of-distribution and few-shot performance . |
| Outcome: | The proposed model is among the first to apply numeracy-focused large language models in a real-world use case with a positive impact. |
ConEntail: An Entailment-based Framework for Universal Zero and Few Shot Classification with Supervised Contrastive Pretraining (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing models for text classification are not universally applicable and lack annotated data. |
| Approach: | They propose a framework for universal zero and few shot classification with supervised contrastive pretraining that can generalize to diverse classification tasks in both zero and many shot settings. |
| Outcome: | The proposed framework outperforms baseline models in zero and few shot settings. |
Guide the Learner: Controlling Product of Experts Debiasing Method Based on Token Attribution Similarities (2023.eacl-main)
Copied to clipboard
| Challenge: | Several proposals have been put forward for improving out-of-distribution performance by mitigating dataset biases. |
| Approach: | They propose a fine-tuning strategy that incorporates the similarity between the main and biased model attribution scores in a Product of Experts (PoE) loss function to further improve OOD performance. |
| Outcome: | The proposed method improves OOD performance while maintaining in-distribution performance. |
Task and Sentiment Adaptation for Appraisal Tagging (2023.eacl-main)
Copied to clipboard
| Challenge: | Appraisal framework in linguistics defines the framework for fine-grained evaluations and opinions. |
| Approach: | They propose to use language models to automatically identify and annotate text segments for appraisal. |
| Outcome: | The proposed model achieves superior performance than baseline adapter-based models and other neural classification models for cross-domain and cross-language settings. |
DREEAM: Guiding Attention with Evidence for Improving Document-Level Relation Extraction (2023.eacl-main)
Copied to clipboard
| Challenge: | Document-level relation extraction (DocRE) is a task of identifying relations between entities in a document. evidence retrieval (ER) in DocRE faces two major issues: high memory consumption and limited availability of annotations. |
| Approach: | They propose a memory-efficient approach that uses evidence as the supervisory signal . they propose er self-training to learn ER from automatically-generated evidence . |
| Outcome: | The proposed method exhibits state-of-the-art performance on the DocRED benchmark . it uses evidence as the supervisory signal and self-trains on massive data without annotations . |
Span-based Named Entity Recognition by Generating and Compressing Information (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing work on Named Entity Recognition (NER) only used generative or information compression models to improve performance. |
| Approach: | They propose to combine two types of IB models into one system to enhance Named Entity Recognition (NER) they incorporate unsupervised generative components span reconstruction and synonym generation into a span-based NER system. |
| Outcome: | The proposed model focuses on learning span representation, which is applicable not only to span-based NER but also to other span-related tasks such as event coreference resolution and question answering. |
An In-depth Analysis of Implicit and Subtle Hate Speech Messages (2023.eacl-main)
Copied to clipboard
| Challenge: | Explicit hate speech is more easily identifiable by recognizing hateful words, but subtle messages are harmful . subtle messages contain linguistically subtle and implicit forms of HS, such as circumlocution, metaphors and sarcasm . social media have faced pressure from civil rights groups demanding to monitor and limit online hate speech . |
| Approach: | They propose to use a fine-grained definition of implicit and subtle messages to detect HS . they then experiment with neural network architectures to detect subtle content . |
| Outcome: | The proposed models perform satisfactory on explicit messages, but fail to detect subtle content. |
MTEB: Massive Text Embedding Benchmark (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing text embeddings are evaluated on a small set of datasets, not covering their possible applications to other tasks. |
| Approach: | They propose a benchmarking framework that evaluates 8 embedding tasks covering 58 datasets and 112 languages. |
| Outcome: | The proposed model is the most comprehensive benchmark of text embeddings to date. |
Step by Step Loss Goes Very Far: Multi-Step Quantization for Adversarial Text Attacks (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing gradient-based attacks quantize all tokens in a text at once, which creates a significant gap between adversarial loss for continuous and discrete text representations. |
| Approach: | They propose a gradient-based attack that quantizes tokens one by one and reoptimizes adversarial example after each quantization. |
| Outcome: | The proposed method outperforms other approaches on various natural language processing tasks. |
TwiRGCN: Temporally Weighted Graph Convolution for Question Answering over Temporal Knowledge Graphs (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent years have witnessed interest in Temporal Question Answering over Knowledge Graphs (TKGQA) but these methods are highly engineered and do not automatically discover relevant parts of the KG during multi-hop reasoning. |
| Approach: | They propose a scheme to modulate the messages passed through a KG edge during convolution based on the relevance of its associated period to the question. |
| Outcome: | The proposed system outperforms state-of-the-art models on a recent challenging dataset for multi-hop complex temporal QA called TimeQuestions. |
ZELDA: A Comprehensive Benchmark for Supervised Entity Disambiguation (2023.eacl-main)
Copied to clipboard
| Challenge: | Entity disambiguation (ED) is the task of disambiguating named entity mentions in text to unique entries in a knowledge base. |
| Approach: | They propose a benchmark for entity disambiguation that includes a unified training data set, entity vocabulary, candidate lists and challenging evaluation splits covering 8 different domains. |
| Outcome: | The proposed benchmark is based on a unified training data set, entity vocabulary, candidate lists and evaluation splits covering 8 different domains. |
GLADIS: A General and Large Acronym Disambiguation Benchmark (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing acronym disambiguation benchmarks are limited to specific domains . a study on a Microsoft question answering forum found that only 7% of acronyms co-occur with their corresponding long forms, which confuses the readers about the meaning of a text. |
| Approach: | They propose a new acronym disambiguation benchmark with a dictionary and a pre-training corpus . they then pre-train a language model on the constructed corpus and show the challenges . |
| Outcome: | The proposed benchmarks pre-train a language model on the constructed corpus for general acronym disambiguation. |
Probing Cross-Lingual Lexical Knowledge from Multilingual Sentence Encoders (2023.eacl-main)
Copied to clipboard
| Challenge: | Pretrained multilingual language models (LMs) can be 'rewired' into effective multilingual sentence encoders (SEs) however, it remains unclear how to best leverage them to represent sub-sentence lexical items in cross-lingual lexicals. |
| Approach: | They propose a method for exposing cross-lingual lexical knowledge by additional fine-tuning through inexpensive contrastive learning that requires only a small amount of word translation pairs. |
| Outcome: | The proposed method exposes cross-lingual lexical knowledge by additional fine-tuning through inexpensive contrastive learning that requires only a small amount of word translation pairs. |
Pento-DIARef: A Diagnostic Dataset for Learning the Incremental Algorithm for Referring Expression Generation from Examples (2023.eacl-main)
Copied to clipboard
| Challenge: | Using an extensional description of a visual input, we show that a model can produce referring expressions from visual inputs, whereas simpler baselines do not. |
| Approach: | They propose to use a visual dataset to generate referring expressions from visual inputs. |
| Outcome: | The proposed model achieves BLEU@1 score and sentence accuracy, whereas baselines do not. |
Mitigating Exposure Bias in Grammatical Error Correction with Data Augmentation and Reweighting (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing approaches to grammatical error correction (GEC) use sequence-to-sequence models, but there is an exposure bias problem. |
| Approach: | They propose a data manipulation approach to overcome the exposure bias problem in seq2seq GEC . they propose augmentation methods to mimic decoder input and reweighting methods to automatically balance the importance of each kind of augmented samples. |
| Outcome: | The proposed method improves on benchmark GEC datasets. |
Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training (2023.eacl-main)
Copied to clipboard
| Challenge: | Large-scale vision-language pre-trained (VLP) models generate unfaithful or nonsensical texts given the source input, which is called hallucination. |
| Approach: | They propose a VLP loss-based model to mitigate object hallucination by decoupling VLP objectives and a token-level image-text alignment. |
| Outcome: | The proposed model reduces object hallucination by 17.4% on two benchmarks. |
Characterizing the Entities in Harmful Memes: Who is the Hero, the Villain, the Victim? (2023.eacl-main)
Copied to clipboard
Shivam Sharma, Atharva Kulkarni, Tharun Suresh, Himanshi Mathur, Preslav Nakov, Md. Shad Akhtar, Tanmoy Chakraborty
| Challenge: | A common problem associated with meme comprehension lies in detecting the entities referenced and characterizing the role of each of these entities. |
| Approach: | They propose to use a memes dataset on US Politics and Covid-19 memes to characterize the role of harmful entities in memes. |
| Outcome: | The proposed model improves 4% over baseline and 1% over competing models. |
Systematic Investigation of Strategies Tailored for Low-Resource Settings for Low-Resource Dependency Parsing (2023.eacl-main)
Copied to clipboard
| Challenge: | Several strategies have been proposed to enhance performance in low-resource scenarios. |
| Approach: | They propose to use 5 low-resource strategies for dependency parsing for multiple languages . they use ensembled approach on 7 UD low-rsource languages based on their results . |
| Outcome: | The proposed approach improves on a low-resource language Sanskrit. |
Compositional Generalisation with Structured Reordering and Fertility Layers (2023.eacl-main)
Copied to clipboard
| Challenge: | Seq2seq models struggle with compositional generalisation, i.e. generalising to new and potentially more complex structures than seen during training. |
| Approach: | They propose a flexible end-to-end differentiable neural model that composes two structural operations: a fertility step and a reordering step. |
| Outcome: | The proposed model outperforms seq2seq models on compositional splits of realistic semantic parsing tasks. |
Investigating Multi-source Active Learning for Natural Language Inference (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent studies often assume that training and test data are drawn from the same distribution. |
| Approach: | They propose to apply active learning to unlabelled data pools to test for learning and generalisation. |
| Outcome: | The proposed strategies outperform random selection and outperformed hard-to-learn data on the task of natural language inference. |
Towards a Unified Multi-Domain Multilingual Named Entity Recognition Model (2023.eacl-main)
Copied to clipboard
Mayank Kulkarni, Daniel Preotiuc-Pietro, Karthik Radhakrishnan, Genta Indra Winata, Shijie Wu, Lingjue Xie, Shaohua Yang
| Challenge: | Named Entity Recognition is a key task whose performance is sensitive to genre and language. |
| Approach: | They propose a setup for Named Entity Recognition which includes multi-domain and multilingual training and evaluation across 13 domains and 4 languages. |
| Outcome: | The proposed model improves on 13 domains and 4 languages across 13 domain and 4 language domains. |
Do Neural Topic Models Really Need Dropout? Analysis of the Effect of Dropout in Topic Modeling (2023.eacl-main)
Copied to clipboard
| Challenge: | Dropout is a regularization trick used to resolve overfitting in large feedforward neural networks, but there is nil analysis of it for unsupervised models and in particular, VAE-based neural topic models. |
| Approach: | They propose to use dropout to solve overfitting problems in unsupervised neural topic models by stochastically dropping out the activation of neurons to prevent complex co-adaptations of feature vectors. |
| Outcome: | The proposed class of neural topic models can be used to improve the quality and predictive performance of the generated topics. |
A Psycholinguistic Analysis of BERT’s Representations of Compounds (2023.eacl-main)
Copied to clipboard
| Challenge: | 'compound' semantic representations are based on the semantics of constituent words, and are lexical items like any other word. |
| Approach: | They leverage a dataset that includes human judgments on two psycholinguistic measures of compound semantic analysis: lexeme meaning dominance (LMD) and semantic transparency (ST). |
| Outcome: | The proposed representations are based on a dataset that includes human judgments on two psycholinguistic measures of compound semantic analysis: lexeme meaning dominance (LMD) and semantic transparency (ST). |
Measuring Normative and Descriptive Biases in Language Models Using Census Data (2023.eacl-main)
Copied to clipboard
| Challenge: | a new study examines how gender-based distributions of occupations are reflected in pre-trained language models. |
| Approach: | They propose a method to measure to what degree pre-trained language models are aligned to normative and descriptive occupational distributions. |
| Outcome: | The proposed method is language independent and can be extended to other dimensions of census data and demographic variables. |
UDAPTER - Efficient Domain Adaptation Using Adapters (2023.eacl-main)
Copied to clipboard
| Challenge: | Using adapters, unsupervised domain adaptation (UDA) is more parameter efficient and requires large-scale data to be effective. |
| Approach: | They propose to add small bottleneck layers to each layer of a pre-trained language model to make it more parameter efficient by adding adapters. |
| Outcome: | The proposed methods outperform unsupervised domain adaptation methods such as DANN and DSN in natural language inference and sentiment classification tasks. |
Efficient CTC Regularization via Coarse Labels for End-to-End Speech Translation (2023.eacl-main)
Copied to clipboard
| Challenge: | Developing techniques to support end-to-end speech translation is non-trivial because of the speech-text modality gap. |
| Approach: | They propose a coarse labeling approach that merges vocabulary labels via simple heuristic rules . they propose to use 256-bit truncation, division or modulo operations to regularize the encoder . |
| Outcome: | The proposed method can increase training efficiency while delivering better performance. |
Exploring Category Structure with Contextual Language Models and Lexical Semantic Networks (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent work on word embeddings reports low correlations with human ratings . contextual language models (CLMs) have been successful in acquiring semantic and world knowledge. |
| Approach: | They propose to use BERT to probe contextual language models for predicting typicality scores. |
| Outcome: | The proposed methods improve on previous studies on word embeddings and their ability to predict typicality scores. |
An Empirical Study of Clinical Note Generation from Doctor-Patient Encounters (2023.eacl-main)
Copied to clipboard
| Challenge: | Medical doctors spend 52 to 102 minutes per day writing clinical notes from patient encounters. |
| Approach: | They propose to use a new dataset to generate automated and manual clinical notes from doctor-patient conversations in a clinical setting. |
| Outcome: | The proposed model could reduce the time spent writing clinical notes from doctor-patient conversations in a clinical setting. |
Instruction Clarification Requests in Multimodal Collaborative Dialogue Games: Tasks, and an Analysis of the CoDraw Dataset (2023.eacl-main)
Copied to clipboard
| Challenge: | In visual instruction-following dialogue games, players can engage in repair mechanisms in the face of an ambiguous or underspecified instruction. |
| Approach: | They annotate Instruction Clarification Requests (iCRs) in CoDraw, an existing dataset of interactions in a multimodal collaborative dialogue game. |
| Outcome: | The proposed dataset contains lexically and semantically diverse iCRs produced self-motivatedly by players deciding to clarify in order to solve the task successfully. |
Can Synthetic Text Help Clinical Named Entity Recognition? A Study of Electronic Health Records in French (2023.eacl-main)
Copied to clipboard
| Challenge: | In sensitive domains, the sharing of corpora is restricted due to confidentiality, copyrights or trade secrets. |
| Approach: | They use auto-regressive neural models to generate a clinical case corpus annotated with clinical entities and evaluate it for a named entity recognition task. |
| Outcome: | The proposed model can produce clinical case corpus annotated with clinical entities while maintaining confidentiality. |
IRMA: the 335-million-word Italian coRpus for studying MisinformAtion (2023.eacl-main)
Copied to clipboard
| Challenge: | IRMA contains over 600,000 Italian news articles collected from 56 websites classified as ‘untrustworthy’ by professional fact-checkers. |
| Approach: | They present IRMA, a corpus of italian news articles collected from 56 websites classified as ‘untrustworthy’ by professional fact-checkers. |
| Outcome: | The IRMA corpus contains over 600,000 Italian news articles collected from 56 websites classified as ‘untrustworthy’ by professional fact-checkers. |
Parameter-Efficient Korean Character-Level Language Modeling (2023.eacl-main)
Copied to clipboard
| Challenge: | Character-level language modeling has been shown to perform well on highly agglutinative or morphologically rich languages while using only a small fraction of the parameters required by (sub)word models. |
| Approach: | They propose a “three-hot” embedding and decoding scheme that exploits the decomposability of Korean characters to model at the syllable level but using only jamo-level representations. |
| Outcome: | The proposed model reduces the embedding parameters by 99.6% and does not lose translation quality compared to the baseline model. |
Opportunities and Challenges in Neural Dialog Tutoring (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing approaches to designing dialog tutors have been challenging . current approaches perform poorly in constrained learning scenarios, authors find . |
| Approach: | They analyze dialog tutoring models using automatic and human evaluations to understand the new opportunities brought by dialog tutors. |
| Outcome: | The proposed models perform poorly in less constrained learning scenarios, the authors show . they find large number of model reasoning errors in 45% of conversations . |
Evaluating the Robustness of Discrete Prompts (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods that generate discrete prompts from a small set of training instances have reported superior performance, but manual writing prompts that generalize well is challenging due to several reasons. |
| Approach: | They propose to use discrete prompts to learn lexical constructs that would not be encountered in manually-written prompts. |
| Outcome: | The proposed method is robust against perturbations to NLI inputs but sensitive to other types of perturbations such as shuffling and deletion of prompt tokens. |
Assessing Out-of-Domain Language Model Performance from Few Examples (2023.eacl-main)
Copied to clipboard
| Challenge: | Pretrained language models exhibit impressive generalization capabilities, but behave unpredictably under certain domain shifts. |
| Approach: | They propose to incorporate attributions into a few-shot model predicting out-of-domain (OOD) performance task to find out if models agree with pathological heuristics that may indicate worse generalization capabilities. |
| Outcome: | The proposed model-based model-learning model can perform better on a few-shot example set, and incorporate feature attributions to improve it. |
Mind the Labels: Describing Relations in Knowledge Graphs With Pretrained Models (2023.eacl-main)
Copied to clipboard
| Challenge: | Pretrained language models (PLMs) for data-to-text generation produce inaccurate outputs if labels are ambiguous or incomplete, which is often the case in D2T datasets. |
| Approach: | They propose to use a dataset to descib a relation between two entities using relation labels to train pretrained language models. |
| Outcome: | The proposed models are robust to generalizing to out-of-domain domains on a dataset for descibing a relation between two entities. |
Shapley Head Pruning: Identifying and Removing Interference in Multilingual Transformers (2023.eacl-main)
Copied to clipboard
| Challenge: | Multilingual transformer-based models demonstrate remarkable zero and few-shot transfer across languages by learning and reusing language-agnostic features. |
| Approach: | They propose to prune attention heads from a fixed model to reduce interference as a model acquires more languages. |
| Outcome: | The proposed model can reduce interference by pruning language-specific attention heads. |
Why Don’t You Do It Right? Analysing Annotators’ Disagreement in Subjective Tasks (2023.eacl-main)
Copied to clipboard
| Challenge: | Disagreement can reflect different aspects of linguistic annotation, from annotators’ subjectivity to sloppiness or lack of context to interpret a text. |
| Approach: | They propose a taxonomy of possible reasons leading to annotators' disagreement in subjective tasks and manually label part of a Twitter dataset for offensive language detection in english following this taxonomies. |
| Outcome: | The proposed taxonomy of disagreements in linguistic datasets can be used to assess how accurate tweets belonging to different disagreement categories can be classified as offensive or not. |
Analyzing Challenges in Neural Machine Translation for Software Localization (2023.eacl-main)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is a new form of machine translation that reduces the post-editing time of human annotators. |
| Approach: | They propose to use a novel multilingual UI corpus collection to test NMT for user interfaces. |
| Outcome: | The proposed test set evaluates state-of-the-art methods on a UI translation task from English to German and identifies its limitations. |
Bootstrapping Multilingual Semantic Parsers using Large Language Models (2023.eacl-main)
Copied to clipboard
| Challenge: | Despite cross-lingual generalization, translation models require significant amounts of labeled data for many low-resource languages . brittle translation services may be due to domain mismatch between input text and general-purpose text . |
| Approach: | They propose to use large language models to translate English datasets into several languages via few-shot prompting. |
| Outcome: | The proposed method outperforms a strong translation-train baseline on 41 out of 50 languages. |
Modeling Complex Event Scenarios via Simple Entity-focused Questions (2023.eacl-main)
Copied to clipboard
| Challenge: | Event schemas describe a sequence of events in a particular context, but they are difficult to model with standard event language models. |
| Approach: | They propose a question-guided generation framework that generates events as answers to questions about participants. |
| Outcome: | The proposed framework provides better coverage of participants, diverse events within a domain, comparable perplexities for modeling event sequences, and more effective control for interactive schema generation. |
Uncovering Implicit Inferences for Improved Relational Argument Mining (2023.eacl-main)
Copied to clipboard
| Challenge: | Argument mining attempts to extract arguments and their structure from unstructured texts. |
| Approach: | They propose a generative neuro-symbolic approach to finding inference chains that connect argument pairs by using the Commonsense Transformer. |
| Outcome: | The proposed approach outperforms the state-of-the-art by 2-5% in F1 score on three datasets. |
How people talk about each other: Modeling Generalized Intergroup Bias and Emotion (2023.eacl-main)
Copied to clipboard
Venkata Subrahmanyan Govindarajan, Katherine Atwell, Barea Sinno, Malihe Alikhani, David I. Beaver, Junyi Jessy Li
| Challenge: | Current studies of bias in NLP rely on identifying (unwanted or negative) bias towards a specific demographic group, but this is not always practical. |
| Approach: | They extrapolate a notion of bias from social science literature to predict interpersonal group relationship (IGR) using interpersonal emotions as an anchor. |
| Outcome: | The proposed model predicts the interpersonal group relationship (IGR) using interpersonal emotions as an anchor. |
Semantic Parsing for Conversational Question Answering over Knowledge Graphs (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent years have seen an increasing number of applications aiming to build conversational interfaces based on information retrieval and user recommendation. |
| Approach: | They develop a dataset where user questions are annotated with Sparql parses and system answers correspond to execution results thereof. |
| Outcome: | The proposed parsers can be used to ground questions into queries over definitions in a knowledge graph with large vocabularies. |
MAPL: Parameter-Efficient Adaptation of Unimodal Pre-Trained Models for Vision-Language Few-Shot Prompting (2023.eacl-main)
Copied to clipboard
| Challenge: | Large pre-trained models have proved to be remarkable zero- and (prompt-based) few-shot learners in unimodal vision and language tasks. |
| Approach: | They propose to use frozen unimodal models to learn a lightweight mapping between the representation spaces of unimod models using aligned image-text data. |
| Outcome: | The proposed method can generalize to unseen VL tasks from a few in-context examples while training orders of magnitude fewer parameters. |
ComSearch: Equation Searching with Combinatorial Strategy for Solving Math Word Problems with Weak Supervision (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing weakly-supervised methods for solving math word problems are expensive and time-consuming. |
| Approach: | They propose a weakly-supervised approach to solve math word problems . they propose 'comsearch' algorithm which compresses the search space by excluding mathematically equivalent equations. |
| Outcome: | The proposed algorithm can compress the search space by excluding mathematically equivalent equations. |
Towards preserving word order importance through Forced Invalidation (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent studies show pre-trained language models are insensitive to word order . performance on NLU tasks remains unchanged even after permuting the word . |
| Approach: | They propose a simple approach called Forced Invalidation to force the model to identify permuted sequences as invalid samples. |
| Outcome: | The proposed approach significantly improves the sensitivity of the models to word order on English NLU and QA tasks over BERT-based and attention-based models over word embeddings. |
How Many and Which Training Points Would Need to be Removed to Flip this Prediction? (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods to find St using brute-force are intractable. |
| Approach: | They propose a fast approximation method to find St based on influence functions . they propose to identify a minimum subset of training data that one would need to remove . |
| Outcome: | The proposed method can find St based on influence functions for simple classification models. |
Reinforced Sequence Training based Subjective Bias Correction (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for subjective bias correction focus on making one-word edits . a hybrid method to improve the performance of Seq2Seq and transformer-based bias correction models is proposed . |
| Approach: | They propose a reinforced sequence training approach for robust subjective bias correction . it balances bias neutralization with fluency and semantics preservation through reinforcement learning . |
| Outcome: | The proposed method improves subjective bias correction over existing methods . it is cross-trained over multiple sources of bias to be more robust to new styles of biased writing . |
Detecting Lexical Borrowings from Dominant Languages in Multilingual Wordlists (2023.eacl-main)
Copied to clipboard
| Challenge: | Language contact is reflected in the transfer of words from donor to recipient languages. |
| Approach: | They propose to use two classical sequence comparison methods and one machine learning method to detect lexical borrowings in contact situations where dominant languages play an important role. |
| Outcome: | The proposed methods outperform classical methods on a sample of seven Latin American languages. |
Towards Integration of Discriminability and Robustness for Document-Level Relation Extraction (2023.eacl-main)
Copied to clipboard
| Challenge: | Document-level relation extraction (DocRE) predicts relations for entity pairs relying on context-dependent reasoning . a large number of annotation errors can make it difficult to distinguish large semantically close relations . |
| Approach: | They propose a loss function to improve discriminability and robustness for DocRE . they also propose supervised contrastive learning and negative label sampling strategy . |
| Outcome: | The proposed method achieves state-of-the-art results on the DocRED dataset and its recently cleaned version. |
Penguins Don’t Fly: Reasoning about Generics through Instantiations and Exceptions (2023.eacl-main)
Copied to clipboard
| Challenge: | Generics express generalizations about the world that are not universally true . commonsense knowledge bases encode some generic knowledge but rarely enumerate exceptions . |
| Approach: | They propose a framework informed by linguistic theory to generate exemplars for generics . they generate 19k exemplar cases for 650 generics and show they outperform a strong baseline . |
| Outcome: | The proposed framework outperforms a baseline framework by 12.8 precision points. |
Adding Instructions during Pretraining: Effective way of Controlling Toxicity in Language Models (2023.eacl-main)
Copied to clipboard
| Challenge: | Pretrained large language models generate harmful language encompassing hate speech, abusive language, social biases, and threats. |
| Approach: | They propose two strategies that augment pretraining data to reduce model toxicity . MEDA adds raw toxicity score as meta-data and INST adds instructions indicating toxicity to pretraining samples. |
| Outcome: | The proposed strategies reduce toxicity probability up to 61% while preserving accuracy on five benchmark NLP tasks and improving AUC scores on bias detection tasks by 1.3%. |
Multi2Claim: Generating Scientific Claims from Multi-Choice Questions for Scientific Fact-Checking (2023.eacl-main)
Copied to clipboard
Neset Tan, Trung Nguyen, Josh Bensemann, Alex Peng, Qiming Bao, Yang Chen, Mark Gahegan, Michael Witbrock
| Challenge: | Existing scientific fact-checking datasets are limited due to expertise bottleneck . multi2Claim pipeline is a tool to convert multiple-choice questions into fact- checking data . |
| Approach: | They propose a pipeline for automatically converting multiple-choice questions into fact-checking data . they generate two large-scale datasets for scientific-fact-checker tasks . success at this task can help the reader understand scientific topics and promote science . |
| Outcome: | The proposed pipeline improves performance on two large-scale scientific fact-checking datasets. |
On Evaluation of Document Classification with RVL-CDIP (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing document classification benchmarks have label noise, ambiguous documents, and sensitive information. |
| Approach: | They argue that RVL-CDIP is unsuitable for benchmarking document classifiers . they advocate for a new document classification benchmark with ambiguous labels . |
| Outcome: | The RVL-CDIP benchmark is widely used for document classification . the authors argue that its limited scope, presence of errors and lack of diversity make it less than ideal for benchmarking. |
Event Linking: Grounding Event Mentions to Wikipedia (2023.eacl-main)
Copied to clipboard
| Challenge: | a new task for natural language understanding is called Event Linking . the context where an event is mentioned lacks the details of this event . |
| Approach: | They propose a new task to link an article's event mention to the most appropriate Wikipedia page . they collect a training set from Wikipedia and evaluate two models to test the task . |
| Outcome: | The proposed model is based on a dataset and a real-world news domain . it is expected that the most appropriate Wikipedia page will provide rich knowledge about the mention . |
SwitchPrompt: Learning Domain-Specific Gated Soft Prompts for Classification in Low-Resource Domains (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent work shows promising results when prompting pre-trained language models, but in low-resource domains, the domain gap between the pre-training data and the downstream task is too large. |
| Approach: | They propose a method for prompting pre-trained language models using domain-specific keywords with a trainable gated prompt. |
| Outcome: | The proposed prompting method outperforms state-of-the-art prompting methods on three text classification benchmarks and shows that it reduces the need for domain-specific language model pre-training. |
Do dialogue representations align with perception? An empirical study (2023.eacl-main)
Copied to clipboard
| Challenge: | masked language models produce stronger correlations than auto-regressive models, but humans and models make different response selection mistakes. |
| Approach: | They propose to use spoken conversation as a model to measure human comprehension behaviour. |
| Outcome: | The proposed model outperforms the model which produces the strongest correlation with human responses. |
Methods for Measuring, Updating, and Visualizing Factual Beliefs in Language Models (2023.eacl-main)
Copied to clipboard
Peter Hase, Mona Diab, Asli Celikyilmaz, Xian Li, Zornitsa Kozareva, Veselin Stoyanov, Mohit Bansal, Srinivasan Iyer
| Challenge: | Pretrained language models store a large amount of factual information that can be elicited by prompting or finetuning. |
| Approach: | They propose methods to measure model factual beliefs and update incorrect beliefs in models . they propose a new visualization tool that shows relationships between stored model beliefs . |
| Outcome: | The proposed methods improve models' consistency and accuracy, the authors show . their methods outperform existing methods in more difficult settings, the paper shows . |
Improving Sign Recognition with Phonology (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing work does not consider sign language phonology, but none leverages it . a recent study has shown that sign language recognition models lack structure . |
| Approach: | They explicitly recognize the role of phonology in sign production to train models for isolated sign language recognition . they train models that take in pose estimations of a signer producing a single sign to predict its phonological characteristics . |
| Outcome: | The proposed model improves sign recognition accuracy by 9% on the WLASL benchmark . the study could accelerate linguistic research in the domain of signed languages . |
Parameter-efficient Modularised Bias Mitigation via AdapterFusion (2023.eacl-main)
Copied to clipboard
Deepak Kumar, Oleg Lesota, George Zerveas, Daniel Cohen, Carsten Eickhoff, Markus Schedl, Navid Rekabsaz
| Challenge: | Large pre-trained language models contain societal biases and carry along these biase . Current approaches to mitigate these bias impose debiasing by updating model parameters, effectively transferring model to irreversible debiased state. |
| Approach: | They propose to develop stand-alone debiasing functionalities separate from the model, which can be integrated into the model on-demand while keeping the core model untouched. |
| Outcome: | The proposed approach improves or maintains effectiveness of bias mitigation, avoids catastrophic forgetting in a multi-attribute scenario, and maintains on-par task performance while granting parameter-efficiency and easy switching between the original and debiased models. |
LingMess: Linguistically Informed Multi Expert Scorers for Coreference Resolution (2023.eacl-main)
Copied to clipboard
| Challenge: | Current coreference systems use a single pairwise scoring component to assign mentions a score . different kinds of mentions require different information sources to assess their score - a problem that requires many decisions . |
| Approach: | They propose a linguistically motivated categorization of mention-pairs into 6 types of coreference decisions and learn a dedicated scoring function for each category. |
| Outcome: | The proposed model significantly improves the pairwise scorer and overall performance on the English Ontonotes coreference corpus and 5 additional datasets. |
Finding the Law: Enhancing Statutory Article Retrieval via Graph Neural Networks (2023.eacl-main)
Copied to clipboard
| Challenge: | Statutory article retrieval (SAR) is a promising application of legal text processing. |
| Approach: | They propose a graph-augmented dense statute retriever model that incorporates the structure of legislation via a neural network to improve density retrieval performance. |
| Outcome: | The proposed model outperforms baselines on a real-world expert-annotated dataset. |
Behavior Cloned Transformers are Neurosymbolic Reasoners (2023.eacl-main)
Copied to clipboard
| Challenge: | In injecting actions from symbolic modules into the action space of a behavior cloned transformer agent increases performance on four text game benchmarks that test arithmetic, navigation, sorting, and common sense reasoning by an average of 22% . contemporary agents struggle on tasks such as navigation, admetic and other tasks that humans make use of external tools. |
| Approach: | They propose to inject actions from symbolic modules into the action space of a behavior cloned transformer agent to increase performance on four text game benchmarks that test arithmetic, navigation, sorting, and common sense reasoning by an average of 22% . |
| Outcome: | The proposed method improves performance on four text game benchmarks that test arithmetic, navigation, sorting, and common sense reasoning by 22%, allowing an agent to reach the highest possible performance on unseen games. |
Bridging the Gap Between BabelNet and HowNet: Unsupervised Sense Alignment and Sememe Prediction (2023.eacl-main)
Copied to clipboard
| Challenge: | Sememes are the minimum semantic units of natural languages, but their use is limited by a lack of available sememe knowledge bases. |
| Approach: | They propose to use sense alignment to connect BabelNet with HowNet by relaxing constraints until a complete alignment is achieved. |
| Outcome: | The proposed method improves on previous supervised methods by 12% . it is based on interpretable propagation of sememe information between lexical resources . |
The StatCan Dialogue Dataset: Retrieving Data Tables through Conversations with Genuine Intents (2023.eacl-main)
Copied to clipboard
| Challenge: | StatCan Dialogue Dataset consists of 19,379 conversation turns between agents and online users . researchers propose two tasks to help knowledge workers find relevant tables for live chat users based on real-world intents . |
| Approach: | They propose two tasks based on 19,379 conversation turns between agents and online users . they investigate the difficulty of each task by establishing strong baselines . |
| Outcome: | The proposed task is based on a dataset of 19,379 conversation turns . the researchers show that the models struggle to generalize to future conversations . |
Question Generation Using Sequence-to-Sequence Model with Semantic Role Labels (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing question generation methods that generate multiple questions from text are labor-intensive and do not capture the complexity of ways a human asks questions. |
| Approach: | They propose a question generation method that combines the benefits of rule-based and neural sequence-to-sequence (Seq2Sequen) models. |
| Outcome: | The proposed method significantly improves the state-of-the-art neural question generation approaches on three real-world data sets. |
StyLEx: Explaining Style Using Human Lexical Annotations (2023.eacl-main)
Copied to clipboard
| Challenge: | Large pre-trained language models often learn spurious domain-specific words to make predictions. |
| Approach: | They propose a model that learns from human annotated explanations of stylistic features and jointly predicts them as model explanations. |
| Outcome: | The proposed model can provide human like stylistic lexical explanations without sacrificing performance on in-domain and out-of-domain datasets. |
Comparing Intrinsic Gender Bias Evaluation Measures without using Human Annotated Examples (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods to evaluate gender biases in pre-trained language models have been limited by the cost and difficulties of recruiting human annotators. |
| Approach: | They propose a method to compare intrinsic gender bias evaluation measures without relying on human annotated examples. |
| Outcome: | The proposed method compares gender-based gender bias evaluation measures without human annotators without human input. |
Faithfulness-Aware Decoding Strategies for Abstractive Summarization (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing studies on faithfulness of abstractive summarization have focused on decoding strategies. |
| Approach: | They propose two faithfulness-aware generation methods to further improve faithfulness . they propose to use a distillation approach to generate faithful summaries with greedy decoding . |
| Outcome: | The proposed methods improve faithfulness across two datasets as evaluated by automatic faithfulness metrics and human evaluation. |
Dynamic Benchmarking of Masked Language Models on Temporal Concept Drift with Multiple Views (2023.eacl-main)
Copied to clipboard
| Challenge: | Temporal concept drift is a problem of data changing over time. |
| Approach: | They benchmark 11 pretrained masked language models on a series of tests to evaluate temporal concept drift. |
| Outcome: | The proposed framework evaluates 11 pretrained masked language models on a series of tests . it aims to reveal how robust an MLM is over time and provide a signal in case it has become outdated . |
Real-Time Visual Feedback to Guide Benchmark Creation: A Human-and-Metric-in-the-Loop Workflow (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent research has shown that language models exploit ‘artifacts’ in benchmarks to solve tasks, rather than learning them, leading to inflated model performance. |
| Approach: | They propose a benchmark creation paradigm for NLP that focuses on guiding crowdworkers and provides realtime visual feedback to improve sample quality. |
| Outcome: | The proposed paradigm decreases effort, frustration, mental, and temporal demands of crowdworkers and analysts, while increasing the performance of both user groups. |
COMPS: Conceptual Minimal Pair Sentences for testing Robust Property Knowledge and its Inheritance in Pre-trained Language Models (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing pre-trained language models (PLMs) lack robustness in demonstrating simple reasoning, despite having the prerequisite knowledge. |
| Approach: | They propose to test pre-trained language models' ability to attribute properties to concepts and their ability to demonstrate property inheritance behavior. |
| Outcome: | The proposed model can easily distinguish between concepts on the basis of a property when they are trivially different, but find it relatively difficult when concepts are related on the base of nuanced knowledge representations. |
Probabilistic Robustness for Data Filtering (2023.eacl-main)
Copied to clipboard
| Challenge: | Modern machine learning works with massive amounts of data on a range of tasks like language modeling, object detection, and data mining. |
| Approach: | They propose a probabilistic robustness rewarded data optimization approach to enhance the model's generalization power by selecting training data that optimizes probabilistic metrics. |
| Outcome: | The proposed approach achieves +17.2% increase of accuracy and -28.05 decrease of perplexity on unknown-domain test sets. |
Unsupervised Improvement of Factual Knowledge in Language Models (2023.eacl-main)
Copied to clipboard
| Challenge: | Masked language modeling (MLM) is often dominated by high-frequency words that are sub-optimal for learning factual knowledge. |
| Approach: | They propose an approach that forces the model to prioritize informative words in a fully unsupervised way. |
| Outcome: | The proposed approach significantly improves the performance of pretrained language models on factual recall, question answering, sentiment analysis, and natural language inference in a closed-book setting. |
Learning to Ignore Adversarial Attacks (2023.eacl-main)
Copied to clipboard
| Challenge: | Despite the strong performance of current NLP models, they can be brittle against adversarial inputs. |
| Approach: | They propose a rationale model that explicitly learns to ignore adversarial tokens . their approach leads to sizable improvements in robustness over baseline models . |
| Outcome: | The proposed model outperforms data augmentation with adversarial examples and closes the gap between model performance and an attacked test set. |
Should You Mask 15% in Masked Language Modeling? (2023.eacl-main)
Copied to clipboard
| Challenge: | Masked language models (MLMs) traditionally mask 15% of tokens due to the belief that more masking would leave insufficient context to learn good representations. |
| Approach: | They revisit the 15% masking rate of MLMs to examine the role of masking in linguistic training. |
| Outcome: | The proposed masking rate outperforms BERT-large size models on GLUE and SQUAD while maintaining 95% accuracy. |
How do Words Contribute to Sentence Semantics? Revisiting Sentence Embeddings with a Perturbation Method (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing studies on sentence representation learning focus on human annotation, but they neglect the critical property that essential contents should contribute to sentence semantics more than non-essential contents when encoding a sentence. |
| Approach: | They propose a perturbation method for unsupervised semantic analysis that uses a sentence compression metric to adapt sentence compression datasets for automatic evaluation. |
| Outcome: | The proposed method can capture the main semantics of sentences better than several SOTA unsupervised sentence embedding models. |
AutoTriggER: Label-Efficient and Robust Named Entity Recognition with Auxiliary Trigger Extraction (2023.eacl-main)
Copied to clipboard
Dong-Ho Lee, Ravi Kiran Selvam, Sheikh Muhammad Sarwar, Bill Yuchen Lin, Fred Morstatter, Jay Pujara, Elizabeth Boschee, James Allan, Xiang Ren
| Challenge: | Named entity recognition models have shown impressive results in overcoming label scarcity and generalizing to unseen entities by leveraging distant supervision and auxiliary information such as explanations. |
| Approach: | They propose a framework that automatically generates and leverages “entity triggers” which are human-readable cues in the text that help guide the model to make better decisions. |
| Outcome: | The proposed framework outperforms the RoBERTa-CRF baseline by nearly 0.5 F1 points on three well-studied datasets. |
Incorporating Task-Specific Concept Knowledge into Script Learning (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing work on Goal-Oriented Scripts ignore usage context and personal preferences . proposed tasks are restrictive and rely on overly simplified assumptions . |
| Approach: | They propose a novel approach to Goal-Oriented Script Completion that uses concept prompting and script-oriented contrastive learning to improve performance. |
| Outcome: | The proposed approach improves on a WikiHow-based dataset. |
DeepMaven: Deep Question Answering on Long-Distance Movie/TV Show Videos with Multimedia Knowledge Extraction and Synthesis (2023.eacl-main)
Copied to clipboard
| Challenge: | Long video content understanding poses a challenging set of research questions as it involves long-distance, cross-media reasoning and knowledge awareness. |
| Approach: | They propose a framework which extracts events, entities, and relations from the rich multimedia content in long videos to pre-construct movie knowledge graphs. |
| Outcome: | The proposed framework performs competitively for both the new DeepMovieQA and the pre-existing MovieQA dataset. |
Salient Span Masking for Temporal Understanding (2023.eacl-main)
Copied to clipboard
| Challenge: | Salient Span Masking (SSM) has shown to be effective for closed-book question answering . authors of this study found that SSM alone improves performance on temporal tasks . |
| Approach: | They introduce Temporal Span Masking (TSM) to improve performance on temporal tasks . they find that SSM alone improves the downstream performance by +5.8 points . |
| Outcome: | The proposed approach improves performance on three temporal tasks by +5.8 points . the additional targeted spans achieved by adding the TSM task are the best . |
PECO: Examining Single Sentence Label Leakage in Natural Language Inference Datasets through Progressive Evaluation of Cluster Outliers (2023.eacl-main)
Copied to clipboard
| Challenge: | Efforts to debias NLI have led to datasets that exhibit different kinds of bias than those shown before. |
| Approach: | They propose a new technique to detect and reduce single sentence label leakage . leakage is a problem with many modern NLI datasets, they argue . future work must prioritize reducing this problem, they write . |
| Outcome: | a new model-driven technique can detect leakage and detect subpopulations in the datasets which exhibit it . the proposed technique is based on the progressive evaluation of cluster outliers (PECO) . it allows objective measurement of leakage, and automatic detection of subpopulations in the data which exhibit leakage. |
Weakly-Supervised Questions for Zero-Shot Relation Extraction (2023.eacl-main)
Copied to clipboard
| Challenge: | Zero-Shot Relation Extraction (ZRE) is a task where the training and test sets have no shared relation types. |
| Approach: | They propose to learn a model that can translate relation descriptions into relevant questions, which are then leveraged to generate the correct tail entity. |
| Outcome: | The proposed model outperforms the state-of-the-art on the fewrel and WikiZSL datasets by more than 16 F1 points without using gold question templates. |
DiffQG: Generating Questions to Summarize Factual Changes (2023.eacl-main)
Copied to clipboard
Jeremy R. Cole, Palak Jain, Julian Martin Eisenschlos, Michael J.Q. Zhang, Eunsol Choi, Bhuwan Dhingra
| Challenge: | Existing methods to identify factual changes between paired documents are limited . specialized entailment-like resources and models have been applied to fact verification . |
| Approach: | They propose to represent factual changes between paired documents as question-answer pairs . they propose to generate a discriminating question given an answer span such that the question is answerable by one passage but not the other . |
| Outcome: | The proposed model can flexibly and concisely capture the updated contents of paired documents. |
Contextual Dynamic Prompting for Response Generation in Task-oriented Dialog Systems (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing studies show that large pre-trained language models can be adapted to task-oriented dialog systems. |
| Approach: | They propose to use contextual dynamic prompting to generate prompts in dialogs . they propose to distill useful prompting signals from dialog contexts based on contextual dynamic . |
| Outcome: | The proposed approach improves response generation by 3 points and 17 points when dialog states are incorporated. |
Why Can’t Discourse Parsing Generalize? A Thorough Investigation of the Impact of Data Diversity (2023.eacl-main)
Copied to clipboard
| Challenge: | Discourse parsing performance is not reliable for high-resource languages such as English . a heterogeneous training regime is critical for stable and generalizable models . |
| Approach: | They investigate the impact of genre diversity on RST parsing stability . they use two largest RST corpora of English with text from multiple genres . |
| Outcome: | The proposed model can generalize to text types unseen during training, but it is not reliable for high-resource languages. |
Enriching Biomedical Knowledge for Low-resource Language Through Large-scale Translation (2023.eacl-main)
Copied to clipboard
| Challenge: | Biomedical data and benchmarks are highly valuable but limited in low-resource languages such as English. |
| Approach: | They propose a translation model in Vietnamese that trains a pretrained Encoder-Decoder Transformer model on 20 million translated abstracts. |
| Outcome: | The proposed model can translate and produce both pretrained and supervised biomedical data in two biomedically important domains. |
Syntax-guided Neural Module Distillation to Probe Compositionality in Sentence Embeddings (2023.eacl-main)
Copied to clipboard
| Challenge: | Past work on sentence embedding models faces issues determining the causal impact of implicit syntax representations. |
| Approach: | They construct a neural module net based on a transformer model and train it end-to-end to approximate the sentence’s embedding. |
| Outcome: | The proposed model captures whether syntax is a strong model of its compositional ability. |
Closed-book Question Generation via Contrastive Learning (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent studies on open-book QG have achieved promising progress, but generating natural questions under a more practical closed-book setting remains a challenge. |
| Approach: | They propose a QG model that stores more information in its parameters through contrastive learning and an answer reconstruction module. |
| Outcome: | The proposed model outperforms baselines in automatic evaluation and human evaluation on a public dataset and a new WikiCQA dataset. |
A Hybrid Detection and Generation Framework with Separate Encoders for Event Extraction (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent work on event extraction tasks has been based on classification-based methods . a new generation-based method is being developed to extract event triggers and event arguments from plain text. |
| Approach: | They propose to use independent encoders to model event detection and event argument extraction, respectively, and use token-level features to precisely control the fusion between two encoder. |
| Outcome: | The proposed method avoids feature interference and achieves joint training . it is compared with other methods and achieved competitive results on standard benchmarks . |
Path Spuriousness-aware Reinforcement Learning for Multi-Hop Knowledge Graph Reasoning (2023.eacl-main)
Copied to clipboard
| Challenge: | Multi-hop reasoning is a common approach for query answering, but can be biased to spurious paths which coincidentally lead to the correct answer with poor explanation. |
| Approach: | They propose a method that quantitatively estimates to what extent a path is spurious by a metric called Path Spuriousness (PS) they propose KG reasoning, which infers new facts along existing paths in KGs. |
| Outcome: | The proposed model significantly improves the agent’s ability to prevent spurious paths while keeping comparable to state-of-the-art performance. |
Self-Adaptive Named Entity Recognition by Retrieving Unstructured Knowledge (2023.eacl-main)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is costly because of lack of training data and domain experts. |
| Approach: | They propose a self-adaptive neural model that retrieves external knowledge from unstructured text to learn the usages of entities that have not been learned well. |
| Outcome: | The proposed model outperforms strong baselines on cross-neuro-ner datasets by 2.35 points in F1 metric. |
When Do Pre-Training Biases Propagate to Downstream Tasks? A Case Study in Text Summarization (2023.eacl-main)
Copied to clipboard
Faisal Ladhak, Esin Durmus, Mirac Suzgun, Tianyi Zhang, Dan Jurafsky, Kathleen McKeown, Tatsunori Hashimoto
| Challenge: | Existing studies have shown that large language models contain linguistic and societal biases, but it is unclear how these biase amplify to downstream tasks. |
| Approach: | They investigate how name-nationality bias propagates from pre-training to downstream tasks . they show that these biases manifest themselves as hallucinations in summarization . |
| Outcome: | The proposed model can reduce the rate of hallucinations, but does not change the types of biases that do appear. |
BERT Shows Garden Path Effects (2023.eacl-main)
Copied to clipboard
| Challenge: | Garden path sentences are sentences that readers incorrectly parse, requiring partial or total re-analysis of the sentence structure. |
| Approach: | They assess transformer language models which have been fine-tuned on a question-answering task and evaluate their performance on comprehension questions based on garden path and control sentences. |
| Outcome: | The proposed models have low performance in certain instances of question answering based on garden path contexts, and incorrectly assign semantic roles aligning for the most part with human performance. |
Models Teaching Models: Improving Model Accuracy with Slingshot Learning (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods to label datasets are expensive and require human labor . a semi-supervised method that augments a small dataset with labels reduces the cost of using simpler methods . |
| Approach: | They propose a semi-supervised method to augment a human-labeled dataset with labels from a teacher model to slingshot the performance of a student model. |
| Outcome: | The proposed method reduces the accuracy trade-off required to use simpler methods without disrupting their benefits. |
A Federated Approach for Hate Speech Detection (2023.eacl-main)
Copied to clipboard
| Challenge: | Despite the scale of social media content, privacy preservation in hate speech detection has remained understudied. |
| Approach: | They propose to use federated machine learning to address privacy concerns in hate speech detection by obtaining a 6.81% improvement in F1-score. |
| Outcome: | The proposed method improves the F1-score of hate speech detection by 6.81% while maintaining public data privacy. |
Learning the Legibility of Visual Text Perturbations (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing adversarial attacks in NLP perturb text to produce visually similar strings ('ergo', 'rgo') which are legible to humans but degrade model performance. |
| Approach: | They use a human-annotated dataset comprising the legibility of visually perturbed text to build models that predict the legible inputs and rank them based on their legibility. |
| Outcome: | The proposed models achieve an F score of 0.91 and an accuracy of 0.86 in predicting which of two perturbations is more legible. |
DyLoRA: Parameter-Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptation (2023.eacl-main)
Copied to clipboard
| Challenge: | Pre-training/fine-tuning of pre-training models has become more expensive and resource-hungry. |
| Approach: | They propose a low-rank adaptation technique that trains LoRA blocks for a range of ranks instead of a single rank. |
| Outcome: | The proposed method trains LoRA blocks for a range of ranks instead of a single rank . it can train dynamic search-free models with DyLoRA at least 4 to 7 times faster than LoRA . |
Conversational Emotion-Cause Pair Extraction with Guided Mixture of Experts (2023.eacl-main)
Copied to clipboard
| Challenge: | Emotion-Cause Pair Extraction (ECPE) task aims to pair all emotions and corresponding causes in documents. |
| Approach: | They propose a new Emotion-Cause Pair Extraction task in dialogue . they employ a ECPE dataset with more emotion-cause pairs in documents than news articles . |
| Outcome: | The proposed model improves on a new english dialogue dataset with more emotion-cause pairs than news articles. |
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent advances in the capacity of large language models to generate human-like text have prompted a heated discourse around the risks of societal harms they introduce. |
| Approach: | They propose a taxonomy of interventions organized around the different phases where they can be adopted to mitigate harms. |
| Outcome: | The proposed methods are based on several prior works’ taxonomies of language model risks and provide an overview of strategies for detecting and ameliorating different kinds of risks/harms. |
TraVLR: Now You See It, Now You Don’t! A Bimodal Dataset for Evaluating Visio-Linguistic Reasoning (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing visio-linguistic (V+L) models do not represent visual and linguistic concepts in a unified space. |
| Approach: | They propose to use cross-modal transfer to evaluate the extent to which visio-linguistic (V+L) representations are represented in a unified space. |
| Outcome: | The proposed evaluation settings include cross-modal transfer and a global accuracy score on the entire dataset making the specific sources of success and failure difficult to diagnose. |
Paraphrase Acquisition from Image Captions (2023.eacl-main)
Copied to clipboard
| Challenge: | Using image captions, we hypothesize that different captions for the same image naturally form a set of mutual paraphrases. |
| Approach: | They propose to use image captions as a previously underutilized resource for paraphrases . they analyze captions in the English Wikipedia to find common paraphrase similarities . |
| Outcome: | The proposed dataset compares known paraphrase corpora with their syntactic and semantic similarity to the existing dataset. |
Generation-Based Data Augmentation for Offensive Language Detection: Is It Worth It? (2023.eacl-main)
Copied to clipboard
| Challenge: | generative data augmentation has been shown to be effective in offensive language detection but the potential for bias injection has not been investigated. |
| Approach: | They propose to investigate the robustness of models trained on generated data in a variety of data augmentation setups and analyze models using the HateCheck suite. |
| Outcome: | The proposed model training setups on four English offensive language datasets are robust and robust, while the generative DA setups do not present bias injection issues. |
Quantifying Context Mixing in Transformers (2023.eacl-main)
Copied to clipboard
| Challenge: | Self-attention weights and their transformed variants have been used for analyzing token-to-token interactions in Transformer-based models, but they are not faithful to the models’ decisions as they are only one part of an encoder block. |
| Approach: | They propose a new context mixing score customized for Transformers that provides us with a deeper understanding of how information is mixed at each encoder layer. |
| Outcome: | The proposed score outperforms other methods in linguistically informed rationales, probing, and faithfulness analysis. |
KGVL-BART: Knowledge Graph Augmented Visual Language BART for Radiology Report Generation (2023.eacl-main)
Copied to clipboard
| Challenge: | Timely generation of radiology reports and diagnoses is a challenge worldwide due to the enormous number of cases and shortage of radiologists. |
| Approach: | They propose a Knowledge Graph Augmented Vision Language BART model that takes two chest X-ray images and outputs a report with patient-specific findings. |
| Outcome: | The proposed model outperforms state-of-the-art transformer-based models on scoring metrics. |
A simple but effective model for attachment in discourse parsing with multi-task learning for relation labeling (2023.eacl-main)
Copied to clipboard
| Challenge: | a discourse parsing model for conversation trained on the STAC is hard due to the complexity of discourse graphs and the frequent lack of surface cues provided by EDUs. |
| Approach: | They propose a discourse parsing model for conversation trained on the STAC that encodes discourse units and uses a multitask setting to predict relation labels. |
| Outcome: | The proposed model outperforms state-of-the-art models for discourse attachment prediction with no loss in performance for attachment. |
How Far Can It Go? On Intrinsic Gender Bias Mitigation for Text Classification (2023.eacl-main)
Copied to clipboard
| Challenge: | a growing interest in exploring how gender bias pertains in contextualized language models has been generated . intrinsic mitigation strategies and bias metrics have been proposed to mitigate gender bias in contextualised language models . |
| Approach: | They propose to use different intrinsic bias mitigation strategies to mitigate gender bias in contextualized language models. |
| Outcome: | The proposed probe shows that some mitigation techniques can hide gender bias . the probe also shows that not all mitigation techniques fool extrinsic bias despite their use . |
Multimodal Event Transformer for Image-guided Story Ending Generation (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods focus on cross-modal feature fusion but overlook reasoning and mining implicit information from story plots and ending image. |
| Approach: | They propose a multimodal event transformer framework for image-guided story ending generation. |
| Outcome: | The proposed method achieves state-of-the-art performance for image-guided story ending generation. |
Improving Cross-modal Alignment for Text-Guided Image Inpainting (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods allocate most of computation to visual encoding, while light computation on modeling modality interactions. |
| Approach: | They propose a novel model for text-guided image inpainting by improving cross-modal alignment knowledge by using a vision-language encoder and an image generator. |
| Outcome: | The proposed model achieves state-of-the-art performance compared with other strong competitors on two vision-language datasets. |
Semantic Specialization for Knowledge-based Word Sense Disambiguation (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for knowledge-based Word Sense Disambiguation (WSD) use only lexical knowledge to adapt contextualized embeddings. |
| Approach: | They propose a semantic specialization where contextualized embeddings are adapted to the WSD task using only lexical knowledge. |
| Outcome: | The proposed method outperforms previous studies that adapt contextualized embeddings while controlling deviations from the original embeddables. |
Concept-based Persona Expansion for Improving Diversity of Persona-Grounded Dialogue (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing approaches to improve the quality of persona-grounded dialogues are limited to a few informative words. |
| Approach: | They propose a concept-based persona expansion framework that takes the original persona as input and generates expanded personas that contain conceptually rich content. |
| Outcome: | The proposed framework improves the quality of persona-grounded dialogue responses in diversity and richness. |
RPTCS: A Reinforced Persona-aware Topic-guiding Conversational System (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing systems that control concept transitions in a conversation lack a persona-aware topic transition dataset. |
| Approach: | They propose a persona-aware topic-guiding conversational system that leads the conversation to drift to a set of target concepts depending on the persona of the speaker and the context of the conversation. |
| Outcome: | The proposed system produces fluent responses with no useful information and is based on a conversational dataset with a human-in-loop only quality checks. |
What Did You Learn To Hate? A Topic-Oriented Analysis of Generalization in Hate Speech Detection (2023.eacl-main)
Copied to clipboard
| Challenge: | Hate speech detection datasets often use different annotation guidelines, resulting in inconsistencies . authors propose a topic-oriented approach to study generalization across popular hate speech datasets . |
| Approach: | They propose a topic-oriented approach to study generalization across popular hate speech datasets . they compare Transformer-based models in capturing topic-generic and topic-specific knowledge . |
| Outcome: | The proposed approach improves the reliability of hate speech detection on social media platforms. |
End-to-end Case-Based Reasoning for Commonsense Knowledge Base Completion (2023.eacl-main)
Copied to clipboard
| Challenge: | Pretrained language models have been shown to store knowledge in their parameters and have achieved reasonable performance in knowledge-intensive tasks. |
| Approach: | They propose to provide retrieved passages that contain relevant knowledge as additional input to the commonsense knowledge base completion (CKBC) task. |
| Outcome: | The proposed framework generates more valid, informative, and novel knowledge than the state-of-the-art COMET model for commonsense knowledge base completion (CKBC) tasks. |
Exploring Segmentation Approaches for Neural Machine Translation of Code-Switched Egyptian Arabic-English Text (2023.eacl-main)
Copied to clipboard
| Challenge: | Code-switching (CS) is a problem in machine translation, but its performance is not investigated for CS settings. |
| Approach: | They propose to use morphological segmentation techniques for machine translation tasks . they compare morphology-based and frequency-based segmentation for MT tasks based on data size . |
| Outcome: | The proposed approach performs best in MT tasks but under-performs in other languages. |
Identifying the limits of transformers when performing model-checking with natural language (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent studies have focused on transformer models’ ability to perform reasoning on text, but the above question has not been adequately answered. |
| Approach: | They investigated the problem of model-checking with natural language to determine whether transformers can comprehend logical semantics in natural language. |
| Outcome: | The proposed model-checking problem is suited to address this issue but is untouched in natural language inference research. |
Improving the Generalizability of Collaborative Dialogue Analysis With Multi-Feature Embeddings (2023.eacl-main)
Copied to clipboard
| Challenge: | Using a multi-feature embedding improves the generalizability of conflict prediction models trained on dialogues. |
| Approach: | They propose a multi-feature embedding that leverages textual, structural, and semantic information from dialogues by incorporating lexical, dialogue acts, and sentiment features. |
| Outcome: | The proposed model is excellent domain-agnostic representation for meta-pretraining a few-shot model on collaborative multiparty dialogues. |
MetaQA: Combining Expert Agents for Multi-Skill Question Answering (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent explosion of question-answering datasets and models has increased interest in generalization of models across multiple domains and formats. |
| Approach: | They propose to combine expert agents with a flexible and training-efficient architecture that considers questions, answer predictions, and answer-prediction confidence scores to select the best answer among a list of answer predictions. |
| Outcome: | The proposed model outperforms previous multi-agent and multi-dataset approaches and is highly data-efficient to train and adaptable to any QA format. |
BERT Is Not The Count: Learning to Match Mathematical Statements with Proofs (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing work on mathematical article analysis uses natural language processing to solve complex mathematical articles. |
| Approach: | They propose a bilinear similarity model and two decoding methods to match statements to proofs effectively. |
| Outcome: | The proposed model matches proofs to statements without being aware of proofs, but it follows a relatively shallow symbolic analysis and matching to achieve that performance. |
Lessons Learned from a Citizen Science Project for Natural Language Processing (2023.eacl-main)
Copied to clipboard
Jan-Christoph Klie, Ji-Ung Lee, Kevin Stowe, Gözde Şahin, Nafise Sadat Moosavi, Luke Bates, Dominic Petrak, Richard Eckart De Castilho, Iryna Gurevych
| Challenge: | Annotations are expensive and difficult to obtain, which is why many NLP systems outsource their work to paid crowdworkers. |
| Approach: | They propose to use Citizen Science to re-annotate parts of a pre-existing crowdsourced dataset to gain high-quality annotations. |
| Outcome: | The proposed approach yields high-quality annotations and motivated volunteers, but requires consideration of scalability, participation over time, and legal and ethical issues. |
Contrastive Learning with Keyword-based Data Augmentation for Code Search and Code Question Answering (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent work on code search proposes data augmentation of queries for contrastive learning. |
| Approach: | They propose to augment query-code pairs with key words to preserve key words . they use keyDAC to fine-tune various pre-trained language models . |
| Outcome: | The proposed approach outperforms the current state-of-the-art in code search and question answering tasks. |
Large Scale Multi-Lingual Multi-Modal Summarization Dataset (2023.eacl-main)
Copied to clipboard
| Challenge: | a large dataset of document-image pairs and annotated multi-modal summarization data is needed for multi-lingual modeling . encoder-decoder models represent information comprising multiple modalities. |
| Approach: | They propose to use a multi-lingual summarization dataset to analyze multi-modal summarizing using multi-linguistic annotated data. |
| Outcome: | The proposed dataset is the largest multi-lingual multi-modal summarization dataset for 13 languages and consists of cross-lingual summarizing data for 2 languages. |
External Knowledge Acquisition for End-to-End Document-Oriented Dialog Systems (2023.eacl-main)
Copied to clipboard
| Challenge: | End-to-end neural models for conversational AI often assume that a response can be generated by considering only the knowledge acquired during training. |
| Approach: | They propose an architecture for document-oriented conversations with access to external knowledge sources. |
| Outcome: | The proposed architecture outperforms baseline models on the Wizard of Wikipedia dataset by 10.3% and 7.4%. |
In-Depth Look at Word Filling Societal Bias Measures (2023.eacl-main)
Copied to clipboard
| Challenge: | Language models (LMs) are ubiquitous in current NLP and have brought undeniable performance improvements for many tasks. |
| Approach: | They propose to use word filling prompts to evaluate language models' behavior to find out if they are valid. |
| Outcome: | The proposed measures produce unexpected and illogical results when appropriate control group samples are constructed. |
Retrieval-augmented Image Captioning (2023.eacl-main)
Copied to clipboard
| Challenge: | Experimental results show that image captioning can be effectively formulated from this new perspective. |
| Approach: | They propose a pretrained visual and language decoders for image captioning that generate sentences from the input image and a set of captions retrieved from a datastore. |
| Outcome: | The proposed model generates sentences given the input image and retrieved captions, while the decoder attends to the multimodal encoder representations. |
Automatic Evaluation and Analysis of Idioms in Neural Machine Translation (2023.eacl-main)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) struggles with the translation of rare multi-word expressions (MWEs). |
| Approach: | They propose a metric for automatically measuring the frequency of literal translation errors without human involvement. |
| Outcome: | The proposed metric measures the frequency of literal translation errors without human involvement with the models trained in different conditions and across a wide range of metrics and test sets. |
Representation biases in sentence transformers (2023.eacl-main)
Copied to clipboard
| Challenge: | argued that transformer-based models are not well suited for sentence-level downstream tasks. |
| Approach: | They propose to use sentence transformers to produce full-sentence representations . they propose to combine transformers with a training regime that embeds tokens into the model . |
| Outcome: | The proposed model performs better on downstream tasks than the vanilla model and its variants. |
AbLit: A Resource for Analyzing and Generating Abridged Versions of English Literature (2023.eacl-main)
Copied to clipboard
| Challenge: | Creating an abridged version of a text requires shortening it while maintaining its linguistic qualities. |
| Approach: | They propose an abridgement task that requires shortening and maintaining linguistic qualities of a text while maintaining its linguistic quality. |
| Outcome: | The proposed dataset captures passage-level alignments between original and abridged texts . it can be used to generate a bridge and shorten the original text . |
Self-training Reduces Flicker in Retranslation-based Simultaneous Translation (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing approaches to reduce flicker in simultaneous translation have increased the latency through masking and specialised inference, thus losing the simplicity of the approach. |
| Approach: | They propose to train a machine translation system to reduce flicker by controlling monotonicity and biased beam search to achieve the same flicker-latency tradeoff. |
| Outcome: | The proposed approach reduces flicker by controlling monotonicity while maintaining similar translation quality to the original. |
Social Commonsense for Explanation and Cultural Bias Discovery (2023.eacl-main)
Copied to clipboard
| Challenge: | Social commonsense contains many human biases due to social and cultural influence. |
| Approach: | They aim to identify cultural biases in data that strongly influence model decisions . they use social commonsense knowledge to augment large-scale language models . |
| Outcome: | The proposed method shows that social commonsense knowledge can explain model behavior on two social tasks. |
Counter-GAP: Counterfactual Bias Evaluation through Gendered Ambiguous Pronouns (2023.eacl-main)
Copied to clipboard
| Challenge: | a number of studies have focused on gender bias in language models, but these methods fail to detect it. |
| Approach: | They propose to use gender bias in coreference resolution to evaluate gender bias . they propose to construct an annotated quadruple-level dataset with 4008 instances . |
| Outcome: | The proposed method is able to detect gender bias in a quadruple dataset . previous methods failed to detect bias or cancel it, the authors argue . |
The NLP Task Effectiveness of Long-Range Transformers (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing benchmarks on long-range attention models have not been sufficient to develop efficient Transformers and their practical application on complex NLP tasks. |
| Approach: | They propose to benchmark 7 Transformer variants on 5 difficult NLP tasks and 7 datasets to examine their capacity for long-range attention. |
| Outcome: | The proposed models have advantages on content selection and query-guided decoding, but they come with previously unrecognized drawbacks such as insufficient attention to distant tokens and accumulated approximation error. |
Creation and evaluation of timelines for longitudinal user posts (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for segmenting user posts into timelines improve quality and cost of manual annotation. |
| Approach: | They propose a set of methods for segmenting longitudinal user posts into timelines likely to contain interesting moments of change in a user’s behaviour based on their online posting activity. |
| Outcome: | The proposed framework is able to evaluate two different social media datasets and compares with existing models. |
Semi-supervised New Event Type Induction and Description via Contrastive Loss-Enforced Batch Attention (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for event extraction use annotated event types but are expensive and time-consuming. |
| Approach: | They propose a semi-supervised approach to learning new event types using a masked contrastive loss. |
| Outcome: | The proposed method learns similarities between clusters by enforcing an attention mechanism over the data minibatch. |
Multilingual Content Moderation: A Case Study on Reddit (2023.eacl-main)
Copied to clipboard
| Challenge: | a growing need for AI moderators to safeguard users and protect mental health of human moderator from traumatic content. |
| Approach: | They propose to use a multilingual dataset to study the challenges of content moderation . they propose to analyze 1.8 million Reddit comments in English, german, spanish and french . |
| Outcome: | The proposed dataset highlights the challenges and suggests related research problems . it shows that the proposed model can be used to predict the violated rule . |
GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Models (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent work aimed to improve task performance of large language models by rewriting or tuning them manually, but manual rewrite is time-consuming and requires subjective interpretation. |
| Approach: | They propose a gradient-free, edit-based search approach for improving task instructions for large language models. |
| Outcome: | The proposed approach outperforms manual rewriting and purely example-based prompts while allowing for API-based tuning. |
DiscoScore: Evaluating Text Generation with BERT and Discourse Coherence (2023.eacl-main)
Copied to clipboard
| Challenge: | DiscoScore is a parametrized discourse metric that uses BERT to model discourse coherence . it is weak when operated at system level, and is therefore not reliable in a way to spot improvements . |
| Approach: | They propose a parametrized discourse metric which uses BERT to model discourse coherence from different perspectives. |
| Outcome: | The proposed model outperforms existing models on document-level machine translation and summarization. |
Know your audience: specializing grounded language models with listener subtraction (2023.eacl-main)
Copied to clipboard
| Challenge: | Effective communication requires adapting to the idiosyncrasies of each communicative context. |
| Approach: | They propose a method for specializing grounded language models without supervision . they fine-tune an attention-based adapter between a CLIP vision encoder and a large language model . |
| Outcome: | The proposed method allows a speaker to adapt to the idiosyncracies of the listeners without supervision. |
Meeting the Needs of Low-Resource Languages: The Value of Automatic Alignments via Pretrained Models (2023.eacl-main)
Copied to clipboard
Abteen Ebrahimi, Arya D. McCarthy, Arturo Oncevay, John E. Ortega, Luis Chiruzzo, Gustavo Giménez-Lugo, Rolando Coto-Solano, Katharina Kann
| Challenge: | Large multilingual models have inspired a new class of word alignment methods, which work well for pretraining languages. |
| Approach: | They propose to use transformer-based word alignment methods to extract alignments from massive pretrained models. |
| Outcome: | The proposed methods outperform traditional methods for languages unseen to pretraining models, and are competitive with each other. |