Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers)

Injecting Wiktionary to improve token-level contextual representations using contrastive learning (2024.eacl-short)

Challenge:	Hallucinations and off-target translations remain unsolved problems in machine translation, especially for low-resource languages and massively multilingual models.
Approach:	They propose two methods to mitigate hallucinations and off-target translations with a modified decoding objective without retraining or external models.
Outcome:	The proposed methods reduce translation errors with segment-level chrF2 below 10 by 67-83% on average across 57 tested translation directions.

Copied to clipboard

Anna Mosolova, Marie Candito, Carlos Ramisch

Multilingual Gradient Word-Order Typology from Universal Dependencies (2024.eacl-short)

Challenge:	lexical semantics tasks require contextual word embeddings that are not blind to context, despite the fact that vectors of the same meaning are too different.
Approach:	They propose to fine-tune pre-trained language models by using automatically self-augmented examples to target contextual word embeddings.
Outcome:	The proposed method achieves significant improvements on the original WiC test set and in two new tests.

Copied to clipboard

Emi Baylor, Esther Ploeger, Johannes Bjerva

Evaluating the Factuality of Zero-shot Summarizers Across Varied Domains (2024.eacl-short)

Challenge:	Existing typological databases, including WALS and Grambank, suffer from inconsistencies due to categorical format.
Approach:	They propose a new seed dataset that uses continuous-valued data instead of categorical data to better reflect the variability of language.
Outcome:	The proposed dataset can be easily adapted to generate data for a broader set of features and languages.

Copied to clipboard

Sanjana Ramprasad, Kundan Krishna, Zachary Lipton, Byron Wallace

Leveraging Implicit Feedback from Deployment Data in Dialogue (2024.eacl-short)

Challenge:	Recent work has shown that large language models can generate zero-shot summaries without explicit supervision that are often comparable or even preferred to manually composed reference summary.
Approach:	They evaluate large language models (LLMs) that generate zero-shot summaries without explicit supervision that are often comparable to manual reference summary . they acquire annotations from domain experts to identify inconsistencies in summaires and categorize errors.
Outcome:	The proposed model outperforms fine-tuned models in biomedical articles and legal bills across specialized domains.

Copied to clipboard

Richard Yuanzhe Pang, Stephen Roller, Kyunghyun Cho, He He, Jason Weston

Natural Language Interfaces

Challenge:	Xu et al., 2023) and Bai ed., 2019) use crowdworkers to collect signals from natural dialogue episodes.
Approach:	They use the publicly released BlenderBot deployment data to extract signals from conversations to implicitly measure the quality of a machine-generated utterance.
Outcome:	The proposed model improves over baseline models, but some proxy signals can lead to undesirable generations.

Characterizing the Confidence of Large Language Model-Based Automatic Evaluation Metrics (2024.eacl-short)

Copied to clipboard

Rickard Stureborg, Dimitris Alikaniotis, Yoshi Suhara

Equipping Language Models with Tool Use Capability for Tabular Data Analysis in Finance (2024.eacl-short)

Challenge:	Recent studies have focused on using Large Language Models (LLMs) to evaluate NLP tasks automatically.
Approach:	They characterize LLM evaluators’ confidence in ranking candidate NLP models and develop a configurable Monte Carlo simulation method to compensate for loss of correlation.
Outcome:	The proposed method can reach 95% confidence rankings of candidate models with reasonable evaluation set sizes.

Copied to clipboard

Adrian Theuma, Ehsan Shareghi

Multimodality

Challenge:	Large language models (LLMs) have an array of reasoning capabilities but face limitations such as error propagation and hallucination.
Approach:	They propose to use a LLAMA-2 13B CHAT model to act as a task router and task solver to offload certain reasoning steps to external tools that are more suited for the task.
Outcome:	The proposed model improves by 35.2% and 5.06% over baseline models and strong GPT-3.5 results.

Commonsense-augmented Memory Construction and Management in Long-term Conversations via Context-aware Persona Refinement (2024.eacl-short)

Copied to clipboard

Hana Kim, Kai Ong, Seoyeon Kim, Dongha Lee, Jinyoung Yeo

Dialogue Systems & Conversational Agents

Challenge:	Memorizing and utilizing speakers’ personas is a common practice for response generation in long-term conversations, yet human-authored datasets often provide uninformative persona sentences that hinder response quality.
Approach:	They propose a framework that leverages commonsense-based persona expansion to address such issues in long-term conversations.
Outcome:	The proposed framework facilitates better response generation via human-like persona refinement.

Investigating the Potential of Task Arithmetic for Cross-Lingual Transfer (2024.eacl-short)

Copied to clipboard

Marinela Parović, Ivan Vulić, Anna Korhonen

Cross-Lingual Transfer

Challenge:	Massively multilingual Transformer-based models (MMTs) can learn representations which have a degree of cross-lingual alignment despite being trained using purely unsupervised objectives.
Approach:	They propose a modular approach to cross-lingual transfer using task arithmetic . they show that modularity can be achieved even with full model fine-tuning .
Outcome:	The proposed approach shows strong performance on multilingual benchmarks encompassing both high-resource and low-resourced languages.

On the Benefits of Fine-Grained Loss Truncation: A Case Study on Factuality in Summarization (2024.eacl-short)

Copied to clipboard

Lorenzo Jaime Flores, Arman Cohan

Evaluating Unsupervised Argument Aligners via Generation of Conclusions of Structured Scientific Abstracts (2024.eacl-short)

Challenge:	Text summarization and simplification are among the most widely used applications of NLP, but they are prone to hallucination due to training on unaligned data.
Approach:	They propose a loss-truncation approach to modify the standard log loss to adaptively remove noisy examples during training to improve model performance.
Outcome:	The proposed approach yields a considerable number of hallucinated entities on various datasets.

Copied to clipboard

Yingqiang Gao, Nianlong Gu, Jessica Lam, James Henderson, Richard Hahnloser

Over-Reasoning and Redundant Calculation of Large Language Models (2024.eacl-short)

Challenge:	Scientific abstracts provide a concise summary of research findings.
Approach:	They evaluate unsupervised approaches for extracting scientific arguments as aligned premise-conclusion pairs . they find mutual information outperforms other measures on this task .
Outcome:	The proposed methods outperform language models on the task of extracting scientific arguments from abstracts.

Copied to clipboard

Cheng-Han Chiang, Hung-yi Lee

Multimodal Fallacy Classification in Political Debates (2024.eacl-short)

Challenge:	Large language models (LLMs) can solve problems step-by-step, but it is unclear whether they know when to use CoT and whether they are always necessary.
Approach:	They propose to use LLMs to generate redundant calculations and reasoning on a manually constructed math QA dataset, GSM8K-Zero.
Outcome:	The proposed model generates redundant calculations and reasoning on a manually constructed math QA dataset, but it is unclear whether it is necessary to use CoT reasoning.

Copied to clipboard

Eleonora Mancini, Federico Ruggeri, Paolo Torroni

Text Classification

Challenge:	Recent advances in NLP suggest that some tasks, such as argument detection and relation classification, are better framed in a multimodal perspective.
Approach:	They propose to use multimodal argument mining to capture paralinguistic aspects of fallacious arguments.
Outcome:	The proposed multimodal argument mining improves argument detection and relation classification in political debates.

The Parrot Dilemma: Human-Labeled vs. LLM-augmented Data in Classification Tasks (2024.eacl-short)

Copied to clipboard

Anders Møller, Arianna Pera, Jacob Dalsgaard, Luca Aiello

Information Retrieval

Challenge:	Large Language Models (LLMs) are not perfect generalists as they often underperform traditional fine-tuning methods.
Approach:	They compare human-labeled and synthetically generated data in CSS classification tasks . they leverage large language models such as OpenAI's GPT-4 for zero-shot classification .
Outcome:	The proposed models perform better on human-labeled data than synthetically augmented models on rare classes within multi-class tasks.

Language Model Sentence Completion with a Parser-Driven Rhetorical Control Method (2024.eacl-short)

Copied to clipboard

Joshua Zingale, Jugal Kalita

”It’s how you do things that matters”: Attending to Process to Better Serve Indigenous Communities with Language Technologies (2024.eacl-short)

Challenge:	Large Language Models (LLMs) generate nonfacts and deviate from desired criteria for text generation.
Approach:	They propose a controlled text generation algorithm that enforces adherence toward specific rhetorical relations in an LLM sentence-completion context by a parser-driven decoding scheme.
Outcome:	The proposed method generates sentences that satisfy desired rhetorical relations in an LLM.

Copied to clipboard

Ned Cooper, Courtney Heldreth, Ben Hutchinson

Challenge:	Indigenous languages are historically under-served by natural language processing (NLP) but this is changing with the recent scaling of large multilingual models and an increased focus by the NLP community on endangered languages.
Approach:	They propose to build NLP technologies for Indigenous languages that should primarily serve Indigenous communities.
Outcome:	The proposed approach is based on interviews with 17 researchers working in or with Aboriginal and/or Torres Strait Islander communities on language technology projects in Australia.

Source Identification in Abstractive Summarization (2024.eacl-short)

Copied to clipboard

Yoshi Suhara, Dimitris Alikaniotis

From Partial to Strictly Incremental Constituent Parsing (2024.eacl-short)

Challenge:	Existing studies define input sentences that contain essential information in the generated summary as source sentences.
Approach:	They define input sentences that contain essential information in the generated summary as source sentences and analyze the source sentences to determine how abstractive summaries are made.
Outcome:	The proposed method performs well in abstractive settings, while similarity-based methods perform robustly in extractive settings.

Copied to clipboard

Ana Ezquerro, Carlos Gómez-Rodríguez, David Vilares

Challenge:	Incremental NLP aims to learn and adapt partial representations as information unfolds, but studies on incremental approaches have focused on non-incremental approaches.
Approach:	They build incremental constituent parsers that adhere to a strong definition of incrementality across languages and compare them against non-incremental and partially incremental models.
Outcome:	The proposed models are compared against non-incremental and partially incremental models.

Predict the Next Word: <Humans exhibit uncertainty in this task and language models _____> (2024.eacl-short)

Copied to clipboard

Evgenia Ilia, Wilker Aziz

A Prompt Response to the Demand for Automatic Gender-Neutral Translation (2024.eacl-short)

Challenge:	Language models (LMs) are statistical models trained to assign probability to human-generated text.
Approach:	They evaluate language models' ability to reproduce variability that humans exhibit in the ‘next word prediction’ task.
Outcome:	The language models are trained to assign probability to human-generated text . they exhibit low calibration to human uncertainty, and advise against it .

Copied to clipboard

Beatrice Savoldi, Andrea Piergentili, Dennis Fucci, Matteo Negri, Luisa Bentivogli

Interpreting Predictive Probabilities: Model Confidence or Human Label Variation? (2024.eacl-short)

Challenge:	Advancements in machine translation (MT) are hindered by the lack of dedicated parallel data, which are necessary to adapt MT systems to satisfy neutral constraints.
Approach:	They propose to use GPT-4 to generate GNTs that avoid bias and undue binary assumptions by comparing MT with the popular GPT-3 model.
Outcome:	The proposed model outperforms the existing model and provides valuable insights into the potential and challenges associated with prompting for neutrality.

Copied to clipboard

Joris Baan, Raquel Fernández, Barbara Plank, Wilker Aziz

Smaller Language Models are Better Zero-shot Machine-Generated Text Detectors (2024.eacl-short)

Challenge:	In modern NLP, neural networks are the de-facto standard to predict complex probability measures from available context.
Approach:	They propose to use a single predictive distribution to evaluate models with disentangled representations of uncertainty about predictions and uncertainty about human labels.
Outcome:	The proposed models are crucial for trustworthy and fair NLP systems, but exploiting a single distribution is limiting.

Copied to clipboard

Niloofar Mireshghallah, Justus Mattern, Sicun Gao, Reza Shokri, Taylor Berg-Kirkpatrick

CharSpan: Utilizing Lexical Similarity to Enable Zero-Shot Machine Translation for Extremely Low-resource Languages (2024.eacl-short)

Challenge:	Using large language models to detect machine generated text is difficult for humans to distinguish between human-written and machine-generated text.
Approach:	They propose to use one language model to detect machine-generated text produced by another language model in a zero-shot way.
Outcome:	The proposed methods can detect machine-generated text without additional training/data.

Copied to clipboard

Kaushal Maurya, Rahul Kejriwal, Maunendra Desarkar, Anoop Kunchukuttan

Robust Neural Machine Translation for Abugidas by Glyph Perturbation (2024.eacl-short)

Challenge:	Existing models for ELRLs lack parallel corpora and monolingual corporata . authors propose novel character-span noise argumentation model to facilitate cross-lingual transfer .
Approach:	They propose a character-span noise argumentation model to facilitate cross-lingual transfer . they use character-size noise argumentations to regularize training data of HRL .
Outcome:	The proposed model outperforms baselines on closely related HRL-ELRL pairs from three different language families.

Copied to clipboard

Hour Kaing, Chenchen Ding, Hideki Tanaka, Masao Utiyama

Translation Errors Significantly Impact Low-Resource Languages in Cross-Lingual Learning (2024.eacl-short)

Challenge:	Neural machine translation systems are vulnerable when trained on limited data.
Approach:	They propose to add noise to the training phase to increase robustness of NMT systems trained on limited data.
Outcome:	The proposed training strategy overcomes noise and improves robustness for low-resource tasks for abugida glyphs.

Copied to clipboard

Ashish Agrawal, Barah Fazili, Preethi Jyothi

Less is More for Long Document Summary Evaluation by LLMs (2024.eacl-short)

Challenge:	XNLI benchmarks use parallel versions of English evaluation sets in multiple target languages . a recent study found that translation errors exist in some low-resource languages resulting in incorrect estimates of cross-lingual transfer .
Approach:	They propose to measure the gap in performance between zero-shot evaluations on human-translated and machine-transcribed target text across multiple target languages.
Outcome:	The proposed benchmarks show that translation errors exist for Hindi and Urdu . the results corroborate previous studies that found translation errors in Hindi and urdu despite translation errors.

Copied to clipboard

Yunshu Wu, Hayate Iso, Pouya Pezeshkpour, Nikita Bhutani, Estevam Hruschka

Challenge:	Large Language Models (LLMs) have shown promising performance in summary evaluation tasks, yet they face challenges such as high computational cost and the Lost-in-the-middle problem where important information in the middle of long documents is often overlooked.
Approach:	They propose a novel method which extracts key sentences from a long source document and then evaluates the summary by prompting LLMs.
Outcome:	The proposed method significantly reduces evaluation costs and exhibits a higher correlation with human evaluations.

Leveraging ChatGPT in Pharmacovigilance Event Extraction: An Empirical Study (2024.eacl-short)

Copied to clipboard

Zhaoyue Sun, Gabriele Pergola, Byron Wallace, Yulan He

Information Extraction & Text Mining

Challenge:	pharmacovigilance event extraction is a key field of healthcare that involves identifying, evaluating, understanding, and preventing adverse effects.
Approach:	They investigate the ability of large language models (LLMs) to extract adverse events from medical text.
Outcome:	The proposed model performs reasonably well with demonstration selection strategies, but falls short compared to fully fine-tuned small models.

A Comparative Analysis of Conversational Large Language Models in Knowledge-Based Text Generation (2024.eacl-short)

Copied to clipboard

Phillip Schneider, Manuel Klettner, Elena Simperl, Florian Matthes

Extreme Fine-tuning: A Novel and Fast Fine-tuning Approach for Text Classification (2024.eacl-short)

Challenge:	Generating natural language text from graph-structured data is essential for conversational information seeking.
Approach:	They conduct an empirical analysis of conversational large language models in generating natural language text from semantic triples using a WebNLG dataset.
Outcome:	The proposed models improve their ability to generate natural language text from semantic triples using few-shot prompting, post-processing, and efficient fine-tuning techniques.

Copied to clipboard

Boonnithi Jiaramaneepinit, Thodsaporn Chay-intr, Kotaro Funakoshi, Manabu Okumura

Information Extraction & Text Mining

Challenge:	Existing methods for fine-tuning pre-trained models require massive computational resources and time.
Approach:	They propose a novel approach for fine-tuning a pre-trained model using backpropagation and an iterative extreme learning machine for training a classifier.
Outcome:	The proposed approach outperforms state-of-the-art approaches in training-time measurement and performance with comparable model performance.

Flow Matching for Conditional Text Generation in a Few Sampling Steps (2024.eacl-short)

Copied to clipboard

Vincent Hu, Di Wu, Yuki Asano, Pascal Mettes, Basura Fernando, Björn Ommer, Cees Snoek

Corpus-Steered Query Expansion with Large Language Models (2024.eacl-short)

Challenge:	Current diffusion models face multiple drawbacks including slow sampling, noise schedule sensitivity, and misalignment between training and sampling stages.
Approach:	They propose a method which leverages flow matching for conditional text generation.
Outcome:	The proposed method can generate text in a few steps by training with a novel anchor loss, alleviating the need for expensive hyperparameter optimization of the noise schedule prevalent in diffusion models.

Copied to clipboard

Yibin Lei, Yu Cao, Tianyi Zhou, Tao Shen, Andrew Yates

Defending Against Disinformation Attacks in Open-Domain Question Answering (2024.eacl-short)

Challenge:	Recent studies show query expansions generate hypothetical documents that answer queries as expansions.
Approach:	They propose a corpus-steered query expansion to promote incorporation of knowledge embedded within the corpus.
Outcome:	et al. analyzed corpus-based Query Expansion (CSQE) using LLMs to generate hypothetical documents that answer the query.

Copied to clipboard

Orion Weller, Aleem Khan, Nathaniel Weir, Dawn Lawrie, Benjamin Van Durme

Sentence Representations via Gaussian Embedding (2024.eacl-short)

Challenge:	Existing methods to defend against data poisoning attacks in open-domain question answering are lacking .
Approach:	They propose a method that uses query augmentation to find diverse passages that could answer the original question but are less likely to have been poisoned.
Outcome:	The proposed method provides gains of nearly 20% exact match across varying levels of data poisoning/knowledge conflicts.

Copied to clipboard

Shohei Yoda, Hayato Tsukagoshi, Ryohei Sasano, Koichi Takeda

Representation Learning

Challenge:	Sentence embeddings represent a sentence's meaning as a point in a vector space and primarily use symmetric measures such as the cosine similarity to measure the similarity between sentences, they cannot capture asymmetric relationships between two sentences, such as entailment and hierarchical relations.
Approach:	They propose a Gaussian-distribution-based contrastive learning framework for sentence embedding that can handle asymmetric inter-sentential relations and a similarity measure for identifying entailment relations.
Outcome:	The proposed framework performs comparable to that of previous methods on natural language inference tasks and estimates direction of entailment relations, which is difficult with point representations.

STORiCo: Storytelling TTS for Hindi with Character Voice Modulation (2024.eacl-short)

Copied to clipboard

Pavan Tankala, Preethi Jyothi, Preeti Rao, Pushpak Bhattacharyya

Multimodality

Challenge:	Existing datasets for read speech for Hindi lack expressiveness and character voice consistency.
Approach:	They propose to use a Hindi text-to-speech (TTS) dataset to train a multi-speaker model on the single-sector data and propose to improve expressiveness and character voice consistency.
Outcome:	The proposed model improves expressiveness and character voice consistency compared to the baseline single-speaker model.

Rethinking Loss Functions for Fact Verification (2024.eacl-short)

Copied to clipboard

Yuta Mukobara, Yutaro Shigeto, Masashi Shimbo

Reasoning

Challenge:	Existing objective functions for fact verification fail to capture heterogeneity among verdict classes . cross-entropy loss treats all misclassification types uniformly, which is problematic .
Approach:	They propose two task-specific objective functions that capture the heterogeneity among verdict classes . they use a dictionary-based objective function to classify Wikipedia sentences into three verdict classes.
Outcome:	The proposed objectives outperform the standard cross-entropy loss objective . the proposed objectives are combined with simple class weighting to overcome imbalance .

A Dataset for Metaphor Detection in Early Medieval Hebrew Poetry (2024.eacl-short)

Copied to clipboard

Michael Toker, Oren Mishali, Ophir Münz-Manor, Benny Kimelfeld, Yonatan Belinkov

Sentiment Analysis

Challenge:	a corpus of late antique and medieval Hebrew poetry is rich in metaphors and similes . scholars in the humanities need to distinguish between figurative and literal language .
Approach:	They present a corpus of late antique and medieval Hebrew poetry with expert annotations of metaphor . they hope to facilitate further research in this area .
Outcome:	The proposed dataset includes a corpus of late antique and medieval Hebrew poetry with expert annotations of metaphor.

SOCIALITE-LLAMA: An Instruction-Tuned Model for Social Scientific Tasks (2024.eacl-short)

Copied to clipboard

Gourab Dey, Adithya V Ganesan, Yash Kumar Lal, Manal Shah, Shreyashee Sinha, Matthew Matero, Salvatore Giorgi, Vivek Kulkarni, H. Schwartz

Pre-Training Methods for Question Reranking (2024.eacl-short)

Challenge:	Social science NLP tasks require large data to capture semantics and implicit pragmatics.
Approach:	They propose an open-source instruction tuning tool for social science NLP tasks that captures implicit pragmatic cues from text.
Outcome:	The proposed model matches or improves on a state-of-the-art, multi-task finetuned model on 80% of social tasks.

Copied to clipboard

Stefano Campese, Ivano Lauriola, Alessandro Moschitti

Information Retrieval

Challenge:	Existing methods for Question Answering to search for semantically similar questions are not suitable for new questions.
Approach:	They propose an unsupervised method for retrieving and ranking questions . they use a question retrieval model and a selection model to rerank questions based on their relevance .
Outcome:	The proposed method achieves state-of-the-art performance on QRC and Quora-match datasets . it provides better and cheaper access to answers than the system generated them .

Dynamic Masking Rate Schedules for MLM Pretraining (2024.eacl-short)

Copied to clipboard

Zachary Ankner, Naomi Saphra, Davis Blalock, Jonathan Frankle, Matthew Leavitt