Findings of the Association for Computational Linguistics: EACL 2023
Using Punctuation as an Adversarial Attack on Deep Learning-Based NLP Systems: An Empirical Study (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies show that insertions of a few symbols are a general attack mechanism, but grammar checks do not mitigate them. |
| Approach: | They propose to use punctuation insertions as adversarial attacks on NLP systems to create a toolbox of methods to attack models while also pointing out flaws. |
| Outcome: | The results show that punctuation insertions outperform word-level attacks in settings with a limited number of word synonyms and queries to the victim’s model. |
Self-Supervised Unimodal Label Generation Strategy Using Recalibrated Modality Representations for Multimodal Sentiment Analysis (2023.findings-eacl)
Copied to clipboard
| Challenge: | Multimodal sentiment analysis (MSA) has gained much attention over the last few years due to a lack of unimodal annotations in benchmark datasets. |
| Approach: | They propose a framework which integrates multimodal and unimodal tasks to optimize learning representations from multimodal data. |
| Outcome: | The proposed model learns to weight features differently based on features of other modalities and auto-generates unimodal annotations via a unimodule. |
Fighting FIRe with FIRE: Assessing the Validity of Text-to-Video Retrieval Benchmarks (2023.findings-eacl)
Copied to clipboard
Pedro Rodriguez, Mahmoud Azab, Becka Silvert, Renato Sanchez, Linzy Labson, Hardik Shah, Seungwhan Moon
| Challenge: | Existing benchmarks for text-to-video retrieval are incomplete, resulting in false negatives . a recent state-of-the-art model gains 25% recall points, but this is not the case for TVR. |
| Approach: | They propose to retire video captioning datasets as TVR benchmarks . they propose to annotate and release additional caption-video pairs to mitigate this flaw . |
| Outcome: | The proposed method fails to accurately reflect reality, despite lack of purpose-built benchmarks. |
Improving Numeracy by Input Reframing and Quantitative Pre-Finetuning Task (2023.findings-eacl)
Copied to clipboard
| Challenge: | Innumeracy is a problem in pretrained language models, but it is not discussed in this paper . Numerals are an indispensable part of narratives and provide much fine-grained information. |
| Approach: | They propose a method to solve innumeracy in pretrained language models by exploring the notation of numbers. |
| Outcome: | The proposed method improves performance in three benchmark datasets containing quantitative-related tasks. |
Visualize Before You Write: Imagination-Guided Open-Ended Text Generation (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing tools for text-to-image synthesis can visualize machine imaginations for a given context. |
| Approach: | They propose a framework that uses machine-generated images to guide language models in open-ended text generation. |
| Outcome: | The proposed framework is effective on open-ended text generation tasks while showing minor degeneration. |
ImaginE: An Imagination-Based Automatic Evaluation Metric for Natural Language Generation (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing evaluation methods for natural language generation rely on token-level or embedding-level comparisons with text references. |
| Approach: | They propose to use text-to-image generator to generate an image as the embodied imagination for the text snippet and compute the imagination similarity using contextual embeddings. |
| Outcome: | The proposed metric improves existing evaluation metrics’ correlations with human similarity judgments in both reference-based and reference-free scenarios. |
Entity-Aware Dual Co-Attention Network for Fake News Detection (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing models for fake news detection are limited in their ability to detect it from different aspects. |
| Approach: | They propose a Dual Co-Attention Network (Dual-CAN) for fake news detection that takes news content, social media replies, and external knowledge into consideration. |
| Outcome: | The proposed model outperforms existing models in two benchmark datasets. |
CIKQA: Learning Commonsense Inference with a Unified Knowledge-in-the-loop QA Paradigm (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing commonsense reasoning datasets target different knowledge types, modalities, and formats, but how to help machines acquire and infer over commonsensical knowledge is still unclear. |
| Approach: | They propose a commonsense reasoning benchmark to motivate commonsensing progress from two perspectives: (1) Evaluating whether models can distinguish knowledge quality by predicting if the knowledge is enough to answer the question or not. |
| Outcome: | The proposed model outperforms existing models in evaluating their generalization capabilities across tasks while demonstrating that distinguishing knowledge quality remains challenging for current models. |
Data-Efficient Methods For Improving Hate Speech Detection (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for hate speech detection are data-hungry and require large datasets. |
| Approach: | They propose an input-level data augmentation technique EasyMix to improve hate speech detection in english and multilingual datasets. |
| Outcome: | The proposed method improves the performance across english and multilingual datasets by 1% and 2-8%. |
Learning the Effects of Physical Actions in a Multi-modal Environment (2023.findings-eacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are trained on large corpora of disembodied texts. |
| Approach: | They propose a multi-modal task of predicting the outcomes of actions solely from realistic sensory inputs (images and text). They extend an LLM to model latent representations of objects to better predict action outcomes in an environment. |
| Outcome: | The proposed model can capture commonsense when augmented with visual information and generalize and learn commonsensical reasoning better. |
FVQA 2.0: Introducing Adversarial Samples into Fact-based Visual Question Answering (2023.findings-eacl)
Copied to clipboard
| Challenge: | Fact-based Visual Question Answering (FVQA) is a visual question answering task that requires information retrieval using common sense knowledge graphs to answer. |
| Approach: | They propose a new test question with adversarial variants to address this imbalance by using a KB-VQA dataset that is small and contains only one answer per question. |
| Outcome: | The proposed version reduces the vulnerability of the original FVQA dataset without human annotations. |
Revisiting Intermediate Layer Distillation for Compressing Language Models: An Overfitting Perspective (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for knowledge distillation (KD) are prone to overfitting to training datasets . recent advances in NLP have shown that using PLMs such as BERT and RoBERTa on downstream tasks is effective. |
| Approach: | They propose a consistency-regularized knowledge distillation method which mitigates overfitting of existing methods. |
| Outcome: | The proposed method outperforms existing methods on the GLUE benchmark and synthetic datasets. |
Implicit Temporal Reasoning for Evidence-Based Fact-Checking (2023.findings-eacl)
Copied to clipboard
| Challenge: | Temporal reasoning is implicit since models learn from data how to leverage temporal information. |
| Approach: | They propose to ground claims and associated evidence on shared timelines using publication dates and time expressions extracted from their text. |
| Outcome: | The proposed model outperforms existing models that explicitly model temporal relations between evidence and the document by up to 9% Micro F1 and 15% Macro F1 on the MultiFC dataset. |
Active PETs: Active Data Annotation Prioritisation for Few-Shot Claim Verification with Pattern Exploiting Training (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent work on few-shot classification has addressed the issue of data prioritisation of unlabelled data. |
| Approach: | They propose a weighted approach that uses a set of pattern-exploiting training models to actively select unlabelled data as candidates for annotation. |
| Outcome: | The proposed approach shows consistent improvement over baseline methods on two technical fact-checking datasets and using six different pretrained language models. |
Plan-then-Seam: Towards Efficient Table-to-Text Generation (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent work explicitly decomposes the generation process into content planning and surface generation stages, employing two autoregressive networks for them respectively. |
| Approach: | They propose a non-parallelelizable table-to-text model that produces outputs in parallel with one network. |
| Outcome: | The proposed model achieves 3.0 5.6 times speedup for inference time, reducing 50% parameters, while maintaining as least comparable performance against strong two-stage table-to-text competitors. |
A corpus of metaphors as register markers (2023.findings-eacl)
Copied to clipboard
| Challenge: | Using corpus annotation, we show huge differences in metaphor usage between different registers and specific properties of registers. |
| Approach: | They present their work on corpus annotation for metaphor in germany . they focus on metaphors that can serve as register markers and be reliably indentified . |
| Outcome: | The proposed corpus annotations show huge differences in metaphor usage between different registers and specific properties of registers. |
Translate First Reorder Later: Leveraging Monotonicity in Semantic Parsing (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing approaches that model alignments between sentences fail at compositional generalization tasks, resulting in a resurgence of such approaches. |
| Approach: | They propose a two-step approach that first translates input sentences monotonically and then reorders them to obtain the correct output. |
| Outcome: | The proposed approach improves compositional generalization over existing models and other approaches that exploit gold alignment annotations. |
PePe: Personalized Post-editing Model utilizing User-generated Post-edits (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing neural machine translation models ignore personal style in their translations, but in these studies the definition of personal style is over-simplified. |
| Approach: | They propose a personalized automatic post-editing framework that generates sentences considering distinct personal behaviors by collecting post-edited data from a live machine translation system and combining a discriminator module and user-specific parameters. |
| Outcome: | The proposed model outperforms baseline models on four different metrics including BLEU, TER, YiSi-1, and human evaluation. |
Infusing Context and Knowledge Awareness in Multi-turn Dialog Understanding (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing work on multi-turn dialog understanding does not model multi-turned dynamics, instead leaving them for updating dialog states only. |
| Approach: | They propose to equip a BERT-based framework with knowledge and context awareness to model multi-turn dialog dynamics by detecting intents and slots within each user utterance. |
| Outcome: | The proposed framework can detect intents and slots within a dialog and extract key slot information as 'semantic frames' however, humans usually associate relevant background knowledge with the current dialog contexts to better illustrate slot semantics revealed from word connotations . |
MCoNaLa: A Benchmark for Code Generation from Multiple Natural Languages (2023.findings-eacl)
Copied to clipboard
| Challenge: | MCoNaLa benchmarks natural language code generation in languages that are not native to English. |
| Approach: | They propose to benchmark natural language code generation from natural language commands extending beyond English by using a multilingual dataset. |
| Outcome: | The proposed dataset compares natural language commands with code generation systems in three languages. |
Augmenting pre-trained language models with audio feature embedding for argumentation mining in political debates (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies on the integration of multimodality with text and audio in natural language processing tasks have focused on the use of image and text for emotion recognition, fake news detection and document image classification. |
| Approach: | They propose to integrate audio features with text in a task of argumentation mining using a previously reported dataset and an audio-enhanced version. |
| Outcome: | The proposed model outperforms text-based models on a dataset of 28,850 utterances on 'argumentation mining' with limited data. |
Improving Retrieval Augmented Neural Machine Translation by Controlling Source and Fuzzy-Match Interactions (2023.findings-eacl)
Copied to clipboard
| Challenge: | a general-domain model has access to customer or domain specific parallel data at inference time, but not during training. |
| Approach: | They propose a zero-shot adaptation approach where a general-domain model has access to customer or domain specific parallel data at inference time, but not during training. |
| Outcome: | The proposed architecture outperforms existing architectures in two language pairs . it consistently improves BLEU across language pair, domain, and number k of fuzzy matches . |
CALM-Bench: A Multi-task Benchmark for Evaluating Causality-Aware Language Models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent advances in foundation language models have shown the efficacy of pre-trained models across diverse QA tasks. |
| Approach: | They propose a multi-task benchmark for evaluating causality-aware language models to unify causal QA research. |
| Outcome: | The proposed model outperforms single-task fine-tuned models on the CALM-Bench tasks. |
ezCoref: Towards Unifying Annotation Guidelines for Coreference Resolution (2023.findings-eacl)
Copied to clipboard
Ankita Gupta, Marzena Karpinska, Wenlong Zhao, Kalpesh Krishna, Jack Merullo, Luke Yeh, Mohit Iyyer, Brendan O’Connor
| Challenge: | Existing datasets vary in definition of coreferences and are curated for linguistic experts. |
| Approach: | They propose to use ezCoref to create a crowdsourcing-friendly coreference annotation methodology that teaches annotators only cases that are treated similarly across existing datasets. |
| Outcome: | The proposed method reannotates 240 passages from seven existing english coreference datasets while teaching annotators only cases that are treated similarly across them. |
PREME: Preference-based Meeting Exploration through an Interactive Questionnaire (2023.findings-eacl)
Copied to clipboard
Negar Arabzadeh, Ali Ahmadvand, Julia Kiseleva, Yang Liu, Ahmed Hassan Awadallah, Ming Zhong, Milad Shokouhi
| Challenge: | Recent studies show that providing meeting summaries does not align with current approaches to document summarization. |
| Approach: | They propose a framework for generating questionnaires for preference-based meeting exploration . they measure how much questions are answerable to ensure factual correctness . |
| Outcome: | The proposed framework provides a list of suggested questions reflecting user preferences . it measures how much questions are answerable to ensure factual correctness . |
Sentence Identification with BOS and EOS Label Combinations (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for preprocessing sentences only use the end of the sentence (EOS) however, real-world texts often contain non-sentential units (NSUs) such as metadata, sentence fragments, etc. |
| Approach: | They propose a task of sentence identification where the goal is to identify SUs while excluding NSUs in a given text. |
| Outcome: | The proposed method outperforms baselines which only use EOS labels on the sentence identification task. |
Gauging the Gap Between Human and Machine Text Simplification Through Analytical Evaluation of Simplification Strategies and Errors (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent studies on text simplification have focused on lexical and syntactic simplification, but few studies have attempted to assess what kind of editing operations are performed by the systems in concrete terms. |
| Approach: | They develop an analytical evaluation framework for neural text simplification systems that includes fine-grained taxonomies of simplification strategies and errors. |
| Outcome: | The framework was used to evaluate TS models produced by human editors and multiple neural TS systems and found that human editors perform deletions and local substitutions while excessively omitting important information. |
Bridging the Gap between Pre-Training and Fine-Tuning for Commonsense Generation (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods focusing on this task usually concatenate the concatened concepts words as the inputs of a pre-trained language model (PLM) however, in pre-training, the input is often corrupted sentences with correct word order. |
| Approach: | They propose a two-stage framework to improve the ability of pre-trained language models to deal with masked sentences with incorrect word order and a special token to make the input distribution more similar to the one used in pre-training. |
| Outcome: | The proposed method is able to generate a sentence containing all given concepts and correctly describe the relations between concepts. |
LED: A Dataset for Life Event Extraction from Dialogs (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for extracting life events from conversations are limited. |
| Approach: | They propose a dataset containing fine-grained life event annotations on conversational data. |
| Outcome: | The proposed dataset combines three information extraction frameworks to extract life events from conversations. |
Reading and Reasoning over Chart Images for Evidence-based Automated Fact-Checking (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies on detecting manipulated or fake images focus on identifying manipulated and false images. |
| Approach: | They propose a novel task, chart-based fact-checking, to validate textual, structural and visual information of charts to determine the veracity of textual claims. |
| Outcome: | The proposed model outperforms existing models in vision-language tests with 63.8% accuracy. |
Causal Reasoning of Entities and Events in Procedural Texts (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing work on entity state tracking or event reasoning is limited to procedural texts. |
| Approach: | They propose a benchmark for causal reasoning of event plausibility and entity states . they represent entities as programming languages while prompting language models . |
| Outcome: | The proposed model outperforms existing models on human reasoning and event reasoning. |
Few-Shot Structured Policy Learning for Multi-Domain and Multi-Task Dialogues (2023.findings-eacl)
Copied to clipboard
| Challenge: | Reinforcement learning is widely adopted to model dialogue managers in task-oriented dialogues, but the user simulator provided by state-of-the-art dialogue frameworks are only rough approximations of human behaviour. |
| Approach: | They propose to use structured policies to improve sample efficiency when learning on multi-domain and multi-task environments. |
| Outcome: | The proposed policies improve sample efficiency and performance on multi-domain and multi-task environments. |
Transfer Knowledge from Natural Language to Electrocardiography: Can We Detect Cardiovascular Disease Through Language Models? (2023.findings-eacl)
Copied to clipboard
Jielin Qiu, William Han, Jiacheng Zhu, Mengdi Xu, Michael Rosenberg, Emerson Liu, Douglas Weber, Ding Zhao
| Challenge: | Recent advances in Large Language Models (LLMs) have shown powerful ability in various downstream applications. |
| Approach: | They propose an approach for cardiovascular disease diagnosis and automatic ECG diagnosis report generation. |
| Outcome: | The proposed approach generates high-quality cardiac diagnosis reports and achieves competitive zero-shot classification performance even compared with supervised baselines. |
Practical Takes on Federated Learning with Pretrained Language Models (2023.findings-eacl)
Copied to clipboard
| Challenge: | federated learning with pretrained language models for language tasks entails data privacy constraints when learning from diverse data domains. |
| Approach: | They propose to use pretrained language models to learn from diverse data domains . they elaborate hypotheses over the components in federated NLP architectures based on three tasks . |
| Outcome: | The proposed model can generalize by adapting to the different domains. |
Paper Bullets: Modeling Propaganda with the Help of Metaphor (2023.findings-eacl)
Copied to clipboard
| Challenge: | We hypothesize that it can be beneficial to model propaganda and metaphor together . we identify propaganda using loaded language and name-calling . |
| Approach: | They hypothesize that it can be beneficial to model propaganda and metaphor together . they use two datasets to identify propaganda techniques in news articles and memes . |
| Outcome: | The proposed model improves performance for the two most common propaganda techniques, especially loaded language and name-calling. |
Lexical Semantics with Large Language Models: A Case Study of English “break” (2023.findings-eacl)
Copied to clipboard
| Challenge: | Large neural language models (LLMs) can be powerful tools for research in lexical semantics. |
| Approach: | They argue that large neural language models can be powerful tools for research in lexical semantics by capturing known sense distinctions and identifying informative new sense combinations. |
| Outcome: | The proposed models capture many of the sense distinctions found in the English verb break and can be used to identify informative new sense combinations for further analysis. |
SWING: Balancing Coverage and Faithfulness for Dialogue Summarization (2023.findings-eacl)
Copied to clipboard
Kung-Hsiang Huang, Siffi Singh, Xiaofei Ma, Wei Xiao, Feng Nan, Nicholas Dingwall, William Yang Wang, Kathleen McKeown
| Challenge: | Existing approaches to dialogue summarization rely on features of conversation data. |
| Approach: | They propose to use natural language inference models to improve coverage and faithfulness . they use fine-grained training signals to encourage model to generate missing content . |
| Outcome: | The proposed model achieves higher faithfulness and coverage while maintaining conciseness compared to prior methods. |
Language-Aware Multilingual Machine Translation with Self-Supervised Learning (2023.findings-eacl)
Copied to clipboard
| Challenge: | Multilingual machine translation (MMT) is a challenging multitask optimization problem because of lack of a framework to learn language-specific parameters. |
| Approach: | They propose a self-supervised learning task that denies monolingual data to MMT . they then propose 'intra-distillation' task that co-trains with MMT task . |
| Outcome: | The proposed approach outperforms three state-of-the-art methods on 8-language and 15-language benchmarks. |
Cloze Quality Estimation for Language Assessment (2023.findings-eacl)
Copied to clipboard
| Challenge: | Cloze tests are widely used in language proficiency tests, but they suffer from low quality and low reliability. |
| Approach: | They propose a task to evaluate whether a cloze test is of sufficient "high-quality" they use a dataset that includes English clozing tests and corresponding evaluations by native English speakers. |
| Outcome: | The proposed method could contribute to the CQE task, but the task is still challenging. |
Bag of Tricks for In-Distribution Calibration of Pretrained Transformers (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained language models (PLMs) often predict over-confidently. |
| Approach: | They propose to use ensemble learning and data augmentation to improve confidence calibration for PLMs by combining calibration techniques with a trade-off between accuracy and classification. |
| Outcome: | The proposed calibration method improves classification accuracy and confidence in pre-trained language models by combining several calibration techniques. |
Fine-Tuning Deteriorates General Textual Out-of-Distribution Detection by Distorting Task-Agnostic Features (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for detecting out-of-distribution inputs are underexplored . detecting semantic and non-semantic shifts is difficult for pre-tuned pre-trainers . |
| Approach: | They propose a general OOD score that integrates confidence scores from task-agnostic and task-specific representations to improve detecting semantic and non-semantic shifts. |
| Outcome: | The proposed method improves on two cross-task benchmarks with semantic and non-semantic shifts. |
A Question of Style: A Dataset for Analyzing Formality on Different Levels (2023.findings-eacl)
Copied to clipboard
| Challenge: | Using machine learning, we can produce contextually appropriate language. |
| Approach: | They present a dataset of German sentence-level formality assessed on a continuous informal-formal scale. |
| Outcome: | The proposed dataset compares sentences from a wide range of genres assessed on a continuous informal-formal scale. |
Task-specific Compression for Multi-task Language Models using Attribution-based Pruning (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing compression methods for multi-task language models use large number of parameter parameters even when performing only a specific task. |
| Approach: | They propose a training-free compression method for multi-task language models using pruning method . they use an attribution method to determine which neurons are essential for performing a specific task . |
| Outcome: | The proposed method outperforms baseline pruning methods on six widely-used datasets. |
Zero-shot Transfer of Article-aware Legal Outcome Classification for European Court of Human Rights Cases (2023.findings-eacl)
Copied to clipboard
| Challenge: | Legal Judgment Prediction (LJP) is a classification task that uses textual descriptions of case facts as the input. |
| Approach: | They propose to use legal reasoning to map article text to specific case fact text to improve the model's generalization to zero-shot settings. |
| Outcome: | The proposed model outperforms straightforward fact classification and improves zero-shot transfer performance. |
Abstractive Document Summarization with Summary-length Prediction (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing abstractive summarization models do not consider summarizing-specific information such as the target summary length. |
| Approach: | They propose a method for enabling a model to understand summarization-specific information by predicting the summary length in the encoder and generating a summary of the predicted length in fine-tuning. |
| Outcome: | The proposed method improves ROUGE scores on the WikiHow, NYT, and CNN/DM datasets. |
Hierarchical Label Generation for Text Classification (2023.findings-eacl)
Copied to clipboard
| Challenge: | None Hierarchical text classification (HTC) aims to assign the most relevant labels with their structure for a given document. |
| Approach: | They propose a method that captures the label hierarchy for real-world classification applications by using a taxonomic hierarchy. |
| Outcome: | The proposed method can generate unseen labels in subword level. |
Active Learning for Multilingual Semantic Parser (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing multilingual semantic parsing datasets are limited in translation effort due to data imbalance. |
| Approach: | They propose a first active learning procedure for multilingual semantic parsing (AL-MSP) it selects only a subset from existing datasets to be translated, they propose . |
| Outcome: | The proposed method significantly reduces translation costs with ideal selection methods. |
Joint Word and Morpheme Segmentation with Bayesian Non-Parametric Models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Language documentation often requires segmenting transcriptions of utterances into words and morphemes . a long tradition of nonparametric Bayesian models is used to handle these tasks . |
| Approach: | They propose a Bayesian model for simultaneously segmenting utterances at two levels . they use two under-resourced languages to better understand the value of weak supervision . |
| Outcome: | The proposed model can be used to identify language documents with weak supervision. |
Cross-Lingual Transfer of Cognitive Processing Complexity (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent studies indicate that multilingual language models utilize structural similarities between languages to facilitate cross-lingual transfer. |
| Approach: | They propose a multilingual model that uses structural similarities between languages to facilitate cross-lingual transfer by a meaningful bias towards sentence length and cross-linguistic differences. |
| Outcome: | The proposed model can predict varied patterns for 13 languages, despite being fine-tuned only on English data. |
Does Transliteration Help Multilingual Language Modeling? (2023.findings-eacl)
Copied to clipboard
| Challenge: | Script diversity presents a challenge to Multilingual Language Models by reducing lexical overlap . Script diversification can be used to improve performance of MLLMs by transliterating closely related languages to a common script. |
| Approach: | They empirically measure the effect of transliteration on MLLMs by focusing on Indic languages . they find that transliterations benefit low-resource languages without negatively affecting high-resourced ones . |
| Outcome: | The proposed transliteration-based model learns sentences that are more similar across languages. |
A Multilingual Dataset of Racial Stereotypes in Social Media Conversational Threads (2023.findings-eacl)
Copied to clipboard
Tom Bourgeade, Alessandra Teresa Cignarella, Simona Frenda, Mario Laurent, Wolfgang Schmeisser-Nieto, Farah Benamara, Cristina Bosco, Véronique Moriceau, Viviana Patti, Mariona Taulé
| Challenge: | a new corpus-based study addresses racial stereotypes in social media conversations . a multilingual corpus of rhs is used to investigate how they are spread . |
| Approach: | They propose a corpus-based method for multilingual racial stereotype identification in social media conversational threads. |
| Outcome: | The proposed method sheds light on how racial hoaxes are spread and allows identification of negative stereotypes that reinforce them. |
Detecting Contextomized Quotes in News Headlines by Contrastive Learning (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing research has found a link between the use of direct quotations and fake news. |
| Approach: | They propose a contrastive learning framework that allows embedding news quotes based on domain-driven positive and negative samples to identify such an editorial strategy. |
| Outcome: | The proposed framework maximizes the semantic similarity between the headline quote and the matched quote in the body text while minimizing similarity for other unmatched quotes in the same or other articles. |
Zero-Shot On-the-Fly Event Schema Induction (2023.findings-eacl)
Copied to clipboard
| Challenge: | a new approach to event processing uses large language models to generate source documents that can be curated without manual data collection. |
| Approach: | They propose a framework that generates a graphical representation of events in documents . they show that the model is more complete than previous supervised methods . |
| Outcome: | The proposed model is more complete than human-curated schemas in most scenarios. |
BanglaNLG and BanglaT5: Benchmarks and Resources for Evaluating Low-Resource Natural Language Generation in Bangla (2023.findings-eacl)
Copied to clipboard
| Challenge: | 'BanglaNLG' is a comprehensive benchmark for evaluating natural language generation models in Bangla, a widely spoken yet low-resource language. |
| Approach: | They propose to aggregate six conditional text generation tasks under the BanglaNLG benchmark and introduce a new dataset on dialogue generation in the process. |
| Outcome: | The proposed model outperforms several multilingual models by 9% absolute gain and 32% relative gain on all of these tasks. |
It’s about Time: Rethinking Evaluation on Rumor Detection Benchmarks using Chronological Splits (2023.findings-eacl)
Copied to clipboard
| Challenge: | Current rumor detection benchmarks use random splits as training, development and test sets which results in topical overlaps. |
| Approach: | They propose to use chronological rather than random splits for rumor classification . they propose to always use chronological splits to minimize topical overlaps . |
| Outcome: | The proposed model overestimates performance on four popular rumor detection benchmarks considering chronological instead of random splits. |
MUTANT: A Multi-sentential Code-mixed Hinglish Dataset (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods to identify code-mixed text are difficult to scale effectively and efficiently on multi-sentential data. |
| Approach: | They propose to identify multi-sentential code-mixed text (MCT) from multilingual articles using a token-level language-aware pipeline. |
| Outcome: | The proposed dataset includes 67k articles with 85k identified Hinglish MCTs. |
Bridging the Gap between Native Text and Translated Text through Adversarial Learning: A Case Study on Cross-Lingual Event Extraction (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent research in cross-lingual learning has found that combining large-scale pretrained multilingual language models with machine translation can yield good performance. |
| Approach: | They propose a model architecture that jointly encodes a source language input sentence with its translation to the target language during training and takes a target language sentence with it as input during evaluation. |
| Outcome: | The proposed model architecture can integrate machine translation to improve event extraction while adding machine-translated data yields unstable performance due to representational gap. |
Scalable Prompt Generation for Semi-supervised Learning with Language Models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Prompt-based learning methods in semi-supervised learning (SSL) settings have been shown to be effective on multiple natural language understanding datasets and tasks. |
| Approach: | They propose to use a set of prompt tokens to create diverse prompt models and a varying number of soft prompt token to encourage language models to learn different prompts. |
| Outcome: | The proposed method achieves the best average accuracy of 71.5% in different few-shot learning settings. |
Novel Feature Discovery for Task-Oriented Dialog Systems (2023.findings-eacl)
Copied to clipboard
| Challenge: | Prior work on novelty detection limits the scope of features represented by novel single intents to those represented by multiple user-perceived fine-grained features belonging to the same intent. |
| Approach: | They propose to use a feature discovery technique to discover novel features from user utterances rather than single intent discovery to classify them into slots. |
| Outcome: | The proposed approach consistently detects novel features from user utterances on two datasets. |
Context Generation Improves Open Domain Question Answering (2023.findings-eacl)
Copied to clipboard
Dan Su, Mostofa Patwary, Shrimai Prabhumoye, Peng Xu, Ryan Prenger, Mohammad Shoeybi, Pascale Fung, Anima Anandkumar, Bryan Catanzaro
| Challenge: | Existing closed-book question answering methods do not fully exploit the parameterized knowledge. |
| Approach: | They propose a closed-book QA framework which uses a coarse-to-fine approach to extract the relevant knowledge and answer a question. |
| Outcome: | The proposed method outperforms open-book QA methods on three QA benchmarks. |
RedHOT: A Corpus of Annotated Medical Questions, Experiences, and Claims on Social Media (2023.findings-eacl)
Copied to clipboard
| Challenge: | Social media platforms such as Reddit are vulnerable to misinformation and disinformation. |
| Approach: | They propose a method to automatically derive (noisy) supervision for retrieval of trustworthy evidence relevant to a given claim made on social media. |
| Outcome: | The proposed method outperforms baseline models in the retrieval task performed by medical doctors. |
Paparazzi: A Deep Dive into the Capabilities of Language and Vision Models for Grounding Viewpoint Descriptions (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing language and vision models can be used for language understanding in 3D environments . however, existing models lack specific properties and biases that limit their performance . |
| Approach: | They propose a framework that uses a camera to generate images from different viewpoints and evaluate them in terms of their similarity to natural language descriptions. |
| Outcome: | The proposed model performs poorly on most canonical views and fine-tunes using hard negative sampling and random contrasting yields good results even under conditions with little available training data. |
PLACES: Prompting Language Models for Social Conversation Synthesis (2023.findings-eacl)
Copied to clipboard
Maximillian Chen, Alexandros Papangelis, Chenyang Tao, Seokhwan Kim, Andy Rosenbaum, Yang Liu, Zhou Yu, Dilek Hakkani-Tur
| Challenge: | Currently, collecting high quality conversational data is expensive and infeasible for many applications . a promising direction is to generate synthetic dialogues by prompting large language models . |
| Approach: | They propose to use expert-written conversations as in-context examples to generate synthetic dialogues by prompting large language models. |
| Outcome: | The proposed approach is generalizable to multi-party conversations, compared to human-collected conversations. |
FedPerC: Federated Learning for Language Generation with Personal and Context Preference Embeddings (2023.findings-eacl)
Copied to clipboard
| Challenge: | federated learning is a decentralized learning paradigm that assumes no access to a large labeled dataset and instead leverages averaged parameter updates across all users of the system. |
| Approach: | They propose a method to personalize federated learning with personal embeddings and shared context embeddables. |
| Outcome: | The proposed approach achieves 50% improvement in test-time perplexity using 0.001% of the memory required by baseline approaches and greater sample- and compute-efficiency. |
A Neural CRF-based Hierarchical Approach for Linear Text Segmentation (2023.findings-eacl)
Copied to clipboard
Inderjeet Nair, Aparna Garimella, Balaji Vasan Srinivasan, Natwar Modani, Niyati Chhaya, Srikrishna Karanam, Sumit Shekhar
| Challenge: | Existing methods to segment unformatted text and transcripts explicitly train to predict segment boundaries, but they fail to provide a large annotated dataset. |
| Approach: | They propose a method to generate hierarchical segmentation structures based on Wikipedia annotations by using a neural conditional random field. |
| Outcome: | The proposed method outperforms or achieves competitive performance when compared to previous state-of-the-art algorithms. |
MultiFin: A Dataset for Multilingual Financial NLP (2023.findings-eacl)
Copied to clipboard
| Challenge: | Multilingual models are needed to process financial text, which is produced across the world and requires a large dataset. |
| Approach: | They propose to annotate a publicly available financial dataset using a hierarchical label structure and an annotation schema based on a real-world application. |
| Outcome: | The proposed model can be used in high-resource languages, but there is room for improvement in low-resourced languages. |
MLASK: Multimodal Summarization of Video-based News Articles (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent studies on multimodal summarization have shown that the benefits of pre-training and using additional modalities in the input are not orthogonal. |
| Approach: | They propose to use a dataset to train a multimodal article summarization model by automatically crawling several news websites. |
| Outcome: | The proposed dataset can be used to model multimodal summarization by training a Transformer-based neural model. |
Going beyond research datasets: Novel intent discovery in the industry setting (2023.findings-eacl)
Copied to clipboard
| Challenge: | Novel intent discovery automates grouping of similar messages to identify previously unknown intents. |
| Approach: | They propose to use question-only data to improve the intent discovery pipeline . they propose to utilize conversational structure of real-life datasets for clustering . |
| Outcome: | The proposed method gives 33pp performance boost over state-of-the-art model for question only . it also gives 13pp performance increase over the naive baseline model . |
DATScore: Evaluating Translation with Data Augmented Translations (2023.findings-eacl)
Copied to clipboard
| Challenge: | Experimental results show that DATScore correlates better with human meta-evaluations than the other recent state-of-the-art metrics. |
| Approach: | They propose to use data augmented translations to improve the evaluation of machine translations by using two new scoring strategies. |
| Outcome: | The proposed metric improves on 3 NLG tasks other than translation. |
How do decoding algorithms distribute information in dialogue responses? (2023.findings-eacl)
Copied to clipboard
| Challenge: | Using different decoding algorithms, we find that human dialogue generation is beneficial for adherence to the Uniform Information Density principle. |
| Approach: | They investigate whether decoding algorithms implicitly follow the Uniform Information Density principle by distributing information evenly in utterances. |
| Outcome: | The proposed method encourages non-uniform responses, but under low/high surprisal conditions, resulting in poor quality responses. |
Benchmarking Long-tail Generalization with Likelihood Splits (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods to test out of distribution generalization have their own strengths and weaknesses. |
| Approach: | They propose a method to create challenging benchmarks that require generalizing to the tail of the distribution by re-splitting existing datasets. |
| Outcome: | The proposed approach can be customized to construct meaningful splits for a wide range of tasks. |
Exploring Enhanced Code-Switched Noising for Pretraining in Neural Machine Translation (2023.findings-eacl)
Copied to clipboard
| Challenge: | Multilingual pretraining approaches to denoise synthetic code-switched data have shown that they generate the noise using non-contextual, one-to-one word translations obtained from lexicons. |
| Approach: | They propose an approach where contextual, many-to-many word translations are generated using a ‘base’ NMT model. |
| Outcome: | The proposed approach improves on 3 different language families and shows that small models can perform better than massive models like mBART50 and mRASP2 . |
XQA-DST: Multi-Domain and Multi-Lingual Dialogue State Tracking (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for capturing dialogue data are expensive and limited in their application. |
| Approach: | They propose a domain-agnostic extractive question answering approach with shared weights across domains to disentangle complex domain information in ToDs. |
| Outcome: | The proposed model can efficiently leverage domain-agnostic QA datasets while being domain-scalable and open vocabulary in DST. |
Improving Prediction Backward-Compatiblility in NLP Model Upgrade with Gated Fusion (2023.findings-eacl)
Copied to clipboard
| Challenge: | a regression error during model upgrade often outweighs the benefits of accuracy gain . a novel method that promotes backward compatibility during model upgrades is proposed . |
| Approach: | They propose a method that promotes backward compatibility via learning to mix predictions between old and new models. |
| Outcome: | The proposed method outperforms existing methods and achieves negative flip rate reductions by 73.2% on two model upgrade scenarios. |
AmbiCoref: Evaluating Human and Model Sensitivity to Ambiguous Coreference (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing models do not have welldefined target behavior for coreferential ambiguity. |
| Approach: | They propose to use AmbiCoref to test whether coreference resolution models are sensitive to ambiguity. |
| Outcome: | The proposed model is more sensitive to ambiguity than existing models. |
Improving Unsupervised Out-of-domain detection through Pseudo Labeling and Learning (2023.findings-eacl)
Copied to clipboard
| Challenge: | Unsupervised OOD detection is a task aimed at discriminating whether given samples are from the in-domain (IND) . previous studies adopted the one-class classification approach, assuming that the training samples come from a single domain. |
| Approach: | They propose a framework that leverages latent categorical information to improve representation learning for textual OOD detection. |
| Outcome: | The proposed framework significantly outperforms baseline models on three datasets. |
How Many Data Samples is an Additional Instruction Worth? (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent introduced instruction-paradigm empowers non-expert users to leverage NLP resources by defining a new task in natural language. |
| Approach: | They propose to define a task in natural language without creating task-specific datasets or building models. |
| Outcome: | The proposed model outperforms multitask learning models but is far from state-of-the-art task-specific models. |
[MASK] Insertion: a robust method for anti-adversarial attacks (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies have focused on adversarial defenses against pretrained language models. |
| Approach: | They propose an adversarial defensing algorithm that inserts tokens into input sequences . they show an improvement in accuracy between 3.2 and 11.1 absolute points . |
| Outcome: | The proposed algorithm improves model accuracy on clean and polluted inputs compared with state-of-the-art models . |
ViDeBERTa: A powerful pre-trained language model for Vietnamese (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing models for Vietnamese that perform well on downstream tasks, such as Question answering, are based on Transformer. |
| Approach: | They propose a pre-trained monolingual Vietnamese model with three versions . they fine-tune and evaluate the model on three important natural language downstream tasks, Part-of-speech tagging, Named-entity recognition, and Question answering. |
| Outcome: | The proposed model outperforms the existing model on three important natural language downstream tasks, Part-of-speech tagging, Named-entity recognition, and Question answering. |
NapSS: Paragraph-level Medical Text Simplification via Narrative Prompting and Sentence-matching Summarization (2023.findings-eacl)
Copied to clipboard
| Challenge: | a recent study shows that accessing medical literature is difficult for laypeople because it is written for specialists and contains medical jargon. |
| Approach: | They propose a two-stage strategy to identify relevant content to be simplified . they first generate reference summaries via sentence matching between the original and simplified abstracts . |
| Outcome: | The proposed approach improves on a seq2seq-based test set on an English medical corpus . it also improves the SARI score by 1.1% . |
Long-tailed Extreme Multi-label Text Classification by the Retrieval of Generated Pseudo Label Descriptions (2023.findings-eacl)
Copied to clipboard
| Challenge: | Extreme Multi-label text classification (XMTC) is a tough challenge due to the sheer size of the label spaces and the severe data scarcity problem associated with the long tail of rare labels in highly skewed distributions. |
| Approach: | They propose to use a trained bag-of-words classifier to generate pseudo label descriptions from a training bag- of-word classifier. |
| Outcome: | The proposed approach outperforms the existing models in the tail label prediction problem and achieves state-of-the-art (SOTA) performance on XMTC benchmark datasets. |
Unsupervised Keyphrase Extraction via Interpretable Neural Networks (2023.findings-eacl)
Copied to clipboard
Rishabh Joshi, Vidhisha Balachandran, Emily Saldanha, Maria Glenski, Svitlana Volkova, Yulia Tsvetkov
| Challenge: | Prior approaches for unsupervised keyphrase extraction relied on heuristic notions of phrase importance via embedding clustering or graph centrality. |
| Approach: | They propose an approach which defines keyphrases as document phrases that are salient for predicting the topic of the document. |
| Outcome: | The proposed method alleviates the need for ad-hoc heuristics and achieves state-of-the-art results in scientific publications and news articles. |
Large Language Models are few(1)-shot Table Reasoners (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent literature has shown that large language models are excellent few-shot reasoners to solve text reasoning tasks. |
| Approach: | They evaluated LLMs on popular table QA and fact verification datasets like WikiTableQuestion, FetaQA, TabFact, and FEVEROUS and found they are competent at complex reasoning over table structures. |
| Outcome: | The proposed models are more competent at complex reasoning over table structures than tuned T5-large models. |
Realistic Citation Count Prediction Task for Newly Published Papers (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies on citation count prediction assume that future citation counts of academic papers have not had enough time pass since publication. |
| Approach: | They propose to use citation counts of newly published papers as a realistic citation count prediction task and to use them to leverage the citations of papers shortly after publication. |
| Outcome: | The proposed methods significantly improve the performance of citation count prediction for newly published papers in a realistic setting. |
“Why do I feel offended?” - Korean Dataset for Offensive Language Identification (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for detecting offensive content rely on labeled datasets, but few consider low-resource languages with relatively less data available for training. |
| Approach: | They propose to use Korean as a dataset for offensive language identification . they propose to perform abusive language detection and sentiment analysis to help identify offensive languages. |
| Outcome: | The proposed datasets improve the performance of offensive language identification in Korean, while the existing methods are limited. |
Empirical Investigation of Neural Symbolic Reasoning Strategies (2023.findings-eacl)
Copied to clipboard
Yoichi Aoki, Keito Kudo, Tatsuki Kuribayashi, Ana Brassard, Masashi Yoshikawa, Keisuke Sakaguchi, Kentaro Inui
| Challenge: | Neural reasoning accuracy improves when generating intermediate reasoning steps. |
| Approach: | They decompose the reasoning strategy w.r.t. step granularity and chaining strategy. |
| Outcome: | The proposed reasoning strategy significantly affects performance in a symbolic reasoning dataset. |
Analyzing the Effectiveness of the Underlying Reasoning Tasks in Multi-hop Question Answering (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies have utilized underlying reasoning (UR) tasks in multi-hop question answering datasets to explain the predicted answers and evaluate models' reasoning abilities. |
| Approach: | They analyze UR tasks in QA datasets to determine their effectiveness . they find that UR task is helpful in preventing reasoning shortcuts . |
| Outcome: | The proposed model improves QA performance, reasoning shortcuts, and robustness on adversarial questions. |
PubMedCLIP: How Much Does CLIP Benefit Visual Question Answering in the Medical Domain? (2023.findings-eacl)
Copied to clipboard
| Challenge: | Medical visual question answering is a multimodal task that requires a system to understand both medical images and textual questions and infer associations between them. |
| Approach: | They propose a fine-tuned version of CLIP for the medical domain based on PubMed articles. |
| Outcome: | The proposed model improves accuracy up to 3% on two MedVQA benchmark datasets. |
Multilingual BERT has an accent: Evaluating English influences on fluency in multilingual models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Multilingual models can improve NLP performance on low-resource languages by leveraging higher-resourced languages, but they also reduce average performance on all languages. |
| Approach: | They propose a method to evaluate multilingual models by asking if models predict languages with an 'English accent' they propose to use grammatical structure bias to determine if multilingual model is biased toward English-like setting . |
| Outcome: | The proposed method compares the fluency of multilingual models to the fluencies of monolingual Spanish and Greek models. |
Reassessing Evaluation Practices in Visual Question Answering: A Case Study on Out-of-Distribution Generalization (2023.findings-eacl)
Copied to clipboard
Aishwarya Agrawal, Ivana Kajic, Emanuele Bugliarello, Elnaz Davoodi, Anita Gergely, Phil Blunsom, Aida Nematzadeh
| Challenge: | Visual question answering (VQA) is a task of answering open-ended questions about images. |
| Approach: | They evaluate two vision-and-language (V&L) models under different settings . they find they tend to learn to solve the benchmark rather than the skills required by VQA . |
| Outcome: | The proposed models exhibit poor generalization under out-of-distribution settings. |
Our kind of people? Detecting populist references in political debates (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing literature on populism has only limited agreement on its exact properties . |
| Approach: | They propose a cross-lingual dataset to identify populist rhetoric in text . they propose 'hierarchical' annotation procedure to annotate populist references . |
| Outcome: | The proposed dataset can be used to investigate how political actors talk about The Elite and The People and to study how populist rhetoric is used as a strategic device. |
SharPT: Shared Latent Space Prompt Tuning (2023.findings-eacl)
Copied to clipboard
| Challenge: | Prompt tuning is an efficient method for adapting large language models, but it is difficult and expensive to identify the source task that provides optimal prompts. |
| Approach: | They propose to learn a shared latent space which captures a set of basis skills from a mixture of source tasks and then transfer them to target tasks. |
| Outcome: | The proposed method outperforms previous methods on NLI, sentence completion, QA, conference resolution, word sense disambiguation and on various model scales. |
Mini But Mighty: Efficient Multilingual Pretraining with Linguistically-Informed Data Selection (2023.findings-eacl)
Copied to clipboard
| Challenge: | AfriBERTa shows that training transformer models from scratch on 1GB of data from many unrelated African languages outperforms massively multilingual models on downstream NLP tasks. |
| Approach: | They propose that training on smaller amounts of data but from related languages could match the performance of models trained on large, unrelated data. |
| Outcome: | The proposed model outperforms models trained on large, unrelated datasets on downstream NLP tasks. |
Long Document Summarization with Top-down and Bottom-up Inference (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent models infer latent representations of words or tokens with a transformer encoder, which is bottom-up and thus does not capture long-distance context well. |
| Approach: | They propose a method to infer latent representations of words or tokens in documents . they assume a hierarchical structure of a document where top-level captures long range dependency . |
| Outcome: | The proposed model can summarize an entire book and achieve competitive performance on a wide range of document summarization benchmarks. |
Open Information Extraction with Entity Focused Constraints (2023.findings-eacl)
Copied to clipboard
| Challenge: | Open Information Extraction (OIE) is the task of extracting tuples from unstructured corpora without any knowledge of the type and lexical form of the subject, the object, or the subject. |
| Approach: | They exploit domain knowledge to inject constraints into the extraction through constrained inference and constraint-aware training. |
| Outcome: | The proposed approach improves the CaRB and WIRe57 metric and achieves a 29.17% improvement in the CARB and 24.37% improvement on the WIRe56 metric. |
Hierarchical3D Adapters for Long Video-to-text Summarization (2023.findings-eacl)
Copied to clipboard
| Challenge: | a recent study shows that multimodal summarization is not efficient for long inputs and outputs. |
| Approach: | They extend a TV episode transcript summarization dataset and create a multimodal variant by collecting full-length videos. |
| Outcome: | The proposed model can be tuned to perform multimodal summarization tasks efficiently using adapter modules augmented with a hierarchical structure while tuning only 3.8% of model parameters. |
An Intra-Class Relation Guided Approach for Code Comment Generation (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent work in code comment generation assumes that all information required to generate comments is encoded in the target function itself, yet in most realistic situations, it is hard to understand a function in isolation from the surrounding context. |
| Approach: | They propose a graph-based learning framework to capture various relations among functions in a class file. |
| Outcome: | The proposed method outperforms baseline models on automatic and human evaluation metrics on a Java dataset collected from real-world projects. |
Spelling convention sensitivity in neural language models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Various long-distance dependencies have been investigated using neural language models. |
| Approach: | They examine whether large neural language models learn the long-distance dependency of British versus American spelling conventions . a large T5 language model does internalize consistency, but only with respect to observed lexical items . |
| Outcome: | The proposed model internalizes consistency with the training corpora, but only with respect to observed lexical items. |
Modelling Language Acquisition through Syntactico-Semantic Pattern Finding (2023.findings-eacl)
Copied to clipboard
| Challenge: | Usage-based theories of language acquisition have documented the processes by which children acquire language through communicative interaction. |
| Approach: | They propose a method for learning grammars based on similarities and differences in linguistic observations alone. |
| Outcome: | The proposed method is able to learn compositional lexical and item-based constructions of variable extent and degree of abstraction, along with a network of emergent syntactic categories. |
Benchmark Data and Evaluation Framework for Intent Discovery Around COVID-19 Vaccine Hesitancy (2023.findings-eacl)
Copied to clipboard
Shai Gretz, Assaf Toledo, Roni Friedman, Dan Lahav, Rose Weeks, Naor Bar-Zeev, João Sedoc, Pooja Sangha, Yoav Katz, Noam Slonim
| Challenge: | As COVID-19 vaccines were rolled out, they were met with widespread hesitancy. |
| Approach: | They propose a new framework for intent discovery that leverages existing intent classifiers to provide a real-world conversational dataset of conversations conducted by actual users with VIRA. |
| Outcome: | The proposed framework enables users to find out what they are doing and why they are hesitant. |
Learning Disentangled Representations for Natural Language Definitions (2023.findings-eacl)
Copied to clipboard
| Challenge: | Disentangling the encodings of neural models is a fundamental aspect for improving interpretability, semantic control and downstream task performance in Natural Language Processing. |
| Approach: | They propose to use syntactic and semantic regularities in textual data to provide models with both structural biases and generative factors. |
| Outcome: | The proposed model outperforms baselines on several qualitative and quantitative benchmarks and improves the results in the downstream task of definition modeling. |
Distinguishability Calibration to In-Context Learning (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent studies have shown that pre-trained language models generate similar output embeddings which makes it difficult to discriminate for the prompt-based classifier. |
| Approach: | They propose a calibration method which rotates the embedding feature into a new metric space and adapts the ratio of each dimension to a uniform distribution. |
| Outcome: | The proposed method improves the distinguishability of learning embeddings on three datasets under various settings. |
Investigating anatomical bias in clinical machine learning algorithms (2023.findings-eacl)
Copied to clipboard
| Challenge: | Barriers for realisation of the algorithms’ full potential include bias which is systematic and unfair discrimination against certain individuals or groups of individuals in favor of others. |
| Approach: | They define anatomical bias as unfair algorithmic outcomes against patients with medical conditions in specific anatomic locations. |
| Outcome: | The proposed method detects anatomical bias in clinical text algorithms across two machine learning models and two Danish clinical text classification tasks. |
Topic Ontologies for Arguments (2023.findings-eacl)
Copied to clipboard
| Challenge: | Many computational argumentation tasks, such as stance classification, are topic-dependent. |
| Approach: | They map the argumentation landscape using the World Economic Forum, Wikipedia and Debatepedia as sources for argument topics. |
| Outcome: | The argument ontology is the first comprehensive assessment of argument topics in argument corpora. |
Longtonotes: OntoNotes with Longer Coreference Chains (2023.findings-eacl)
Copied to clipboard
Kumar Shridhar, Nicholas Monath, Raghuveer Thirukovalluru, Alessandro Stolfo, Manzil Zaheer, Andrew McCallum, Mrinmaya Sachan
| Challenge: | Using Ontonotes, documents in certain genres were split into smaller parts for ease of annotation. |
| Approach: | They propose to merge annotations from documents split into smaller parts in Ontonotes for ease of annotation. |
| Outcome: | The proposed corpus restores documents to their original form, revealing dramatic increases in length in certain genres. |
More Robust Schema-Guided Dialogue State Tracking via Tree-Based Paraphrase Ranking (2023.findings-eacl)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) can track only slots drawn from a database or domain ontology. |
| Approach: | They propose a framework for generating synthetic schemas which uses tree-based ranking to optimise lexical diversity and semantic faithfulness. |
| Outcome: | The proposed framework improves the generalisation of strong baselines by augmenting training data with prompts generated by the framework. |
Language Model Decoding as Likelihood–Utility Alignment (2023.findings-eacl)
Copied to clipboard
Martin Josifoski, Maxime Peyrard, Frano Rajič, Jiheng Wei, Debjit Paul, Valentin Hartmann, Barun Patra, Vishrav Chaudhary, Emre Kiciman, Boi Faltings
| Challenge: | Existing studies only compare decoding algorithms in narrow scenarios, and their findings do not generalize across tasks. |
| Approach: | They propose a taxonomy of misalignment mitigation strategies to provide a unifying view of decoding as a tool for alignment. |
| Outcome: | The proposed taxonomy combines likelihood and utility assumptions to provide general statements about decoding as a tool for alignment across tasks. |
Lightweight Spatial Modeling for Combinatorial Information Extraction From Documents (2023.findings-eacl)
Copied to clipboard
Yanfei Dong, Lambert Deng, Jiazheng Zhang, Xiaodong Yu, Ting Lin, Francesco Gelli, Soujanya Poria, Wee Sun Lee
| Challenge: | Existing datasets do not cover documents with complex spatial structures and a lack of spatial information for document entity classification. |
| Approach: | They propose a new spatial bias in attention calculation based on the K-nearest-neighbor graph of document entities that limits entities’ attention to their local radius. |
| Outcome: | The proposed model outperforms baselines in most entity types and is highly parameter-efficient compared to existing methods. |
On the Generalization Ability of Retrieval-Enhanced Transformers (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent work on retrieval-augmented language models has shown impressive results . performance gains from retrieval to a large extent originate from overlapping tokens between the database and test data, suggesting less of non-trivial generalization than previously assumed. |
| Approach: | They propose to off-load memory from trainable weights to a retrieval database and compare it to larger models with a larger model. |
| Outcome: | The proposed model outperforms GPT-3 and Jurassic-1 on the Pile at 4% of the model parameters. |
Assessing Monotonicity Reasoning in Dutch through Natural Language Inference (2023.findings-eacl)
Copied to clipboard
| Challenge: | a novel dataset for natural language inference (NLI) is used to study monotonicity reasoning in Dutch. |
| Approach: | They investigate monotonicity reasoning in Dutch using a novel dataset . they find that models struggle with downward entailing contexts . |
| Outcome: | The proposed dataset shows that models struggle with downward entailing contexts, and argue that this is due to a poor understanding of negation. |
Noisy Parallel Data Alignment (2023.findings-eacl)
Copied to clipboard
| Challenge: | Optical character recognition (OCR) is used to convert endangered language documents into machine-readable data, but its noisy outputs are a challenge for many under-resourced languages. |
| Approach: | They propose to use optical character recognition (OCR) to convert endangered language documents into machine-readable data by using noisy alignment models. |
| Outcome: | The proposed model reduces alignment error rate on a state-of-the-art neural-based alignment model up to 59.6%. |
Enhancing Dialogue Generation with Conversational Concept Flows (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies show that explicitly modeling concept flows with a large commonsense knowledge graph improves response quality, but there is a gap between the knowledge graph and the conversation. |
| Approach: | They propose to model human conversational concept flows with a commonsense knowledge graph . they extract abundant concepts and relations from natural conversations and build a conversation-aware knowledge graph. |
| Outcome: | The proposed method performs better than baselines on a large-scale reddit conversation dataset. |
SMHD-GER: A Large-Scale Benchmark Dataset for Automatic Mental Health Detection from Social Media in German (2023.findings-eacl)
Copied to clipboard
| Challenge: | Mental health problems are a challenge to our modern society, and their prevalence is predicted to increase worldwide. |
| Approach: | They propose a large-scale, carefully constructed dataset for MHC detection built on high-precision patterns and the approach proposed for English. |
| Outcome: | The proposed model leverages engineered (psycho-)linguistic features as well as BERT-German to facilitate further research and conduct extensive experiments. |
Exploring Data Augmentation for Code Generation Tasks (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent advances in natural language processing have impacted how models are trained for programming language tasks. |
| Approach: | They propose to use augmentation methods that yield consistent improvements in code translation and summarization by up to 6.9% and 7.5% respectively. |
| Outcome: | The proposed methods improve translation and summarization by 6.9% and 7.5% respectively. |
Stabilized In-Context Learning with Pre-trained Language Models for Few Shot Dialogue State Tracking (2023.findings-eacl)
Copied to clipboard
| Challenge: | Prompt-based methods with large pre-trained language models have shown impressive unaided performance across many NLP tasks. |
| Approach: | They propose a meta-learning scheme to stabilize the ability of the model to perform well under various prompts and introduce a saliency model to limit dialogue text length. |
| Outcome: | The proposed model improves on large pre-trained language models with labeled in-context exemplars and can be used to generate more exemplar queries. |
Can Demographic Factors Improve Text Classification? Revisiting Demographic Adaptation in the Age of Transformers (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies show that incorporating demographic factors in language representations improves performance on downstream NLP tasks. |
| Approach: | They use continuous language modeling and dynamic multi-task learning to adapt pre-trained Transformers to incorporate demographic information into their representations. |
| Outcome: | The proposed model shows that the results are consistent with previous studies. |
JBLiMP: Japanese Benchmark of Linguistic Minimal Pairs (2023.findings-eacl)
Copied to clipboard
| Challenge: | In this paper, we compare syntactic knowledge of language models across different languages. |
| Approach: | They introduce a dataset for targeted syntactic evaluations of language models in Japanese. |
| Outcome: | The proposed dataset compares the syntactic knowledge of language models across languages. |
SMATCH++: Standardized and Extended Evaluation of Semantic Graphs (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing graph-alignment metrics that measure graph distances are not reliable, we show . metric is spread out and does not provide upper bounds for extended tasks. |
| Approach: | They propose a metric to measure a distance between graphs by aligning nodes and counting matching graph triples. |
| Outcome: | The proposed method reduces search space and improves scoring by reducing the number of errors. |
An Extended Sequence Tagging Vocabulary for Grammatical Error Correction (2023.findings-eacl)
Copied to clipboard
| Challenge: | Current sequence-to-sequence and sequence-tagging approaches treat GEC as a machine-translation problem. |
| Approach: | They propose to introduce specialised tags for spelling correction and morphological inflection using the SymSpell and LemmInflect algorithms. |
| Outcome: | The proposed approach outperforms existing methods on the BEA benchmark. |
Cheating to Identify Hard Problems for Neural Machine Translation (2023.findings-eacl)
Copied to clipboard
| Challenge: | We find that the most frequent tokens are not necessarily the most accurately translated due to these often being function words and punctuation that can be used more flexibly in translation. |
| Approach: | They propose a method to provide a compressed representation of the target as an input and a second method to fine-tune a standard transformer model. |
| Outcome: | The proposed method produces models with higher scores, but with more errors . the proposed method is based on two forms of cheating: providing a compressed representation of the target as an input and fine-tuning on the test set. |
Model-Agnostic Bias Measurement in Link Prediction (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing work investigating social bias in factual knowledge graphs has focused on knowledge graph embeddings, so more recent classes of models achieving superior results by fine-tuning Transformers have not yet been investigated. |
| Approach: | They propose a model-agnostic approach for bias measurement leveraging fairness metrics to compare bias in knowledge graph embedding-based predictions (KG only) with models that use pre-trained, Transformer-based language models (KG+LM). |
| Outcome: | The proposed model-agnostic approach compares gender bias in occupation predictions with models that use pre-trained, Transformer-based language models (KG+LM). |
Divergence-Based Domain Transferability for Zero-Shot Classification (2023.findings-eacl)
Copied to clipboard
| Challenge: | a recent study shows that fine-tuning of neural models can improve performance on language-based tasks without brute-force searching effective task combinations. |
| Approach: | They propose to use divergence measures to estimate whether one task pair will perform better than another . they use 58 tasks and 6,600 task pair combinations to study the effect of different tuning methods . |
| Outcome: | The proposed method reduces end-to-end runtime by 40% by estimating transferability . the proposed method is based on 58 tasks and over 6,600 task pair combinations . |
EDU-level Extractive Summarization with Varying Summary Lengths (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies on extractive summarization use finer-grained elementary discourse units . few studies exploited finer grained EDUs with little analysis and justification for the extractive unit selection . |
| Approach: | They propose an extractive model with Varying summary lengths that extracts fixed top-k salient sentences from the document as a summary. |
| Outcome: | The proposed model performs better on ROUGE scores than state-of-the-art models. |
“Chère maison” or “maison chère”? Transformer-based prediction of adjective placement in French (2023.findings-eacl)
Copied to clipboard
| Challenge: | Prior work has shown that transformer-based language models are insensitive to permutated word order, but this is not the case with pretraining. |
| Approach: | They conduct experiments to assess whether transformer-based language models are able to learn the adjective position in noun phrases in French. |
| Outcome: | The proposed model is weaker with complex structures and fixed expressions, but favors context and global syntactic roles. |
On the Role of Reviewer Expertise in Temporal Review Helpfulness Prediction (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for detecting helpful reviews focus on review text and ignore the two key factors of (1) who post the reviews and (2) when the reviews are posted. |
| Approach: | They propose to integrate reviewer's expertise and temporal dynamics to predict helpfulness for unreliable and cold-start reviews. |
| Outcome: | The proposed model improves on existing models and compares with baselines. |
Towards a Unified Model for Generating Answers and Explanations in Visual Question Answering (2023.findings-eacl)
Copied to clipboard
| Challenge: | Current explanation generation models are trained to select the best answers from Multiple-Choice questions or to classify single-word answers to a predetermined vocabulary. |
| Approach: | They propose a multitask learning approach towards a Unified Model for Answer and Explanation generation (UMAE) UMAE models surpass the prior state-of-the-art answer accuracy on A-OKVQA by 10 15%, show competitive results on OK-VQA and VCR, and demonstrate promising out-of domain performance on VQA-X. |
| Outcome: | The proposed model outperforms the state-of-the-art model on A-OKVQA and VCR and shows promising out-of domain performance on VQA-X. |
Machine Translation between Spoken Languages and Signed Languages Represented in SignWriting (2023.findings-eacl)
Copied to clipboard
| Challenge: | Yin et al. ( 2021) calls for including sign language processing (SLP) in natural language processing research. |
| Approach: | They propose to use a sign language writing system to parse, factorize, decode and evaluate signed languages. |
| Outcome: | The proposed method achieves over 30 BLEU in a bilingual setup and over 20 BLUE in two multilingual setups. |
A Multi-dimensional Evaluation of Tokenizer-free Multilingual Pretrained Models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent work on tokenizer-free models shows promising results in cross-lingual transfer . previous work focused on reporting accuracy on a limited set of tasks and data settings . |
| Approach: | They compare tokenizer-free and subword-based models using various dimensions . they find subword models are still the most practical choice in many settings . |
| Outcome: | The proposed model improves cross-lingual transfer and reduces engineering overhead. |
Neural Ranking with Weak Supervision for Open-Domain Question Answering : A Survey (2023.findings-eacl)
Copied to clipboard
| Challenge: | Neural ranking models require substantial amounts of relevance annotations, which is costly to scale. |
| Approach: | They propose to train a NR model with weak supervision instead of annotations . they use a structured overview of standard WS signals used for training a model . |
| Outcome: | The proposed approach reduces the cost of annotations by using weak supervision instead of a parametric model. |
Double Retrieval and Ranking for Accurate Question Answering (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent work shows that answer verification models can improve the state of the art in Question Answering . despite the fact that the supporting candidates are ranked only according to the relevancy with the question, the model still lacks the support needed for other answer candidates. |
| Approach: | They propose a double reranking model that selects the best support for each target answer . they propose 'second neural retrieval stage' to encode question and answer pair as query . |
| Outcome: | The proposed approach improves the state of the art in Question Answering . the proposed model ranked candidates according to relevancy and not the answer . but the proposed approach fails to provide the best support . |
Evaluating the Diversity, Equity, and Inclusion of NLP Technology: A Case Study for Indian Languages (2023.findings-eacl)
Copied to clipboard
| Challenge: | In order for NLP technology to be widely applicable, fair, and useful, it needs to serve a diverse set of speakers across the world’s languages, be equitable, not unduly biased towards any particular language, and be inclusive of all users. |
| Approach: | They propose to use Gini coefficient to assess NLP across all three dimensions to assess diversity, equity, and inclusion across all languages. |
| Outcome: | The proposed evaluation paradigm assesses NLP technologies across all three dimensions and identifies the need for regional-specific choices in model building and dataset creation. |
Joint Reasoning on Hybrid-knowledge sources for Task-Oriented Dialog (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing systems for task oriented dialog use knowledge present only in structured knowledge sources to generate responses. |
| Approach: | They propose a model that assumes that information is always present in a structured knowledge base . they also refine the model to take into account the fact that it can fuse information from structured and unstructured knowledge sources. |
| Outcome: | The proposed model is robust to perturbations to knowledge modality and can fuse information from structured and unstructured knowledge to generate responses. |
Revisiting Offline Compression: Going Beyond Factorization-based Methods for Transformer Language Models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent transformer language models achieve outstanding results on many downstream tasks, but their enormous size often makes them impractical on memory-constrained devices. |
| Approach: | They propose an offline compression approach that reduces the complexity of the model by enabling collaboration between modules. |
| Outcome: | The proposed approach outperforms commonly used factorization-based offline compression methods on various NLP tasks. |
PriMeSRL-Eval: A Practical Quality Metric for Semantic Role Labeling Systems Evaluation (2023.findings-eacl)
Copied to clipboard
Ishan Jindal, Alexandre Rademaker, Khoi-Nguyen Tran, Huaiyu Zhu, Hiroshi Kanayama, Marina Danilevsky, Yunyao Li
| Challenge: | Existing evaluation scripts for semantic role labeling do not consider error propagation . existing evaluation script does not consider argument independent of predicate sense . |
| Approach: | They propose a more strict SRL evaluation metric PriMeSRL to address these issues . they propose to use a metric that measures the quality of the underlying SRL models . |
| Outcome: | The proposed metric reduces quality evaluation of all SoTA SRL models and penalizes failures. |
Prompt-based Learning for Text Readability Assessment (2023.findings-eacl)
Copied to clipboard
| Challenge: | Using a pre-trained seq2seq model, we can discern which text is more difficult from two given texts (pairwise). |
| Approach: | They propose to adapt a pre-trained seq2seq model to discern which text is more difficult from two given texts (pairwise). |
| Outcome: | The proposed model can be adapted to discern which text is more difficult from two given texts (pairwise). |
Best Practices in the Creation and Use of Emotion Lexicons (2023.findings-eacl)
Copied to clipboard
| Challenge: | Inappropriate and incorrect use of emotion lexicons can lead to harmful inferences . |
| Approach: | They propose to present some of the practical and ethical considerations involved in the creation and use of emotion lexicons. |
| Outcome: | The proposed lexicons can lead to harmful inferences and sub-optimal results . the aim is to provide a comprehensive set of practical and ethical considerations . |
The Role of Semantic Parsing in Understanding Procedural Text (2023.findings-eacl)
Copied to clipboard
| Challenge: | Inferring actions and their impact on entities involved in a procedural text can be challenging in various aspects. |
| Approach: | They propose a symbolic parser and semantic role labeling as two sources of semantic parsing knowledge. |
| Outcome: | The proposed framework integrates semantic parsing knowledge into state-of-the-art neural models and shows that it improves procedural understanding. |
Named Entity Recognition in a Very Homogenous Domain (2023.findings-eacl)
Copied to clipboard
| Challenge: | Developing models that perform well on several domains is important, but domain is vague and can be adapted to a new domain. |
| Approach: | They find that even news articles from the same newspaper in English can be considered different domains. |
| Outcome: | The proposed model performs better on out-of-domain data than on specialized data. |
Crawling The Internal Knowledge-Base of Language Models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for representing factual knowledge in a language model are insufficient. |
| Approach: | They propose a procedure for “crawling” the internal knowledge-base of a language model by expanding a knowledge-graph around it. |
| Outcome: | The proposed method yields high precision graphs (82-92%) while emitting a reasonable number of facts per entity. |
Intent Identification and Entity Extraction for Healthcare Queries in Indic Languages (2023.findings-eacl)
Copied to clipboard
| Challenge: | Currently, there is a lack of data and technology for resource-poor languages in developing countries like India. |
| Approach: | They propose to use two different datasets to analyze query intents and entities in healthcare. |
| Outcome: | The proposed model is useful to identify query intents and entities in real-world scenarios. |
Text-Derived Knowledge Helps Vision: A Simple Cross-modal Distillation for Video-based Action Anticipation (2023.findings-eacl)
Copied to clipboard
| Challenge: | Prior work on action anticipation models treat it as a vision modality problem, but knowledge about action sequences can be obtained from textual data. |
| Approach: | They show how knowledge in pretrained language models can be adapted and distilled into vision based action anticipation models. |
| Outcome: | The proposed model achieves a 3.5% relative gain on EGTEA-GAZE+ and 7.2% relative gain for two action anticipation datasets. |
Simple Yet Effective Synthetic Dataset Construction for Unsupervised Opinion Summarization (2023.findings-eacl)
Copied to clipboard
| Challenge: | generating aspect-specific and general opinion summaries is challenging due to the lack of annotated data. |
| Approach: | They propose two unsupervised approaches to generate aspect-specific and general opinion summaries by training on synthetic datasets constructed with aspect-related review contents. |
| Outcome: | The proposed method outperforms existing methods on space and Oposum+ and on other metrics. |
Towards Fine-tuning Pre-trained Language Models with Integer Forward and Backward Propagation (2023.findings-eacl)
Copied to clipboard
Mohammadreza Tayaranian Hosseini, Alireza Ghaffari, Marzieh S. Tahaei, Mehdi Rezagholizadeh, Masoud Asgharian, Vahid Partovi Nia
| Challenge: | Previously, researchers focused on lower bit-width integer data types for forward propagation and backward propagation of language models to save memory and computation. |
| Approach: | They use integer arithmetic for both forward and back propagation in the fine-tuning of BERT. |
| Outcome: | The proposed method improves on the GLUE and SQUAD benchmarks. |
Data Augmentation for Radiology Report Simplification (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing approaches to improve radiology reports are limited due to the high cost of manual simplification. |
| Approach: | They propose a data augmentation approach to generate simplifications of unlabeled radiology sentences using a pre-trained language model and paraphrasing of labeled radiologists sentences. |
| Outcome: | The proposed model generates simplifications of unlabeled radiology sentences and paraphrases labeled radiologists sentences. |
Embedding Recycling for Language Models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies on embedding recycling have not adequately account for overhead costs. |
| Approach: | They propose to reuse contextualized embeddings from previous runs to speed training and inference of future ones. |
| Outcome: | The proposed technique speeds training and inference with no impact on accuracy. |
Trained on 100 million words and still in shape: BERT meets British National Corpus (2023.findings-eacl)
Copied to clipboard
| Challenge: | masked language models are trained on ever larger corpora, but pre-training on a modestly-sized but representative, well-balanced, and publicly available corpus can reach better performance than the original BERT model. |
| Approach: | They propose an optimized LM architecture called LTG-BERT that can be used to train a competitive language model on a small and standardizable corpus. |
| Outcome: | The proposed architecture outperforms the original English BERT model on a representative, well-balanced and publicly available corpus. |
Generating Synthetic Speech from SpokenVocab for Speech Translation (2023.findings-eacl)
Copied to clipboard
| Challenge: | End-to-end speech-totext translation (ST) models require large amounts of data to train, but their size is considerably smaller than text-based MT data. |
| Approach: | They propose a method to convert MT data to ST data via text-to-speech systems. |
| Outcome: | The proposed method improves translation quality by an average of 1.83 BLEU score while performing equally well as TTS-generated speech in improving translation quality. |
Bounding the Capabilities of Large Language Models in Open Text Generation with Prompt Constraints (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing and potential applications of open-ended text generation are farreaching, spanning domains such as QA, story generation, open-end dialogue, and ChatGPT 1 . |
| Approach: | They propose a prompt-centric approach to analyzing and bounding the abilities of open-ended generative models by a set of structural and stylistic prompts. |
| Outcome: | The proposed method can be generalized to other large models like BLOOM and OPT. |
Learning to Retrieve Engaging Follow-Up Queries (2023.findings-eacl)
Copied to clipboard
Christopher Richardson, Sudipta Kar, Anjishnu Kumar, Anand Ramachandran, Zeynab Raeesy, Omar Khan, Abhinav Sethy
| Challenge: | Open domain conversational agents can answer a wide range of targeted queries, but knowledge exploration is a lengthy task. |
| Approach: | They propose a retrieval based system for predicting the next questions that the user might have . they train ranking models on a dataset called the Follow-up Query Bank . |
| Outcome: | The proposed system can proactively assist users in knowledge exploration leading to a more engaging dialog. |
Selective-LAMA: Selective Prediction for Confidence-Aware Evaluation of Language Models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent studies suggest that neural language models learn and store a large amount of facts and commonsense knowledge from training data. |
| Approach: | They propose a benchmark task that evaluates the amount of relational knowledge stored in pre-trained language models. |
| Outcome: | The proposed evaluations show that the selection of confidence functions is more robust to simple guesses than the accuracy-based evaluation. |
Multi-View Source Ablation for Faithful Summarization (2023.findings-eacl)
Copied to clipboard
| Challenge: | MuFaSSa is a metric for evaluating faithfulness of abstractive summaries . it uses different strategies to remove information from source document to form multiple ablated views . |
| Approach: | They propose a metric for evaluating faithfulness of abstractive summaries using multiple ablated views. |
| Outcome: | The proposed metric outperforms existing models on summarization tasks and human-annotated faithfulness labels. |
Mining Effective Features Using Quantum Entropy for Humor Recognition (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies on humor recognition do not understand the mechanisms that generate humor. |
| Approach: | They propose to use quantum entropy to represent the semantic uncertainty of the setup and punchline as features for humor recognition. |
| Outcome: | The proposed features are more effective than baselines for recognizing humorous and non-humorous texts on the SemEval2021 task 7 dataset. |
AdapterSoup: Weight Averaging to Improve Generalization of Pretrained Language Models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Pretrained language models often need to specialize to specific domains. |
| Approach: | They propose an approach that performs weight-space averaging of adapters trained on different domains. |
| Outcome: | The proposed approach improves performance to new domains without extra training. |
Towards End-to-End Open Conversational Machine Reading (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing approaches to the problem of open-retrieval conversational machine reading (OR-CMR) use two separate modules to approach the problem's two successive sub-tasks. |
| Approach: | They propose to model OR-CMR as a unified text-to-text task in a fully end-to end style and propose to use a text-based approach to solve the problem. |
| Outcome: | Experiments on the ShARC and OR-ShARC dataset show that the proposed framework can generalize to different backbone models. |
Generative Knowledge Selection for Knowledge-Grounded Dialogues (2023.findings-eacl)
Copied to clipboard
| Challenge: | Knowledge selection is the key in knowledge-grounded dialogues (KGD), which aims to select an appropriate knowledge snippet to be used in the utterance based on dialogue history. |
| Approach: | They propose a generative approach for knowledge selection called GenKS that learns to select snippets by generating their identifiers with a sequence-to-sequence model. |
| Outcome: | The proposed approach captures intra-knowledge interaction inherently through attention mechanisms while generating their identifiers with a sequence-to-sequence model. |
Evaluating the Tradeoff Between Abstractiveness and Factuality in Abstractive Summarization (2023.findings-eacl)
Copied to clipboard
| Challenge: | Abstractive summarization models generate fluent and well-formed output but lack semantic faithfulness, or factuality, with respect to the input documents. |
| Approach: | They propose new factuality metrics that adjust for the degree of abstractiveness . they propose to visualize the rates of change in factual as we gradually increase abstractiveity . |
| Outcome: | The proposed models generate fluent and well-formed summaries but lack semantic faithfulness, or factuality, with respect to the input documents. |
Fairness in Language Models Beyond English: Gaps and Challenges (2023.findings-eacl)
Copied to clipboard
| Challenge: | Language models are inequitable at encoding and re-presentation, but there is much to be studied and criticism for the existing research that remains to be addressed. |
| Approach: | They propose to survey fairness in multilingual and non-English contexts . they argue that it is infeasible to achieve comprehensive coverage in terms of fairness datasets based on English . |
| Outcome: | The proposed methods are infeasible to scale across languages and cultures, the authors argue . they argue that the current methods are too narrowly focused on specific dimensions and types of biases and cannot scale across cultures. |
Global-Local Modeling with Prompt-Based Knowledge Enhancement for Emotion Inference in Conversation (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies on emotion recognition focus on recognizing emotions through a speaker’s utterance, while research on emotion inference predicts emotions of addressees through previous utterations. |
| Approach: | They propose a global-local modeling method based on recurrent neural networks and pre-trained language models to do emotion inference in conversation. |
| Outcome: | The proposed method achieves state-of-the-art on three datasets. |
Headline Token-based Discriminative Learning for Subheading Generation in News Article (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing models that generate news subheadings rely on topical headline information to capture topical knowledge from the article. |
| Approach: | They propose a model that uses topical headline information to generate news subheadings using masked headline tokens. |
| Outcome: | The proposed model outperforms the comparative models on three news datasets written in two languages and performs robustly on a small dataset and various masking ratios. |
Decipherment as Regression: Solving Historical Substitution Ciphers by Learning Symbol Recurrence Relations (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for solving substitution ciphers use character-level language models to find key . a Transformer-based causal language model can be used to learn recurrences between characters in a ciphered text . |
| Approach: | They propose a Transformer-based causal language model for decipherment that learns recurrences between characters in a ciphertext. |
| Outcome: | The proposed method achieves strong results on synthetic 1:1 and homophonic ciphers . it learns recurrence relations between cipheries and recovers keys in its self-attention . |
A Survey on Recent Advances in Keyphrase Extraction from Pre-trained Language Models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Keyphrase extraction is a key component in Natural Language Processing (NLP) systems for selecting a set of phrases from the document that could summarize the important information discussed in the source document. |
| Approach: | They propose to use supervised and unsupervised keyphrase extraction techniques to investigate the state-of-the-art models for keyphrase extracting. |
| Outcome: | The proposed keyphrase extraction system can significantly accelerate the speed of retrieval and help people get first-hand information from a long document quickly and accurately. |
Prompting for explanations improves Adversarial NLI. Is this true? {Yes} it is {true} because {it weakens superficial cues} (2023.findings-eacl)
Copied to clipboard
| Challenge: | Explanation prompts are used to generate an explanation for a given input . they are also used to improve model performance on adversarial benchmarks . |
| Approach: | They propose to use explanation prompts to generate an explanation that supports a label . they argue that prompting for explanations weakens superficial cues . |
| Outcome: | The proposed explanation prompts outperform label-only prompts on adversarial benchmarks. |
JobXMLC: EXtreme Multi-Label Classification of Job Skills with Graph Neural Networks (2023.findings-eacl)
Copied to clipboard
Nidhi Goyal, Jushaan Kalra, Charu Sharma, Raghava Mutharaju, Niharika Sachdeva, Ponnurangam Kumaraguru
| Challenge: | Existing approaches to predict missing skills are limited to contextual modelling and do not exploit inter-relational structures like job-job and job-skill relationships. |
| Approach: | They propose a skill prediction framework that exploits structural relationships to predict missing skills using job descriptions. |
| Outcome: | The proposed framework outperforms the state-of-the-art approaches by 6% in precision and 3% in recall on real-world recruitment datasets. |
ViLPAct: A Benchmark for Compositional Generalization on Multimodal Human Activities (2023.findings-eacl)
Copied to clipboard
Terry Yue Zhuo, Yaqing Liao, Yuecheng Lei, Lizhen Qu, Gerard de Melo, Xiaojun Chang, Yazhou Ren, Zenglin Xu
| Challenge: | a vision-language benchmark for human activity planning is designed for humans . the task is easy for humans, but challenging for SOTA deep learning models . |
| Approach: | They propose a vision-language benchmark for human activity planning that extends Charades with intents and builds on a multi-choice question test set. |
| Outcome: | The proposed benchmark evaluates the ability of systems to anticipate and plan human actions in a multimodal visionlanguage setting. |
Grammatical Error Correction through Round-Trip Machine Translation (2023.findings-eacl)
Copied to clipboard
| Challenge: | A decade ago the idea of using round-trip MT to guide grammatical error correction was not feasible due to the low quality of MT systems of the day. |
| Approach: | They propose to use round-trip machine translation to guide grammatical error correction to preserve meaning while mapping its surface form from one language into another. |
| Outcome: | The proposed system is re-examined across five languages and models of various sizes and yields consistent improvements. |
Does Masked Language Model Pre-training with Artificial Data Improve Low-resource Neural Machine Translation? (2023.findings-eacl)
Copied to clipboard
| Challenge: | Pre-training masked language models with artificial data has been proven beneficial for several natural language processing tasks, however, it has been less explored for neural machine translation (NMT). |
| Approach: | They pre-trained masked language models with random sequences and created artificial data mimicking token frequency information from the real world. |
| Outcome: | The results show that pre-training models with artificial data improves translation performance in low-resource situations. |
Performance and Risk Trade-offs for Multi-word Text Prediction at Scale (2023.findings-eacl)
Copied to clipboard
Aniket Vashishtha, S Sai Prasad, Payal Bajaj, Vishrav Chaudhary, Kate Cook, Sandipan Dandapat, Sunayana Sitaram, Monojit Choudhury
| Challenge: | Large Language Models (LLMs) generate ethically inappropriate texts even for seemingly innocuous contexts. |
| Approach: | They propose to use large language models to detect and filter toxic content in text prediction tasks by evaluating their toxicity detection approaches against a manually crafted CheckList of harms. |
| Outcome: | The proposed methods are compared against a checklist of harms targeted at different groups and different levels of severity in English. |
Searching for Better Database Queries in the Outputs of Semantic Parsers (2023.findings-eacl)
Copied to clipboard
| Challenge: | generating a database query from a question in natural language is a longstanding task . the task is amplified when the system needs to generalize to databases unseen at training. |
| Approach: | They propose to generalize a query to databases unseen at training . they use state-of-the-art semantic parsers to find queries that meet the criterion . |
| Outcome: | The proposed approach finds that many queries pass all tests on different datasets. |
Style-Aware Contrastive Learning for Multi-Style Image Captioning (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing multi-style image captioning methods focus on visual content and style . existing methods overlook the relationship between linguistic style and visual content. |
| Approach: | They propose a style-aware visual encoder with contrastive learning to mine potential visual content relevant to style and a triplet contrast objective to distinguish whether the image, style and caption matched. |
| Outcome: | The proposed method achieves state-of-the-art performance and an extensive analysis to verify its effectiveness. |
Strategize Before Teaching: A Conversational Tutoring System with Pedagogy Self-Distillation (2023.findings-eacl)
Copied to clipboard
| Challenge: | Conversational tutoring systems (CTSs) aim to help students master educational material with natural language interaction in the form of a dialog. |
| Approach: | They propose to jointly predict teaching strategies and generate tutor responses accordingly to help students master educational material through dialog. |
| Outcome: | The proposed framework is based on three dialog tutoring datasets and is more realistic than previous models that generate responses given the strategies as input. |
ICA-Proto: Iterative Cross Alignment Prototypical Network for Incremental Few-Shot Relation Classification (2023.findings-eacl)
Copied to clipboard
Wangjie Jiang, Zhihao Ye, Bang Liu, Ruihui Zhao, Jianguang Zheng, Mengyao Li, Zhiyong Li, Yujiu Yang, Yefeng Zheng
| Challenge: | Existing methods for relation classification suffer from the scarcity of manually annotated data. |
| Approach: | They propose a novel relation classification model that incorporates query representation into the encoding of novel prototypes and utilizes iteratively to achieve more interaction. |
| Outcome: | The proposed model outperforms the state-of-the-art model on two benchmark datasets. |
A Large-Scale Multilingual Study of Visual Constraints on Linguistic Selection of Descriptions (2023.findings-eacl)
Copied to clipboard
| Challenge: | a multilingual study examines how vision constrains linguistic choice . we use existing annotations to investigate the effect of different visual conditions on numeral expressions in captions . |
| Approach: | They propose a method that leverages existing corpora of images with captions written by native speakers to constrain linguistic choice. |
| Outcome: | The proposed method covers four languages and five linguistic properties, including verb transitivity and use of numerals. |
How Much Syntactic Supervision is “Good Enough”? (2023.findings-eacl)
Copied to clipboard
| Challenge: | RNNGs with syntactic supervision underperformed RNNs with some syntaktic supervision, whereas RNNS with mild supervision achieved the best performance comparable to the state-of-the-art GPT-2-XL. |
| Approach: | They propose a method where syntactic LMs are gradually ablated from full syntatic supervision to zero syntastic supervision by preserving NP, VP, PP, SBAR nonterminal symbols. |
| Outcome: | The proposed method underperforms the RNNGs with zero syntactic supervision, and the LMs with mild syntaktic supervision perform better than the state-of-the-art GPT-2-XL. |
Are the Best Multilingual Document Embeddings simply Based on Sentence Embeddings? (2023.findings-eacl)
Copied to clipboard
| Challenge: | obtaining document embeddings at document level is challenging due to computational requirements and lack of appropriate data. |
| Approach: | They compare methods to produce document-level representations from sentences based on LASER, LaBSE, and Sentence BERT pre-trained multilingual models. |
| Outcome: | The proposed methods produce document-level representations from sentences in 8 languages . the results show that a clever combination of sentence embeddings is usually better than encoding the full document as a single unit. |
Improving User Controlled Table-To-Text Generation Robustness (2023.findings-eacl)
Copied to clipboard
| Challenge: | In experiments, models perform well on test sets coming from the same distribution as the train data but their performance drops when evaluated on realistic noisy user inputs. |
| Approach: | They propose a user controlled table-to-text generation task where users explore the content in a table by selecting cells and reading a natural language description thereof. |
| Outcome: | The proposed model gains 4.85 BLEU points on user noisy test cases and 1.4 on clean test cases. |
Better Pre-Training by Reducing Representation Confusion (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods to improve pre-trained language models address information confusion in position encoding and model representations. |
| Approach: | They propose two techniques to improve pre-trained language models by decoupling directions and auxiliary regularizers. |
| Outcome: | The proposed techniques can improve pre-trained language models on GLUE benchmarks. |
MAFiD: Moving Average Equipped Fusion-in-Decoder for Question Answering over Tabular and Textual Data (2023.findings-eacl)
Copied to clipboard
| Challenge: | Experimental results show that Transformer-based questions have a "long" hybrid sequence over tabular and textual elements, causing long-range reasoning problems. |
| Approach: | They propose a moving average-equipped fusion-in-decoder to handle long-range reasoning problems . they use FiD and EMA to combine different levels of reasoning . |
| Outcome: | Experimental results show that the proposed model increases exact matching and F1 by 1.1 and 1.7 on the blind test set. |
Transformer-based Models for Long-Form Document Matching: Challenges and Empirical Analysis (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent advances in the area of long document matching have primarily focused on using transformer-based models for long document encoding and matching. |
| Approach: | They propose to use simple neural models and simple embeddings to improve document matching by taking significantly less training time, energy, and memory. |
| Outcome: | The proposed models outperform the more complex BERT-based models while taking significantly less training time, energy, and memory. |
Simple and Effective Multi-Token Completion from Masked Language Models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Pre-trained neural masked language models are limited to predicting a single token . recent pre-tried LMs like T5 do allow predicting multi-token completions, but are more expensive to train and run. |
| Approach: | They propose two ways to adapt pre-trained masked language models to produce multi-token completions. |
| Outcome: | The proposed method surpasses current state-of-the-art models while being more parameter efficient. |
A Survey on Dynamic Neural Networks for Natural Language Processing (2023.findings-eacl)
Copied to clipboard
| Challenge: | Dynamic neural networks can scale up pretrainable models with sub-linear increases in computation and time. |
| Approach: | They summarize the progress of three types of dynamic neural networks in NLP . skimming, mixtures of experts, and early exit are among the most popular . |
| Outcome: | The proposed models can scale up with sub-linear increases in computation and time . skimming, mixture of experts, and early exit are the most popular approaches . |
Transformers with Learnable Activation Functions (2023.findings-eacl)
Copied to clipboard
| Challenge: | Activation functions can reduce the topological complexity of input data and improve model performance. |
| Approach: | They propose to consider data as a topology with its own shape to simplify its complexity and make it linearly separable in the output space. |
| Outcome: | The RAF-based Transformer model outperforms its FAF-based counterpart on the GLUE benchmark by 5.71 points and 2.05 points on SQuAD with all available data. |
The Solvability of Interpretability Evaluation Metrics (2023.findings-eacl)
Copied to clipboard
| Challenge: | Feature attribution methods are often evaluated on metrics such as comprehensiveness and sufficiency. |
| Approach: | They propose to use beam search to define problem of optimizing an explanation for a metric . they also propose to evaluate the metric on one or more metrics to determine its solvability . |
| Outcome: | The proposed explainer can solve the problem of optimizing an explanation for a metric by beam search. |
Reliable Gradient-free and Likelihood-free Prompt Tuning (2023.findings-eacl)
Copied to clipboard
| Challenge: | Large pre-trained language models are often offered as black-box APIs due to privacy or commercial constraints. |
| Approach: | They propose to tune the soft prompts without requiring gradient computation and extend the model to include a distribution over prompts. |
| Outcome: | The proposed methods are competitive with gradient-based approaches with full access to the PLM. |
Combining Psychological Theory with Language Models for Suicide Risk Detection (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing models for suicide prevention are limited in domains and are not available in low-resource languages. |
| Approach: | They propose a computational model that combines pre-trained language models with a fixed set of manually crafted suicidal cues and a two-stage fine-tuning process to detect suicide risk. |
| Outcome: | The proposed model outperforms baseline models even early on in the conversation and performs well across genders and age groups. |
Cross-Lingual Question Answering over Knowledge Base as Reading Comprehension (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing high-quality xMRC datasets can be further utilized to fine-tune our model. |
| Approach: | They propose a cross-lingual question answering over knowledge base approach that converts KB subgraphs into passages to narrow the gap between KB schemas and questions. |
| Outcome: | The proposed approach outperforms baselines and achieves strong few-shot and zero-shot performance on two xKBQA datasets in 12 languages. |
Delving Deeper into Cross-lingual Visual Question Answering (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies on cross-lingual VQA have reported poor zero-shot transfer performance of current multilingual multimodal Transformers . lack of multilingual resources has hindered development and evaluation of VQA methods beyond the English language . |
| Approach: | They analyze cross-lingual VQA across different question types of varying complexity . they show that simple modifications to the standard training setup can substantially reduce the transfer gap to monolingual English performance. |
| Outcome: | The proposed model significantly reduces the transfer gap to monolingual English performance . the proposed model also improves on question types and languages . |
Bridging Argument Quality and Deliberative Quality Annotations with Adapters (2023.findings-eacl)
Copied to clipboard
| Challenge: | Assessing the quality of an argument is a complex, highly subjective task . argument quality dimensions are complex and dependent on the context in which it is assessed . |
| Approach: | They propose a multi-task learning framework that incorporates knowledge about related dimensions into the learning process. |
| Outcome: | The proposed framework improves quality prediction in an extrinsic, out-of-domain task. |
Interventional Probing in High Dimensions: An NLI Case Study (2023.findings-eacl)
Copied to clipboard
| Challenge: | Probing strategies have been shown to detect the presence of various linguistic features inlarge language models; in particular, semantic features intermediate to the “natural logic”fragment of the NLI. |
| Approach: | They propose to use amnesic probing and mnestic probing to investigate the effect of these semantic fea-tures on NLI classification by examining the effects of a mnemonic probing variation on the model. |
| Outcome: | The proposed methods have been shown to detect features intermediate to the “natural logic”fragment of the Natural Language Inferencetask (NLI). |
Program Synthesis for Complex QA on Charts via Probabilistic Grammar Based Filtered Iterative Back-Translation (2023.findings-eacl)
Copied to clipboard
Shabbirhussain Bhaisaheb, Shubham Paliwal, Rajaswa Patil, Manasi Patwardhan, Lovekesh Vig, Gautam Shroff
| Challenge: | Current chart-based Question Answering approaches address structural, visual or simple data retrieval-type questions with fixed-vocabulary answers. |
| Approach: | They employ a neural semantic parser to transform NL questions into SQL programs . they use a probabilistic context-free grammar to generate NL queries from a schema . |
| Outcome: | The proposed approach achieves State-of-the-Art (SOTA) results on reasoning-based queries. |
Exploiting Language Characteristics for Legal Domain-Specific Language Model Pretraining (2023.findings-eacl)
Copied to clipboard
| Challenge: | Pretraining large language models has resulted in tremendous performance improvement for many natural language processing tasks. |
| Approach: | They propose to incorporate pretraining objectives that explicitly exploit domain specific language characteristics into the model. |
| Outcome: | The proposed objectives target token-level feature representation and incorporate sentence level semantics. |
Global Constraints with Prompting for Zero-Shot Event Argument Classification (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing zero-shot trigger extraction models require annotations, which is not practical for open-domain applications. |
| Approach: | They propose to use global constraints with prompting to tackle event argument classification without annotation and task-specific training. |
| Outcome: | The proposed model outperforms the best zero-shot baselines by 12.5% and 10.9% F1 on ACE and ERE with given argument spans and by 4.3% and 3.3% F1 without given argument spas. |
Distillation of encoder-decoder transformers for sequence labelling (2023.findings-eacl)
Copied to clipboard
| Challenge: | despite the strong trend in NLP to explore the use of large language models, there is still limited work evaluating prompting and decoding mechanisms for SL tasks. |
| Approach: | They propose a hallucination-free framework for sequence tagging that is especially suited for distillation. |
| Outcome: | The proposed framework performs well across multiple sequence labelling datasets and in a few-shot learning scenario. |
Predicting Desirable Revisions of Evidence and Reasoning in Argumentative Writing (2023.findings-eacl)
Copied to clipboard
| Challenge: | Using the essay context of the revision and feedback from students prior to the revision, we identify desirable and undesirable revisions. |
| Approach: | They propose to use the essay context of the revision and the feedback students received before the revision to improve classifier performance. |
| Outcome: | The proposed models improve over baseline models, while models utilizing context improve over the baseline models. |
Discourse Structure Extraction from Pre-Trained and Fine-Tuned Language Models in Dialogues (2023.findings-eacl)
Copied to clipboard
| Challenge: | Discourse processing suffers from data sparsity, especially for dialogues . a variety of discourse frameworks have been proposed to extract discourse information from dialogues. |
| Approach: | They propose unsupervised and semi-supervised methods to infer latent discourse structures for dialogues based on attention matrices from Pre-trained Language Models. |
| Outcome: | The proposed methods achieve encouraging results on the STAC corpus, with F1 scores of 57.2 and 59.3 for the unsupervised and semi-supervised methods, respectively. |
Relation Extraction with Weighted Contrastive Pre-training on Distant Supervision (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods ignore the intrinsic noise of distant supervision during the pre-training stage. |
| Approach: | They propose a weighted contrastive learning method that explicitly reduces noise . they leverage supervised data to estimate reliability and reduce noise compared to non-weighted baselines . |
| Outcome: | The proposed method reduces the noise of distant supervision and estimates reliability of pre-training instances. |
CK-Transformer: Commonsense Knowledge Enhanced Transformers for Referring Expression Comprehension (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing frameworks for referring expression comprehension with commonsense knowledge are lacking in the field of multimodal referring . |
| Approach: | They propose a framework for commonsense knowledge Enhanced Transformers which integrates commonsensible knowledge into representations of objects in an image. |
| Outcome: | The proposed framework improves on the existing state of the art in referring expression comprehension with commonsense knowledge (CK-Transformer) it achieves 3.14% accuracy over the existing framework. |
Curricular Next Conversation Prediction Pretraining for Transcript Segmentation (2023.findings-eacl)
Copied to clipboard
| Challenge: | Prior research on document segmentation has focused on segmenting documents such as Wikipedia articles. |
| Approach: | They propose to pretrain a model to identify consecutive conversations to address these challenges . they introduce a curriculum to Advanced NCP to make the task more relevant to the downstream task . |
| Outcome: | The proposed model outperforms previous models in speech recognition errors and is robust to speech recognition. |