Findings of the Association for Computational Linguistics: ACL 2023
Investigating Glyph-Phonetic Information for Chinese Spell Checking: What Works and What’s Next? (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained Chinese language models have shown impressive performance on a wide range of NLP tasks, but the generalization ability of these models has not been well understood. |
| Approach: | They propose to use glyph-phonetic information to improve Chinese spell checking models . they propose a new, more challenging, and practical setting for testing the generalizability of CSC models. |
| Outcome: | The proposed model incorporates glyph-phonetic information and is more challenging and practical. |
A Self-Supervised Integration Method of Pretrained Language Models and Word Definitions (2023.findings-acl)
Copied to clipboard
| Challenge: | DefBERT integrates word definition modeling with word semantics in dictionaries. |
| Approach: | They propose a method that integrates pretrained models with word semantics in dictionaries. |
| Outcome: | The proposed method is effective on word similarity and short text classification tasks. |
Conformal Nucleus Sampling (2023.findings-acl)
Copied to clipboard
| Challenge: | Modern language generation methods employ one of a handful of standard decoding strategies . greedy search and other methods generate dull text or degenerate text . |
| Approach: | They employ a calibration procedure to calibrate the parameter p as a function of the entropy of the next word distribution. |
| Outcome: | The proposed method overconfidently selects the word with the highest probability . the greedy method and its beam search variations tend to return dull text or degenerate text . |
DiscoPrompt: Path Prediction Prompt Tuning for Implicit Discourse Relation Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing works on implicit discourse relation recognition focus on syntax features and lack of connectives. |
| Approach: | They propose a prompt-based path prediction method that integrates the interactive information and intrinsic senses among the hierarchy in IDRR. |
| Outcome: | The proposed method shows significant improvement against competitive baselines. |
Modularized Zero-shot VQA with Pre-trained Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work on zero-shot visual question answering does not explicitly consider multi-step reasoning chains, making them less interpretable compared with a decomposition-based approach. |
| Approach: | They propose a modularized zero-shot network that explicitly decomposes questions into sub reasoning steps and is highly interpretable. |
| Outcome: | The proposed model decomposes questions into sub reasoning steps and is highly interpretable. |
TimelineQA: A Benchmark for Question Answering over Timelines (2023.findings-acl)
Copied to clipboard
Wang-Chiew Tan, Jane Dwivedi-Yu, Yuliang Li, Lambert Mathias, Marzieh Saeidi, Jing Nathan Yan, Alon Halevy
| Challenge: | Existing question answering techniques for lifelogs do not provide accurate answers . augmented reality glasses have led to the creation of personal assistants . |
| Approach: | They propose to use a benchmark to query lifelogs to find out what happened in real life . they find that extractive QA systems out-perform retrieval-augmented QA techniques . |
| Outcome: | The proposed method outperforms state-of-the-art retrieval-augmented QA systems in atomic queries and multi-hop queries. |
Abstractive Text Summarization Using the BRIO Training Paradigm (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing abstractive summarization models rely heavily on reference summaries and lack control over their performance. |
| Approach: | They propose a BRIO paradigm to reduce the dependence on reference summaries by fine-tuning pre-trained language models and training them with the paradigm. |
| Outcome: | The proposed paradigm outperforms existing models on Vietnamese and CNNDM datasets while maintaining the main content of the original text. |
Modeling the Q-Diversity in a Min-max Play Game for Robust Optimization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for minimizing the worst-case loss of annotated groups are lacking in practice due to expensive annotations and privacy issues. |
| Approach: | They propose a distributionally robust optimization framework that relaxes group identification into direct parameterization by using an interactive training mode. |
| Outcome: | The proposed method outperforms state-of-the-art methods on synthetic and real-world text classification tasks. |
Pre-training Language Model as a Multi-perspective Course Learner (2023.findings-acl)
Copied to clipboard
Beiduo Chen, Shaohan Huang, Zihan Zhang, Wu Guo, Zhenhua Ling, Haizhen Huang, Furu Wei, Weiwei Deng, Qi Zhang
| Challenge: | Experimental results show that our method significantly improves ELECTRA’s average performance by 2.8% and 3.2% absolute points respectively on GLUE and SQuAD 2.0 benchmarks. |
| Approach: | They propose a multi-perspective course learning method to fetch many degrees and visual angles for sample-efficient pre-training and to fully leverage the relationship between generator and discriminator. |
| Outcome: | The proposed method improves ELECTRA's performance on GLUE and SQuAD 2.0 benchmarks and overshadows recent advanced ELECL-style models under the same settings. |
Layerwise universal adversarial attack on NLP models (2023.findings-acl)
Copied to clipboard
| Challenge: | a new technique for layerwise UATs searches hidden layers of a network for universal adversarial triggers . a previous study showed that adversarials can fool models by perturbing samples that leave the ground truth label unchanged but can modify model prediction drastically. |
| Approach: | They propose a new approach to construct layerwise UATs by perturbing hidden layers of a network and propose LUATs that are more efficient than vanilla UAT methods. |
| Outcome: | The proposed method provides better transferability in a model-to-model setting with an average gain of 9.3% in fooling rate over baseline. |
Scene-robust Natural Language Video Localization via Learning Domain-invariant Representations (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have focused on improving performance with the assumption of independently identical data distribution while ignoring out-of-distribution data. |
| Approach: | They propose a scene-robust NLVL problem and a generalizable framework to learn a robust model. |
| Outcome: | The proposed model learns generalizable domain-invariant representations by alignment and decomposition. |
Exploiting Pseudo Image Captions for Multimodal Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to multimodal summarization with multimodal output (MSMO) lack reference images for training, and exposure of image captions during training is inconsistent with MSMO’s task settings. |
| Approach: | They propose a coarse-to-fine image-text alignment mechanism to identify the most relevant sentence of each image in a document, resembling the role of image captions in capturing visual knowledge. |
| Outcome: | The proposed method sets up state-of-the-art on all intermodality and intramodality metrics and improves on image recommendation precision. |
Cross-Lingual Transfer with Target Language-Ready Task Adapters (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing frameworks for (zero-shot) cross-lingual transfer employ separate language and task adapters which can be arbitrarily combined to perform any task to any target language. |
| Approach: | They propose to fine-tune 'target language-ready' adapters to the target language to achieve better transfer performance without sacrificing the modularity of MAD-X. |
| Outcome: | The proposed adapters outperform MAD-X and BAD-X on most tasks and languages while maintaining the modularity of MAD. |
DynaMiTE: Discovering Explosive Topic Evolutions with User Guidance (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing Dynamic topic models are either fully supervised, requiring expensive human annotations, or fully unsupervised, producing topic evolutions that often do not cater to a user’s needs. |
| Approach: | They propose to use a framework that ensembles semantic similarity, category indicative, and time indicative scores to produce informative topic evolutions. |
| Outcome: | The proposed framework can be used to discover topic evolutions from temporal corpora that align with user-provided category names and uniquely capture topics at each time step. |
Boost Transformer-based Language Models with GPU-Friendly Sparsity and Quantization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have shown that transformer-based language models can achieve state-of-the-art performance on various natural language processing tasks. |
| Approach: | They propose a GPU-friendly 2:4 fine-grained structured sparsity and quantization scheme to maximize the GPU's acceleration constraints. |
| Outcome: | The proposed scheme achieves state-of-the-art compression on TLM model of various encoder and decoder blocks with negligible accuracy degradation on SQuAD, GLUE, CNN-DM & XSum and WikiText benchmarking tasks. |
RMSSinger: Realistic-Music-Score based Singing Voice Synthesis (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for singing voice synthesis are limited to fine-grained music scores . manual adjustment destroys regularity of note durations, making fine-grain music scores "crushed" |
| Approach: | They propose a method to synthesize singing voices given realistic music scores . they use real-music-score-based Singing Voice Synthesis to generate high-quality voices . |
| Outcome: | The proposed method eliminates manual annotation and simplifies phoneme-level mel-note alignment. |
Zero-Shot Prompting for Implicit Intent Prediction and Recommendation with Commonsense Reasoning (2023.findings-acl)
Copied to clipboard
| Challenge: | Currently, intelligent assistants require explicit user requests to perform tasks or services, leading to lengthy and complex conversations. |
| Approach: | They propose a framework that automatically infers implicit intents from user utterances and prompts a large pre-trained language model to suggest suitable task-oriented bots. |
| Outcome: | The proposed framework reduces interaction complexity and integrates domains and tasks. |
MTGP: Multi-turn Target-oriented Dialogue Guided by Generative Global Path with Flexible Turns (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches focus on global planning, which plans toward the target before the conversation. |
| Approach: | They propose to generate a global path as a natural language sentence instead of a sequence of nodes. |
| Outcome: | The proposed method has fewer turns, more coherent semantics, and higher success rate than baselines. |
The Larger they are, the Harder they Fail: Language Models do not Recognize Identifier Swaps in Python (2023.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are used for programming tasks but lack a deep understanding of the content they manipulate. |
| Approach: | They show that LLMs fail to correctly generate correct Python code when default function names are swapped . they also show that they become more confident in their incorrect predictions as the model size increases . |
| Outcome: | The proposed models fail to generate correct Python code when default function names are swapped, and become more confident in their incorrect predictions as the model size increases. |
Class Lifelong Learning for Intent Detection via Structure Consolidation Networks (2023.findings-acl)
Copied to clipboard
Qingbin Liu, Yanchao Hao, Xiaolong Liu, Bo Li, Dianbo Sui, Shizhu He, Kang Liu, Jun Zhao, Xi Chen, Ningyu Zhang, Jiaoyan Chen
| Challenge: | Existing intent detection models can only handle predefined intent classes in the offline environment. |
| Approach: | They propose a method that continually learns new intent classes from new data . structure-based retrospection and contrastive knowledge distillation are used to solve these problems . |
| Outcome: | The proposed method outperforms existing models on three benchmarks. |
On Evaluating and Mitigating Gender Biases in Multilingual Settings (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks and resources for evaluating gender biases in multilingual settings are limited. |
| Approach: | They propose to extend DisCo to different Indian languages using human annotations to evaluate gender biases in multilingual models. |
| Outcome: | The proposed benchmarks and mitigation techniques are extended beyond English to evaluate gender biases in multilingual models. |
Rethinking Round-Trip Translation for Machine Translation Evaluation (2023.findings-acl)
Copied to clipboard
| Challenge: | Automatic evaluation methods for translation often require model training and the availability of parallel corpora limits their applicability to low-resource settings. |
| Approach: | They revisit the statistical machine translation technique and use it to improve translation quality. |
| Outcome: | The proposed method improves translation quality estimation models and identifies adversarial competitors in shared tasks via cross-system verification. |
G3R: A Graph-Guided Generate-and-Rerank Framework for Complex and Cross-domain Text-to-SQL Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to complex and cross-domain Text-to-SQL generation lack domain knowledge . domain knowledge is not incorporated to enhance their ability to generalise to unseen databases. |
| Approach: | They propose a framework called G3R for complex and cross-domain Text-to-SQL generation . they propose re-ranking SQL queries based on domain knowledge and a graph-guided SQL generator . |
| Outcome: | The proposed framework achieves state-of-the-art results on the Spider and Spider-DK benchmarks. |
A Unified Knowledge Graph Augmentation Service for Boosting Domain-specific NLP Tasks (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing domain-specific pre-trained language models lack domain knowledge in domain-focused training. |
| Approach: | They propose a unified domain language model development service to inject domain knowledge into the PLM fine-tuning stage. |
| Outcome: | Experiments on domain-specific text classification and QA tasks verify the effectiveness and generalizability of KnowledgeDA. |
Dialogue Planning via Brownian Bridge Stochastic Process for Goal-directed Proactive Dialogue (2023.findings-acl)
Copied to clipboard
| Challenge: | Goal-directed dialogue systems aim to proactively reach a pre-determined target through multi-turn conversations. |
| Approach: | They propose a coherent dialogue planning approach that uses a stochastic process to model the temporal dynamics of dialogue paths. |
| Outcome: | The proposed approach generates more coherent utterances and achieves the goal with a higher success rate. |
A Match Made in Heaven: A Multi-task Framework for Hyperbole and Metaphor Detection (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to detect metaphor and hyperbole independently have not explored their relationship computationally. |
| Approach: | They propose a multi-task deep learning framework to detect hyperbole and metaphor simultaneously by annotating two hyperbolic datasets with metaphor labels. |
| Outcome: | The proposed framework improves state-of-the-art hyperbole detection by 12% over existing methods. |
Prompt Tuning for Unified Multimodal Pretrained Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Prompt tuning has demonstrated success in natural language pretraining and even vision pretraining. |
| Approach: | They propose to apply prompt tuning to a unified sequence-to-sequence pretrained model by adding a sequence of learnable embeddings to each layer and finetuning the pretrained models on downstream tasks. |
| Outcome: | The proposed method outperforms other parameter-efficient tuning methods on multimodal models and is robust against adversarial attacks. |
Learning Joint Structural and Temporal Contextualized Knowledge Embeddings for Temporal Knowledge Graph Completion (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods that incorporate time information into static knowledge graph embedding ignore the contextual nature of the TKG structure. |
| Approach: | They propose a method that employs pre-trained language models to learn joint Structural and Temporal Contextualized Knowledge Embeddings. |
| Outcome: | The proposed method is superior to existing methods that ignore the contextual nature of the TKG structure. |
A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets (2023.findings-acl)
Copied to clipboard
Md Tahmid Rahman Laskar, M Saiful Bari, Mizanur Rahman, Md Amran Hossen Bhuiyan, Shafiq Joty, Jimmy Huang
| Challenge: | Currently, the evaluation of large language models (LLMs) such as ChatGPT in academic datasets is difficult due to the difficulty of evaluating the generative outputs produced by this model against the ground truth. |
| Approach: | They evaluate ChatGPT across 140 tasks and analyze 255K responses it generates in academic datasets. |
| Outcome: | The proposed model performs well on 140 tasks and generates 255K responses in these datasets. |
Generating Deep Questions with Commonsense Reasoning Ability from the Text by Disentangled Adversarial Inference (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for commonsense question generation produce shallow questions that can be answered by simple word matching. |
| Approach: | They propose a task of commonsense question generation that aims to yield deep-level questions from the text. |
| Outcome: | The proposed model can yield deep-level and to-the-point questions from the text. |
TADA: Efficient Task-Agnostic Domain Adaptation for Transformers (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained transformer-based language models are limited in their expressiveness and domain knowledge. |
| Approach: | They propose a task-agnostic domain adaptation method which is modular, parameter-efficient, and data-efficient. |
| Outcome: | The proposed method is efficient and modular, parameter-efficient, and data-efficient. |
Robust Natural Language Understanding with Residual Attention Debiasing (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing ensemble-based debiasing methods do not address unintended dataset biases . attention plays a crucial role in providing robust prediction in NLU models . |
| Approach: | They propose an end-to-end debiasing method that mitigates unintended biases from attention. |
| Outcome: | The proposed method improves the OOD performance of BERT-based models on three benchmarks. |
MoNET: Tackle State Momentum via Noise-Enhanced Training for Dialogue State Tracking (2023.findings-acl)
Copied to clipboard
| Challenge: | Experimental results show that MoNET outperforms previous DST methods in alleviating state momentum issues and improving the anti-noise ability. |
| Approach: | They propose to use previous state of each turn in training data as input to learn to predict current state. |
| Outcome: | The proposed model outperforms existing methods on multiWOZ datasets and shows that it can update and correct slot values and improve anti-noise ability. |
PAL: Persona-Augmented Emotional Support Conversation Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work has demonstrated the effectiveness of dialogue models in providing emotional support due to the lack of human resources for mental health support. |
| Approach: | They propose a framework for dynamically inferring and modeling seekers’ persona from the conversation history and a model that leverages persona information to provide personalized emotional support. |
| Outcome: | The proposed model outperforms baseline models on the studied benchmark. |
Farewell to Aimless Large-scale Pretraining: Influential Subset Selection for Language Model (2023.findings-acl)
Copied to clipboard
Xiao Wang, Weikang Zhou, Qi Zhang, Jie Zhou, SongYang Gao, Junzhe Wang, Menghan Zhang, Xiang Gao, Yun Wen Chen, Tao Gui
| Challenge: | Pretrained language models have achieved remarkable success in various natural language processing tasks. |
| Approach: | They propose to use end-task knowledge to select a tiny subset of pretraining corpus to influence performance. |
| Outcome: | The proposed model outperforms pretrained models on eight datasets covering four domains with 0.45% of the data and a three-orders-of-magnitude lower computational cost. |
Exclusive Supermask Subnetwork Training for Continual Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Continual Learning (CL) methods focus on accumulating knowledge over time while preventing catastrophic forgetting. |
| Approach: | They propose a CL method that finds a supermask for each new task that keeps or removes each weight to produce a subnetwork. |
| Outcome: | The proposed method outperforms strong previous methods on NLP and Vision domains while preventing forgetting. |
Transferring General Multimodal Pretrained Models to Text Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for text recognition rely on large-scale pretraining on human-annotated or synthetic data. |
| Approach: | They propose a method to transfer multimodal pretrained models to text recognition using image captioning. |
| Outcome: | The proposed method outperforms the baselines and achieves state-of-the-art performance in the Chinese text recognition benchmark. |
A Formal Perspective on Byte-Pair Encoding (2023.findings-acl)
Copied to clipboard
| Challenge: | Byte-Pair Encoding (BPE) is a popular algorithm used for tokenizing data in NLP, but the underlying optimization problem that BPE seeks to solve has not yet been laid down. |
| Approach: | They propose an algorithm which is a 1/sigma*(1-e(-sigma))-approximation of an optimal merge sequence. |
| Outcome: | The proposed algorithm improves the runtime complexity from O(NM) to O(N log M) and the lower bound of the approximation is approx0.37. |
Automatic Named Entity Obfuscation in Speech (2023.findings-acl)
Copied to clipboard
| Challenge: | Identifying, replacing and inserting replacement named entities synthesized using voice cloning into original audio reduces the likelihood of deanonymization. |
| Approach: | They propose to use a named entity recognition system built directly on speech to replace a masked language model and synthesize the replacement named entities using voice cloning. |
| Outcome: | The proposed system is built on the English speech NER dataset and tested on a sample of the LibriSpeech corpus. |
Recursion of Thought: A Divide-and-Conquer Approach to Multi-Context Reasoning with Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to generate intermediate steps (CoT) are limited by the maximum context size due to various reasons. |
| Approach: | They propose a new inference framework that introduces several special tokens that the models can output to trigger context-related operations. |
| Outcome: | Extensive experiments with multiple architectures including GPT-3 show that the proposed framework significantly improves LMs’ inference capability. |
UniS-MMC: Multimodal Classification via Unimodality-supervised Multimodal Contrastive Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing multimodal fusion methods ignore inter-modality relationship, treat each modality equally, suffer sensor noise, and thus reduce multimodal learning performance. |
| Approach: | They propose a multimodal contrastive method to explore more reliable multimodal representations under the weak supervision of unimodal predicting. |
| Outcome: | The proposed method outperforms current state-of-the-art multimodal learning methods on image-text classification benchmarks UPMC-Food-101 and N24News. |
Robustness-Aware Word Embedding Improves Certified Robustness to Adversarial Word Substitutions (2023.findings-acl)
Copied to clipboard
| Challenge: | Embedding interval bound constraint is important for NLP models to be certified robust, but adversarial examples can be crafted by synonym substitutions. |
| Approach: | They propose a triplet loss to train robustness-aware word embeddings for better certified robustness. |
| Outcome: | The proposed method outperforms state-of-the-art certified defense baselines and generalizes well to unseen substitutions. |
Exploring the Compositional Generalization in Context Dependent Text-to-SQL Parsing (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models struggle on the text-to-SQL benchmarks, but we propose a method to improve their generalization ability. |
| Approach: | They propose a method to improve the combinatorial generalization of Text-to-SQL models by aligning previous SQL statements with the input utterance. |
| Outcome: | The proposed method improves the generalization ability of Text-to-SQL models. |
Towards Generative Event Factuality Prediction (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for event factuality prediction focus on author's presentation of factuity . a novel end-to-end generative task is proposed to predict event factuality holders, targets, and their associated factual values. |
| Approach: | They propose a generative task and system for predicting event factuality holders, targets, and their associated factual values. |
| Outcome: | The proposed system improves on the FactBank corpus and other corpora . it can predict presentation of factuality of nested sources alongside their target events . |
Can Language Models Be Specific? How? (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing pre-trained language models have a preference for more specific answers . however, there may exist multiple answers for a query, while not all answers are equally specific. |
| Approach: | They propose to build a benchmark for specificity testing by forming masked token prediction tasks with prompts. |
| Outcome: | The proposed methods improve the specificity of pre-trained language models without additional training. |
The Web Can Be Your Oyster for Improving Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Pretrained language models encode a large amount of knowledge, but knowledge is frozen at the time of training, and the models become static and limited by training data. |
| Approach: | They propose an adaptive search engine assisted learning method that can self-evaluate the confidence level of PLM’s predictions and adaptively determine when to refer to the web for more data. |
| Outcome: | The proposed model outperforms retrieval-augmented methods on 16 knowledge-intensive tasks on a wide range of knowledge-related tasks. |
Enhancing Few-shot Cross-lingual Transfer with Target Language Peculiar Examples (2023.findings-acl)
Copied to clipboard
| Challenge: | We use multilingual mask language models to train a model with hard examples, but if no labeled data are available, they need to be created through human annotations. |
| Approach: | They devised a metric to select annotation candidates from an unlabeled data pool that efficiently enhance accuracy for few-shot cross-lingual transfer. |
| Outcome: | The proposed model improves accuracy for few-shot cross-lingual transfer in 20 languages and 6 tasks with high peculiarity examples. |
Overcoming Catastrophic Forgetting in Massively Multilingual Continual Learning (2023.findings-acl)
Copied to clipboard
Genta Winata, Lingjue Xie, Karthik Radhakrishnan, Shijie Wu, Xisen Jin, Pengxiang Cheng, Mayank Kulkarni, Daniel Preotiuc-Pietro
| Challenge: | Existing methods to handle catastrophic forgetting fail to retain knowledge learnt in the past when sudden shifts occur in training data distributions. |
| Approach: | They propose a learning rate scheduling method that preserves new information without strongly overwriting past knowledge. |
| Outcome: | The proposed method preserves new information without overwriting past knowledge in a multilingual continuous learning framework. |
UniFine: A Unified and Fine-grained Approach for Zero-shot Vision-Language Understanding (2023.findings-acl)
Copied to clipboard
| Challenge: | supervised methods for vision-language tasks have been well-studied, but they lack the fine-grained information needed for semantics understanding. |
| Approach: | They propose a framework to take advantage of fine-grained information for zero-shot vision-language learning, covering multiple tasks such as VQA, SNLI-VE, and VCR. |
| Outcome: | The proposed framework outperforms previous zero-shot methods on VQA and achieves substantial improvement on SNLI-VE and VCR. |
Aligning Instruction Tasks Unlocks Large Language Models as Zero-Shot Relation Extractors (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work has shown that fine-tuning large language models on large instruction-following datasets improves their performance on a wide range of NLP tasks, but they fail to outperform small LMs on relation extraction (RE), a fundamental information extraction task. |
| Approach: | They propose a framework that aligns RE with question answering (QA), a predominant task in instruction-tuning datasets. |
| Outcome: | The proposed framework outperforms small LLMs on relation extraction (RE), a fundamental information extraction task, by a large margin. |
TADA : Task Agnostic Dialect Adapters for English (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work on dialectal English NLP is task-specific, using manual annotated dialect data, weak supervision, or data augmentation. |
| Approach: | They propose a method for task-agnostic dialect adaptation by aligning non-SAE dialects with task-specific adapters from SAE. |
| Outcome: | The proposed method improves dialectal robustness on 4 dialectal variants of the GLUE benchmark without task-specific supervision. |
Generative Zero-Shot Prompt Learning for Cross-Domain Slot Filling with Inverse Prompting (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to slot filling only learn surface mapping of slot types between D S and D T and get poor generalization capability or robustness. |
| Approach: | They propose a generative zero-shot prompt learning framework for cross-domain slot filling which improves generalization and robustness than previous work. |
| Outcome: | The proposed framework improves generalization and robustness on unseen slots and an efficient prompt tuning strategy boosts performance. |
Re-appraising the Schema Linking for Text-to-SQL (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work has shown that schema linking reduces robustness of text-to-SQL models . EMSL is used to correlate natural language queries with the given database schema . |
| Approach: | They propose a grammar linking module to help model align grammar references with SQL keywords. |
| Outcome: | The proposed language model improves performance without using EMSL, the authors show . their language model is more robust, and the proposed grammar linking improves interoperability . |
Echoes from Alexandria: A Large Resource for Multilingual Book Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent research in text summarization has focused on news stories, where texts are typically short and have strong layout features. |
| Approach: | They propose a resource for multilingual book summarization that uses a new extractive-then-abstractive baseline to compare the results. |
| Outcome: | The proposed resource is the largest and first to be multilingual, featuring 5 languages and 25 language pairs. |
When Gradient Descent Meets Derivative-Free Optimization: A Match Made in Black-Box Scenario (2023.findings-acl)
Copied to clipboard
| Challenge: | Large pre-trained language models (PLMs) are expensive and may not be open-sourced due to commercial considerations and potential risks of misuse. |
| Approach: | They propose to introduce gradient descent into black-box tuning scenario . they propose a method which integrates gradient descent and derivative-free optimization . |
| Outcome: | The proposed method achieves significant performance gains over previous state-of-the-art methods. |
Align-then-Enhance: Multilingual Entailment Graph Enhancement with Soft Predicate Alignment (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to learn typed entailment graphs with predicates as nodes and enttailment relations as edges are incomplete. |
| Approach: | They propose a task to utilize entailment information from one EG to enhance another in a different language. |
| Outcome: | The proposed framework outperforms existing graphs in multilingual entailment graph enhancement tasks. |
Few-shot Classification with Hypersphere Modeling of Prototypes (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for fewshot learning use embeddings in space, but they lack expressivity and are difficult to perform statistically. |
| Approach: | They propose a method where class information is represented by hyperspheres with dynamic sizes with two sets of learnable parameters: the hypersphere’s center and the radius. |
| Outcome: | The proposed method is much more expressive than embeddings and performs better than statistical modeling. |
Structured Mean-Field Variational Inference for Higher-Order Span-Based Semantic Role Labeling (2023.findings-acl)
Copied to clipboard
| Challenge: | Span-based semantic role labeling is traditionally tackled by BIObased sequence labeling approaches. |
| Approach: | They propose to decompose the edge from predicate word to argument span into three different edges, enabling higher-order inference. |
| Outcome: | The proposed model outperforms vanilla MFVI on span-based semantic role labeling benchmarks. |
AQE: Argument Quadruplet Extraction via a Quad-Tagging Augmented Generative Approach (2023.findings-acl)
Copied to clipboard
| Challenge: | Argument mining involves multiple subtasks, but each one is insufficient for understanding argumentative structure and reasoning process. |
| Approach: | They propose a quadruplet extraction task that extracts four argumentative components . they use a generative quadragging module to augment the training of the generative framework . |
| Outcome: | The proposed method can extract arguments from a large-scale dataset. |
The Dangers of trusting Stochastic Parrots: Faithfulness and Trust in Open-domain Conversational Question Answering (2023.findings-acl)
Copied to clipboard
Sabrina Chiesurin, Dimitris Dimakopoulos, Marco Antonio Sobrevilla Cabezudo, Arash Eshghi, Ioannis Papaioannou, Verena Rieser, Ioannis Konstas
| Challenge: | Empirical studies of dialogue have shown that people use different kinds of context-dependent linguistic behavior to indicate grounding, including use of fragments, ellipsis and pronominal reference. |
| Approach: | They propose to use open-domain question answering systems as test-bed for task based dialog generation and compare open- and closed-book models to test their hypothesis. |
| Outcome: | The proposed model parrots user input while providing an unfaithful response. |
Discrete Prompt Optimization via Constrained Generation for Zero-shot Re-ranker (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies suggest re-rankers by fine-tuning pre-trained language models . however, manual search for discrete prompts is expensive and sub-optimal in transferability . |
| Approach: | They propose a discrete prompt optimization method that guides the generated texts toward optimal prompts . they propose to use large-scale language models as a zero-shot re-ranker . |
| Outcome: | The proposed method improves the performance of the re-ranker against baselines and human prompts. |
Triggering Multi-Hop Reasoning for Question Answering in Language Models using Soft Prompts and Random Walks (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods that decompose multi-hop questions into single hop sub-questions are difficult to implement. |
| Approach: | They propose to use random-walks to guide pre-trained language models to map multi-hop questions to random-walked paths that lead to the answer. |
| Outcome: | The proposed methods improve on two T5 LMs. |
Multimedia Generative Script Learning for Task Planning (2023.findings-acl)
Copied to clipboard
| Challenge: | Goal-oriented generative script learning aims to generate subsequent steps to reach a specific goal . ability to capture historical states in visual modalities provides detailed information not covered by text . |
| Approach: | They propose a goal-oriented generative script learning task to generate subsequent steps by tracking historical states in both text and vision modalities. |
| Outcome: | The proposed task outperforms baselines in three aspects of the current task. |
Label Agnostic Pre-training for Zero-shot Text Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to text classification assume a fixed set of labels . however, in real-world applications, there exists an infinite label space for describing a given text . |
| Approach: | They propose two new methods that inject aspect-level understanding into pre-trained models at train time to improve zero-shot generalization. |
| Outcome: | The proposed methods improve zero-shot generalization on a set of challenging datasets. |
Click: Controllable Text Generation with Sequence Likelihood Contrastive Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Current language models have shown impressive capability of generating fluent and grammatical text, but they often produce behaviors misaligned with human expectations. |
| Approach: | They propose a new language model called Leo for controllable text generation which employs a contrastive loss on sequence likelihood which fundamentally decreases the generation probability of negative samples. |
| Outcome: | The proposed model outperforms baselines on language detoxification, sentiment steering, and repetition reduction tasks. |
Improving Embedding-based Unsupervised Keyphrase Extraction by Incorporating Structural Information (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing unsupervised keyphrase extraction models ignore the indicative role of the highlights in certain locations, leading to wrong keyphrases extraction. |
| Approach: | They propose a Highlight-Guided Unsupervised Keyphrase Extraction model that models phrase-document relevance via the highlights of documents and calculates cross-phrase relevance between all candidate phrases. |
| Outcome: | The proposed model outperforms the state-of-the-art unsupervised keyphrase extraction models on three benchmarks. |
Towards Reasoning in Large Language Models: A Survey (2023.findings-acl)
Copied to clipboard
| Challenge: | Reasoning is a fundamental aspect of human intelligence that plays a crucial role in many intellectual activities. |
| Approach: | They propose to improve LLMs' ability to elicit reasoning by providing exemplars or prompts to model reasoning. |
| Outcome: | This paper provides a comprehensive overview of the state of knowledge on reasoning in large language models. |
Transitioning from benchmarks to a real-world case of information-seeking in Scientific Publications (2023.findings-acl)
Copied to clipboard
| Challenge: | Scientific publications are one of the primary means by which researchers disseminate their findings and discoveries to the community, but the amount of information to go through can easily become daunting and challenging. |
| Approach: | They propose a zero-shot text search case for information seeking in scientific papers that uses a biomedical IR benchmark and an industrial case of vitamin B's impact on health. |
| Outcome: | The proposed model is based on a zero-shot text search case in scientific publications . it shows that benchmarks are not faithfully representative of the task at hand . |
CLIPText: A New Paradigm for Zero-shot Text Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | Experimental results show that CLIP can be applied to zero-shot text classification tasks. |
| Approach: | They propose a CLIP model for zero-shot text classification that integrates prompt into CLIPText to better derive knowledge from CLIP. |
| Outcome: | The proposed model can be applied to a text-image matching problem and show that it can be used for language tasks. |
Rethinking Dictionaries and Glyphs for Chinese Language Pre-training (2023.findings-acl)
Copied to clipboard
| Challenge: | Large-scale pre-trained language models (PLMs) such as BERT and GPT have revolutionized various research fields in natural language processing (NLP) |
| Approach: | They propose a new learning paradigm that enhances the semantics understanding ability of Chinese PLMs with dictionary knowledge and structure of Chinese characters. |
| Outcome: | The proposed model improves on both modern Chinese understanding benchmark CLUE and ancient Chinese understanding. |
One Embedder, Any Task: Instruction-Finetuned Text Embeddings (2023.findings-acl)
Copied to clipboard
Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A. Smith, Luke Zettlemoyer, Tao Yu
| Challenge: | a new method for embedding text is developed for tasks that require specialized encoders . INSTRUCTOR is a single embedder that can generate text embeddables tailored to different tasks and domains based on instruction finetuning . |
| Approach: | They introduce a new method for computing text embeddings given task instructions . they first annotate instructions for 330 diverse tasks and train INSTRUCTOR on this multitask mixture . |
| Outcome: | The proposed method improves on 70 embedding evaluation tasks with fewer parameters than the previous best model. |
Towards Speech Dialogue Translation Mediating Speakers of Different Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | a new task is proposed to mediate speakers of different languages using speech dialogue translation . we consider context as an important aspect that needs to be addressed in this task . speech translation (ST) has also recently shown success in monologue translation - but no study has focused on ST of dialogues . |
| Approach: | They propose a task to mediate speakers of different languages using speech dialogue translation . they construct a speechBSD dataset and conduct baseline experiments . |
| Outcome: | The proposed task mediates speakers of different languages using speech dialogue translation dataset . it shows that bilingual context performs better in our settings . |
Adaptation Approaches for Nearest Neighbor Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Semi-parametric Nearest Neighbor Language Models (kNN-LMs) have produced impressive gains over purely parametric LMs, however, there has been little investigation into adapting such models for new domains. |
| Approach: | They propose to adapt kNN-LMs to expand neighborhood retrieval over an additional adaptation datastore and adapt the weights of retrieved neighbors using a learned Rescorer module. |
| Outcome: | The proposed approach outperforms purely parametric adaptation and zero-shot models and achieves perplexity improvements of 17.1% and 16% across domains. |
Language Models for German Text Simplification: Overcoming Parallel Data Scarcity through Style-specific Pre-training (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to train automatic text simplification systems for languages other than English are limited by the lack of parallel data. |
| Approach: | They propose to use German Easy Language as a corpus of automatic text simplification systems to fine-tune language models to the style characteristics of the language. |
| Outcome: | The proposed language models adapt to the style characteristics of Easy Language and output more accessible texts. |
Client-Customized Adaptation for Parameter-Efficient Federated Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models have a large memory footprint and are difficult to use in federated learning (FL) |
| Approach: | They propose a hypernetwork-based FL framework that generates client-specific adapters by conditioning the client information. |
| Outcome: | The proposed framework maximizes the utility of shared model parameters while minimizing divergence caused by client heterogeneity. |
FolkScope: Intention Knowledge Graph Construction for E-commerce Commonsense Discovery (2023.findings-acl)
Copied to clipboard
Changlong Yu, Weiqi Wang, Xin Liu, Jiaxin Bai, Yangqiu Song, Zheng Li, Yifan Gao, Tianyu Cao, Bing Yin
| Challenge: | Existing intention-based studies on recommendation tasks are limited and use models to implicitly model the intention memberships. |
| Approach: | They propose a framework that leverages the generation power of large language models and human-in-the-loop annotation to semi-automatically construct the intention knowledge graph. |
| Outcome: | The proposed framework can model e-commerce knowledge and have many potential applications. |
I am PsyAM: Modeling Happiness with Cognitive Appraisal Dimensions (2023.findings-acl)
Copied to clipboard
| Challenge: | Emotions are an indicator of psychological states, such as happiness, which can be modelled through cognitive appraisal theory (CAT) |
| Approach: | They propose to use adaptor modules in a sequential multi-task learning setup to generate high-dimensional feature representations of hedonic well-being (momentary happiness) they propose to apply feature fusion methods to model emotion in text . |
| Outcome: | The proposed framework has cross-task validity and generalizability and is robust against traditional methods and BERT baselines. |
Value type: the bridge to a better DST model (2023.findings-acl)
Copied to clipboard
| Challenge: | Value type of the slots can provide lots of useful information for DST tasks. however, it has been ignored in most previous works. |
| Approach: | They propose a new framework for DST task based on slot value type . they propose to extract the type of token from each turn and train a Ner model to extract corresponding type-entity from each conversation according to the token. |
| Outcome: | The proposed framework is effective on two multi-domain task-oriented conversation datasets. |
Hypothetical Training for Robust Machine Reading Comprehension of Tabular Context (2023.findings-acl)
Copied to clipboard
| Challenge: | Counterfactual training is expensive because of the complexity of tabular data. |
| Approach: | They propose a hypothetical training framework that uses paired examples with different hypothetical questions to supervise the direction of model gradient towards the counterfactual answer change. |
| Outcome: | The proposed framework improves on tabular MRC datasets. |
BanglaBook: A Large-scale Bangla Dataset for Sentiment Analysis from Book Reviews (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing literature on Bangla Sentiment Analysis (SA) has limited data and cross-domain adaptability. |
| Approach: | They present a large-scale dataset of Bangla book reviews with 158,065 samples . they employ a range of machine learning models to establish baselines including SVM, LSTM, and Bangla-BERT. |
| Outcome: | The proposed model improves performance over models that rely on manual features. |
Risks and NLP Design: A Case Study on Procedural Document QA (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing language models that answer recipes better than humans can mitigate risks to users. |
| Approach: | They propose to specialize the analysis to more concrete applications and their plausible users. |
| Outcome: | The proposed model answers recipes as well or better than humans who answered the questions on the web. |
The Diminishing Returns of Masked Language Models to Science (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have shown that masked language models can improve downstream tasks by pretraining larger models for longer on more data. |
| Approach: | They empirically evaluate the extent to which these results extend to tasks in science by using 14 domain-specific transformer-based masked language models. |
| Outcome: | The proposed model can improve on 12 scientific tasks, but not all. |
Causal Matching with Text Embeddings: A Case Study in Estimating the Causal Effects of Peer Review Policies (2023.findings-acl)
Copied to clipboard
| Challenge: | Using a method that uses text embeddings, we find that human judges prefer manuscript matches from our method in 70% of cases. |
| Approach: | They propose to use text embeddings to integrate a method to estimate the causal effects of peer review policies in publication venues that shift policies from single-blind to double-blinded. |
| Outcome: | The proposed method is compared with a widely-used method of stratified propensity score matching and a baseline of randomly selected matches. |
Learning to Generalize for Cross-domain QA (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for QA are hampered by increased training costs . current methods suffer significant performance degradation when applied to out-of-domain examples. |
| Approach: | They propose a method that combines prompting methods and linear probing with fine-tuning strategy, which does not entail additional cost. |
| Outcome: | The proposed method outperforms state-of-the-art baselines with an average increase in F1 score of 4.5%-7.9%. |
Enhanced Chart Understanding via Visual Language Pre-training on Plot Table Pairs (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to understand chart plots are difficult to apply to visual-language tasks. |
| Approach: | They propose a V+L model that learns how to interpret table information from chart images via cross-modal pre-training on plot table pairs. |
| Outcome: | The proposed model outperforms state-of-the-art models on the chartQA benchmark by over 8% performance gains. |
Importance of Synthesizing High-quality Data for Text-to-SQL Parsing (2023.findings-acl)
Copied to clipboard
Yiqun Hu, Yiyun Zhao, Jiarong Jiang, Wuwei Lan, Henghui Zhu, Anuj Chauhan, Alexander Hanbo Li, Lin Pan, Jun Wang, Chung-Wei Hang, Sheng Zhang, Jiang Guo, Mingwen Dong, Joseph Lilien, Patrick Ng, Zhiguo Wang, Vittorio Castelli, Bing Xiang
| Challenge: | Existing text-to-SQL parsers lack the data to perform well with augmented synthetic data. |
| Approach: | They propose a framework that imposes strong typing constraints and incorporates key relationships from schema. |
| Outcome: | The proposed framework improves on the high-quality synthesized SQL and natural language question (NLQ) models have significant accuracy boosts and achieve new state-of-the-art performance on spider. |
Exploring Schema Generalizability of Text-to-SQL (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing text-to-SQL models are limited in their generalizability, despite their performance being over-estimated. |
| Approach: | They propose a framework to generate novel text-to-SQL data via automatic and synchronous (DS, SQL) pair altering. |
| Outcome: | The proposed framework generates text-to-SQL data via automatic and synchronous (DS, SQL) pair altering. |
Enhancing Cross-lingual Natural Language Inference by Soft Prompting with Multilingual Verbalizer (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to cross-lingual natural language inference lack annotated parallel corpora. |
| Approach: | They propose a new prompt learning framework with the Multilingual Verbalizer for XNLI that uses a multilingual verbalizer to align the representations of original and augmented multilingual questions into a unified semantic space with consistency regularization. |
| Outcome: | The proposed framework outperforms existing methods under few-shot and full-shot cross-lingual transfer settings. |
A Confidence-based Partial Label Learning Model for Crowd-Annotated Named Entity Recognition (2023.findings-acl)
Copied to clipboard
Limao Xiong, Jie Zhou, Qunxi Zhu, Xiao Wang, Yuanbin Wu, Qi Zhang, Tao Gui, Xuanjing Huang, Jin Ma, Ying Shan
| Challenge: | Existing models for named entity recognition (NER) are based on large-scale labeled datasets, which always obtain using crowdsourcing. |
| Approach: | They propose a CONfidence-based partial Label Learning method to integrate prior and posterior confidences for crowd-annotated named entity recognition models. |
| Outcome: | The proposed model improves on real-world and synthetic datasets compared with baselines. |
Towards Zero-Shot Persona Dialogue Generation with In-Context Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to improve persona consistency on high-quality human-labeled persona datasets face high cost and poor scalability. |
| Approach: | They propose a method to improve zero-shot persona consistency via in-context learning by pre-training a persona-augmented dialogue generation model and then using in-constant prompting mechanism to realize zero- shot persona customization. |
| Outcome: | The proposed method improves persona consistency without compromising coherence and informativeness in zero-shot settings. |
Grammar-based Decoding for Improved Compositional Generalization in Semantic Parsing (2023.findings-acl)
Copied to clipboard
| Challenge: | Sequence-to-sequence (seq2sequ) models have been successful in semantic parsing tasks but struggle on out-of-distribution data. |
| Approach: | They propose to use a large-scale dialogue dataset to evaluate compositional generalization of semantic parsing. |
| Outcome: | The proposed model outperforms BART- and T5-based models on the SMCalflow-CS dataset on the zero-shot learning task. |
Exploiting Rich Textual User-Product Context for Improving Personalized Sentiment Analysis (2023.findings-acl)
Copied to clipboard
| Challenge: | Typical approaches do not exploit the potential of historical reviews or do not make full use of user/product associations. |
| Approach: | They propose to use historical reviews to initialize user and product representations and incorporate textual associations via a user-product cross-context module. |
| Outcome: | The proposed method outperforms existing state-of-the-art models on IMDb, Yelp and Longformer benchmarks. |
Efficient Out-of-Domain Detection for Sequence to Sequence Models (2023.findings-acl)
Copied to clipboard
Artem Vazhentsev, Akim Tsvigun, Roman Vashurin, Sergey Petrakov, Daniil Vasilev, Maxim Panov, Alexander Panchenko, Artem Shelmanov
| Challenge: | Sequence-to-sequence (seq2sequ) models are a ubiquitous tool for text generation but they are not suitable for many other tasks. |
| Approach: | They propose to use UE techniques to identify out-of-domain (OOD) inputs where the model is susceptible to errors. |
| Outcome: | The proposed methods outperform heavyweight ensembles on the task of OOD detection. |
Emotion Cause Extraction on Social Media without Human Annotation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have focused on extracting emotion causes from news articles, but lack of fine-grained annotations has limited the ECE task. |
| Approach: | They propose a new ECE framework that extracts emotion causes from social media data without relying on human annotations. |
| Outcome: | The proposed framework achieves high extraction performance and generalizability without relying on human annotations. |
Pseudo Outlier Exposure for Out-of-Distribution Detection using Pretrained Transformers (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to detect out-of-distribution (OOD) samples are overconfident for real-world language applications. |
| Approach: | They propose a method that constructs a surrogate OOD dataset by sequentially masking tokens related to ID classes. |
| Outcome: | The proposed method can train a rejection network with ID and diverse outlier samples but requires additional data collection overhead. |
Adversarial Multi-task Learning for End-to-end Metaphor Detection (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to learn basic sense discrimination (BSD) are limited in training data. |
| Approach: | They propose a multi-task learning framework to transfer MD knowledge to basic sense discrimination using word sense disambiguation. |
| Outcome: | The proposed framework can mitigate the data scarcity problem in metaphor detection. |
SERENGETI: Massively Multilingual Language Models for Africa (2023.findings-acl)
Copied to clipboard
| Challenge: | Pretrained models acquire valuable, generalizable linguistic information during pretraining and have advanced the state of the art on task-specific finetuning. |
| Approach: | They develop a set of massively multilingual language models that covers 517 African languages and language varieties. |
| Outcome: | The proposed models outperform 4 models that cover 4-23 African languages on eight natural language understanding tasks, achieving 82.27 average F_1. |
Prompt- and Trait Relation-aware Cross-prompt Essay Trait Scoring (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing systems assume to grade essays on same prompt as used in training and assign only a holistic score. |
| Approach: | They propose a prompt- and trait relation-aware cross-prompt essay trait scorer that encodes prompt-awful essay representation by essay-promotion attention and utilizing the topic-coherence feature extracted by the topic model. |
| Outcome: | The proposed model shows state-of-the-art results for all prompts and traits. |
AugESC: Dialogue Augmentation with Large Language Models for Emotional Support Conversation (2023.findings-acl)
Copied to clipboard
| Challenge: | Crowdsourced dialogue corpora are limited in scale and topic coverage due to the expensive cost of data curation. |
| Approach: | They construct an augmented dataset for the emotional support conversation task using large language models for dialogue augmentation. |
| Outcome: | The proposed approach outperforms baselines of dialogue augmentation and improves the model's generalization ability to open-domain topics. |
2*n is better than n2: Decomposing Event Coreference Resolution into Two Tractable Problems (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for training coreference systems sample from a largely skewed distribution, making it difficult to learn coreference beyond surface matching. |
| Approach: | They propose a heuristic to efficiently filter out a large number of non-coreferent pairs and a training approach on a balanced set of coreferent and non- coreferente mention pairs. |
| Outcome: | The proposed approach significantly reduces compute requirements on two popular ECR datasets while reducing the computational complexity. |
SCCS: Semantics-Consistent Cross-domain Summarization via Optimal Transport Alignment (2023.findings-acl)
Copied to clipboard
Jielin Qiu, Jiacheng Zhu, Mengdi Xu, Franck Dernoncourt, Trung Bui, Zhaowen Wang, Bo Li, Ding Zhao, Hailin Jin
| Challenge: | Existing methods for multimodal summarization ignore the structure and semantics of the whole video and article. |
| Approach: | They propose a semantic-consistent cross-domain summarization model that extracts features from video and article and uses fusion methods to select representative one. |
| Outcome: | The proposed model produces high-quality multimodal summaries on three MSMO datasets. |
General-to-Specific Transfer Labeling for Domain Adaptable Keyphrase Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Large distribution shifts among different domains hinder transferability of keyphrase generation models. |
| Approach: | They propose a pipeline which guides KPG models’ learning focus from general syntactical features to domain-related semantics in a data-efficient manner. |
| Outcome: | The proposed pipeline can produce good quality keyphrases in new domains and achieve consistent improvements after adaptation with limited in-domain annotated data. |
E-NER: Evidential Deep Learning for Trustworthy Named Entity Recognition (2023.findings-acl)
Copied to clipboard
Zhen Zhang, Mengting Hu, Shiwan Zhao, Minlie Huang, Haotian Wang, Lemao Liu, Zhirui Zhang, Zhe Liu, Bingzhe Wu
| Challenge: | Named entity recognition (NER) systems focus on improving model performance, ignoring the need to quantify model uncertainty. |
| Approach: | They propose to introduce two uncertainty-guided loss terms to the conventional EDL and a series of uncertainty-guiding training strategies to solve these challenges. |
| Outcome: | The proposed method achieves better OOV/OOD detection performance and generalization ability on OOV entities compared to state-of-the-art methods. |
LMCap: Few-shot Multilingual Image Captioning by Retrieval Augmented Language Model Prompting (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing captioning models are mostly available on English or are specialised for other high-resource languages. |
| Approach: | They propose an image-blind few-shot multilingual captioning model that prompts a language model with retrieved captions. |
| Outcome: | The proposed model is competitive with fully-supervised multilingual captioning models without requiring any supervised training on captioning data. |
Boosting Text Augmentation via Hybrid Instance Filtering Framework (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing text augmentation methods generate instances with shifted feature spaces, which leads to a drop in performance on large datasets. |
| Approach: | They propose a hybrid instance-filtering framework that generates instances with shifted feature spaces, which leads to a drop in performance on augmented data. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on three classification tasks and nine public datasets. |
Gradient-Boosted Decision Tree for Listwise Context Model in Multimodal Review Helpfulness Prediction (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have shown that FCNNs perform inefficient splitting for review features, making it difficult to clearly differentiate helpful from unhelpful reviews. |
| Approach: | They propose a listwise attention network that captures the MRHP ranking context and a pairwise optimization objective that enhances model generalization. |
| Outcome: | The proposed framework achieves state-of-the-art results and polished generalization performance on two large-scale MRHP benchmark datasets. |
Extract and Attend: Improving Entity Translation in Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to improve entity translation in Neural machine translation still suffer from inaccurate translation of entities due to the lack of entity training instances. |
| Approach: | They propose an extract-and-tend approach to enhance entity translation in NMT by extracting entities from a dictionary and attending to them with a prefix. |
| Outcome: | Experiments on En-Zh and En-Ru show that the proposed approach improves translation accuracy and translation quality. |
Real-World Compositional Generalization with Disentangled Sequence-to-Sequence Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to compositional generalization have been designed with semantic parsing in mind. |
| Approach: | They propose a disentangled sequence-to-sequence model which encourages more disentanglement and improves its compute and memory efficiency. |
| Outcome: | The proposed model improves generalization performance across existing tasks and datasets and a new machine translation benchmark. |
Cross-lingual AMR Aligner: Paying Attention to Cross-Attention (2023.findings-acl)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) graphs embed the semantics of a sentence in a directed acyclic graph, where concepts are represented by nodes, semantic relations between concepts by edges, and the co-references by reentrant nodes. |
| Approach: | They propose a novel aligner for Abstract Meaning Representation graphs that scales cross-lingually and can align units and spans in sentences of different languages. |
| Outcome: | The proposed aligner achieves state-of-the-art in the benchmarks and can scale cross-lingually. |
Zero-Shot Text Classification via Self-Supervised Tuning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing solutions to zero-shot text classification use pre-trained language models or large-scale annotated data. |
| Approach: | They propose a self-supervised learning paradigm to solve zero-shot text classification tasks by tuning the language models with unlabeled data. |
| Outcome: | The proposed model outperforms the state-of-the-art models on 7 out of 10 tasks and is less sensitive to prompt design. |
Logical Transformers: Infusing Logical Structures into Pre-Trained Language Models (2023.findings-acl)
Copied to clipboard
Borui Wang, Qiuyuan Huang, Budhaditya Deb, Aaron Halfaker, Liqun Shao, Daniel McDuff, Ahmed Hassan Awadallah, Dragomir Radev, Jianfeng Gao
| Challenge: | Existing pre-trained language models that ignore the logical structures underlying natural language text often lack the ability to capture and encode key logical information in the input sequences. |
| Approach: | They propose to construct logic-aware input embeddings for transformer language models through logic detection, logic mapping and hierarchical logical projections and then develop a new modeling paradigm that can upgrade existing transformer language model into logical transformers to boost their performance. |
| Outcome: | The proposed model can achieve superior performance on four important and challenging tasks. |
Large Language Models with Controllable Working Memory (2023.findings-acl)
Copied to clipboard
Daliang Li, Ankit Singh Rawat, Manzil Zaheer, Xin Wang, Michal Lukasik, Andreas Veit, Felix Yu, Sanjiv Kumar
| Challenge: | Large language models (LLMs) have led to a series of breakthroughs in natural language processing due to the massive amounts of world knowledge they memorize during pretraining. |
| Approach: | They propose a method to inject counterfactual and irrelevant contexts into standard supervised datasets to strengthen both controllability and robustness. |
| Outcome: | The proposed method improves controllability and robustness across model architectures and sizes. |
A Unified Evaluation Framework for Novelty Detection and Accommodation in NLP with an Instantiation in Authorship Attribution (2023.findings-acl)
Copied to clipboard
| Challenge: | State-of-the-art natural language processing models have been shown to achieve remarkable performance in ‘closed-world’ settings where all the labels in the evaluation set are known at training time. |
| Approach: | They propose a multi-stage task to evaluate a system's performance on pipelined novelty ‘detection’ and ‘accommodation’ tasks. |
| Outcome: | The proposed model performs poorly in ‘closed-world’ settings where all the labels in the evaluation set are known at training time. |
CDA: A Contrastive Data Augmentation Method for Alzheimer’s Disease Detection (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for detecting AD are challenging and time-consuming due to lack of data and generalizability of the models. |
| Approach: | They propose a contrastive data augmentation method which simulates the cognitive impairment of a patient by randomly deleting a proportion of text from the transcript to create negative samples. |
| Outcome: | The proposed method achieves the best performance among language-based models on the benchmark ADReSS Challenge dataset. |
Disentangling Aspect and Stance via a Siamese Autoencoder for Aspect Clustering of Vaccination Opinions (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to mining public opinions about vaccines from social media make direct usage of supervision information constraining the models to predefined aspect classes while still not distinguishing those aspects from users’ stances. |
| Approach: | They propose a model for vaccination opinion mining from social media that disentangles users’ stances from opinions via a disentangling attention mechanism and a Swapping-Autoencoder. |
| Outcome: | The proposed model outperforms existing methods on aspect-based opinion mining. |
Temporal Relation Classification using Boolean Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | a new approach for temporal relation classification (TRC) is proposed . a boolean question answering model is used to classify temporal relations between two events . |
| Approach: | They propose an efficient approach for temporal relation classification using a boolean question answering model based on TRC annotation guidelines. |
| Outcome: | The proposed model outperforms state-of-the-art models by 2.4% on questions designed by human annotation experts. |
Are Synonym Substitution Attacks Really Synonym Substitution Attacks? (2023.findings-acl)
Copied to clipboard
| Challenge: | In synonym substitution attacks, an adversarial sample is constructed by substituting words in the original sentence with their synonyms. |
| Approach: | They examine how synonym substitution attacks replace words in the original sentence and show that there are still unresolved obstacles that make current SSAs generate invalid adversarial samples. |
| Outcome: | The proposed methods generate large fractions of invalid substitution words that are ungrammatical or do not preserve the original sentence’s semantics. |
DivHSK: Diverse Headline Generation using Self-Attention based Keyword Selection (2023.findings-acl)
Copied to clipboard
| Challenge: | Diverse headline generation is an NLP task where the goal is to generate multiple headlines that are true to the content of the article but are different among themselves. |
| Approach: | They propose a novel model that generates multiple diverse headlines by using a pre-trained encoder and a cluster-specific keyword set. |
| Outcome: | The proposed model outperforms existing literature and their strong baselines and emerges as a state-of-the-art model. |
Similarity-Based Content Scoring - A more Classroom-Suitable Alternative to Instance-Based Scoring? (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work suggests that similarity-based content scoring methods can yield comparable results to instance-based supervised learning. |
| Approach: | They propose to use similarity-based scoring to achieve similar results . they compare different instance-based and similarity based methods on multiple data sets . |
| Outcome: | The proposed approach has a lower need for annotated training data and better zero-shot performance, but the results are not consistent with previous studies. |
Pragmatic Inference with a CLIP Listener for Contrastive Captioning (2023.findings-acl)
Copied to clipboard
| Challenge: | a new method for contrastive captioning generates discriminative captions that distinguish target images from very similar alternative distractor images. |
| Approach: | They propose a pragmatic inference procedure that formulates captioning as a reference game between a speaker and a listener. |
| Outcome: | The proposed method outperforms previous methods for discriminative captioning by 11% to 15% accuracy in human evaluations. |
A Statistical Exploration of Text Partition Into Constituents: The Case of the Priestly Source in the Books of Genesis and Exodus (2023.findings-acl)
Copied to clipboard
Gideon Yoffe, Axel Bühler, Nachum Dershowitz, Thomas Romer, Eli Piasetzky, Israel Finkelstein, Barak Sober
| Challenge: | Existing versions of the Hebrew Bible are a result of various editorial actions, such as additions, redactions, and other factors. |
| Approach: | They propose a statistical stylometric pipeline for a hypothesized partition of a text. |
| Outcome: | The proposed pipeline detects literary features yielding the optimal overlap between hypothesized and unsupervised partitions while conserving implicit correlations between units of text that are more likely to be grouped. |
A Language-First Approach for Procedure Planning (2023.findings-acl)
Copied to clipboard
| Challenge: | Developing intelligent agents requires the ability to produce plans on the fly based on visual observations. |
| Approach: | They propose a language-first procedure planning framework with a modularized design . they first align current and goal observations with corresponding steps and then use a pre-trained LM to predict intermediate steps. |
| Outcome: | The proposed framework matches state-of-the-art procedures on COIN and CrossTask benchmarks. |
An Empirical Analysis of Leveraging Knowledge for Low-Resource Task-Oriented Semantic Parsing (2023.findings-acl)
Copied to clipboard
Mayank Kulkarni, Aoxiao Zhong, Nicolas Guenon des mesnards, Sahar Movaghati, Mukund Sridhar, He Xie, Jianhua Lu
| Challenge: | Task-oriented semantic parsing is a new approach to represent the meaning of user requests with arbitrarily nested semantics. |
| Approach: | They propose to use knowledge-enhanced encoders to parse user requests with arbitrarily nested semantics. |
| Outcome: | The proposed model improves performance in low-resource and low-compute settings. |
TempLM: Distilling Language Models into Template-Based Generators (2023.findings-acl)
Copied to clipboard
| Challenge: | Pretrained language models (PLMs) have greatly improved text generation, but they have also been known to produce unfaithful or inappropriate content. |
| Approach: | They propose a pretrained language model that is a template-based generator and uses it to generate a text. |
| Outcome: | The proposed model is more faithful than the original PLM and more fluent than prior template systems. |
Incorporating Graph Information in Transformer-based AMR Parsing (2023.findings-acl)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) is a semantic graph abstraction for text representations. |
| Approach: | They propose a model and method that incorporates graph information into the learned representations of AMR by word-to-node alignment. |
| Outcome: | The proposed model improves AMR parsing performance by embedding graph information into the encoder at training time. |
Rethinking the Word-level Quality Estimation for Machine Translation from Human Judgement (2023.findings-acl)
Copied to clipboard
| Challenge: | Word-level Quality Estimation (QE) of Machine Translation aims to detect potential translation errors in the translated sentence without reference. |
| Approach: | They propose to use a human-generated translation judgment to generate a word-level quality estimate (QE) using a translation error rate toolkit to detect translation errors without reference. |
| Outcome: | The proposed dataset is more consistent with human judgment and confirms the effectiveness of the proposed tag-correcting strategies. |
PV2TEA: Patching Visual Modality to Textual-Established Information Extraction (2023.findings-acl)
Copied to clipboard
| Challenge: | Empirical results show up to 11.74% absolute (20.97% relative) increase over unimodal baselines. |
| Approach: | They propose to patch the visual modality to the textual-established attribute in- formation extractor. |
| Outcome: | Empirical results show up to 11.74% absolute (29.9% relative) increase over unimodal baselines. |
Structural Contrastive Pretraining for Cross-Lingual Comprehension (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to train multilingual language models using pretraining tasks like mask language modeling have yielded promising results on a wide range of downstream tasks. |
| Approach: | They propose a new task to align the structural words in a parallel sentence, enhancing models’ ability to comprehend cross-lingual representations. |
| Outcome: | The proposed task improves model's ability to comprehend cross-lingual representations by increasing the frequency of negative pairings. |
Reducing Sensitivity on Speaker Names for Text Generation from Dialogues (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models are sensitive to nuances, resulting in unfairness in real-world applications. |
| Approach: | They propose to quantitatively measure a model's sensitivity on speaker names and comprehensively evaluate a number of known methods for reducing speaker name sensitivity. |
| Outcome: | The proposed approach reduces speaker name sensitivity and improves quality of generation. |
Topic and Style-aware Transformer for Multimodal Emotion Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that visual modality makes minimal contribution to multimodal emotion recognition due to its high dimensionality. |
| Approach: | They propose to leverage the strong multimodality backbone VATT to project the visual signal to the common space with language and acoustic signals. |
| Outcome: | The proposed model outperforms SOTA results and integrates visual signals and handles subjectivity issues by serving as content "normalization" previous studies show that visual modality makes minimal contribution to the performance of multimodal emotion recognition tasks due to high dimensionality. |
Exploiting Abstract Meaning Representation for Open-Domain Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work attempts to address these challenges using Pretrained Language Models (PLMs) but the diversity of surface form expressions can hinder the model’s ability to capture accurate correlations, especially when the context is lengthy and complex. |
| Approach: | They propose a method known as Graph-as-Token (GST) to incorporate AMRs into PLMs to assist the model in understanding complex semantic information. |
| Outcome: | The proposed method outperforms existing methods and significantly improves performance on both Natural Questions and TriviaQA. |
Nonparametric Masked Language Modeling (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing language models (LMs) predict tokens with a softmax over a finite vocabulary, which can make it difficult to predict rare tokens or phrases. |
| Approach: | They introduce a nonparametric masked language model that replaces a softmax with a distribution over every phrase in a reference corpus and uses an in-batch approximation to train it. |
| Outcome: | The proposed model outperforms larger parametric models on 16 tasks including classification, fact probing and question answering. |
Pay More Attention to Relation Exploration for Knowledge Base Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches focus on entity representation and final answer reasoning, which results in limited supervision for this task. |
| Approach: | They propose a framework that utilizes relations to enhance entity representation and introduce additional supervision. |
| Outcome: | The proposed framework improves the F1 score on two benchmark datasets by 5.8% . it improves by 6.7% on WebQSP, better than state-of-the-art methods . |
Speaking Multiple Languages Affects the Moral Bias of Language Models (2023.findings-acl)
Copied to clipboard
Katharina Haemmerl, Bjoern Deiseroth, Patrick Schramowski, Jindřich Libovický, Constantin Rothkopf, Alexander Fraser, Kristian Kersting
| Challenge: | Pre-trained multilingual language models are often better on English than other languages . however, they are trained on varying amounts of data for each language . |
| Approach: | They apply the MORALDIRECTION framework to multilingual models and analyse their results . they find that PMLMs encode differing moral biases, but these do not correspond to cultural differences or commonalities in human opinions. |
| Outcome: | The proposed model captures moral norms from English and imposes them on other languages. |
Retrieving Relevant Context to Align Representations for Cross-lingual Event Detection (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to cross-lingual transfer learning for event detection are mixed with event-discriminative context. |
| Approach: | They propose a method where representations are augmented with additional context to bridge the gap between languages while enriching contextual information to facilitate ED. |
| Outcome: | The proposed model performs well on three languages. |
NormNet: Normalize Noun Phrases for More Robust NLP (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to debunk false features in deep NLP models are inadequate . previous work suggests that models learn spurious features instead of the true signals of the task . |
| Approach: | They propose a normalization strategy to eliminate false features caused by the textual surfaces of noun phrases. |
| Outcome: | The proposed normalization strategy eliminates false features caused by the textual surfaces of noun phrases. |
Cross Encoding as Augmentation: Towards Effective Educational Text Classification (2023.findings-acl)
Copied to clipboard
Hyun Seung Lee, Seungtaek Choi, Yunsung Lee, Hyeongdon Moon, Shinhyeok Oh, Myeongho Jeong, Hyojun Go, Christian Wallraven
| Challenge: | Existing methods to improve text classification in education suffer from data scarcity . authors propose a retrieval approach that provides effective learning in educational text classification. |
| Approach: | They propose a retrieval approach that provides effective learning in educational text classification by introducing cross-encoder style texts to a bi-encoding architecture. |
| Outcome: | The proposed method is effective in multi-label scenarios and low-resource tags compared to state-of-the-art models. |
Adversarial Robustness of Prompt-based Few-Shot Learning for Natural Language Understanding (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent few-shot learning methods focus on improving downstream task performance, but there is limited understanding of the adversarial robustness of such methods. |
| Approach: | They evaluate prompt-based FSL methods against fully fine-tuned models to better understand the impact of various factors towards robustness. |
| Outcome: | The proposed methods show that they are less robust in the face of adversarial perturbations than fully fine-tuned models. |
This prompt is measuring <mask>: evaluating bias evaluation in language models (2023.findings-acl)
Copied to clipboard
| Challenge: | a growing body of work uses prompts and templates to assess bias in language models . authors examine the scope of possible bias types and identify those under-researched . |
| Approach: | They draw on a measurement modelling framework to create a bias taxonomy . they show that bias tests are often unstated or ambiguous, carry implicit assumptions . |
| Outcome: | The proposed taxonomy shows that bias tests are often unstated or ambiguous . the analysis illuminates the scope of possible bias types the field can measure . |
Towards Open Environment Intent Prediction (2023.findings-acl)
Copied to clipboard
| Challenge: | Out-of-Domain (OOD) Intent Classification and New Intent Discovering are two tasks in the Task-Oriented Dialogue System. |
| Approach: | They propose a task paradigm to extend Out-of-Domain (OOD) Intent Classification and New Intent Discovering tasks in the Task-Oriented Dialogue System. |
| Outcome: | The proposed scheme improves on existing OOD intent classification and discovery datasets. |
Teamwork Is Not Always Good: An Empirical Study of Classifier Drift in Class-incremental Information Extraction (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for learning incrementally do not address the problem of class-incremental learning. |
| Approach: | They propose a framework that can continuously learn new classes from a data stream without forgetting previously learned classes. |
| Outcome: | The proposed framework shows significant improvement over the state-of-the-art frameworks with up to 44.7% absolute F-score gain. |
C-XNLI: Croatian Extension of XNLI Dataset (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing parallel datasets limit multilingual evaluations due to the nature of linguistic annotation, which is tedious, subjective, and costly. |
| Approach: | They extend the Cross-lingual Natural Language Inference corpus with Croatian and use Facebook's 1.2B parameter m2m_100 model to analyze the train set and compare its quality with the existing machine-translated German set. |
| Outcome: | The proposed model is consistent with other XNLI dubs and is compared with the existing machine-translated German train set. |
AVATAR: A Parallel Corpus for Java-Python Program Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Program translation is a time-consuming and costly process that requires expertise in both the source and target languages. |
| Approach: | They present a collection of 9,515 programming problems and their solutions written in Java and Python. |
| Outcome: | The proposed model lacks in generating functionally accurate code. |
On Dataset Transferability in Active Learning for Transformers (2023.findings-acl)
Copied to clipboard
| Challenge: | Active learning (AL) aims to reduce labeling costs by querying the examples most beneficial for model learning. |
| Approach: | They propose to query examples most beneficial for model learning by querying data points most informative for labeling. |
| Outcome: | The proposed method reduces labeling costs by querying the examples most beneficial for model learning. |
Structured Persuasive Writing Support in Legal Education: A Model and Tool for German Legal Case Solutions (2023.findings-acl)
Copied to clipboard
| Challenge: | An annotation approach for capturing structured components and arguments in legal case solutions of German students is proposed based on the appraisal style, which dictates the structured way of persuasive writing in law. |
| Approach: | They propose an annotation scheme with annotation guidelines that identify structured writing in legal case solutions. |
| Outcome: | The proposed approach captures the structure of a persuasive legal text and can be predicted using transformer-based models. |
Characterizing the Impacts of Instances on Robustness (2023.findings-acl)
Copied to clipboard
Rui Zheng, Zhiheng Xi, Qin Liu, Wenbin Lai, Tao Gui, Qi Zhang, Xuanjing Huang, Jin Ma, Ying Shan, Weifeng Ge
| Challenge: | Existing defense approaches focus on developing new model structures or training algorithms, but they do little to tap the potential of training instances. |
| Approach: | They propose a method that can distinguish between robust and non-robust instances according to the model’s sensitivity to perturbations on individual instances during training. |
| Outcome: | The proposed method can distinguish between robust and non-robust instances according to the model’s sensitivity to perturbations on individual instances during training. |
Generate then Select: Open-ended Visual Question Answering Guided by World Knowledge (2023.findings-acl)
Copied to clipboard
Xingyu Fu, Sheng Zhang, Gukyeong Kwon, Pramuditha Perera, Henghui Zhu, Yuhao Zhang, Alexander Hanbo Li, William Yang Wang, Zhiguo Wang, Vittorio Castelli, Patrick Ng, Dan Roth, Bing Xiang
| Challenge: | Open-ended Visual Question Answering (VQA) requires models to reason over visual and natural language inputs using world knowledge. |
| Approach: | They propose a new VQA pipeline that deploys a generate-then-select strategy guided by world knowledge for the first time. |
| Outcome: | The proposed pipeline expands the knowledge coverage from in-domain training data by 4.1% on OK-VQA, without additional computation cost. |
Hence, Socrates is mortal: A Benchmark for Natural Language Syllogistic Reasoning (2023.findings-acl)
Copied to clipboard
Yongkang Wu, Meng Han, Yutao Zhu, Lei Li, Xinyu Zhang, Ruofei Lai, Xiaoguang Li, Yuanhang Ren, Zhicheng Dou, Zhao Cao
| Challenge: | SylloBase is a benchmark for syllogistic reasoning, a critical capability widely required in natural language understanding tasks, such as text entailment and question answering. |
| Approach: | They propose to use a benchmark to learn syllogistic reasoning on a set of templates and to use them to generate and understand slogisms. |
| Outcome: | The proposed benchmark covers a complete taxonomy of syllogism reasoning patterns, and contains both automatically and manually constructed samples. |
Categorial grammar induction from raw data (2023.findings-acl)
Copied to clipboard
| Challenge: | a new model for categorial grammar induction is based on raw data without part-of-speech information. |
| Approach: | They propose a grammar induction model that learns from raw data without part-of-speech information. |
| Outcome: | a new model for inducing a basic categorial grammar is developed . the model attains a recall-homogeneity of 0.33 on average, and a bias toward forward function application is added . |
Attribute Controlled Dialogue Prompting (2023.findings-acl)
Copied to clipboard
| Challenge: | Prompt-tuning is an increasingly popular parameter-efficient method for adapting large pretrained language models to downstream tasks. |
| Approach: | They propose an instance-specific prompt-tuning algorithm for dialog generation that generates prompts based on instance-level control code rather than the conversation history. |
| Outcome: | The proposed prompt-tuning module is a fraction of the size of the pretrained language model and saves memory and expensive storage space. |
Open-World Factually Consistent Question Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for question generation suffer from factual inconsistencies and incorrect entities and are not answerable from the input paragraph. |
| Approach: | They propose a data processing technique based on de-lexicalization for consistent question generation across domains and a model that is generic across question-generation models. |
| Outcome: | The proposed method produces entity-level factually consistent questions without significant impact on traditional metrics. |
Contrastive Learning of Sociopragmatic Meaning in Social Media (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent progress in representation and contrastive learning in NLP has not considered the class of sociopragmatic meaning (i.e., meaning in interaction within different language communities). |
| Approach: | They propose a framework for learning task-agnostic representations transferable to a wide range of sociopragmatic tasks. |
| Outcome: | The proposed framework outperforms other contrastive learning frameworks for both in-domain and out-of-domain data, across both the general and few-shot settings. |
Noisy Positive-Unlabeled Learning with Self-Training for Speculative Knowledge Graph Reasoning (2023.findings-acl)
Copied to clipboard
Ruijie Wang, Baoyu Li, Yichen Lu, Dachun Sun, Jinning Li, Yuchen Yan, Shengzhong Liu, Hanghang Tong, Tarek Abdelzaher
| Challenge: | State-of-the-art methods fail in speculative reasoning task on knowledge graphs . state-of the-art approaches assume correctness of fact is determined by its presence in KG . |
| Approach: | They propose a speculative reasoning task on real-world knowledge graphs . they propose nPUGraph that estimates correctness of both collected and uncollected facts . |
| Outcome: | The proposed framework improves the robustness of a label posterior-aware graph encoder against false positive links and identifies missing facts to provide high-quality grounds of reasoning. |
ACROSS: An Alignment-based Framework for Low-Resource Many-to-One Cross-Lingual Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies ignore data imbalance in multilingual settings and do not utilize monolingual data. |
| Approach: | They propose a cross-lingual summarization model that aligns cross-linguistic data with high-resource monolingual data via contrastive and consistency loss. |
| Outcome: | The proposed model outperforms baseline models and consistently dominates on 45 language pairs. |
RFiD: Towards Rational Fusion-in-Decoder for Open-Domain Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | Open-domain Question Answering (ODQA) systems rely on spurious features instead of genuine causal relationships to generate answers. |
| Approach: | They propose a model that leverages the encoders of FiD to distinguish between causal relationships and spurious features and guides the decoder to generate answers informed by this discernment. |
| Outcome: | The proposed model improves on two ODQA datasets and shows that it can identify causal relationships and identify spurious features. |
Unsupervised Keyphrase Extraction by Learning Neural Keyphrase Set Function (2023.findings-acl)
Copied to clipboard
| Challenge: | Unsupervised keyphrase extraction is a task of extracting a keyphrase set that provides readers with highlevel information about the key ideas or important topics described in the document. |
| Approach: | They propose an unsupervised keyphrase extraction task that is a document-set matching problem instead of modeling the relevance between an individual phrase and the document. |
| Outcome: | The proposed model outperforms the state-of-the-art unsupervised keyphrase extraction baselines by a large margin. |
Diffusion Theory as a Scalpel: Detecting and Purifying Poisonous Dimensions in Pre-trained Language Models Caused by Backdoor or Bias (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to fine-tune pre-trained language models (PLMs) are not safe, since the fine-uning process is invisible to the user. |
| Approach: | They propose a technique to study the dynamic process of fine-tuning for finding poisonous dimensions using diffusion theory. |
| Outcome: | The proposed approach can detect poisonous dimensions with abnormal dynamics, purify them and fine-tune them on a clean dataset. |
Retrieving Multimodal Prompts for Generative Visual Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | Visual question answering (VQA) is a multimodal machine learning problem that challenges a model to answer a question posed about an image. |
| Approach: | They propose a generative model enhanced by multimodal prompt retrieval that integrates retrieved prompts and multimodal features to generate answers in free text. |
| Outcome: | The proposed model outperforms its non-retrieval counterpart by 30% on medical VQA tasks. |
InfoSync: Information Synchronization across Multilingual Semi-structured Tables (2023.findings-acl)
Copied to clipboard
| Challenge: | Information Synchronization of semi-structured data across languages is challenging . culture differences, topic preferences, and editing inconsistency lead to information mismatches . |
| Approach: | They propose a method for tabular synchronization that uses information from Wikipedia tables in one language with tables in another language. |
| Outcome: | The proposed method achieves an acceptance rate of 77.28% on Wikipedia . english articles across the web are more timely updated than other languages . |
T2IAT: Measuring Valence and Stereotypical Biases in Text-to-Image Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in text-to-image generative models have produced high quality images with a breakthrough of inference speed. |
| Approach: | They propose a text-to-image association test framework that quantifies implicit stereotypes between concepts and valence and those in images. |
| Outcome: | The proposed framework quantifies implicit stereotypes between concepts and valence and those in images. |
An Investigation of Evaluation Methods in Automatic Medical Note Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies show that doctors can save significant amounts of time when using automatic note generation. |
| Approach: | They propose task-specific metrics for automatic note generation from medical conversation summarization and generation, including knowledge-graph embedding-based metrics, customized model-based measures with domain-specific weights, and ensemble metrics. |
| Outcome: | The proposed evaluation metrics are compared to existing models and can have different behaviors on different types of clinical notes datasets. |
Rethinking Translation Memory Augmented Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to enhance neural machine translation (NMT) by using a TM have been reported to be effective. |
| Approach: | They propose a translation memory augmented neural machine translation model that is good at fitting data but more sensitive to fluctuations in training data. |
| Outcome: | The proposed model achieves consistent gains over conventional and existing models under two variance-preferable scenarios as well as the high resource scenario. |
Controlling Styles in Neural Machine Translation with Activation Prompt (2023.findings-acl)
Copied to clipboard
| Challenge: | Earlier studies on controlling styles in neural machine translation (NMT) have focused on regulating the level of formality, but they still encounter two major challenges. |
| Approach: | They propose a method to control the style of neural machine translation by retrieving prompts from stylized monolingual corpus. |
| Outcome: | The proposed method can control the style of translation and achieve remarkable performance. |
Focusing, Bridging and Prompting for Few-shot Nested Named Entity Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work on few-shot named entity recognition (NER) addresses flat entities instead of nested entities. |
| Approach: | They propose a method based on focusing, bridging and prompting for few-shot nested NER without using source domain data. |
| Outcome: | The proposed method outperforms baseline models on four benchmark datasets and outperformed several competing models on F1-score by 9.33% on ACE2004, 6.17% on ace2005, 9.40% on GENIA and 5.12% on KBP2017. |
Together We Make Sense–Learning Meta-Sense Embeddings (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing sense embeddings do not cover all senses of ambiguous words equally well due to discrepancies in their training resources. |
| Approach: | They propose a meta-sense embedding method that preserves sense neighbourhoods by combining multiple independently trained source sense embeddables. |
| Outcome: | The proposed method outperforms several baselines on Word Sense Disambiguation and Word-in-Context tasks. |
Multimodal Prompt Learning for Product Title Generation with Extremely Limited Labels (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to generate informative titles for products with limited labels are inadequate for novel products. |
| Approach: | They propose a prompt-based approach to generate attractive titles for novel products . they use multimodal prompts to preserve characteristics and writing styles of novel products. |
| Outcome: | The proposed approach achieves state-of-the-art results on novel product categories with limited labels. |
Large Language Models are Built-in Autoregressive Search Engines (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing dual-encoder dense retrievers obtain representations for questions and documents independently, allowing only shallow interactions between them. |
| Approach: | They propose to use large language models to generate URLs for document retrieval by following human instructions. |
| Outcome: | The proposed method achieves better retrieval performance than existing retrieval approaches on open-domain question answering benchmarks. |
Beyond Triplet: Leveraging the Most Data for Multimodal Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Multimodal machine translation (MMT) aims to improve translation quality by incorporating information from other modalities, such as vision. |
| Approach: | They propose a framework for multimodal machine translation that utilizes large-scale non-triple data and a multimodal translation dataset. |
| Outcome: | The proposed method can significantly improve translation performance with more non-triple data. |
From chocolate bunny to chocolate crocodile: Do Language Models Understand Noun Compounds? (2023.findings-acl)
Copied to clipboard
| Challenge: | Noun compound interpretation is the task of expressing a noun compound in a free-text paraphrase that makes the relationship between the constituent nouns explicit. |
| Approach: | They propose modifications to the standard task and propose a new task that solves it. |
| Outcome: | The proposed task solves the standard task of paraphrasing a noun compound in a free-text paraphrase that makes the relationship between the constituent nouns explicit. |
Measuring Intersectional Biases in Historical Documents (2023.findings-acl)
Copied to clipboard
Nadav Borenstein, Karolina Stanczak, Thea Rolskov, Natacha Klein Käfer, Natália da Silva Perez, Isabelle Augenstein
| Challenge: | digitised historical documents suffer from errors introduced by optical character recognition (OCR) and are written in an archaic language. |
| Approach: | They investigate the continuities and transformations of bias in Caribbean historical newspapers during the colonial era . they use distributional semantics models and word embeddings to measure gender, race, and intersectional biases. |
| Outcome: | The authors show that gender and racial biases are interdependent and their intersection triggers distinct effects. |
Incomplete Utterance Rewriting by A Two-Phase Locate-and-Fill Regime (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models with incomplete utterances have too large search space, resulting in poor quality of rewriting results. |
| Approach: | They propose a 2-phase rewriting framework which predicts empty slots in the utterance that need to be completed and generates the part to be filled into each position. |
| Outcome: | The proposed framework achieves state-of-the-art results on several public rewriting datasets. |
Exploring Variation of Results from Different Experimental Conditions (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent research has shown that reproducibility of NLP experiments is not guaranteed by arbitrary factors like random seed and different data splits. |
| Approach: | They propose to run the training and testing pipeline for three neural text simplification models under varying experimental conditions using the same data and train/dev/test set splits. |
| Outcome: | The proposed pipeline yields a large number of results for three neural text simplification models under varying experimental conditions. |
Playing the Part of the Sharp Bully: Generating Adversarial Examples for Implicit Hate Speech Detection (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing algorithms for hate speech detection focus on explicit forms of hate speech, but they fail to properly detect subtle and implicit HS messages. |
| Approach: | They propose a framework for generating adversarial implicit HS short-text messages using Auto-regressive language models and a strategy to group the generated messages in complexity levels. |
| Outcome: | The proposed framework shows that iteratively retraining on HARD messages significantly improves implicit HS benchmarks. |
X-RiSAWOZ: High-Quality End-to-End Multilingual Dialogue Datasets and Few-shot Agents (2023.findings-acl)
Copied to clipboard
Mehrad Moradshahi, Tianhao Shen, Kalika Bali, Monojit Choudhury, Gael de Chalendar, Anmol Goel, Sungkyun Kim, Prashant Kodali, Ponnurangam Kumaraguru, Nasredine Semmar, Sina Semnani, Jiwon Seo, Vivek Seshadri, Manish Shrivastava, Michael Sun, Aditya Yadavalli, Chaobin You, Deyi Xiong, Monica Lam
| Challenge: | X-RiSAWOZ dataset has more than 18,000 human-verified dialogue utterances for each language . Xiaoping and Xinhui are the main challenges for task-oriented dialogue research . |
| Approach: | They develop a toolkit to accelerate the post-editing of a new language dataset after translation . their dataset, code, and toolkit are released open-source . |
| Outcome: | The proposed toolkit accelerates the post-editing of a new language dataset after translation. |
Subword Segmental Machine Translation: Unifying Segmentation and Target Sentence Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Subword segmenters are used in neural machine translation, but are not used in high-resource settings. |
| Approach: | They propose a subword segmental machine translation (SSMT) that unifies subword and MT in a single trainable model. |
| Outcome: | The proposed model improves chrF scores for morphologically rich agglutinative languages and is more robust on a test set constructed for evaluating morphology generalisations. |
Measuring and Mitigating Local Instability in Deep Neural Networks (2023.findings-acl)
Copied to clipboard
| Challenge: | Uncertain details like random initialization can change the outputs of a trained system with potentially disastrous consequences. |
| Approach: | They propose a model stability problem by studying how the predictions of a deep neural network change as a consequence of stochasticity in the training process. |
| Outcome: | The proposed method outperforms data-agnostic methods and is 90% cheaper than the gold standard. |
What Knowledge Is Needed? Towards Explainable Memory for kNN-MT Domain Adaptation (2023.findings-acl)
Copied to clipboard
| Challenge: | kNN-MT builds an external datastore, which saves all target language token occurrences in the parallel corpus. |
| Approach: | They propose a new paradigm for domain adaptation by building an external datastore which usually saves all target language token occurrences in the parallel corpus. |
| Outcome: | The proposed model can be easily pruned according to local correctness, and it is more explainable. |
Measuring Your ASTE Models in The Wild: A Diversified Multi-domain Dataset For Aspect Sentiment Triplet Extraction (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing ASTE datasets are limited in their ability to represent real-world scenarios, hindering progress in this area. |
| Approach: | They propose a new ASTE dataset that is manually annotated to better fit real-world scenarios by providing more diverse and realistic reviews. |
| Outcome: | The proposed dataset is manually annotated to better fit real-world scenarios. |
Grounding the Lexical Substitution Task in Entailment (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing definitions of lexical substitutes are vague or inconsistent with the gold annotations. |
| Approach: | They propose a new definition which is grounded in the relation of entailment . they empirically validate the definition and create a dataset from existing semantic resources . |
| Outcome: | The proposed method improves the performance of existing lexical substitution systems on the existing benchmarks. |
Operator Selection and Ordering in a Pipeline Approach to Efficiency Optimizations for Transformers (2023.findings-acl)
Copied to clipboard
| Challenge: | Natural language processing tasks rely on complex neural models . transformer-based models are typically slow to execute, making it a non-trivial challenge to apply them in real-world applications. |
| Approach: | They propose to consider an efficiency method as an operator applied on a model . they find that the commutativity and cumulativeness of efficiency operators are plausible . |
| Outcome: | The proposed method is commutative and cumulative, and the results are estimated by combining methods. |
AraMUS: Pushing the Limits of Data and Model Scale for Arabic Natural Language Processing (2023.findings-acl)
Copied to clipboard
Asaad Alghamdi, Xinyu Duan, Wei Jiang, Zhenhai Wang, Yimeng Wu, Qingrong Xia, Zhefeng Wang, Yi Zheng, Mehdi Rezagholizadeh, Baoxing Huai, Peilun Cheng, Abbas Ghaddar
| Challenge: | Developing monolingual large Pre-trained Language Models (PLMs) is shown to be very successful in handling different tasks in Natural Language Processing (NLP). |
| Approach: | They present AraMUS, the largest Arabic PLM with 11B parameters trained on 529GB of high-quality Arabic textual data. |
| Outcome: | The proposed model achieves state-of-the-art performance on a diverse set of Arabic classification and generative tasks. |
Leveraging Explicit Procedural Instructions for Data-Efficient Action Prediction (2023.findings-acl)
Copied to clipboard
| Challenge: | a paper presents a data-efficient solution to constructing task-oriented dialogue systems . large language models have shown success in modeling such dialogues, but they require large quantities of data . |
| Approach: | They propose a system that leverages explicit instructions from agent guidelines . they propose dialogue-document matching and action-oriented masked language modeling . |
| Outcome: | The proposed system improves accuracy predicting in- and out-of-distribution actions while preserving high performance in settings with low or sparse data. |
Quantifying Train-Evaluation Overlap with Nearest Neighbors (2023.findings-acl)
Copied to clipboard
| Challenge: | Using a novel metric, we quantify train-evaluation overlap in benchmark datasets to evaluate model generalization. |
| Approach: | They quantify train-evaluation overlap as a measure of an individual dataset’s adequacy to evaluate model generalization over a wide range of datasets. |
| Outcome: | The proposed metric quantifies train-evaluation overlap, providing insights for constructing datasets to study generalization. |
Unsupervised Mapping of Arguments of Deverbal Nouns to Their Corresponding Verbal Labels (2023.findings-acl)
Copied to clipboard
| Challenge: | Deverbal nouns are nominal forms of verbs used in English texts to describe events or actions . many NLP systems neglect to handle nominalized constructions, resulting in limited applications . |
| Approach: | They propose to map arguments of deverbal nouns to universal-dependency relations of verbal constructions . they propose to use the same labels as verbal cases to map the arguments . |
| Outcome: | The proposed approach maps arguments of nominalized nouns to the corresponding verbal constructions. |
The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challenges (2023.findings-acl)
Copied to clipboard
| Challenge: | Code-Switching is a common phenomenon in written text and conversation . it is not so common to observe code-switching in spoken language and not in written language . |
| Approach: | They present a systematic survey on code-switching research in natural language processing to understand the progress of the past decades and conceptualize the challenges and tasks on the topic. |
| Outcome: | The proposed model combines linguistic theories and machine learning techniques to understand the code-switching phenomenon. |
Learning to Predict Persona Information for Dialogue Personalization without Explicit Persona Description (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to personalize dialogue agents rely on explicit persona descriptions during inference, which severely limits their application in real-world scenarios. |
| Approach: | They propose a method that learns to predict persona information based on the dialogue history to personalize dialogue agents without relying on explicit persona descriptions during inference. |
| Outcome: | The proposed method improves the consistency and engagingness of generated responses when conditioning on the predicted profile of the dialogue agent. |
Automated Refugee Case Analysis: A NLP Pipeline for Supporting Legal Practitioners (2023.findings-acl)
Copied to clipboard
| Challenge: | In Canada, retrieving similar cases and their analysis is a key part of legal work . long processing times are due to a significant backlog and to the amount of work required from counsels . |
| Approach: | They propose to extend existing neural named-entity recognition models to retrieve 19 categories of items from refugee cases. |
| Outcome: | The proposed pipeline achieves a superior F1- score on five of the targeted categories and superior to 80% on an additional 4 categories. |
Recurrent Attention Networks for Long-text Modeling (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to encoding long documents using self-attention have been limited by quadratic computational complexities and limited application in long text processing. |
| Approach: | They propose a long-document encoding model that allows the recurrent operation of self-attention. |
| Outcome: | The proposed model extracts global semantics in token-level and document-level representations, making it inherently compatible with both sequential and sequential tasks. |
Exploring the Relationship between Alignment and Cross-lingual Transfer in Multilingual Transformers (2023.findings-acl)
Copied to clipboard
| Challenge: | despite lack of explicit cross-lingual training data, multilingual models can achieve cross-linguistic transfer. |
| Approach: | They find alignment is significantly correlated with cross-lingual transfer . they advocate for further research on realignment methods for smaller models . |
| Outcome: | The proposed method outperforms XLM-R Large in POS-tagging between English and Arabic by +15.8 accuracy. |
Aerial Vision-and-Dialog Navigation (2023.findings-acl)
Copied to clipboard
| Challenge: | Aerial visionand-dialling navigation (AVDN) is a new approach to autonomous drones that can converse with humans and follow natural language commands to complete tasks. |
| Approach: | They propose to use Aerial Visionand-Dialog Navigation (AVDN) to navigate a drone via natural language conversation by collecting a dataset of over 3k recorded navigation trajectories with asynchronous human-human dialogs between commanders and followers. |
| Outcome: | The proposed system can converse with humans and follow natural language commands to fly to the expected destination. |
Improved Logical Reasoning of Language Models via Differentiable Symbolic Programming (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained large language models struggle to perform logical reasoning reliably despite advances in scale and compositionality. |
| Approach: | They propose a Differentiable Symbolic Reasoning framework that uses symbolic programming to improve LMs' logical reasoning abilities. |
| Outcome: | The proposed framework outperforms competitive baselines when faced with systematic changes in sequence length. |
B2T Connection: Serving Stability and Performance in Deep Transformers (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to prevent the vanishing gradient problem in deep neural networks are not effective. |
| Approach: | They propose a method that can equip both higher stability and effective training by a simple modification from Post-LN. |
| Outcome: | The proposed method outperforms Pre-LN and Post-Ln on a wide range of tasks. |
Boosting Zero-shot Cross-lingual Retrieval by Training on Artificially Code-Switched Data (2023.findings-acl)
Copied to clipboard
| Challenge: | Using zero-shot rankers, cross-lingual IR models are limited by their language coverage. |
| Approach: | They propose to train ranking models on artificially code-switched data instead of using a dictionary. |
| Outcome: | The proposed approach is robust towards the ratio of code-switched tokens and extends to unseen languages. |
Domain-specific Attention with Distributional Signatures for Multi-Domain End-to-end Task-Oriented Dialogue (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to construct multi-domain task-oriented dialogue systems are difficult to extend to new domains due to high cost of data annotation and scarcity of labeled dialogue data. |
| Approach: | They propose a domain attention module that uses distributional signatures to construct multi-domain dialogue systems with limited data. |
| Outcome: | The proposed method outperforms baseline models on most metrics while keeping smaller model scale. |
CKDST: Comprehensively and Effectively Distill Knowledge from Machine Translation to End-to-End Speech Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | End-to-end speech-totext translation (ST) data are limited due to the limited resources. |
| Approach: | They propose a knowledge distillation framework for speech translation that integrates knowledge from machine translation and decouples knowledge from non-target class knowledge. |
| Outcome: | The proposed framework outperforms state-of-the-art models on a benchmark dataset. |
Follow the leader(board) with confidence: Estimating p-values from a single test set with item and response variance (2023.findings-acl)
Copied to clipboard
| Challenge: | Among the problems with leaderboard culture in NLP has been the widespread lack of confidence estimation in reported results. |
| Approach: | They propose a framework and simulator for estimating p-values for comparisons between the results of two systems using variance found naturally (though rarely reported) in test set items and individual labels on an item (responses). |
| Outcome: | The proposed framework and simulator are used to estimate p-values for comparisons between the results of two systems under the assumption that the null hypothesis is true. |
Parallel Data Helps Neural Entity Coreference Resolution (2023.findings-acl)
Copied to clipboard
| Challenge: | Current neural coreference models are trained on monolingual annotated data but annotating such coreference information is expensive and challenging. |
| Approach: | They propose a simple yet effective model to exploit coreference knowledge from parallel data. |
| Outcome: | The proposed model improves on the OntoNotes 5.0 English dataset by 1.74 percentage points . it is based on an unsupervised module learning coreference from annotations . |
Towards Open-Domain Twitter User Profile Inference (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to user profile inference focus on limited attributes and can reveal users' private information. |
| Approach: | They propose a prompt-based generation method which can infer values implicitly mentioned in Twitter user profiles. |
| Outcome: | The proposed method can infer more comprehensive user profiles than baseline extraction-based methods, but limitations remain to be applied for real-world use. |
Eliciting Affective Events from Language Models by Multiple View Co-prompting (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to generate training data using weakly labeled data are costly and limited . |
| Approach: | They propose a method for acquiring and labeling affective events with multiple view co-prompting using pre-trained language models. |
| Outcome: | The proposed approach improves state-of-the-art affective event classifier on two datasets. |
ZeroAE: Pre-trained Language Model based Autoencoder for Transductive Zero-shot Text Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for text classification use only encoders or decoders that do not allow for the use of labels in unseen domains. |
| Approach: | They propose an autoencoder that encodes text into two disentangled spaces and decodes it to generate text with labels in the unseen domains. |
| Outcome: | The proposed model outperforms the existing methods in label-partially-unseen and label-fully-un-seeen scenarios and even outperfects the SOTA methods. |
PRAM: An End-to-end Prototype-based Representation Alignment Model for Zero-resource Cross-lingual Named Entity Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to address the named entity recognition problem are limited and lack explicit optimization specific to the task. |
| Approach: | They propose a prototype-based representation alignment model for a cross-lingual named entity recognition task using labeled source language data. |
| Outcome: | The proposed model outperforms existing state-of-the-art methods in some challenging scenarios. |
It Takes Two to Tango: Navigating Conceptualizations of NLP Tasks and Measurements of Performance (2023.findings-acl)
Copied to clipboard
| Challenge: | a meta-analysis and survey of practitioners reveal that benchmarks suffer from operationalization disagreements. |
| Approach: | They propose a taxonomy of disagreement to explain disagreements in NLP benchmarks . they propose defining how tasks are conceptualized and operationalizing benchmarks to document their limitations. |
| Outcome: | The proposed taxonomy identifies two types of disagreements among NLP practitioners . it shows that benchmarks are not clearly conceptualized and suffer from operationalization disagreements . |
Task-adaptive Label Dependency Transfer for Few-shot Named Entity Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for Named Entity Recognition (NER) ignore label dependency, resulting in suboptimal performance. |
| Approach: | They propose a meta-learning method to make label dependency transferable by learning general initialization and individual parameter update rule for label dependency. |
| Outcome: | The proposed method improves existing methods by learning general initialization and individual parameter update rule for label dependency. |
WYWEB: A NLP Evaluation Benchmark For Classical Chinese (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for classical Chinese are inadequate to evaluate performance of different NLP models. |
| Approach: | They propose an evaluation benchmark for classical Chinese NLP, which evaluates existing models. |
| Outcome: | The proposed benchmark evaluates the performance of existing models in classical Chinese. |
A Fused Gromov-Wasserstein Framework for Unsupervised Knowledge Graph Entity Alignment (2023.findings-acl)
Copied to clipboard
| Challenge: | Entity alignment is the task of identifying corresponding entities across different knowledge graphs (KGs). |
| Approach: | They propose an unsupervised entity alignment framework that leverages the Fused Gromov-Wasserstein distance to compare KG semantics and KG structural information. |
| Outcome: | The proposed framework surpasses 21 competitive baselines, including cutting-edge methods, without supervision or hyper-parameter tuning. |
Two Examples are Better than One: Context Regularization for Gradient-based Prompt Tuning (2023.findings-acl)
Copied to clipboard
| Challenge: | Prompting has gained tremendous attention as an efficient method for the adaptation of large-scale language models. |
| Approach: | They propose a regularization method that guides a prompt to produce a task context properly. |
| Outcome: | The proposed method improves prediction performance in a zero-shot in-context learning setting without demonstration examples for in-constitu learning. |
An Investigation of Noise in Morphological Inflection (2023.findings-acl)
Copied to clipboard
| Challenge: | Neural morphological inflection systems can be used for languages with very little supervised data, but are often less likely to have clean, goldstandard data. |
| Approach: | They propose an error taxonomy and annotation pipeline for inflection training data and propose a character-level masked language modeling (CMLM) pretraining objective. |
| Outcome: | The proposed pipeline is based on error taxonomy and annotation pipelines for unsupervised morphological paradigm completion. |
Graph Reasoning for Question Answering with Triplet Retrieval (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to answer complex questions require reasoning over knowledge graphs (KGs) state-of-the-art methods constrain retrieved knowledge in local subgraphs and discard more diverse triplets that are disconnected but useful for question answering. |
| Approach: | They propose a method to retrieve the most relevant triplets from KGs and then rerank them, which are then concatenated with questions to be fed into language models. |
| Outcome: | The proposed method outperforms state-of-the-art methods on commonsenseQA and OpenbookQA datasets with 4.6% absolute accuracy. |
End-to-End Argument Mining over Varying Rhetorical Structures (2023.findings-acl)
Copied to clipboard
| Challenge: | Rhetorical Structure Theory implies no single discourse interpretation of a text . inconsistent parsing of similar structures can result in inconsistent argumentation analysis . |
| Approach: | They propose a deep dependency parsing model to assess the connection between rhetorical and argument structures. |
| Outcome: | The proposed model allows for end-to-end argumentation analysis using a rhetorical tree instead of a word sequence. |
Unsupervised Task Graph Generation from Instructional Video Transcripts (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to task graph generation use instruction-tuned language models to generate accurate task graphs. |
| Approach: | They propose an unsupervised approach that combines the reasoning capabilities of instruction-tuned language models with clustering and ranking components to generate accurate task graphs. |
| Outcome: | The proposed approach generates more accurate task graphs than a supervised learning approach on tasks from the ProceL and CrossTask datasets. |
Exploiting Hierarchically Structured Categories in Fine-grained Chinese Named Entity Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Named Entity Recognition (CNER) is a widely used technology in various applications. |
| Approach: | They propose a method that uses a custom-designed relevance scoring function to learn the potential relevance between different flattened hierarchical labels. |
| Outcome: | The proposed method outperforms the state-of-the-art on the FiNE dataset. |
Adversarial Textual Robustness on Visual Dialog (2023.findings-acl)
Copied to clipboard
| Challenge: | a recent study evaluated the robustness of visual dialog models against textual attacks. |
| Approach: | They aim to understand how multimodal input components contribute to robustness . they also evaluate how to generate adversarial test examples which fool the model . |
| Outcome: | The proposed model is more robust when it encodes dialog history than when it does not. |
Language Model Analysis for Ontology Subsumption Inference (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies focus on simple, triple-based, relational KBs but omit more sophisticated, logic-based conceptualised KB. |
| Approach: | They propose to use ontology subsumption axioms to probe LMs' knowledge of ontologies by probing datasets from atomic and complex concepts. |
| Outcome: | The proposed methods encode less background knowledge of Subsumption Inference (SI) than traditional Natural Language Inference but can improve on SI significantly when a small number of samples are given. |
Exploring Automatically Perturbed Natural Language Explanations in Relation Extraction (2023.findings-acl)
Copied to clipboard
| Challenge: | ExpBERT is a new approach to guide neural networks with explanations . previous studies have shown that explanations provide valuable inductive biases that guide models, improving generalization ability and data efficiency. |
| Approach: | They propose to use a new inductive bias-based model to guide models with explanations. |
| Outcome: | The proposed explanations achieve comparable results to annotated explanations, but with a significant increase in computational efficiency, 20-30 times faster. |
Varta: A Large-Scale Headline-Generation Dataset for Indic Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | Varta dataset includes more than 41 million pairs of headlines and articles in 14 different Indic languages (and English) |
| Approach: | They present a large-scale multilingual dataset for headline generation in Indic languages. |
| Outcome: | The Varta dataset includes more than 41 million pairs of headlines and articles in 14 different Indic languages (and English) the data can be used to train strong language models that outperform competitive baselines in both NLU and NLG benchmarks. |
Better Zero-Shot Reasoning with Self-Adaptive Prompting (2023.findings-acl)
Copied to clipboard
| Challenge: | Modern large language models (LLMs) have demonstrated impressive capabilities at sophisticated tasks, often through step-by-step reasoning similar to humans. |
| Approach: | They propose a new method that uses a set of examples from the LLM zero-shot outputs to improve performance. |
| Outcome: | The proposed method improves performance up to 15% compared to baselines and matches or exceeds few-shot baselines at a range of reasoning tasks. |
Multimodal Recommendation Dialog with Subjective Preference: A New Challenge and Benchmark (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing multimodal task-oriented dialog data fails to demonstrate the diverse expressions of user subjective preferences and recommendation acts in the real-life shopping scenario. |
| Approach: | They propose a multimodal task-oriented dialog dataset with subjective preferences and recommendation acts that is well-annotated with sales experts. |
| Outcome: | The proposed model is powered by a state-of-the-art multimodal model for these tasks. |
ANALOGICAL - A Novel Benchmark for Long Text Analogy Evaluation in Large Language Models (2023.findings-acl)
Copied to clipboard
Thilini Wijesiriwardene, Ruwan Wickramarachchi, Bimal Gajera, Shreeyash Gowaikar, Chandan Gupta, Aman Chadha, Aishwarya Naresh Reganti, Amit Sheth, Amitava Das
| Challenge: | Modern large language models are evaluated on extrinsic measures based on benchmarks such as GLUE and SuperGLUE. |
| Approach: | They propose a benchmark to intrinsically evaluate large language models across a taxonomy of analogies of long text with six levels of complexity. |
| Outcome: | The proposed benchmark evaluates LLMs across a taxonomy of analogies of long text with six levels of complexity. |
Financial Numeric Extreme Labelling: A dataset and benchmarking (2023.findings-acl)
Copied to clipboard
Soumya Sharma, Subhendu Khatuya, Manjunath Hegde, Afreen Shaikh, Koustuv Dasgupta, Pawan Goyal, Niloy Ganguly
| Challenge: | In 2019, the SEC mandates that all public companies file periodic financial statements that should contain numerals annotated with a particular label. |
| Approach: | They propose to use a dataset to automate the assignment of a label to a particular numeral span in a sentence from an extremely large label set. |
| Outcome: | The proposed solution outperforms the previous approaches but is less frequent than the pipeline solution. |
Multilingual Summarization with Factual Consistency Evaluation (2023.findings-acl)
Copied to clipboard
| Challenge: | Abstractive summarization models generate factually inconsistent summaries, reducing their utility for real-world applications. |
| Approach: | They propose to use data filtering and controlled generation to detect hallucinations in machine generated summaries. |
| Outcome: | The proposed models detect factual inconsistencies in machine generated summaries, but they focus on English only. |
Enhancing Out-of-Vocabulary Estimation with Subword Attention (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to learn OOV word representations use advanced architectures like attention on the context of the word, but they tend to use simple structures like ngram addition or character based convolutional neural networks (CNN) |
| Approach: | They propose a transformer-based OOV estimation model that uses attention mechanisms on both the context and the subwords to learn OOV representations. |
| Outcome: | The proposed model outperforms current state-of-the-art models on OOV representations based on attention mechanisms on the context and subwords . |
Encoder and Decoder, Not One Less for Pre-trained Language Model Sponsored NMT (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for neural machine translation (NMT) use encoder-only enhancement or rely on specific multilingual PLMs. |
| Approach: | They propose a monolingual PLM-sponsored NMT model that lets both encoder and decoder enjoy PLM enhancement. |
| Outcome: | The proposed model improves on IWSLT14 En-De, De-En, WMT14 En -De and En-Fr tasks and achieves new state-of-the-art. |
TransGEC: Improving Grammatical Error Correction with Translationese (2023.findings-acl)
Copied to clipboard
| Challenge: | Experimental results show that data augmentation improves accuracy over strong baselines. |
| Approach: | They propose to use translationese as input for GEC data augmentation to overcome stylistic discrepancies . they propose to obtain human-translated texts with a more similar style to non-native texts . |
| Outcome: | The proposed method improves correction accuracy over strong baselines on four GEC benchmarks. |
NewsDialogues: Towards Proactive News Grounded Conversation (2023.findings-acl)
Copied to clipboard
Siheng Li, Yichun Yin, Cheng Yang, Wangjie Jiang, Yiwei Li, Zesen Cheng, Lifeng Shang, Xin Jiang, Qun Liu, Yujiu Yang
| Challenge: | Hot news is one of the most popular topics in daily conversations. |
| Approach: | They propose a task where a dialogue system can lead the conversation based on key topics of the news. |
| Outcome: | The proposed method can lead conversations based on key topics of the news . it can also be used in information-seeking and chit-chat scenarios . |
Task-aware Retrieval with Instructions (2023.findings-acl)
Copied to clipboard
Akari Asai, Timo Schick, Patrick Lewis, Xilun Chen, Gautier Izacard, Sebastian Riedel, Hannaneh Hajishirzi, Wen-tau Yih
| Challenge: | Existing models that learn intents from labeled data are complicated and require a vast number of annotated examples to train a model. |
| Approach: | They propose a general-purpose task-aware retrieval system with instructions that can adapt to a new task without any parameter updates. |
| Outcome: | The proposed system outperforms two benchmarks on a set of domains and tasks on X2-Retrieval. |
Non-Repeatable Experiments and Non-Reproducible Results: The Reproducibility Crisis in Human Evaluation in NLP (2023.findings-acl)
Copied to clipboard
| Challenge: | reproducibility of human evaluations is rarely queried in NLP . authors estimate that just 5% of humanevaluations are repeatable . |
| Approach: | They propose to make human evaluations more repeatable and more reproducible . they estimate that just 5% of human evaluation experiments are repeatable . |
| Outcome: | The results show that human evaluations are rarely queried or formally tested in NLP . the authors estimate that just 5% of human evaluation experiments are repeatable . |
Define, Evaluate, and Improve Task-Oriented Cognitive Capabilities for Instruction Generation Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work examines the cognitive capabilities of language models through psychological tests designed for humans. |
| Approach: | They propose to use human-like cognitive capabilities to evaluate language models . they propose to augment language models with better listeners to improve their performance . |
| Outcome: | The proposed method boosts language models with better models of the listener and improves them. |
Robustness of Multi-Source MT to Transcription Errors (2023.findings-acl)
Copied to clipboard
| Challenge: | In multilingual settings, the same content may be available in various languages via simultaneous interpreting, dubbing or subtitling. |
| Approach: | They hypothesize that leveraging multiple sources will improve translation quality if the sources complement one another in terms of correct information they contain. |
| Outcome: | The proposed method is robust to speech recognition errors on a 10-hour ESIC corpus. |
Not The End of Story: An Evaluation of ChatGPT-Driven Vulnerability Description Mappings (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing data proves that ChatGPT performs no less than humans in text generation and knowledge Q&A. |
| Approach: | They propose to use ChatGPT to map vulnerabilities to common weakness enumeration (CWE), common attack pattern ennumeration and classification (ATT&CK) techniques and other classifications. |
| Outcome: | The proposed method performs better than human experts on many tasks, but it can't replace professional security engineers in vulnerability analysis. |
Multi3NLU++: A Multilingual, Multi-Intent, Multi-Domain Dataset for Natural Language Understanding in Task-Oriented Dialogue (2023.findings-acl)
Copied to clipboard
| Challenge: | Task-oriented dialogue systems are typically constructed for a single domain or language and do not generalise well beyond this. |
| Approach: | They constructed a multilingual, multi-intent, multi domain dataset to support work on Natural Language Understanding (NLU) in ToD across multiple languages and domains simultaneously. |
| Outcome: | The proposed dataset extends the English-only dataset to include manual translations into a range of high, medium, and low resource languages in two domains (banking and hotels). |
A Robust Information-Masking Approach for Domain Counterfactual Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Domain shift is a big challenge in NLP, but many approaches fail to leverage domain-specific nuances relevant to the task at hand. |
| Approach: | They propose a method that uses frequency-based masking to transform a text from the source domain to a target domain. |
| Outcome: | The proposed method outperforms baselines on 10 out of 12 domain-counterfactual classification settings with an average of 1.7% improvement in accuracy metric. |
Misleading Relation Classifiers by Substituting Words in Texts (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to generate adversarial examples for relation classification are vulnerable to adversarials. |
| Approach: | They propose a method that uses most important parts of speech to substitute words with synonyms or hyponyms to generate adversarial texts of high quality. |
| Outcome: | The proposed method can generate adversarial texts of high quality and most relationships can be correctly identified in the process of human evaluation. |
Automatic Table Union Search with Tabular Representation Learning (2023.findings-acl)
Copied to clipboard
Xuming Hu, Shen Wang, Xiao Qin, Chuan Lei, Zhengyuan Shen, Christos Faloutsos, Asterios Katsifodimos, George Karypis, Lijie Wen, Philip S. Yu
| Challenge: | Existing methods to identify uniability based on column representations are insufficient to reveal latent relational features to describe column relation between pair of columns. |
| Approach: | They propose a self-supervised table union search framework called AutoTUS to learn column relational representations in a multi-stage manner. |
| Outcome: | The proposed framework improves on the SOTA baseline and on real-world datasets. |
Bidirectional Transformer Reranker for Grammatical Error Correction (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained seq2seq models suffer from a prediction bias due to their unidirectional decoding. |
| Approach: | They propose a bidirectional Transformer reranker that re-estimates the probability of each candidate sentence generated by pre-trained seq2seq models. |
| Outcome: | The proposed model improves on the original model and gives a 59.52 GLEU score on the JFLEG corpus. |
Not Enough Data to Pre-train Your Language Model? MT to the Rescue! (2023.findings-acl)
Copied to clipboard
| Challenge: | In recent years, transformer-based language models (LMs) have become the default approach for many NLP tasks. |
| Approach: | They compare the performance of transformer-based language models with machine-translated corpora. |
| Outcome: | The proposed model can be improved with real data, but further research is needed. |
UMSE: Unified Multi-scenario Summarization Evaluation (2023.findings-acl)
Copied to clipboard
| Challenge: | Summarization quality evaluation is a non-trivial task in text summarization. |
| Approach: | They propose a unified multi-scenario summarization evaluation model that shares cross-sceenario knowledge and uses a self-supervised training paradigm to optimize the model without extra human labeling. |
| Outcome: | The proposed model can achieve comparable performance with existing methods for three evaluation scenarios. |
Maximum Entropy Loss, the Silver Bullet Targeting Backdoor Attacks in Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing backdoor defense paradigms focus on detecting and removing poisoned samples at pre-training or inference time. |
| Approach: | They propose a new approach where the backdoor attack is directly reversed by incorporating maximum entropy loss into training to neutralize the minimal cross-entropiness loss fine-tuning on poisoned data. |
| Outcome: | The proposed model significantly lowers the attack success rate on classification tasks and reduces the risk of backdoor attacks on clean data. |
Improving Named Entity Recognition via Bridge-based Domain Adaptation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for named entity recognition use pre-training language models to represent words, leading to entity type misclassification. |
| Approach: | They propose a model-agnostic framework called MoCL for cross-domain named entity recognition to refine the original representations and combine it with two distinct cross- domain NER methods and two pre-training language models to explore its generalization ability. |
| Outcome: | The proposed framework is model-agnostic and can be used to generalize and refine existing models. |
SANTA: Separate Strategies for Inaccurate and Incomplete Annotation Noise in Distantly-Supervised Named Entity Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Distantly-Supervised Named Entity Recognition (DS-NER) is widely used in the supervised setting, but the context-free matching process and the limited coverage of knowledge bases introduce inaccurate and incomplete annotation noise respectively. |
| Approach: | They propose to handle two types of noise separately with Memory-smoothed Focal Loss and Entity-aware KNN to relieve the entity ambiguity problem caused by inaccurate annotation and a noise-tolerant loss to improve the model’s robustness. |
| Outcome: | The proposed model achieves a new state-of-the-art on five public datasets. |
The State of Profanity Obfuscation in Natural Language Processing Scientific Publications (2023.findings-acl)
Copied to clipboard
| Challenge: | obfuscation is used for English but not other languages, and even then, unevenly. |
| Approach: | They propose a multilingual community resource called PrOf to standardize profanity obfuscation processes. |
| Outcome: | The proposed tool can help scientific publications to make hate speech work accessible and comparable, irrespective of language. |
Teacher and Student Models of Offensive Language in Social Media (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to identify offensive language online use large pre-trained transformer models. however, the inference time, disk, and memory requirements of these models are prohibitively large. |
| Approach: | They propose to transfer knowledge from large transformer models to much smaller neural models to make predictions at the token- and post-level. |
| Outcome: | The proposed model performs 100 times better than transformer models but with 100 times less parameters and much less memory usage. |
A Simple Yet Strong Domain-Agnostic De-bias Method for Zero-Shot Sentiment Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | a recent study shows that large language models are biased to their pre-training data, leading to poor performance in prompt templates. |
| Approach: | They propose a domain-agnostic data construction method to de-bias a given prompt template . they show that domain-based generic responses are superior to in-domain ground-truth data . |
| Outcome: | The proposed method improves sentiment analysis tasks across domains and domains . it also yields better performance than existing in-domain models . |
Balancing the Effect of Training Dataset Distribution of Multiple Styles for Multi-Style Text Transfer (2023.findings-acl)
Copied to clipboard
| Challenge: | Text style transfer requires a high-quality paired dataset and quality training data. |
| Approach: | They propose to use a pseudo-parallel dataset to adjust the style distribution in training data to balance the style transfer model. |
| Outcome: | The proposed model produces more effective control effects over multiple styles than an imbalanced or skewed one. |
A Benchmark on Extremely Weakly Supervised Text Classification: Reconcile Seed Matching and Prompting Approaches (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for XWS-TC rely on minimal human guidance . X-WS-tc methods require no humanannotated datasets . |
| Approach: | They propose a benchmarking method to compare two approaches to XWS-TC . they use seed-matching and prompting a language model with instructions to decode label words . |
| Outcome: | The proposed methods are more tolerant to human guidance and more robust to model-based methods. |
Ambiguity Meets Uncertainty: Investigating Uncertainty Estimation for Word Sense Disambiguation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing supervised methods treat word sense disambiguation as a classification task but ignore uncertainty estimation (UE) in the real-world setting, the data is always noisy and out of distribution. |
| Approach: | They propose to use word sense disambiguation to determine an appropriate sense for a word given its context to determine the most appropriate sense. |
| Outcome: | The proposed model reflects data uncertainty satisfactorily but underestimates model uncertainty. |
Zemi: Learning Zero-Shot Semi-Parametric Language Models from Multiple Tasks (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing semi-parametric language models lack the capacity to perform zero-shot tasks . large language models have shown impressive zero-shoot ability, but huge model size costs . semi-parametric language model can be used to augment a smaller language model with retrieved background knowledge . |
| Approach: | They propose a semi-parametric language model for zero-shot task generalization that augments a smaller language model with retrieved related background knowledge. |
| Outcome: | The proposed model outperforms T0-3B by 16% across seven diverse evaluation tasks while being 3.8x smaller in scale. |
Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers (2023.findings-acl)
Copied to clipboard
| Challenge: | Large pretrained language models have shown surprising in-context learning ability . despite the great success in performance, its working mechanism remains unclear . |
| Approach: | They explain language models as meta-optimizers and understand in-context learning as implicit finetuning . they find that Transformer attention has a dual form of gradient descent . |
| Outcome: | The proposed model can predict labels for unseen inputs without parameter updates . the proposed model outperforms smaller models with a single parameter update . |
Dramatic Conversation Disentanglement (2023.findings-acl)
Copied to clipboard
| Challenge: | a new dataset is available for studying conversation disentanglement in movies and TV series . a recent study focused on IRC chatroom dialogues, but movies and television show provide a space for study . |
| Approach: | They propose a dataset for studying conversation disentanglement in movies and TV series . they operationalize a conversational thread and apply the best-performing model to 808 movies . |
| Outcome: | The proposed model disentangles 808 movies from 10,033 dialogue turns . the best-performing model is compared with previous models . |
Injecting Comparison Skills in Task-Oriented Dialogue Systems for Database Search Results Disambiguation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing task-oriented dialogue systems handle ambiguity in database search results by randomly selecting one or few results and presenting their names to the user. |
| Approach: | They propose a task that compares properties of multiple database search results . they use a dataset to collect high-quality dialogue data and an augmented version of the SGD dataset . |
| Outcome: | The proposed task compares properties of two entities in a trade-off form based on user preferences . the proposed dataset and code will be publicized . |
Emergent Modularity in Pre-trained Transformers (2023.findings-acl)
Copied to clipboard
Zhengyan Zhang, Zhiyuan Zeng, Yankai Lin, Chaojun Xiao, Xiaozhi Wang, Xu Han, Zhiyuan Liu, Ruobing Xie, Maosong Sun, Jie Zhou
| Challenge: | Existing studies on pre-trained Transformers show that they learn fine-grained neuron functions. |
| Approach: | They examine the presence of modularity in pre-trained Transformers . they focus on Mixture-of-Experts, a promising candidate for modularity . |
| Outcome: | The proposed structure stabilizes at the early stage, which is faster than neuron stabilization. |
Universal Information Extraction with Meta-Pretrained Self-Retrieval (2023.findings-acl)
Copied to clipboard
Xin Cong, Bowen Yu, Mengcheng Fang, Tingwen Liu, Haiyang Yu, Zhongkai Hu, Fei Huang, Yongbin Li, Bin Wang
| Challenge: | Existing methods for IE are task-specific, resulting in specialized and isolated approaches for different tasks. |
| Approach: | They propose a method to retrieve task-specific knowledge from pretrained language models to enhance universal IE by using a Meta-Pretraining Algorithm. |
| Outcome: | The proposed method achieves the new state-of-the-art on 4 IE tasks, 12 datasets under fully-supervised, low-resource and few-shot scenarios. |
SETI: Systematicity Evaluation of Textual Inference (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing pre-trained language models (PLMs) have shown remarkable performance on this task, but little is known about their ability to address compositional generalization. |
| Approach: | They propose a benchmark to evaluate pre-trained language models' systematicity in the domain of textual inference. |
| Outcome: | The proposed benchmark evaluates pre-trained language models on six widely used PLMs. |
Coarse-to-fine Few-shot Learning for Named Entity Recognition (2023.findings-acl)
Copied to clipboard
Ruotian Ma, Zhang Lin, Xuanting Chen, Xin Zhou, Junzhe Wang, Tao Gui, Qi Zhang, Xiang Gao, Yun Wen Chen
| Challenge: | Existing few-shot NER solutions do not consider sub-class discrimination and various granularity of new classes during coarse training. |
| Approach: | They propose a method that uses a cluster-based prototype loss to learn group-wise discriminative representations of coarse-grained classes and a mixture prototype loss for learning the representations. |
| Outcome: | The proposed method shows superior performance over baseline methods on in-domain and cross-domain settings with various target granularity. |
Self-Evolution Learning for Discriminative Language Model Pretraining (2023.findings-acl)
Copied to clipboard
| Challenge: | Random masking does not consider the importance of the different words in the sentence meaning, e.g., entity-level masking requires expensive prior knowledge and generally does not use existing model weights. |
| Approach: | They propose a token masking and learning method that uses a random masking strategy to learn the under-explored tokens. |
| Outcome: | The proposed method improves linguistic knowledge learning and generalization on 10 tasks. |
QueryForm: A Simple Zero-shot Form Entity Query Framework (2023.findings-acl)
Copied to clipboard
Zifeng Wang, Zizhao Zhang, Jacob Devlin, Chen-Yu Lee, Guolong Su, Hao Zhang, Jennifer Dy, Vincent Perot, Tomas Pfister
| Challenge: | Form-like document understanding is a key yet under-investigated problem . endlessly training specialized models on new document types is not scalable in many practical scenarios. |
| Approach: | They propose to use large-scale query-entity pairs generated from form-like webpages to pre-train QueryForm. |
| Outcome: | The proposed framework sets state-of-the-art average F1 score on XFUND and Payment benchmarks. |
Search-Oriented Conversational Query Editing (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing CQR models are not learned toward improving the downstream search performance . existing models generate the rewrite token-by-token from scratch . |
| Approach: | They propose a text editing-based CQR model tailored for conversational search . they propose rewrite tokens are selected from the dialogue in a non-autoregressive fashion . |
| Outcome: | The proposed model outperforms state-of-the-art models on three conversational search benchmarks while having low rewriting latency. |
TAPIR: Learning Adaptive Revision for Incremental Natural Language Understanding with a Two-Pass Model (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent approaches for incremental processing use RNNs or Transformers, which consume whole sequences and are by nature non-incremental. |
| Approach: | They propose a two-pass model for AdaPtIve Revision to obtain an incremental supervision signal for learning an adaptive revision policy. |
| Outcome: | The proposed model has better incremental performance and faster inference speed compared to restart-incremental Transformers while showing little degradation on full sequences. |
Speaking the Language of Your Listener: Audience-Aware Adaptation via Plug-and-Play Theory of Mind (2023.findings-acl)
Copied to clipboard
| Challenge: | Adaptation is a process in human communication by which a speaker tunes its language to that of a listener to achieve communicative success. |
| Approach: | They propose a visual-based referential game between a knowledgeable speaker and a listener with limited visual and linguistic experience to model this adaptation mechanism. |
| Outcome: | The proposed model improves on plug-and-play approaches to controlled language generation without finetuning the speaker’s underlying language model. |
A Semi-Autoregressive Graph Generative Model for Dependency Graph Parsing (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing parsers that capture dependency graphs are lacking in capturing explicit dependencies . graph-based parsing is a popular choice for capturing dependency relationships between words . |
| Approach: | They propose a semi-autoregressive dependency parser that generates dependency graphs by adding nodes and edge groups autoregressively while pouring out all group elements in parallel. |
| Outcome: | The proposed method outperforms baselines on Enhanced Universal Dependencies of multiple languages. |
AMR-TST: Abstract Meaning Representation-based Text Style Transfer (2023.findings-acl)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) is a semantic representation that can enhance natural language generation (NLG) by providing a logical semantic input. |
| Approach: | They propose an AMR-based text style transfer technique that converts source text to an AML graph and generates transferred text based on the AMR graph modified by a TST policy named style rewriting. |
| Outcome: | The proposed method achieves state-of-the-art results compared with baseline models in automatic and human evaluations. |
Understanding the Cooking Process with English Recipe Text (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to translate recipes into a flow graph have performance problems . authors propose a framework to construct a graph from recipe text . |
| Approach: | They propose a framework that can be used to translate recipes into a flow graph representation. |
| Outcome: | The proposed framework can predict the edge label and achieve the overall F1 score of 92.2 on the English recipe flow graph corpus. |
Follow the Wisdom of the Crowd: Effective Text Generation via Minimum Bayes Risk Decoding (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing text decoding methods struggle to produce high-quality text . Greedy and beam search suffer from text degeneration and linguistic diversity issues . |
| Approach: | They propose a family of decoding methods based on minimum bayes risk minimization to address diversity-quality trade-offs in open-ended natural-language generation. |
| Outcome: | The proposed methods improve diversity-quality trade-offs on open-ended natural-language generation tasks. |
RobustQA: Benchmarking the Robustness of Domain Adaptation for Open-Domain Question Answering (2023.findings-acl)
Copied to clipboard
Rujun Han, Peng Qi, Yuhao Zhang, Lan Liu, Juliette Burger, William Yang Wang, Zhiheng Huang, Bing Xiang, Dan Roth
| Challenge: | Existing ODQA datasets consist mainly of Wikipedia corpus, and are insufficient to study models’ generalizability across diverse domains. |
| Approach: | They propose a benchmark to evaluate ODQA's domain robustness using Wikipedia corpus . they annotate QA pairs in retrieval datasets with rigorous quality control . |
| Outcome: | The proposed benchmark improves model performance on annotated QA pairs in retrieval datasets with rigorous quality control. |
SenteCon: Leveraging Lexicons to Learn Human-Interpretable Language Representations (2023.findings-acl)
Copied to clipboard
| Challenge: | In many settings, it is important to understand a model’s decision-making process. |
| Approach: | They propose a method for introducing human interpretability in deep language representations by encoding a passage of text as a layer of interpretable categories. |
| Outcome: | The proposed method outperforms existing interpretable language representations on downstream tasks and on agreement with human characterizations of the text. |
Reinforcement Learning for Topic Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Topic modeling is a method to extract information from documents by grouping topics into topics and linking them with words describing them. |
| Approach: | They propose to replace the variational autoencoder with a continuous action space reinforcement learning policy and modify the neural network architecture to weight the ELBO loss. |
| Outcome: | The proposed model outperforms all other unsupervised models and performs on par with or better than most models using supervised labeling and contrastive learning. |
Contextualized Soft Prompts for Extraction of Event Arguments (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing prompt-based methods for event argument extraction rely on discrete and manually-designed prompts that cannot exploit specific context for each example. |
| Approach: | They propose a prompt-based method that introduces soft prompts to facilitate encoding of individual example context and multiple relevant documents to boost EAE. |
| Outcome: | The proposed method extensively evaluates on benchmark datasets to demonstrate its benefits with state-of-the-art performance. |
TextVerifier: Robustness Verification for Textual Classifiers with Certifiable Guarantees (2023.findings-acl)
Copied to clipboard
| Challenge: | a textual classifier must withstand word-level alteration attacks due to inherent vulnerability. |
| Approach: | They propose a formal verification framework with certifiable guarantees on deep neural networks in natural language processing against word-level alteration attacks. |
| Outcome: | The proposed framework provides an approximation of the maximal safe radius with tight bounds . it yields an efficient speed edge and reliable anytime estimation . |
OASum: Large-Scale Open Domain Aspect-based Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing generic summarization methods generate only one summary for all different requests which is not optimal for diverse demands. |
| Approach: | They use crowd-sourced knowledge on Wikipedia to create a large-scale open-domain aspect-based summarization dataset with 1 million different aspects on 2 million Wikipedia pages. |
| Outcome: | The proposed model can generate diverse aspect-based summarizations on Wikipedia with zero/few-shot and fine-tuning on seven downstream datasets. |
On the Limitations of Simulating Active Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Active learning (AL) is a human-and-model-in-the-loop paradigm that iteratively selects informative unlabeled data for human annotation. |
| Approach: | They propose to simulate active learning by using an already labeled dataset as the pool of unlabeled data. |
| Outcome: | The proposed model-in-the-loop paradigm can be used to perform experiments with human annotations on-the fly. |
Towards Alleviating the Object Bias in Prompt Tuning-based Factual Knowledge Extraction (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to optimize prompts for factual knowledge extraction are undesirable object bias. |
| Approach: | They propose a prompt tuning method that reduces object bias and improves factual knowledge extraction. |
| Outcome: | The proposed method reduces object bias and improves accuracy of factual knowledge extraction. |
vONTSS: vMF based semi-supervised neural topic modeling with optimal transport (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent Neural Topic Models (NTMs) have limited applications in the real world due to the challenge of incorporating human knowledge. |
| Approach: | They propose a semi-supervised neural topic modeling method, vONTSS, which uses von Mises-Fisher variational autoencoders and optimal transport. |
| Outcome: | The proposed method outperforms existing semi-supervised topic modeling methods on multiple aspects. |
Bias Beyond English: Counterfactual Tests for Bias in Sentiment Analysis in Four Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | Sentiment analysis systems are used in hundreds of products and languages . Gender and racial biases are well-studied in English, but understudied elsewhere . |
| Approach: | They build a counterfactual evaluation corpus for gender and racial/migrant bias in four languages. |
| Outcome: | The evaluation corpus reveals which models have less bias and pinpoints changes in model bias behaviour, enabling more targeted mitigation strategies. |
Complementary Explanations for Effective In-Context Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have remarkable capabilities in learning from expla- nations in prompts, but there has been limited understanding of exactly how these explana- tions function or why they are effective. |
| Approach: | They propose a maximal marginal relevance-based exemplar selection approach to construct exemplar sets that are both relevant and comple- mentary. |
| Outcome: | The proposed model improves in- context learning performance across three tasks on multiple LLMs. |
MISMATCH: Fine-grained Evaluation of Machine-generated Text with Mismatch Error Types (2023.findings-acl)
Copied to clipboard
Keerthiram Murugesan, Sarathkrishna Swaminathan, Soham Dan, Subhajit Chaudhury, Chulaka Gunasekara, Maxwell Crouse, Diwakar Mahajan, Ibrahim Abdelaziz, Achille Fokoue, Pavan Kapanipathi, Salim Roukos, Alexander Gray
| Challenge: | Existing evaluation metrics for machine text are inadequate to capture quality of text . a recent study has focused on task-specific evaluation metrics or on properties of machine-generated text based on mismatch errors . |
| Approach: | They propose a new evaluation scheme based on fine-grained mismatch errors . they propose 13 mismatch error types to guide the model for better prediction of human judgments . |
| Outcome: | The proposed evaluation scheme is based on mismatch errors in 7 NLP tasks . the mismatch error types guide the model for better prediction of human judgments . |
RHO: Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing knowledge-grounded dialogue systems generate accurate and informative responses, but they are prone to hallucination problems. |
| Approach: | They propose a method to generate hallucinated responses using knowledge graphs . they propose local knowledge grounding to combine textual embeddings with corresponding KG embeddments . a global knowledge ground technique is also proposed to equip RHO with multi-hop reasoning abilities . |
| Outcome: | The proposed approach outperforms state-of-the-art methods on automatic and human evaluation by a large margin. |
Transformer Language Models Handle Word Frequency in Prediction Head (2023.findings-acl)
Copied to clipboard
| Challenge: | Prediction head is a crucial component of Transformer language models. Despite its direct impact on prediction, its characteristics have been overlooked in previous analyses. |
| Approach: | They examine the inner workings of the prediction head, specifically the bias parameters, and quantify the effect of controlling their frequency biases on text generation. |
| Outcome: | The prediction head is a crucial component of the Transformer language models. |
Prompted LLMs as Chatbot Modules for Long Open-domain Conversation (2023.findings-acl)
Copied to clipboard
| Challenge: | Using pre-trained large language models (LLMs) as individual modules for long-term consistency and flexibility is a challenge for open-domain chatbots due to the computational burden of updating models with billions of parameters and the scarcity of data in the dialogue domain. |
| Approach: | They propose a method that uses pre-trained large language models as individual modules for long-term consistency and flexibility. |
| Outcome: | The proposed method is on par with fine-tuned chatbot models in open-domain conversations, showing it can create consistent and engaging chatbots. |
Prompt to be Consistent is Better than Self-Consistent? Few-Shot and Zero-Shot Fact Verification with Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for few-shot and zero-shot fact verification require a large set of training data. |
| Approach: | They propose a method to prompt pre-trained language models to be consistent to improve the factuality assessment capability of PLMs. |
| Outcome: | The proposed method outperforms state-of-the-art few-shot fact verification models with a small number of unlabeled instances on zero-shot verification. |
Model Analysis & Evaluation for Ambiguous Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | Ambiguous questions are a challenge for Question Answering models as they require answers that cover multiple interpretations of the original query. |
| Approach: | They aim to investigate whether model/data scaling improves the answers’ quality and whether automated metrics align with human judgment. |
| Outcome: | The proposed models can generate long-form answers that combine conflicting information and provide valuable insights into the limitations of the current approaches. |
Debiasing should be Good and Bad: Measuring the Consistency of Debiasing Techniques in Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in deep learning have led to the creation of large Transformer-based language models (LMs). |
| Approach: | They propose a protocol which distinguishes methods that yield desirable results . they apply this protocol to a popular debiasing method, Self-Debiase, and compare it to one called Instructive Debiaser. |
| Outcome: | The proposed protocol provides essential insights into the generalizability and interpretability of debiasing methods that may otherwise go overlooked. |
Critic-Guided Decoding for Controlled Text Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work has demonstrated reinforcement learning and weighted decoding as effective approaches to achieve a higher level of language control and quality with pros and cons. |
| Approach: | They propose a method that combines reinforcement learning and weighted decoding to train a critic from reward models. |
| Outcome: | The proposed method generates more coherent and well-controlled texts than previous methods on three controlled generation tasks, topic control, sentiment control, and detoxification. |
MedNgage: A Dataset for Understanding Engagement in Patient-Nurse Conversations (2023.findings-acl)
Copied to clipboard
| Challenge: | Literature suggests that actively engaged patients are more likely to obtain the full benefits of an intervention and exhibit better outcomes. |
| Approach: | They propose to annotate a dataset of patient-nurse conversations about cancer symptom management using a new framework for patient engagement. |
| Outcome: | The proposed model predicts patient-nurse conversations from socio-affective and cognitive dimensions. |
SEAG: Structure-Aware Event Causality Generation (2023.findings-acl)
Copied to clipboard
Zhengwei Tao, Zhi Jin, Xiaoying Bai, Haiyan Zhao, Chengfeng Dou, Yongqiang Zhao, Fang Wang, Chongyang Tao
| Challenge: | Current methods for extracting event causality are limited by the lack of cross-task dependencies and may cause error propagation. |
| Approach: | They propose an approach for Structure-Aware Event Causality Generation (SEAG) they generate the ECG structure using a pre-trained language model and perform structural discriminative training alongside auto-regressive generation. |
| Outcome: | The proposed method is effective in extracting event causality from text. |
Large Language Models Can be Lazy Learners: Analyze Shortcuts in In-Context Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown great potential for in-context learning, but their robustness and performance on downstream tasks remains limited. |
| Approach: | They propose to examine the reliance of LLMs on shortcuts or spurious correlations within prompts for downstream tasks and find larger models are more likely to utilize shortcuts in prompts during inference. |
| Outcome: | The proposed model is “lazy learner” and more likely to use shortcuts in prompts during inference. |
A Two-Stage Decoder for Efficient ICD Coding (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent automated ICD coding efforts improve performance by encoding medical notes and codes with additional data and knowledge bases. |
| Approach: | They propose a two-stage decoding mechanism to predict ICD codes using hierarchical properties of the codes to split the prediction into two steps: at first, predict the parent code and then predict the child code based on the previous prediction. |
| Outcome: | Experiments on the public MIMIC-III data show that the proposed model performs well in single-model settings without external data or knowledge. |
Asymmetric feature interaction for interpreting model predictions (2023.findings-acl)
Copied to clipboard
| Challenge: | Prior work on feature interaction attribution studies focus on asymmetric interaction that only explains the additional influence of a set of words in combination, which fails to capture asymmetry influence that contributes to model prediction. |
| Approach: | They propose an asymmetric feature interaction attribution explanation model that explores asymmetry higher-order feature interactions in the inference of deep neural NLP models. |
| Outcome: | The proposed model outperforms state-of-the-art models on two sentiment classification datasets. |
Disagreement Matters: Preserving Label Diversity by Jointly Modeling Item and Annotator Label Distributions with DisCo (2023.findings-acl)
Copied to clipboard
Tharindu Cyril Weerasooriya, Alexander Ororbia, Raj Bhensadadia, Ashiqur KhudaBukhsh, Christopher Homan
| Challenge: | a recent study shows that annotator disagreement is common in supervised learning . a simple neural model that learns to predict annotators' labels is competitive with other models that do not model specific annotations. |
| Approach: | They propose a neural model that learns to predict annotator distributions by aggregating over all annotators. |
| Outcome: | The proposed model outperforms models that do not model specific annotators or do not learn label distribution learning. |
Domain Aligned Prefix Averaging for Domain Generalization in Abstractive Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on domain generalization have sophisticated training algorithms. |
| Approach: | They propose a lightweight, weight averaging approach to domain generalization for abstractive summarization using prefix tuning and weight adjusting. |
| Outcome: | The proposed method performs better on four diverse summarization domains compared to baselines. |
ClaimDiff: Comparing and Contrasting Claims on Contentious Issues (2023.findings-acl)
Copied to clipboard
| Challenge: | Using fact verification tasks, however, can not detect subtle differences in factually consistent claims, which might bias the readers. |
| Approach: | They propose a novel dataset that primarily focuses on comparing the nuance between claim pairs. |
| Outcome: | The proposed dataset shows that human-labeled 2,941 claim pairs are weaker than baselines, showing a 19% absolute gap with the baselines. |
Unsupervised Paraphrasing of Multiword Expressions (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for paraphrasing multiword expressions in context are unsupervised . multiwords are notoriously difficult to model because the meaning of the whole can diverge substantially from that of the component words. |
| Approach: | They propose an unsupervised approach to paraphrasing multiword expressions in context using monolingual corpus data and pre-trained language models. |
| Outcome: | The proposed method outperforms all unsupervised systems and rivals supervised systems on the SemEval 2022 idiomatic text similarity task. |
G-Tuning: Improving Generalization of Pre-trained Language Models with Generative Adversarial Network (2023.findings-acl)
Copied to clipboard
| Challenge: | Empirical evaluations on the GLUE benchmark demonstrate that fine-tuning can enhance the generalization performance of pre-trained language models (PLMs) in downstream tasks. |
| Approach: | They propose a fine-tuning framework that transforms the latent representation of pre-trained language models from a universal space to a target space and integrates a generative adversarial network into the fine-untun process. |
| Outcome: | Empirical evaluations on the GLUE benchmark and two additional demanding scenarios show that the proposed framework can improve the generalization performance of pre-trained language models (PLMs) in downstream tasks. |
Unified Language Representation for Question Answering over Text, Tables, and Images (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to answer complex questions are limited to text or structured data. |
| Approach: | They propose a paradigm that transforms images and tables into unified language representations to simplify QA problems. |
| Outcome: | The proposed framework outperforms existing methods on two datasets and the WebQA leaderboard. |
A Set Prediction Network For Extractive Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent approaches to extracting salient sentences from source document are naive and lack dependencies between sentences. |
| Approach: | They propose a set prediction network to detect redundancy relationship between sentences . they use a non-autoregressive decoder to predict sentences in parallel . |
| Outcome: | The proposed method outperforms previous state-of-the-art models on extracted summary datasets. |
Geo-Seq2seq: Twitter User Geolocation on Noisy Data through Sequence to Sequence Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | a new method for Twitter user geolocation rewrites noisy, multilingual location strings into structured English location names. |
| Approach: | They propose a sequence-to-sequence (seq2sequ) model that rewrites noisy location strings into structured English location names. |
| Outcome: | The proposed model can generalize well to unseen temporal data, but performance does vary by language and country. |
Predicting Numerals in Text Using Nearest Neighbor Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | naive language models treat numerals as string tokens, resulting in difficulty in acquiring commonsense . kNN-LM is an extension of pre-trained neural LMs with the k-nearest neighbor (kNN) search . |
| Approach: | They apply k-nearest neighbor LM to a masked numeral prediction task . they found it is effective for fine-grained predictions of numerals from context . |
| Outcome: | The retrieval-based method is effective for fine-grained numeral prediction from context . it improves accuracy for the OOV numerals, the study shows . |
HonestBait: Forward References for Attractive but Faithful Headline Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Current approaches to generating attractive headlines often learn directly from data based on clicks and views . clickbait models fail to reveal how much interest is raised by the writing style and how much is due to the event or topic itself . |
| Approach: | They propose a framework for generating headlines using forward references . they use a dataset containing pairs of fake news and verified news . |
| Outcome: | The proposed framework yields more attractive headlines while maintaining high veracity . the framework is based on a dataset containing fake news with verified news . |
Few Shot Rationale Generation using Self-Training with Dual Teachers (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models that generate free-text explanations for annotated labels are expensive and require a large annotation dataset. |
| Approach: | They propose a self-training approach leveraging both labeled and unlabeled data to further improve few-shot models by combining teacher models and a multi-tasking student model. |
| Outcome: | The proposed model improves on three public datasets and can generate a free-text explanation for predicted labels. |
Towards Accurate Translation via Semantically Appropriate Application of Lexical Constraints (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work has not evaluated LNMT models under challenging real-world conditions. |
| Approach: | They propose a homograph disambiguation module and a model that integrates contextually rich information about unseen lexical constraints from pre-trained language models. |
| Outcome: | The proposed model can cope with “homographs” and “unseen” lexical constraints. |
NoisywikiHow: A Benchmark for Learning with Real-world Noisy Labels in Natural Language Processing (2023.findings-acl)
Copied to clipboard
| Challenge: | Large-scale datasets in the real world often contain label noise, which can cause model overfitting and degrade generalization. |
| Approach: | They propose to use label noise to imitate human errors in annotations . they use a noisy label noise benchmark to evaluate their methods . |
| Outcome: | The proposed benchmarks are different from data with heterogeneous label noises in the real world. |
Sampling Better Negatives for Distantly Supervised Named Entity Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing supervised named entity recognition approaches rely on human annotations. |
| Approach: | They propose a method to select negative samples with high similarities with positive samples . they propose to use automatically labeled training data instead of human annotations . |
| Outcome: | The proposed method achieves consistent performance improvements on four distantly supervised NER datasets. |
Prototype-Based Interpretability for Legal Citation Prediction (2023.findings-acl)
Copied to clipboard
| Challenge: | citation prediction is a key problem in high-stakes decision making areas such as law . experts often require interpretability for automatic systems to be utilized in practical settings . |
| Approach: | They propose to use legal citation prediction to solve a problem with legal experts' feedback . they propose to add a prototype architecture to add interpretability while adhering to legal parameters . |
| Outcome: | The proposed model performs well while adhering to decision parameters used by lawyers. |
LMs stand their Ground: Investigating the Effect of Embodiment in Figurative Language Interpretation by Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Figures are based on the use of words in a way that deviates from their conventional order and meaning. |
| Approach: | They propose to use a figurative language model to interpret embodied metaphors by using larger language models that conceptualise embodies the action of the metaphorical sentence. |
| Outcome: | The proposed model enables interpretation of figurative language when the action of the metaphorical sentence is more embodied. |
Making Better Use of Training Corpus: Retrieval-based Aspect Sentiment Triplet Extraction via Label Interpolation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to extract sentimental triplets are infeasible and counterproductive . aspect Sentiment Triplets Extraction (ASTE) task is an emerging sub-task of Aspect-based Sentimence Analysis . |
| Approach: | They propose a retrieval-based approach to the Aspect Sentiment Triplet Extraction task . they retrieve semantic similar triplets from the training corpus and interpolate their label information . |
| Outcome: | The proposed approach establishes a new state-of-the-art on the Aspect Sentiment Triplet Extraction task. |
Multi-Domain Dialogue State Tracking with Disentangled Domain-Slot Attention (2023.findings-acl)
Copied to clipboard
| Challenge: | Multi-domain dialogue state tracking is a challenge for task-oriented dialogue systems . domains and slots are aggregated into a single query to generate domain-slot specific representations . |
| Approach: | They propose to disentangle domain-slot attention for multi-domain dialogue state tracking by separating query about domains and slots from the attention component. |
| Outcome: | The proposed approach outperforms the standard multi-head attention with aggregated domain-slot query. |
Improved Visual Story Generation with Adaptive Context Modeling (2023.findings-acl)
Copied to clipboard
| Challenge: | a recent work shows that diffusion models generate images of high resolution and semantic consistency to text prompts. |
| Approach: | They propose a method that uses adaptive context modeling to improve leading system . they evaluate their method on pororoSV and FlintstonesSV datasets . |
| Outcome: | The proposed method achieves state-of-the-art FID scores on pororo and Flintstones datasets. |
Question-Interlocutor Scope Realized Graph Modeling over Key Utterances for Dialogue Reading Comprehension (2023.findings-acl)
Copied to clipboard
| Challenge: | Compared to standard RC tasks, dialogue reading comprehension (DRC) has raised challenges because of the complex speaker information and noisy dialogue context. |
| Approach: | They propose a new method for dialogue reading comprehension that extracts answers from dialogues by using key-utterances-extracting methods and a Question-Interlocutor Scope Realized Graph. |
| Outcome: | The proposed method achieves state-of-the-art performance against previous works. |
Speech-to-Speech Translation for a Real-world Unwritten Language (2023.findings-acl)
Copied to clipboard
Peng-Jen Chen, Kevin Tran, Yilin Yang, Jingfei Du, Justine Kao, Yu-An Chung, Paden Tomasello, Paul-Ambroise Duquenne, Holger Schwenk, Hongyu Gong, Hirofumi Inaguma, Sravya Popuri, Changhan Wang, Juan Pino, Wei-Ning Hsu, Ann Lee
| Challenge: | a new study examines speech-to-speech translation (S2ST) that translates speech from one language into another . the research area for unwritten languages remains a research area with little exploration due to the lack of training data. |
| Approach: | They propose a system that translates speech from one language into another . they use Taiwanese Hokkien as an example of an unwritten language . |
| Outcome: | The proposed system can be used to train models in languages without standard writing systems. |
Code Execution with Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
Chenxiao Liu, Shuai Lu, Weizhu Chen, Daxin Jiang, Alexey Svyatkovskiy, Shengyu Fu, Neel Sundaresan, Nan Duan
| Challenge: | Pre-trained code intelligence models ignore the execution trace and only rely on source code and syntactic structures to understand code execution. |
| Approach: | They develop a mutation-based data augmentation technique to create a Python dataset and task for code execution that challenges existing models. |
| Outcome: | The proposed model outperforms existing models on code execution and shows its potential for zero-shot code-to-code search and text-to code generation. |
BertNet: Harvesting Knowledge Graphs with Arbitrary Relations from Pretrained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to construct knowledge graphs are limited to a small set of relations due to manual cost or restrictions in text corpus. |
| Approach: | They propose to automatically construct knowledge graphs (KGs) of diverse new relations from pretrained language models that accept knowledge queries with prompts. |
| Outcome: | The proposed framework extracts knowledge of over 400 new relations from pretrained language models, including RoBERTaNet, with minimal input of a relation definition and a few shot of example entity pairs. |
Sequential Path Signature Networks for Personalised Longitudinal Language Modeling (2023.findings-acl)
Copied to clipboard
| Challenge: | Current work on low-dimensional static user representations or more importantly on dynamic user representation is limited. |
| Approach: | They propose to integrate path signatures from rough path theory into neural sequential models by integrating contextual neural representations and recursive neural networks. |
| Outcome: | The proposed model outperforms state-of-the-art models on macro-average F1 score on two available datasets and outperformed previous models which only have access to historical posts. |
A Multi-modal Debiasing Model with Dynamical Constraint for Robust Visual Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that many well-developed Visual Question Answering systems suffer from bias problem. |
| Approach: | They propose a way to mitigate bias problem by subtracting bias score from standard VQA base score. |
| Outcome: | The proposed method improves on the VQA v2.0 and VQA-CP V2,0 datasets. |
Trigger-Argument based Explanation for Event Detection (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing works on ED use words or phrases to explain models’ inner mechanisms, but for ED, the event structure is more enlightening clues to explain model behaviors. |
| Approach: | They propose a Trigger-Argument based Explanation method which can utilize event structure knowledge to uncover a faithful interpretation for existing ED models at neuron level. |
| Outcome: | The proposed method can reveal the process by which the model predicts on the large-scale MAVEN and the widely-used ACE 2005 datasets. |
Interactive Concept Learning for Uncovering Latent Themes in Large Text Collections (2023.findings-acl)
Copied to clipboard
| Challenge: | Topic modeling is a popular method for identifying emerging themes from text collections. |
| Approach: | They propose a framework that receives and encodes expert feedback at different levels of abstraction. |
| Outcome: | The proposed framework combines automation and manual coding, allowing experts to maintain control while reducing the manual effort required. |
NormMark: A Weakly Supervised Markov Model for Socio-cultural Norm Discovery (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for norm recognition focus only on surface-level features of dialogues and do not take into account the interactions within a conversation. |
| Approach: | They propose a probabilistic generative Markov model to carry latent features throughout a dialogue and trainable on weakly annotated data using the variational technique. |
| Outcome: | The proposed model outperforms current state-of-the-art methods on a weakly annotated dataset, outperforming existing methods, including GPT3. |
VoteTRANS: Detecting Adversarial Text without Training by Voting on Hard Labels of Transformations (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for detecting adversarial attacks need original/adversarial data. |
| Approach: | They propose to detect adversarial text by voting on hard labels from predictions of transformations using a model that uses original/adversarial data. |
| Outcome: | The proposed method detects adversarial text across various state-of-the-art attacks, models, and datasets without training. |
Fusion or Defusion? Flexible Vision-and-Language Pre-Training (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to vision-and-language pretraining (VLP) lack effectiveness and efficiency in downstream multimodal tasks. |
| Approach: | They propose a flexible vision-and-language pre-training model by incorporating cross-modal fusions into a dual-encoder architecture and a cross-module knowledge transfer strategy to guide the training process. |
| Outcome: | The proposed model is well-equipped with effectiveness and efficiency compared with other strong VLP models. |
COCKATIEL: COntinuous Concept ranKed ATtribution with Interpretable ELements for explaining neural net classifiers on NLP (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent debates have shown that attention maps and attribution methods are unreliable. |
| Approach: | They propose a model-agnostic XAI technique that generates meaningful explanations from the last layer of a neural net model trained on an NLP classification task by using Non-Negative Matrix Factorization to discover concepts the model leverages to make predictions. |
| Outcome: | The proposed technique generates meaningful explanations from the last layer of a neural net model trained on an NLP classification task without compromising the accuracy of the underlying model or requiring a new one to be trained. |
Code-Switched Text Synthesis in Unseen Language Pairs (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing efforts on text synthesis for code-switching require training on code-witched texts in the target language pairs. |
| Approach: | They propose a model that synthesizes code-switched texts for language pairs absent from training data by adding an additional code-sharing module to a pre-trained machine translation model. |
| Outcome: | The proposed model synthesizes code-switched texts for language pairs lacking from training data. |
Imagination is All You Need! Curved Contrastive Learning for Abstract Sequence Modeling Utilized on Long Short-Term Dialogue Planning (2023.findings-acl)
Copied to clipboard
| Challenge: | Curved Contrastive Learning is a new approach for learning the relative turn distance between utterance pairs in multi-turn dialogues. |
| Approach: | They propose a technique for learning the relative turn distance between utterance pairs in multi-turn dialogues by projecting goal utterence and reply candidates into a latent space. |
| Outcome: | The proposed technique can guide transformers towards a goal in a zero-shot fashion . it can be used to estimate the likelihood of sequences by the entailment strength . |
Data-Efficient French Language Modeling with CamemBERTa (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in NLP have significantly improved the performance of language models on a variety of tasks. |
| Approach: | They introduce a French DeBERTa model that builds upon the DeBERTAV3 architecture and training objective and evaluate its performance on a variety of French downstream tasks and datasets. |
| Outcome: | The proposed model outperforms BERT-based models on most tasks given the same amount of training tokens and trained on 30% of its input tokens. |
Coupling Large Language Models with Logic Programming for Robust and General Reasoning from Text (2023.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are robust and general, but their reasoning ability is not at a level to compete with the best models trained for specific natural language reasoning problems. |
| Approach: | They propose to use large language models as a few-shot semantic parser to convert natural language sentences into a logical form that serves as input for answer set programs. |
| Outcome: | The proposed model can handle multiple question-answering tasks without requiring retraining for each new task. |
Evaluating the Factual Consistency of Large Language Models Through News Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing LLMs generally assign a higher score to factually consistent summaries than to factualally inconsistent summary. |
| Approach: | They propose a benchmark to measure whether large language models prefer factually consistent continuations of inputs. |
| Outcome: | The proposed benchmark compares the scores an LLM assigns to a factually consistent versus a inconsistent summary for an input news article. |
Text Generation Model Enhanced with Semantic Information in Aspect Category Sentiment Analysis (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for ACSA fail to model relations of target words and opinion words in a sentence including multiple aspects. |
| Approach: | They propose to incorporate AMR into a text generation model to model relations of target words and opinion words in a sentence including multiple aspects. |
| Outcome: | The proposed method outperforms state-of-the-art methods on three datasets. |
Mind the Biases: Quantifying Cognitive Biases in Language Model Prompting (2023.findings-acl)
Copied to clipboard
| Challenge: | Cognitive biases in the human decision making process can lead to flawed responses when we are under uncertainty. |
| Approach: | They propose to expose cognitive biases on results of language model prompting which display bias modes resembling cognitive bias. |
| Outcome: | The proposed methods show that a toning-down transformation of the drug-drug description in a prompt can elicit a bias similar to the framing effect, warning users to distrust when prompting language models for answers. |
CodePrompt: Task-Agnostic Prefix Tuning for Program and Language Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Prompt-tuning methods have been used to solve inefficient parameter update and storage issues in Natural Language Generation tasks. |
| Approach: | They propose a task-agnostic prompt tuning method that reflects the traits of PLM for program language. |
| Outcome: | The proposed method is effective in three PLG tasks, not only in the full-data setting but also in the low-resource setting and cross-domain setting. |
Honey, I Shrunk the Language: Language Model Behavior at Reduced Scale. (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have focused on high-compute settings, leaving the question of when these abilities begin to emerge largely unanswered. |
| Approach: | They investigate whether effects of pre-training can be observed when problem size is reduced, modeling a smaller, reduced-vocabulary language. |
| Outcome: | The proposed model performance is correlated with pre-training perplexity and performance. |
Communication Efficient Federated Learning for Multilingual Neural Machine Translation with Adapter (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing frameworks for federated multilingual neural machine translation (Fed-MNMT) are limited in language resources. |
| Approach: | They propose a framework that keeps PLMs frozen and only transfers lightweight adapter modules between clients. |
| Outcome: | The proposed framework reduces communication cost by over 98% while achieving similar or even better performance compared to baselines. |
Cross-task Knowledge Transfer for Extremely Weakly Supervised Text Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for text classification with extremely weak supervision impose stricter supervision constraints than those under regular weak supervision. |
| Approach: | They propose a framework that creates weak labels by leveraging recent developments in zero-shot text classification. |
| Outcome: | The proposed framework outperforms existing methods on weak labels generated by weakly supervise classification. |
GVdoc - Graph-based Visual DOcument Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | Visual document classifiers have shown impressive performance on in-distribution test sets, but they have a hard time classifying out-of-difference examples. |
| Approach: | They propose a graph-based document classification model that generates a document graph based on its layout and trains a neural network to learn node and graph embeddings. |
| Outcome: | The proposed model outperforms state-of-the-art models on out-of distribution data while maintaining comparable performance on in-distribution data. |
A Sequence-to-Sequence&Set Model for Text-to-Table Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models for text-to-table generation are order-insensitive, but suffer from errors . a novel sequence-tosequence&set model generates table body rows in parallel . |
| Approach: | They propose a sequence-to-sequence generation task that serializes each table into a token sequence during training by concatenating all rows in a top-down order. |
| Outcome: | The proposed model outperforms baselines on commonly-used datasets. |
Automatic Readability Assessment for Closely Related Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | In recent years, the main focus of research on automatic readability assessment (ARA) has shifted towards using expensive deep learning-based methods with the primary goal of increasing models’ accuracy. |
| Approach: | They focus on how linguistic aspects such as mutual intelligibility or degree of language relatedness can improve ARA in a low-resource setting. |
| Outcome: | The inclusion of CrossNGO, a novel feature exploiting n-gram overlap, significantly improves the performance of ARA models compared to the use of off-the-shelf large multilingual language models alone. |
Towards Robust Ranker for Text Retrieval (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for text retrieval are based on a 'retrieval & rerank' pipeline, which uses a fast retriever to fetch a set of top document candidates, while a robust ranker is based upon a weak negative mining during contrastive learning. |
| Approach: | They propose a multi-adversarial training strategy that leverages multiple retrievers as generators to challenge a ranker. |
| Outcome: | The proposed model outperforms the existing de facto ranker training paradigms on the passage retrieval benchmarks using BM25-reranking, full-ranking and retriever distillation. |
Semi-Supervised Domain Adaptation for Emotion-Related Tasks (2023.findings-acl)
Copied to clipboard
| Challenge: | Semi-supervised domain adaptation (SSDA) is a model trained from a label-rich source domain to a new but related domain with a few labels of target data. |
| Approach: | They propose to decompose the semi-supervised domain adaptation framework into two subcomponents of unsupervised domain adaption (UDA) from the source to the target domain and semi-supervised learning (SSL) in the target. |
| Outcome: | The proposed method is based on the co-learning of multiple classifiers for computer vision tasks and is published in the journal Nature. |
Boosting Distress Support Dialogue Responses with Motivational Interviewing Strategy (2023.findings-acl)
Copied to clipboard
| Challenge: | Lack of psychotherapeutic data makes it difficult to train chatbots . lack of mental health workers and stigma further demotivates people from seeking help. |
| Approach: | They propose to rephrase MI non-adherent responses into Advise with permission using a behavioral coding scheme to identify conforming and non-conforming responses. |
| Outcome: | The proposed rephrasers can be built with Blender and GPT3 to rephrase MI non-adherent Advise without permission responses into Adviser with permission. |
ECOLA: Enhancing Temporal Knowledge Embeddings with Contextualized Language Representations (2023.findings-acl)
Copied to clipboard
Zhen Han, Ruotong Liao, Jindong Gu, Yao Zhang, Zifeng Ding, Yujia Gu, Heinz Koeppl, Hinrich Schütze, Volker Tresp
| Challenge: | Existing enhancement approaches cannot be applied to temporal knowledge graphs (tKGs) existing enhancement approaches assume knowledge embedding is time-independent, whereas entity embedded in tKG models evolves . |
| Approach: | They propose to use textual data to enhance temporal knowledge embedding by Enhanced Temporal Knowledge Embeddings with Contextualized Language Representations (ECOLA) to evaluate ECOLA, they introduce three new datasets for training and evaluation. |
| Outcome: | The proposed model significantly improves Hits@1 on the link prediction task. |
Gender-tuning: Empowering Fine-tuning for Debiasing Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for debiasing are resource-intensive and costly. Existing solutions for debiansing require fine-tuning on downstream tasks. |
| Approach: | They propose to integrate Masked Language Modeling (MLM) training objectives into fine-tuning’s training process to debiase the PLMs. |
| Outcome: | The proposed approach outperforms the state-of-the-art baselines in terms of gender bias scores while improving PLMs’ performance solely using the downstream tasks’ dataset. |
TextObfuscator: Making Pre-trained Language Model a Privacy Protector via Obfuscating Word Representations (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing inference services are plagued by privacy concerns, such as sharing sensitive data with service providers. |
| Approach: | They propose a framework for protecting inference privacy by applying random perturbations to clustered representations. |
| Outcome: | The proposed framework protects inference privacy by applying random perturbations to clustered representations. |
Mini-Model Adaptation: Efficiently Extending Pretrained Models to New Languages via Aligned Shallow Training (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to pretrain Masked Language Models (MLMs) are expensive and require a full forward and backward pass over the entire model. |
| Approach: | They propose to learn a shallow mini-model from a fraction of a large model's parameters and plug it into a larger model for rapid cross-lingual transfer. |
| Outcome: | Experiments on XNLI, MLQA and PAWS-X show that mini-model adaptation matches the standard approach using up to 2.3x less compute on average. |
DSP: Discriminative Soft Prompts for Zero-Shot Entity and Relation Extraction (2023.findings-acl)
Copied to clipboard
| Challenge: | Prompt-based methods have shown their efficacy in transferring general knowledge within pre-trained language models (PLMs) however, when applied to zero-shot entity and relation extraction, they struggle with the limited coverage of verbalizers to labels and the slow inference speed. |
| Approach: | They propose a method which reformulates zero-shot tasks into token discrimination tasks without having to construct verbalizers. |
| Outcome: | The proposed method outperforms baselines on two zero-shot entity recognition datasets with higher inference speed and achieves 7.5% improvement over previous state-of-the-art models on Wiki-ZSL and FewRel. |
Exploring Robust Overfitting for Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent literature has revealed their vulnerability to crafted adversarial examples on a wide range of NLP tasks. |
| Approach: | They propose to combine regularization methods with adversarial training to mitigate robust overfitting for pre-trained language models by scaling the model's loss. |
| Outcome: | The proposed methods mitigate robust overfitting upon three top adversarial training methods and further promote adversarially robustness. |
Improving Cross-task Generalization of Unified Table-to-text Models with Compositional Task Configurations (2023.findings-acl)
Copied to clipboard
Jifan Chen, Yuhao Zhang, Lan Liu, Rui Dong, Xinchi Chen, Patrick Ng, William Yang Wang, Zhiheng Huang
| Challenge: | Existing methods for multitask learning typically use a dataset name as input prefix, which limits the effectiveness of multitask training. |
| Approach: | They propose compositional task configurations, a set of prompts prepended to the encoder to improve cross-task generalization of unified models. |
| Outcome: | The proposed model outperforms the UnifiedSKG baseline by noticeable margins in both in-domain and zero-shot settings. |
D-CALM: A Dynamic Clustering-based Active Learning Approach for Mitigating Bias (2023.findings-acl)
Copied to clipboard
| Challenge: | Infusing clustering with active learning with AL can overcome the bias issue of both AL and traditional annotation methods while exploiting AL’s annotation efficiency. |
| Approach: | They propose an algorithm that dynamically adjusts clustering and annotation efforts in response to an estimated classifier error-rate. |
| Outcome: | The proposed algorithm outperforms baseline AL approaches with pretrained transformers and traditional Support Vector Machines on eight datasets for emotion, hatespeech, dialog act, and book type detection tasks. |
Language Anisotropic Cross-Lingual Model Editing (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work studies monolingual model editing, which lacks cross-lingual transferability to perform editing simultaneously across languages. |
| Approach: | They propose a framework to naturally adapt monolingual model editing approaches to the cross-lingual scenario using parallel corpus. |
| Outcome: | The proposed framework adapts monolingual model editing approaches to the cross-lingual scenario using parallel corpus and amplifies different subsets of parameters for each language. |
Diverse Retrieval-Augmented In-Context Learning for Dialogue State Tracking (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work has demonstrated that in-context learning for dialogue state tracking outperforms training methods in the few-shot setting. |
| Approach: | They propose a method for in-context learning for dialogue state tracking that takes into account probabilities of competing surface forms and produces a more accurate dialogue state prediction. |
| Outcome: | The proposed method outperforms trained methods in the few-shot setting and requires little data and zero parameter updates. |
Pre-Trained Language-Meaning Models for Multilingual Parsing and Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) have been used for tasks in computational semantics but meaning representations are not included in PLMs. |
| Approach: | They propose to include meaning representations besides natural language texts in the same model . they propose to use DRSs to improve performance of non-English tasks . |
| Outcome: | The proposed approach achieves the best performance on multilingual parsing and DRS-to-text generation tasks. |
Multi-modal Sarcasm Generation: Dataset and Solution (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on sarcasm generation do not consider generating sarcasastic descriptions for a given image with hashtags that provide the sarkastic target. |
| Approach: | They propose a multi-modal Sarcasm generation task that generates sarcastic descriptions like humans using images, hashtags, and OCR tokens. |
| Outcome: | The proposed method can generate sarcastic descriptions like humans using 5000 images and Twitter text. |
Rethinking Semi-supervised Learning with Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Semi-supervised learning (SSL) is a popular setting to make use of unlabelled data . Currently, there are two popular approaches to make effective use of the unlabelled datasets . |
| Approach: | They compare semi-supervised learning (SSL) and task-adaptive pre-training (TAPT) they find TAPT is a stronger and more robust SSL learner, even when using just a few hundred unlabelled samples . |
| Outcome: | The proposed methods improve model performance across different NLP tasks and data sizes. |
Retrieval-Based Transformer for Table Augmentation (2023.findings-acl)
Copied to clipboard
| Challenge: | Data preparation is one of the most expensive and time-consuming steps when performing analytics or building machine learning models. |
| Approach: | They propose a retrieval augmented transformer model that is self-trained for table augmentation tasks. |
| Outcome: | The proposed model outperforms current state-of-the-art models on EntiTables and WebTables. |
ECG-QALM: Entity-Controlled Synthetic Text Generation using Contextual Q&A for NER (2023.findings-acl)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) requires high-quality labeled datasets. |
| Approach: | They propose a method that uses pre-trained language models to generate entity-controlled text to augment small labeled datasets for downstream NER tasks. |
| Outcome: | The proposed method produces full text samples with desired entities appearing in a controllable way while retaining sentence coherence closest to the real world data. |
Tokenization Impacts Multilingual Language Modeling: Assessing Vocabulary Allocation and Overlap Across Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | Multilingual language models perform surprisingly well in a variety of NLP tasks for diverse languages. |
| Approach: | They propose to evaluate the quality of lexical representation and vocabulary overlap observed in sub-word tokenizers. |
| Outcome: | The proposed criteria show that the overlap of vocabulary across languages can be detrimental to certain downstream tasks. |
The Whole Truth and Nothing But the Truth: Faithful and Controllable Dialogue Response Generation with Dataflow Transduction and Constrained Decoding (2023.findings-acl)
Copied to clipboard
Hao Fang, Anusha Balakrishnan, Harsh Jhamtani, John Bufe, Jean Crawford, Jayant Krishnamurthy, Adam Pauls, Jason Eisner, Jacob Andreas, Dan Klein
| Challenge: | In a task-oriented dialogue system, response generation is a conditional language model, but effective dialogue agents must balance fluent generation with stricter constraints. |
| Approach: | They propose a rule-based content selection model that transduces a dialogue agent’s actions and their results into context-free grammars representing the space of contextually acceptable responses. |
| Outcome: | The proposed architecture outperforms both rule-based and learned approaches in human evaluations of fluency, relevance, and truthfulness. |
Know What I don’t Know: Handling Ambiguous and Unknown Questions for Text-to-SQL (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing text-to-SQL parsers generate a plausible SQL query for arbitrary user questions, thereby failing to handle problematic user questions. |
| Approach: | They propose a weakly supervised DTE model for error detection, localization, and explanation. |
| Outcome: | The proposed model achieves the best result on real-world examples and generated examples compared with baselines. |
Rethinking Document-Level Relation Extraction: A Reality Check (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent efforts push up performance boundaries of document-level relation extraction (DocRE) but these efforts are not promising. |
| Approach: | They construct four types of entity mention attacks to examine model robustness . they also have a close check on model usability in a more realistic setting . |
| Outcome: | The proposed model is based on a strong or untenable assumption in common . the model is robust under four types of mention attacks and usable in a realistic setting . |
Optimizing Test-Time Query Representations for Dense Retrieval (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent developments of dense retrieval rely on quality representations of queries and contexts from pre-trained query and context encoders. |
| Approach: | They propose a test-time optimization of query representations that provides fine-grained pseudo labels over retrieval results. |
| Outcome: | The proposed algorithm improves open-domain question answering accuracy and direct re-ranking by up to 2.0% while running 1.3–2.4x faster with an efficient implementation. |
A Customized Text Sanitization Mechanism with Differential Privacy (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to sanitize texts subject to differential privacy do not work for non-metric semantic similarity measures. |
| Approach: | They propose a customized text sanitization mechanism based on a metric local differential privacy definition. |
| Outcome: | The proposed mechanism achieves better privacy-utility trade-offs than existing mechanisms on benchmark datasets. |
LABO: Towards Learning Optimal Label Regularization via Bi-level Optimization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for regularizing deep neural networks rely on weight decay, dropout, batch/layer normalization to converge faster and generalize. |
| Approach: | They propose a framework for training with label regularization which includes conventional LS but can also model instance-specific variants. |
| Outcome: | The proposed approach consistently yields better results than conventional regularization on seven machine translation and three image classification tasks while maintaining training efficiency. |
Frustratingly Easy Label Projection for Cross-lingual Transfer (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to improve cross-lingual transfer performance are based on word alignment, but no empirical studies have evaluated their effectiveness or limitations. |
| Approach: | They propose a mark-then-translate method that integrates translation and projection by inserting special markers around the labeled spans in the original sentence. |
| Outcome: | The proposed method outperforms word alignment-based methods in 57 languages and three tasks. |
Enhancing Hierarchical Text Classification through Knowledge Graph Integration (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to hierarchical text classification are limited by lack of domain knowledge, which leads to mistakes in a variety of situations. |
| Approach: | They propose a Knowledge-enabled Hierarchical Text Classification model which integrates knowledge graphs into HTC to address the knowledge limitations of traditional methods. |
| Outcome: | The proposed model integrates knowledge graphs into the hierarchical text classification process, addressing the knowledge limitations of traditional methods. |
How Many Answers Should I Give? An Empirical Study of Multi-Answer Reading Comprehension (2023.findings-acl)
Copied to clipboard
| Challenge: | Despite recent progress in multi-answer MRC, there is no systematic analysis of how this phenomenon arises and how to better address it. |
| Approach: | They develop a taxonomy to categorize commonly-seen multi-answer MRC instances and examine how well different paradigms deal with different types of multi-announced questions. |
| Outcome: | The proposed taxonomy categorizes commonly-seen multi-answer instances and analyzes how well different paradigms deal with different types of multi-announced instances. |
An Exploration of Encoder-Decoder Approaches to Multi-Label Classification for Legal and Biomedical Text (2023.findings-acl)
Copied to clipboard
| Challenge: | Standard methods for multi-label text classification rely on encoder-only pre-trained models . encoder decoder models have proven more effective in other classification tasks . |
| Approach: | They compare four methods for multi-label classification based on encoder-only models . they use a pre-trained model for multilabel text classification . |
| Outcome: | The proposed methods outperform encoder-only methods on complex datasets and labeling schemes. |
Domain Incremental Lifelong Learning in an Open World (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to lifelong learning (LL) models require access to task identities in the testing phase or cannot handle samples from unseen tasks. |
| Approach: | They propose a dynamic architecture-based lifelong learning model that tries to learn a sequence of tasks with a prompt-enhanced language model. |
| Outcome: | The proposed model outperforms state-of-the-art models in handling unseen tasks and focuses on task-level prompts to capture knowledge from different granularities. |
Improving Knowledge Graph Completion with Generative Hard Negative Mining (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for knowledge graph completion (KGC) use generative methods with a self-information-enhanced training strategy to generate high-quality negatives. |
| Approach: | They propose to leverage a sequence-to-sequence architecture to generate high-quality hard negatives from the same decoding distributions as the anchor. |
| Outcome: | The proposed method produces high-quality negatives with good hardness and diversity on three KGC benchmarks. |
Visually-Enhanced Phrase Understanding (2023.findings-acl)
Copied to clipboard
| Challenge: | Large-scale vision-language pre-training models generate high-quality textual representations, which often outperform models that are purely text-based, such as BERT. |
| Approach: | They propose to utilize both textual and visual encoders of multi-modal pre-trained models to enhance language understanding tasks by generating an image associated with a textual prompt. |
| Outcome: | The proposed method outperforms models that are purely text-based on visual and textual understanding tasks and significantly improves the entity clustering task. |
Reasoning in Large Language Models Through Symbolic Math Word Problems (2023.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have revolutionized NLP by solving downstream tasks with little to no labeled data. |
| Approach: | They propose a model that uses symbolic expressions to provide a concise explanation of the numeric answer. |
| Outcome: | The proposed model has good accuracy on symbolic word problems and is able to provide a concise and verifiable reasoning and make it more interpretable. |
It’s not Sexually Suggestive; It’s Educative | Separating Sex Education from Suggestive Content on TikTok videos (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing systems for TikTok remove/punish both types of videos, even though they serve different purposes. |
| Approach: | They propose a dataset that contains TikTok videos labeled as sexually suggestive, sex-educational content, or neither . they explore two transformer-based models for classifying the videos to validate their importance . |
| Outcome: | The proposed dataset is useful and invites further study on the subject. |
Dynamic Structured Neural Topic Model with Self-Attention Mechanism (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent topic models that capture the time-series evolution of topics assume that topics evolve independently without interaction. |
| Approach: | They propose a dynamic structured neural topic model which captures topic dependencies while capturing their dependencies. |
| Outcome: | The proposed model outperforms a prior dynamic embedded topic model regarding perplexity and coherence while maintaining sufficient diversity across topics. |
Hybrid-Regressive Paradigm for Accurate and Speed-Robust Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Autoregressive translation (NAT) is less robust in decoding batch size and hardware settings than NAT. |
| Approach: | They propose a two-stage translation prototype that prompts a small number of AT predictions and fills in previously skipped tokens at once. |
| Outcome: | The proposed translation prototype achieves comparable translation quality with AT while having 1.5x faster inference speed regardless of batch size and device. |
Commonsense Knowledge Transfer for Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in pre-trained language models have transformed the landscape of natural language processing. |
| Approach: | They propose a framework to transfer commonsense knowledge stored in a neural commonsensing model to a general-purpose pre-trained language model. |
| Outcome: | Empirical results show that the proposed framework improves the model’s performance on downstream tasks that require commonsense reasoning. |
Shielded Representations: Protecting Sensitive Attributes Through Iterative Gradient-Based Projection (2023.findings-acl)
Copied to clipboard
| Challenge: | Natural language processing models tend to learn and encode social biases present in the data. |
| Approach: | They propose a method for removing non-linear encoded concepts from neural representations by iteratively training neural classifiers to predict a particular attribute, followed by a projection of the representation on a hypersurface. |
| Outcome: | The proposed method removes non-linear encoded concepts from neural representations. |
Focal Training and Tagger Decouple for Grammatical Error Correction (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for Grammatical Error Correction (GEC) are categorized into sequence-to-sequence approaches, tagging-based approaches, and hybrid approaches. |
| Approach: | They propose to decouple error detection layer from label tagging layer and to down-weight label imbalance and tabbing entanglement loss using Focal Loss. |
| Outcome: | The proposed methods are effective over three latest Chinese Grammatical Error Correction datasets. |
LET: Leveraging Error Type Information for Grammatical Error Correction (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for grammatical error correction (GEC) are mainly divided into detection-based and end-to-end generative models. |
| Approach: | They propose an end-to-end framework which Leverages Error Type (LET) information in the generation process to introduce more convincing error type information. |
| Outcome: | The proposed framework outperforms existing methods on various datasets by a clear margin. |
On the Role of Parallel Data in Cross-lingual Transfer Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing multilingual models do not exploit the full potential of monolingual data, a new study finds . prior work has shown that parallel data is beneficial for cross-lingual learning, but it is unclear if it is the data itself or the modeling of parallel interactions that matters. |
| Approach: | They compare unsupervised machine translation to supervised machine translator and gold parallel data to generate synthetic parallel data. |
| Outcome: | The proposed model generated parallel data is better than supervised machine translation and gold parallel data in both general and task-specific settings. |
CoMave: Contrastive Pre-training with Multi-scale Masking for Attribute Value Extraction (2023.findings-acl)
Copied to clipboard
Xinnan Guo, Wentao Deng, Yongrui Chen, Yang Li, Mengdi Zhou, Guilin Qi, Tianxing Wu, Dong Yang, Liubin Wang, Yong Pan
| Challenge: | Existing methods to extract product features from unstructured text still suffer from problems . e-commerce platforms are focusing on multi-scale values, which can be confusing . |
| Approach: | They propose a pre-training technique to automatically obtain attribute value pairs from product descriptions to aid e-commerce. |
| Outcome: | The proposed method improves on the existing token-level masking strategy and achieves state-of-the-art on four benchmarks. |
Phrase Retrieval for Open Domain Conversational Question Answering with Conversational Dependency Modeling via Contrastive Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Open-Domain Conversational Question Answering (ODConvQA) aims to answer questions through a multi-turn conversation . however, such a pipeline approach makes the reader vulnerable to errors propagated from the retriever, which makes it slower since they are not runnable in parallel. |
| Approach: | They propose a method to directly predict answers with a phrase retrieval scheme for a sequence of words. |
| Outcome: | The proposed method outperforms the baselines on two ODConvQA datasets. |
Unlearning Bias in Language Models by Partitioning Gradients (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent research has shown that large-scale pretrained language models exhibit issues relating to racism, sexism, religion bias, and toxicity in general. |
| Approach: | They propose a gray-box method for debiasing pretrained masked language models using partitioned contrastive gradient unlearning (PCGU) aims to optimize only the weights that contribute most to a specific domain of bias by computing a first-order approximation based on the gradients of contrastive sentence pairs. |
| Outcome: | The proposed method is low-cost and can pinpoint the sources of social bias in large pretrained language models. |
Meta-training with Demonstration Retrieval for Efficient Few-shot Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Large language models have impressive fewshot performance on many NLP tasks and domains. |
| Approach: | They propose a meta-training approach that uses demonstration retrieval to train parameter-efficient models that generalize well on a larger variety of tasks. |
| Outcome: | The proposed approach outperforms many parameter-efficient methods on QA, NLI, and text classification tasks. |
VCSUM: A Versatile Chinese Meeting Summarization Dataset (2023.findings-acl)
Copied to clipboard
| Challenge: | Compared to news and chat summarization, meeting summarizing is decelerated by the limited data. |
| Approach: | They propose a Chinese meeting summarization dataset that provides annotations for each transcript and a set of benchmark models to facilitate further research. |
| Outcome: | The proposed model can be used to summarize the content of meeting transcripts in Chinese. |
LEDA: a Large-Organization Email-Based Decision-Dialogue-Act Analysis Dataset (2023.findings-acl)
Copied to clipboard
Mladen Karan, Prashant Khare, Ravi Shekhar, Stephen McQuistin, Ignacio Castro, Gareth Tyson, Colin Perkins, Patrick Healey, Matthew Purver
| Challenge: | Using dialog acts to study decision-making in large distributed organizations is challenging due to the size and distributed nature of such groups. |
| Approach: | They propose a set of dialog acts for the study of decision-making mechanisms in large distributed organizations. |
| Outcome: | The proposed dataset can be used to better understand decision-making in large distributed organizations. |
Negation Scope Refinement via Boundary Shift Loss (2023.findings-acl)
Copied to clipboard
| Challenge: | Negation detection is a complex linguistic phenomenon with long spans . existing methods tend to make wrong predictions around the scope boundaries . |
| Approach: | They propose a model which engages the Boundary Shift Loss to refine the predicted boundary. |
| Outcome: | The proposed model refines the predicted boundary on multiple datasets. |
Towards Diverse and Effective Question-Answer Pair Generation from Children Storybooks (2023.findings-acl)
Copied to clipboard
Sugyeong Eo, Hyeonseok Moon, Jinsung Kim, Yuna Hur, Jeongwook Kim, SongEun Lee, Changwoo Chun, Sungsoo Park, Heuiseok Lim
| Challenge: | Recent advances in QA pair generation (QAG) have raised interest in applying this technique to the educational field. |
| Approach: | They propose a QAG framework that enhances QA type diversity by producing different interrogative sentences and implicit/explicit answers. |
| Outcome: | The proposed framework outperforms state-of-the-art methods by significant margins, achieving improved diversity and quality. |
Pulling Out All The Full Stops: Punctuation Sensitivity in Neural Machine Translation and Evaluation (2023.findings-acl)
Copied to clipboard
| Challenge: | Several studies have documented the sensitivity of machine translation models to noise, including spelling errors and dialects. |
| Approach: | They propose to test machine translation systems for punctuation sensitivity by performing sentence-final perturbation tests with full stops, exclamation and questions marks. |
| Outcome: | The proposed tests show that commercial translation systems are vulnerable to single punctuation insertion or deletion, resulting in unreliable translations. |
Reimagining Retrieval Augmented Language Models for Answering Queries (2023.findings-acl)
Copied to clipboard
Wang-Chiew Tan, Yuliang Li, Pedro Rodriguez, Richard James, Xi Victoria Lin, Alon Halevy, Wen-tau Yih
| Challenge: | Large language models (LLMs) are expensive to train, deploy, and maintain, both financially and in terms of environmental impact. |
| Approach: | They present a reality check on large language models and compare their predictions to retrieval-augmented language models. |
| Outcome: | The proposed models fare better on question answering tasks and have become the foundation of impressive demos like Chat-GPT. |
Numeric Magnitude Comparison Effects in Large Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Prior research on the representational capabilities of LLMs evaluates whether they show human-level performance. |
| Approach: | They ask how well popular LLMs capture the magnitudes of numbers from a behavioral lens. |
| Outcome: | The proposed model captures the magnitudes of numbers from a behavioral lens. |
Multi-Relational Probabilistic Event Representation Learning via Projected Gaussian Embedding (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for event representation learning ignore relations and uncertainty of events . Experimental results show that the proposed approach outperforms other state-of-the-art baselines on both existing and newly constructed datasets. |
| Approach: | They propose a novel approach to learning multi-relational probabilistic event embeddings based on contrastive learning. |
| Outcome: | The proposed method outperforms existing benchmarks on existing and newly constructed datasets. |
PragmatiCQA: A Dataset for Pragmatic Question Answering in Conversations (2023.findings-acl)
Copied to clipboard
| Challenge: | Mars? - PragmatiCQA |
| Approach: | Mars? - The Paper . |
| Outcome: | The proposed dataset features 6873 QA pairs that explores pragmatic reasoning in conversations over a diverse set of topics. |
Modular and On-demand Bias Mitigation with Attribute-Removal Subnetworks (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that pre-trained language models can be used to mitigate societal biases and stereotypes. |
| Approach: | They propose a modular bias mitigation approach that integrates debiasing modules into the core model on-demand at inference time. |
| Outcome: | The proposed approach improves on-par with baseline finetuning on gender, race, and age protected attributes on three classification tasks with gender, age, and race as protected attributes. |
Scientific Fact-Checking: A Survey of Resources and Approaches (2023.findings-acl)
Copied to clipboard
| Challenge: | Fact-checking is the task of assessing the veracity of factual claims based on credible evidence and background knowledge. |
| Approach: | They propose to automate scientific fact-checking using natural language processing to assess the veracity of factual claims based on credible evidence and background knowledge. |
| Outcome: | The proposed methods can help combat the spread of misinformation and help individuals understand new scientific breakthroughs. |
Uni-Encoder: A Fast and Accurate Response Selection Paradigm for Generation-Based Dialogue Systems (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to decode a given context-candidate pair are expensive and time-consuming. |
| Approach: | They propose a new paradigm that keeps full attention over each pair while only encoding the context once. |
| Outcome: | The proposed paradigm achieves new state-of-the-art results on four benchmark datasets with high computational efficiency. |
DLAMA: A Framework for Curating Culturally Diverse Facts for Probing the Knowledge of Pretrained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | a few benchmarking datasets have been released to evaluate the factual knowledge of pretrained language models. |
| Approach: | They propose a framework for curating factual triples from Wikidata that are culturally diverse. |
| Outcome: | The proposed framework is built of factual triples from three pairs of contrasting cultures with 78,259 triples. |
Self-adaptive Context and Modal-interaction Modeling For Multimodal Emotion Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to predict emotion label for a given utterance lack modeling of diverse dependency ranges and inconsistent treatment of contribution for various modalities. |
| Approach: | They propose a multimodal emotion recognition in conversation task that uses context and multiple modalities to predict emotion label for a given utterance. |
| Outcome: | The proposed method outperforms the state-of-the-art methods on three multimodal datasets. |
Structure-Discourse Hierarchical Graph for Conditional Question Answering on Long Documents (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to conditional question answering on long documents ignore document structure and discourse relations between sentences in document sections. |
| Approach: | They construct a Structure-Discourse Hierarchical Graph and conduct bottom-up information propagation to address this issue. |
| Outcome: | The proposed approach outperforms the existing methods on the conditional question answering on long documents by 3.0 EM score and 2.4 F1 score on answer measuring, and 2.2 EM and 1.9 F1 scores on jointly answer and condition measuring. |
COBRA Frames: Contextual Reasoning about Effects and Harms of Offensive Statements (2023.findings-acl)
Copied to clipboard
Xuhui Zhou, Hao Zhu, Akhila Yerukola, Thomas Davidson, Jena D. Hwang, Swabha Swayamdipta, Maarten Sap
| Challenge: | Understanding the harms and offensiveness of statements requires reasoning about the social and situational context in which they are made. |
| Approach: | They propose a context-aware formalism for explaining the intents, reactions, and harms of offensive statements grounded in their social and situational contexts. |
| Outcome: | The proposed framework is the first context-aware formalism for explaining the intents, reactions, and harms of offensive statements grounded in their social and situational context. |
Distilling Calibrated Knowledge for Stance Detection (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for stance detection ignore meaningful signals among categories offered by hard labels. |
| Approach: | They propose to use knowledge distillation to calibrate teacher predictions in each generation step. |
| Outcome: | The proposed method can calibrate teacher predictions in each generation step and improves stance detection accuracy. |
PTCSpell: Pre-trained Corrector Based on Character Shape and Pinyin for Chinese Spelling Correction (2023.findings-acl)
Copied to clipboard
| Challenge: | Chinese spelling correction (CSC) is a task which detects incorrect characters in Chinese text and corrects them. |
| Approach: | They propose to pre-train a Chinese spelling correction corrector under the detector-corrector architecture and propose to capture pronunciation and shape information in Chinese characters. |
| Outcome: | The proposed corrector achieves an average of 5.8% F1 improvements over state-of-the-art methods, verifying its effectiveness. |
Disentangling Text Representation With Counter-Template For Unsupervised Opinion Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches for unsupervised opinion summarization are based on reconstruction model, but selection is too coarse as not all information in each input is equally essential for the summary. |
| Approach: | They propose a framework for unsupervised opinion summarization based on text representation disentanglement with counter-template. |
| Outcome: | The proposed framework outperforms the state-of-the-art models on quality and stability on two benchmark datasets. |
Evaluation of Question Generation Needs More References (2023.findings-acl)
Copied to clipboard
Shinhyeok Oh, Hyojun Go, Hyeongdon Moon, Yunsung Lee, Myeongho Jeong, Hyun Seung Lee, Seungtaek Choi
| Challenge: | Existing evaluations of QG methods rely on single reference-based similarity metrics . multiple (pseudo) references are more effective for QG evaluation . |
| Approach: | They propose to paraphrase the reference question for a more robust QG evaluation. |
| Outcome: | The proposed frameworks show higher correlation with human evaluations than evaluation with a single reference. |
XtremeCLIP: Extremely Parameter-efficient Tuning for Low-resource Vision Language Understanding (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to fine-tune visual-language understanding (VLU) require tasks-specific designs and sufficient training data. |
| Approach: | They propose a simple yet efficient paradigm for low-resource Visual Language Understanding (VLU) they reformulate a series of VLU tasks as an open-book affinity-matching problem. |
| Outcome: | The proposed framework outperforms baselines in low-resource settings. |
FACTUAL: A Benchmark for Faithful and Consistent Textual Scene Graph Parsing (2023.findings-acl)
Copied to clipboard
Zhuang Li, Yuyang Chai, Terry Yue Zhuo, Lizhen Qu, Gholamreza Haffari, Fei Li, Donghong Ji, Quan Hung Tran
| Challenge: | Existing parsers that convert image captions into scene graphs often suffer from errors and inconsistency. |
| Approach: | They propose a dataset that re-annotates image captions using a new intermediate representation called FACTUAL-MR and a metric to measure scene graph similarity. |
| Outcome: | The proposed parser outperforms existing parsers in terms of faithfulness and consistency on multiple benchmark datasets. |
Target-Oriented Relation Alignment for Cross-Lingual Stance Detection (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work on cross-lingual stance detection has ignored the inconsistency in the occurrences and distributions of targets between languages, which consequently degrades the performance of stance detector in low-resource languages. |
| Approach: | They propose a fine-grained method which considers both target-level associations and language-level alignments to learn the in-language and cross-language associations. |
| Outcome: | The proposed method is compared with competing methods under variant settings and shows that it performs better in low-resource languages. |
NonFactS: NonFactual Summary Generation for Factuality Evaluation in Document Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained abstractive summarization models generate fluent summaries that are inconsistent with context document and contain nonfactual information. |
| Approach: | They propose a data generation model that synthesizes nonfactual summaries using human annotations. |
| Outcome: | The proposed model can generate nonfactual summaries and generalize to out-of-domain documents. |
When to Read Documents or QA History: On Unified and Selective Open-domain QA (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work aims to answer factoid questions from an open set of domains using knowledge sources. |
| Approach: | They propose to use QA-pair and document corpora to answer open-domain questions . they propose to apply natural follow-up to both models to find answers . |
| Outcome: | The proposed method is validated on natural questions and TriviaQA. |
Interpretable Automatic Fine-grained Inconsistency Detection in Text Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing factual consistency evaluation approaches for text summarization provide binary predictions and limited insights into the weakness of summarizing systems. |
| Approach: | They propose a task of fine-grained inconsistency detection to detect inconsistent summaries . they represent facts with semantic frames extracted by semantic role labeling and highlight related semantic frames to predict inconsistance . |
| Outcome: | The proposed model outperforms baseline models and provides evidence to support or refute the summary. |
A Multi-dimensional study on Bias in Vision-Language models (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have focused on the issue of bias in joint Vision-Language models . pre-trained models complete a neutral template with a hurtful word 5% of the time . |
| Approach: | They propose to use a multi-dimensional bias metric to investigate bias in English VL models . they use gender, ethnicity, and age as dimensions to analyze bias in VLs . |
| Outcome: | The proposed model is based on gender, ethnicity, and age as dimensions. |
Correction of Errors in Preference Ratings from Automated Metrics for Text Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluation methods are over-confident in assigning significant differences between systems . Currently, the most reliable evaluation methods for text generation are human-based evaluations. |
| Approach: | They propose to combine human ratings with automated ratings to reduce the amount of human ratings needed to arrive at robust results. |
| Outcome: | The proposed evaluation protocol reduces the amount of human ratings by 50% while yielding the same evaluation outcome as the pure human evaluation in 95% of cases. |
PEER: Pre-training ELECTRA Extended by Ranking (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models for pre-training require expensive pre-trainer computation cost . ELECTRA model can perform replaced token detection (RTD) task with reduced pre- training cost compared to current models . |
| Approach: | They propose to extend a discriminator-based replaced token detection task into a ranker-based task . they propose to use a binary classifier to perform a more precise task with negligible additional computation cost. |
| Outcome: | The proposed model outperforms state-of-the-art models with ELECTRA in GLUE tasks given the same cost. |
ML-LMCL: Mutual Learning and Large-Margin Contrastive Learning for Improving ASR Robustness in Spoken Language Understanding (2023.findings-acl)
Copied to clipboard
| Challenge: | Despite efforts to improve ASR robustness, errors from pipeline approaches can lead to error propagation. |
| Approach: | They propose a framework for improving ASR robustness in SLU by using mutual learning and large-margin contrastive learning. |
| Outcome: | The proposed framework outperforms existing models and achieves new state-of-the-art performance on three datasets. |
Guiding Dialogue Agents to Complex Semantic Targets by Dynamically Completing Knowledge Graph (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing knowledge graphs are incomplete in tracking complex semantic relations of the target-oriented dialogue. |
| Approach: | They combine methods of knowledge retrieval and relationship prediction to construct a context-related dynamic KG and a metric to evaluate the tracked path automatically. |
| Outcome: | The proposed method can control the agent more logically and smoothly toward the complex target. |
Chain of Thought Prompting Elicits Knowledge Augmentation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing knowledge augmentation methods require retrieving knowledge from external knowledge sources or developing a reasoner to leverage the logical rules within the external knowledge source. |
| Approach: | They propose a Chain-of-Thought-based method that augments knowledge for deep learning by removing the need for additional knowledge retrieval or knowledge reasoning models. |
| Outcome: | The proposed method outperforms both pure CoT-based methods and the non-augmented method across the majority of 11 publicly available benchmarks for various reasoning tasks. |
TACR: A Table Alignment-based Cell Selection Method for HybridQA (2023.findings-acl)
Copied to clipboard
| Challenge: | Hybrid Question-Answering datasets lack a robust reasoning model for text-based QA. |
| Approach: | They propose a table-question-alignment-based cell-selection and reasoning model for hybrid text and table QA. |
| Outcome: | The proposed model outperforms baselines on HybridQA and WikiTableQuestions datasets on cell selection and argumentation. |
Modeling Cross-Cultural Pragmatic Inference with Codenames Duet (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work on pragmatic reasoning tests using simple word reference games with unidentified speakers and listeners, but speakers' sociocultural background shapes their pragmatic assumptions. |
| Approach: | They propose a dataset which operationalizes sociocultural pragmatic inference in a word reference game. |
| Outcome: | The proposed model improves clue-giving and guessing tasks by accounting for background characteristics and the game context. |
Werewolf Among Us: Multimodal Resources for Modeling Persuasion Behaviors in Social Deduction Games (2023.findings-acl)
Copied to clipboard
Bolin Lai, Hongxin Zhang, Miao Liu, Aryan Pariani, Fiona Ryan, Wenqi Jia, Shirley Anugrah Hayati, James Rehg, Diyi Yang
| Challenge: | Existing studies on persuasive behavior modeling focus on textual dialogues . a multimodal dataset is available for persuasion modeling . |
| Approach: | They propose a multimodal dataset for modeling persuasive behaviors using visual signals. |
| Outcome: | The proposed dataset includes 199 dialogue transcriptions and videos captured in a multi-player social deduction game setting and 26,647 utterance level annotations of persuasion strategy and game level annotation of deduction game outcomes. |
Long to reign over us: A Case Study of Machine Translation and a New Monarch (2023.findings-acl)
Copied to clipboard
| Challenge: | We examine translations between French and English in contexts with ambiguity . with the passing of Queen Elizabeth II, MT systems can produce errors due to linguistic features of both languages and the paucity of references to kings in the training data. |
| Approach: | They examine translations between French and English as they were produced by MT systems . they find that even when human translators would have adequate context, machine translation systems do not always produce the expected output. |
| Outcome: | The proposed model shows that even when human translators have context, machine translation systems do not always produce the expected output. |
A Unified Generative Approach to Product Attribute-Value Identification (2023.findings-acl)
Copied to clipboard
| Challenge: | Product attribute value identification (PAVI) is a core task in the e-commerce industry. |
| Approach: | They propose a generative approach to product attribute-value identification (PAVI) they use product text to decode a set of attribute- value pairs as a target sequence from the given product text. |
| Outcome: | The proposed approach outperforms extraction- and classification-based methods on large-scale real-world datasets. |
K-UniMorph: Korean Universal Morphology and its Feature Schema (2023.findings-acl)
Copied to clipboard
| Challenge: | Previously, the Korean language has been underrepresented in the field of morphological paradigms amongst hundreds of diverse world languages. |
| Approach: | They propose a new Universal Morphology dataset for Korean that preserves its distinct characteristics. |
| Outcome: | The proposed dataset extracts inflected Korean verb forms from the largest annotated corpus for Korean. |
How does the brain process syntactic structure while listening? (2023.findings-acl)
Copied to clipboard
| Challenge: | Syntactic parsing is the task of assigning a syntactical structure to a sentence. |
| Approach: | They investigate the predictive power of the brain encoding models in three settings: individual performance of constituency and dependency parsing based embedding methods, relative effectiveness of each of the syntactic parsers, and relative importance of syntaktic information versus semantic information using BERT embeddngs. |
| Outcome: | The proposed methods explain variance in brain regions, especially in listening tasks, and the relative predictive power of the various syntax parsers is unexplored. |
Towards Imperceptible Document Manipulations against Neural Ranking Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Current approaches to detect vulnerabilities in neural ranking models often introduce noticeable errors and require a well-imitated surrogate NRM to guarantee the attack effect. |
| Approach: | They propose a framework called Imperceptible DocumEnt Manipulation to produce adversarial documents that are less noticeable to both algorithms and humans. |
| Outcome: | The proposed framework outperforms strong baselines while maintaining fluency and correctness of the target documents. |
Ask an Expert: Leveraging Language Models to Improve Strategic Reasoning in Goal-Oriented Dialogue Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing dialogue models may encounter scenarios which are not well-represented in the training data and produce unnatural, inappropriate, or unhelpful responses. |
| Approach: | They propose a framework in which a model is trained with access to an "expert" they propose to optimize the model to selectively utilize (or ignore) advice given context and dialogue history. |
| Outcome: | The proposed framework improves quality across all expert sizes and with fewer parameters than the dialogue model itself. |
SciReviewGen: A Large-scale Dataset for Automatic Literature Review Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing literature review models have addressed literature review generation, but lack of large-scale datasets has been a stumbling block. |
| Approach: | They propose to use a large-scale dataset to evaluate automatic literature review generation models. |
| Outcome: | The proposed model can generate summaries comparable to human-written reviews while lacking detailed information. |
Revisiting Sample Size Determination in Natural Language Understanding (2023.findings-acl)
Copied to clipboard
Ernie Chang, Muhammad Hassan Rashid, Pin-Jie Lin, Changsheng Zhao, Vera Demberg, Yangyang Shi, Vikas Chandra
| Challenge: | Recent work has sought to reduce the annotation costs through the use of active learning and data sampling. |
| Approach: | They propose to estimate the training sample size needed to achieve a targeted model performance based on small amount of training samples. |
| Outcome: | The proposed approach predicts model performance within a small margin of mean absolute error (0.9%) with only 10% data. |
TransESC: Smoothing Emotional Support Conversation via Turn-Level State Transition (2023.findings-acl)
Copied to clipboard
| Challenge: | Emotion Support Conversation (ESC) is a goaldirected task with the goal of reducing the emotional distress of people. |
| Approach: | They propose to take turn-level state Transitions of ESC from three perspectives to generate smooth transitions between utterances. |
| Outcome: | The proposed method generates smoother and more effective responses on automatic and human evaluations. |
Residual Prompt Tuning: improving prompt tuning with residual reparameterization (2023.findings-acl)
Copied to clipboard
| Challenge: | Prompt tuning is one of the most parameter-efficient approaches for parameter-effective tuning of pre-trained language models. |
| Approach: | They propose to reparameterize soft prompt embeddings using a shallow network with a residual connection and use it to tune prompt embeds P. |
| Outcome: | The proposed method outperforms prompt tuning on SuperGLUE, T5-Base and BERT-Bass models and can reduce the prompt length by 10 times without hurting performance. |
Attend, Select and Eliminate: Accelerating Multi-turn Response Selection with Dual-attention-based Content Elimination (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models can be used to perform multi-turn response selection, but they can be expensive. |
| Approach: | They propose a framework and a strategy that progressively selects and eliminates unimportant content under context-response dual-attention. |
| Outcome: | The proposed method can effectively speed-up SOTA models without much performance degradation and shows a better trade-off between speed and performance than previous methods. |
Medical Dialogue Generation via Dual Flow Modeling (2023.findings-acl)
Copied to clipboard
| Challenge: | Medical dialogue systems (MDS) aim to provide patients with medical services, such as diagnosis and prescription. |
| Approach: | They propose a Dual Flow enhanced Medical (DFMed) dialogue generation framework that extracts the medical entities and doctor's dialogue acts used in the dialogue history and models their transitions with an entity-centric graph flow and a sequential act flow. |
| Outcome: | The proposed framework exceeds baselines in both automatic and manual evaluations on two datasets. |
Listen, Decipher and Sign: Toward Unsupervised Speech-to-Sign Language Recognition (2023.findings-acl)
Copied to clipboard
Liming Wang, Junrui Ni, Heting Gao, Jialu Li, Kai Chieh Chang, Xulin Fan, Junkai Wu, Mark Hasegawa-Johnson, Chang Yoo
| Challenge: | Existing supervised sign language recognition systems rely on well-annotated data . instead, an unsupervised speech-to-sign language recognition system learns to translate between spoken and sign languages by observing only non-parallel speech and sign-language corpora. |
| Approach: | They propose an unsupervised speech-to-sign language recognition system that can translate between spoken and sign languages by observing only non-parallel speech and sign-language corpora. |
| Outcome: | The proposed approach outperforms baseline models on sign language corpora by 50% . the proposed approach is available at https://github.com/cactuswiththoughts/UnsupSpeech2Sign.git . |
Distinguishing Address vs. Reference Mentions of Personal Names in Text (2023.findings-acl)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a core task in the NLP community . but not much work has been done to distinguish between addressing and referring to entities . |
| Approach: | They propose an automatic tagger that captures the address vs. reference distinction in English . they demonstrate how this distinction is important in NLP and computational social science applications . |
| Outcome: | The proposed tagger performs at 85% accuracy in distinguishing between address and reference in English . many modern Indo-European languages do not have such vocative case markers . |
“Low-Resource” Text Classification: A Parameter-Free Classification Method with Compressors (2023.findings-acl)
Copied to clipboard
| Challenge: | Text classification is one of the most fundamental tasks in natural language processing (NLP), but deep neural networks are data-hungry and expensive to train. |
| Approach: | They propose a non-parametric alternative to DNNs that uses a compressor like gzip and a k-nearest-neighbor classifier to achieve competitive results. |
| Outcome: | The proposed method outperforms BERT on all five OOD datasets and outperformed other methods on the few-shot setting. |
LR-Sum: Summarization for Less-Resourced Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | LR-Sum contains human-written summaries for 40 languages, many of which are less-resourced. |
| Approach: | They propose to use a permissively-licensed dataset to analyze human-written summaries for 40 languages. |
| Outcome: | The proposed dataset contains human-written summaries for 40 languages . authors describe abstractive and extractive summarization experiments . |
RQUGE: Reference-Free Metric for Evaluating Question Generation by Answering the Question (2023.findings-acl)
Copied to clipboard
Alireza Mohammadshahi, Thomas Scialom, Majid Yazdani, Pouya Yanki, Angela Fan, James Henderson, Marzieh Saeidi
| Challenge: | Existing metrics for evaluating the quality of automatically generated questions are expensive and penalise valid questions that may not have high lexical or semantic similarity to the reference questions. |
| Approach: | They propose a question-answering and span scorer metric based on the answerability of the candidate question given the context. |
| Outcome: | The proposed metric has higher correlation with human judgment without relying on the reference question. |
Unsupervised Semantic Variation Prediction using the Distribution of Sibling Embeddings (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work on semantic variation prediction has focused on comparing an averaged contextualised representation of a word . however, some of the previously associated meanings of . a target word can become obsolete over time, while novel usages of existing words are observed. |
| Approach: | They propose a method that uses the entire cohort of contextualised embeddings of a target word to detect the semantic variation of words. |
| Outcome: | The proposed method outperforms existing methods on a SemEval-2020 benchmark dataset and is comparable to the state-of-the-art. |
TranSFormer: Slow-Fast Transformer for Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Prior work has focused on treating subwords as basic units in developing such systems. |
| Approach: | They propose a slow-fast two-stream learning model that uses a “slow” branch to deal with subword sequences and a "fast" branch to cope with longer character sequences. |
| Outcome: | The proposed model shows consistent BLEU improvements (larger than 1 BLUE point) on several machine translation benchmarks. |
Mitigating the Learning Bias towards Repetition by Self-Contrastive Training for Open-Ended Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing language models generate repetitive texts with greedy decoding or beam search. |
| Approach: | They propose a self-contrastive training technique to penalize the output of a premature checkpoint of the same model when it incorrectly predicts repetition. |
| Outcome: | The proposed training mitigates repetition while maintaining fluency while minimizing the overestimation of token-level repetition probabilities. |
Digging out Discrimination Information from Generated Samples for Robust Visual Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to solve this problem rely on additional annotations and generate negative samples . |
| Approach: | They propose a method to Dig out Discrimination information from Generated samples to address these limitations. |
| Outcome: | The proposed method improves on the visual question-answering datasets. |
Words as Gatekeepers: Measuring Discipline-specific Terms and Meanings in Scholarly Publications (2023.findings-acl)
Copied to clipboard
| Challenge: | Specialized terminology, or specialized language, can facilitate efficient in-group communication within fields but hinder understanding for out-groups. |
| Approach: | They propose an interpretable approach to measure scholarly jargon from text using word sense induction to identify words that are widespread but overloaded with different meanings across fields. |
| Outcome: | The proposed method is validated by word sense induction and shows that it provides a complementary, yet unique view of jargon alongside word types. |
Trade-Offs Between Fairness and Privacy in Language Modeling (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing research suggests that privacy preservation comes at the price of worsening biases in classification tasks. |
| Approach: | They propose to incorporate privacy preservation and de-biasing techniques into training text generation models to investigate the trade-off between the two dimensions. |
| Outcome: | The proposed model improves on bias detection, privacy attacks, language modeling, and performance on downstream tasks. |
CSS: A Large-scale Cross-schema Chinese Text-to-SQL Medical Dataset (2023.findings-acl)
Copied to clipboard
| Challenge: | a cross-domain text-to-SQL task aims to parse user questions into SQL on complete unseen databases . a single-domain task evaluates the performance on identical databases based on the same domain . |
| Approach: | They propose a cross-domain text-to-SQL task that parses user questions into SQL on unseen databases. |
| Outcome: | The proposed system can parse user questions into SQL on complete unseen databases. |
Silver Syntax Pre-training for Cross-Domain Relation Extraction (2023.findings-acl)
Copied to clipboard
| Challenge: | Relation Extraction (RE) is the task of extracting structured knowledge from unstructured text. |
| Approach: | They exploit the affinity between syntactic structure and semantic RE to obtain low-cost pre-training data. |
| Outcome: | The proposed model outperforms baseline models in five out of six cross-domain setups without additional annotated data. |
FastDiff 2: Revisiting and Incorporating GANs and Diffusion Models in High-Fidelity Speech Synthesis (2023.findings-acl)
Copied to clipboard
| Challenge: | Experimental results show that Generative adversarial networks sacrifice sample diversity for quality and speed, while diffusion models exhibit outperformed sample quality and diversity at a high computational cost. |
| Approach: | They propose to combine GANs and diffusion probabilistic models to achieve better sample quality and diversity. |
| Outcome: | The proposed models outperform GANs and diffusion models in speech synthesis . the proposed models enjoy an efficient 4-step sampling process and exhibit better sample diversity . |
Uncovering Hidden Consequences of Pre-training Objectives in Sequence-to-Sequence Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have found that some pre-training objectives have negligible impact on downstream performance, but they are not effective on all models. |
| Approach: | They propose a zero-shot control method that is only successful on a subset of models and experiment with different pre-training objectives. |
| Outcome: | The proposed method is only successful on a subset of models and unexpected interactions between the pre-training method and downstream controllability are found. |
Exploring Anisotropy and Outliers in Multilingual Language Models for Cross-Lingual Semantic Sentence Similarity (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that contextual language models display outlier dimensions . this is true for monolingual and multilingual models, but little work has been done on multilingual contexts . |
| Approach: | They investigate outlier dimensions and their relationship to anisotropy in multilingual contexts . they focus on cross-lingual semantic similarity tasks . |
| Outcome: | The proposed model improves on cross-lingual semantic similarity tasks. |
Revisiting Sentence Union Generation as a Testbed for Text Consolidation (2023.findings-acl)
Copied to clipboard
| Challenge: | In order to acquire knowledge on a new subject, it is often necessary to consult multiple sources of written information. |
| Approach: | They propose to revisit the sentence union generation task as an effective well-defined testbed for assessing text consolidation capabilities. |
| Outcome: | The proposed evaluation protocol includes human and automatic evaluations. |
Distilling Reasoning Capabilities into Smaller Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | a step-by-step reasoning approach like chain of thought has proved to be effective in eliciting reasoning abilities in large language models. |
| Approach: | They propose a knowledge distillation approach that leverages CoT reasoning capabilities of larger models and distills them into smaller models. |
| Outcome: | The proposed scheme boosts the performance of smaller models over 70% on multiple reasoning datasets. |
AlignSTS: Speech-to-Singing Conversion via Cross-Modal Alignment (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to speech-to-singing voice conversion are difficult to learn in text-free situations. |
| Approach: | They propose an STS model which views speech variance as different modalities . it uses a novel rhythm adaptor to predict the target rhythm representation . they also use the predicted rhythm representation to re-align the content . |
| Outcome: | The proposed model achieves superior performance in terms of objective and subjective metrics. |
A New Task and Dataset on Detecting Attacks on Human Rights Defenders (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets for event extraction cannot extract information from textual sources. |
| Approach: | They propose to use crowdsourced annotations on 500 online news articles to train and evaluate baseline models to predict annotated characteristics. |
| Outcome: | The proposed dataset includes crowdsourced annotations on 500 online news articles and includes fine-grained information about the type and location of attacks, as well as information about victims. |
Improving Language Model Integration for Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to integrate external language models into machine translation systems have been based on the assumption that the external model learns an implicit target-side language model at decoding time. |
| Approach: | They transfer this concept to the task of machine translation and compare it with the most prominent way of including additional monolingual data - namely back-translation. |
| Outcome: | The proposed approach outperforms the most prominent way of including additional monolingual data, namely back-translation. |
Type Enhanced BERT for Correcting NER Errors (2023.findings-acl)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is the task of identifying spans that belong to particular categories, such as person, location, organization, etc. |
| Approach: | They propose a method that integrates named entity’s type information into BERT by an adapter layer and integrates it into a gazetteer. |
| Outcome: | The proposed method outperforms baselines in multiple corpus. |
Bridge the Gap Between CV and NLP! A Gradient-based Textual Adversarial Attack Framework (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for adversarial samples are poorly applied in computer vision . however, textual adversarials are still vulnerable to small perturbations . |
| Approach: | They propose a framework to extend existing adversarial attack methods to textual adversarials by adding optimized perturbations to embedding layer and amplifying them in forward propagation process. |
| Outcome: | The proposed framework achieves better performance even using proxy gradient information and produces more fluent and grammatical adversarial samples compared to baseline methods. |
DUB: Discrete Unit Back-translation for Speech Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Discrete unit back-translation (DUB) is a back-translated speech-to-text translation (ST) technique that can be applied to ST . a modality gap between speech and text makes it difficult to transfer these techniques to ST due to the modality of the speech-text model. |
| Approach: | They propose a method to represent speech with discrete units instead of continuous features in direct ST. |
| Outcome: | The proposed method achieves comparable performance to existing methods that rely on large-scale external data. |
Knowledge Graph Embeddings using Neural Ito Process: From Multiple Walks to Stochastic Trajectories (2023.findings-acl)
Copied to clipboard
Mojtaba Nayyeri, Bo Xiong, Majid Mohammadi, Mst. Mahfuja Akter, Mirza Mohtashim Alam, Jens Lehmann, Steffen Staab
| Challenge: | Existing knowledge graph embeddings have problems expressing knowledge graphs because they model a specific relation r from a head h to tails by transitioning deterministically to exactly one other point in the embeddable space. |
| Approach: | They propose a framework that models relations between nodes by relation-specific, stochastic transitions. |
| Outcome: | The proposed framework is expressive and generic subsuming state-of-the-art models operating on low-dimensional manifolds. |
Leveraging Denoised Abstract Meaning Representation for Grammatical Error Correction (2023.findings-acl)
Copied to clipboard
| Challenge: | Popular GEC models use large-scale synthetic corpora or use a large number of human-designed rules. |
| Approach: | They propose a model that incorporates denoised AMR as additional knowledge to get AMRs more reliable. |
| Outcome: | The proposed model reduces training time by 32% while inference time is comparable. |
Prediction and Calibration: Complex Reasoning over Knowledge Graph with Bi-directional Directed Acyclic Graph Neural Network (2023.findings-acl)
Copied to clipboard
| Challenge: | Knowledge graphs (KGs) organize world knowledge as interlinked triples which describe entities and their relationships. |
| Approach: | They propose a bi-directional Directed Acyclic Graph neural network that splits the reasoning process into prediction and calibration. |
| Outcome: | The proposed model outperforms previous QE models on FB15k, FB16k-237, and NELL995 on prediction and calibration. |
Prompt-Based Metric Learning for Few-Shot NER (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing metric learning methods do not fully incorporate label semantics into modeling. |
| Approach: | They propose a method to largely improve metric learning for few-shot named entity recognition (NER) a pre-defined category is a key natural language understanding task . |
| Outcome: | The proposed method outperforms the previous state-of-the-art (SOTA) method with 16 of 18 settings outperformed previous methods by 9.12% and 34.51% . |
OpenPI-C: A Better Benchmark and Stronger Baseline for Open-Vocabulary State Tracking (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work on state tracking assumes both entities and state space are known, which limits their applicability. |
| Approach: | They propose a two-stage model that refines the state change prediction conditioned on entities predicted from the first stage. |
| Outcome: | The proposed model improves on the cleaned dataset and the evaluation metric on the proposed model. |
I run as fast as a rabbit, can you? A Multilingual Simile Dialogues Datasets (2023.findings-acl)
Copied to clipboard
| Challenge: | A simile is a figure of speech that compares two different things via shared properties. |
| Approach: | They propose a multilingual simile dialogue dataset that can be used to study similes in real-life scenarios. |
| Outcome: | The proposed dataset is the largest manually annotated simile dataset and contains both English and Chinese data. |
Controllable Conversation Generation with Conversation Structures via Diffusion Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Current generation models fail to effectively utilize rich linguistic and world knowledge to generate coherent long text. |
| Approach: | They propose a conversation generation framework that incorporates human knowledge and conversation structures with both controllability and interpretability for better conversation generation. |
| Outcome: | The proposed framework incorporates human knowledge and conversation structures with both controllability and interpretability for better conversation generation. |
Few-shot Low-resource Knowledge Graph Completion with Reinforced Task Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing few-shot learning-based models have difficulty alleviating the long-tail issue on low-resource KGs because of the lack of training tasks. |
| Approach: | They propose a few-shot low-resource knowledge graph completion framework that generates and selects beneficial few- shot tasks that complement current tasks. |
| Outcome: | The proposed framework is based on several real-world knowledge graphs and validates on multiple domains. |
Incomplete Utterance Rewriting as Sequential Greedy Tagging (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies show that users of dialogue systems tend to use incomplete utterances which usually omit (a.k.a. ellipsis) or refer back (a k.k a co-reference) to the concepts that appeared in previous dialogue contexts. |
| Approach: | They propose a sequence tagging-based model which extracts information from context and introduce speaker-aware embedding to model speaker variation. |
| Outcome: | The proposed model outperforms previous models on restoration scores while having other metric scores comparable to state-of-the-art models. |
Exploiting Commonsense Knowledge about Objects for Visual Activity Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing tasks that aim to identify the objects in an image are object detection and image classification, but recent work has focused on more comprehensive image under- standing tasks. |
| Approach: | They propose to incorporate commonsense knowledge about physical objects into a transformer-based model that is trained to predict the actionverb for visual activity recognition. |
| Outcome: | The proposed model incorporates prototypical function knowledge about physical objects to predict the actionverb for visual activity recognition. |
Tucker Decomposition with Frequency Attention for Temporal Knowledge Graph Completion (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models for temporal knowledge graph completion only consider the combination of one relation with one timestamp, ignoring the global nature of the embedding. |
| Approach: | They propose a temporal knowledge Graph Completion model that captures global temporal dependencies between one relation and the entire timestamp. |
| Outcome: | The proposed model outperforms the state-of-the-art models on three standard TKGC datasets on several metrics. |
Another Dead End for Morphological Tags? Perturbed Inputs and Parsing (2023.findings-acl)
Copied to clipboard
| Challenge: | Part-of-speech tags are used for word-contextualized parsers, but their impact is limited to word-based models. |
| Approach: | They propose an adversarial attack to test whether morphological tags contribute to error propagation or correct parsing mistakes. |
| Outcome: | The proposed attack on 14 treebanks shows that if morphological tags were utopically robust against lexical perturbations, they would be able to correct parsing mistakes. |
HeGeL: A Novel Dataset for Geo-Location from Hebrew Text (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets in English for textual geolocation are limited because of the location of the place is implicit. |
| Approach: | They propose to use a Hebrew place description corpus to analyze lingual geospatial reasoning. |
| Outcome: | The Hebrew Geo-Location corpus collects literal Hebrew place descriptions and analyzes lingual geospatial reasoning. |
Modeling Adversarial Attack on Pre-trained Language Models as Sequential Decision Making (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) have shown strong potential in various downstream tasks. |
| Approach: | They propose to model adversarial attack task as a sequential decision-making problem where the whole attack process is sequential with two decision- making problems, i.e., word finder and word substitution. |
| Outcome: | The proposed approach achieves the highest attack success rate with a comparable modification rate and semantic similarity to attack fine-tuned BERT. |
Towards Robust Personalized Dialogue Generation via Order-Insensitive Representation Regularization (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have focused on developing persona consistent dialogue models . order sensitivity affects the quality and consistency of generated response . |
| Approach: | They propose a model-agnostic framework to improve persona consistent dialogue response generation by concatenating persona texts and dialogue history as a single input sequence. |
| Outcome: | The proposed framework outperforms existing models on the Persona-Chat dataset and shows that it is more robust under different persona orders and more consistent with the persona profile. |
Cost-effective Distillation of Large Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing knowledge distillation methods require pretraining of the teacher on task-specific datasets, which can be costly for large and unstable for small datasets. |
| Approach: | They propose an approach to improve knowledge distillation by a loss-agnostic approach to task and model architecture. |
| Outcome: | The proposed method achieves competitive results across a range of tasks, especially for tasks with smaller datasets. |
Task-Optimized Adapters for an End-to-End Task-Oriented Dialogue System (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work on end-to-end dialogue models with pre-trained dialogue corpora shows promising performance in the conversational system. |
| Approach: | They propose an end-to-end TOD system with task-optimized adapters which learn independently per task adding only small number of parameters after fixed layers of pre-trained network. |
| Outcome: | The proposed system achieves state-of-the-art performance on the MultiWOZ benchmark compared to existing models. |
I Spy a Metaphor: Large Language Models and Diffusion Models Co-Create Visual Metaphors (2023.findings-acl)
Copied to clipboard
Tuhin Chakrabarty, Arkadiy Saakyan, Olivia Winn, Artemis Panagopoulou, Yue Yang, Marianna Apidianaki, Smaranda Muresan
| Challenge: | Visual metaphors are powerful rhetorical devices used to communicate creative ideas through images. |
| Approach: | They propose to generate visual metaphors from linguistic metaphors by using large language models and Diffusion models. |
| Outcome: | The proposed task requires the ability to model implicit meaning and compositionality. |
Text Augmentation Using Dataset Reconstruction for Low-Resource Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for text classification use labeled data, but labeles are expensive and difficult to obtain. |
| Approach: | They propose a novel method of data augmentation using the text-generation capabilities of language models. |
| Outcome: | The proposed method improves the current state-of-the-art methods for data augmentation on multi-class datasets. |
LaSQuE: Improved Zero-Shot Classification from Explanations Through Quantifier Modeling and Curriculum Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Several recent approaches have explored training machine learning models via natural language supervision, but they fail to leverage linguistic quantifiers and mimic humans in compositionally learning complex tasks. |
| Approach: | They propose a method that can learn zero-shot classifiers from language explanations by using three new strategies: (1) modeling the semantics of linguistic quantifiers in explanations; (2) aggregating information from multiple explanations using an attention-based mechanism; (3) model training via curriculum learning. |
| Outcome: | The proposed method outperforms previous work showing an absolute gain of up to 7% in generalizing to unseen real-world classification tasks. |
Learned Adapters Are Better Than Manually Designed Adapters (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to improve adapter-based tuning are sub-optimal . a learning framework is proposed to learn the optimal adapter architectures . |
| Approach: | They propose a framework to automatically learn optimal adapter architectures for better task adaptation of pre-trained models. |
| Outcome: | The proposed framework outperforms the previous parameter-efficient tuning baselines while tuning comparable or fewer parameters. |
Automatic Identification of Code-Switching Functions in Speech Transcripts (2023.findings-acl)
Copied to clipboard
| Challenge: | Code-switching, or switching between languages, occurs for many reasons and has important linguistic, sociological, and cultural implications. |
| Approach: | They build a system to identify a wide range of functions for which speakers code-switch in everyday speech with an accuracy of 75% . they use a dataset of Hindi-English code-witched data to analyze their results . |
| Outcome: | The proposed system can identify a wide range of functions for which speakers code-switch in everyday speech, with an accuracy of 75% across all functions. |
Federated Domain Adaptation for Named Entity Recognition via Distilling with Heterogeneous Tag Sets (2023.findings-acl)
Copied to clipboard
| Challenge: | a federated domain adaptation approach is used to learn with NER datasets from multiple platforms while not violating data privacy. |
| Approach: | They propose to use a distillation approach to facilitate knowledge transfer across platforms. |
| Outcome: | The proposed model performs better in the clinic domain. |
Interpreting Sentiment Composition with Latent Semantic Tree (2023.findings-acl)
Copied to clipboard
| Challenge: | Current researches on sentiment classification are shifting from improving model performance to interpretability. |
| Approach: | They propose a new tree form capable of interpreting sentiment composition in a principled way. |
| Outcome: | The proposed tree can explain sentiment composition in a principled way. |
Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models (2023.findings-acl)
Copied to clipboard
Yuhui Zhang, Michihiro Yasunaga, Zhengping Zhou, Jeff Z. HaoChen, James Zou, Percy Liang, Serena Yeung
| Challenge: | Recent studies show that some tasks exhibit inverse scaling, or U-shaped scaling, where the performance degrades as models are scaled up. |
| Approach: | They propose a task that asks questions with negation to show positive scaling . they hypothesize that solving NeQA depends on question answering and negation understanding . |
| Outcome: | The proposed task can exhibit inverse scaling, U-shaped scaling, or positive scaling, and the scaling trends shift as the task is more powerful. |
Contrastive Training Improves Zero-Shot Classification of Semi-structured Documents (2023.findings-acl)
Copied to clipboard
| Challenge: | Xu et al., 2020 focus on semi-structured document classification in a zero-shot setting . positional, layout, and style information play a vital role in interpreting such documents . |
| Approach: | They propose a matching-based approach that relies on a pairwise contrastive objective for pretraining and fine-tuning. |
| Outcome: | The proposed method significantly improves Macro F1 in the zero-shot learning setting. |
Extracting Shopping Interest-Related Product Types from the Web (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing e-commerce products are limited in their ability to assist customers in interest-oriented shopping. |
| Approach: | They propose to extract PTs from Web pages containing hand-crafted PT recommendations for SIs . they propose to use tree-transformer encoders for node classification to improve inter-node dependency modeling . |
| Outcome: | The proposed model outperforms the best baseline model by 2.37 F1 points on a WebPT dataset. |
Multilingual Pre-training with Self-supervision from Global Co-occurrence Information (2023.findings-acl)
Copied to clipboard
| Challenge: | Empirical studies show multilinguality and crosslinguality emerge from MLM pretraining without supervision. |
| Approach: | They propose to use global co-occurrence information as a source of structural information on multilingual corpora. |
| Outcome: | Empirical studies show that MLM-GC pre-training outperforms MLM pre- training for 4 downstream cross-lingual tasks and 1 additional monolingual task. |
Low-Rank Updates of pre-trained Weights for Multi-Task Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Multi-task learning is a popular approach for learning with pre-trained models due to the complexity of the tasks and the challenges associated with fine-tuning large pre-train models. |
| Approach: | They propose a new approach for Multi-task learning which is based on stacking the weights of Neural Networks as a tensor. |
| Outcome: | The proposed approach achieves equivalent performance to the state-of-the-art on the general language understanding evaluation benchmark by training only 0.3 of the parameters per task while not modifying the baseline weights. |
Sequential Integrated Gradients: a simple but effective method for explaining language models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing explanation methods such as Integrated Gradients (IG) produce a path for each word of a sentence simultaneously, which can lead to sentences with no clear meaning or a significantly different meaning compared to the original one. |
| Approach: | They propose to use a sequenced integrated gradient method to fix every word in a sentence and move it along a straight path to the word of interest. |
| Outcome: | The proposed method improves on Integrated Gradients (IG) and DIG, but can produce sentences with different meanings than the original one. |
DiffuDetox: A Mixed Diffusion Model for Text Detoxification (2023.findings-acl)
Copied to clipboard
Griffin Floto, Mohammad Mahdi Abdollah Pour, Parsa Farinneya, Zhenwei Tang, Ali Pesaranghader, Manasa Bharadwaj, Scott Sanner
| Challenge: | Existing text generation models that reduce toxicity of toxic text are inadequate for text detoxification tasks. |
| Approach: | They propose a conditional and unconditional diffusion model for text detoxification . conditional model takes toxic text as condition and reduces its toxicity . experimental results show the model achieves human-level fluency . |
| Outcome: | The proposed model reduces toxic text and produces diverse sentences . it can be used to train other models and ensure fluency . |
Separating Context and Pattern: Learning Disentangled Sentence Representations for Low-Resource Extractive Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Context information is one of the key factors for extractive summarization, but other factors can be used to identify sentence importance. |
| Approach: | They propose to disentangle context and pattern factors for extractive summarization . they separate context and patterns for a better generalization ability in low-resource setting . |
| Outcome: | The proposed model can be used in the zero-shot setting or fine-tuned in the few-shot settings. |
Disentangling Reasoning Capabilities from Language Models with Compositional Reasoning Transformers (2023.findings-acl)
Copied to clipboard
| Challenge: | ReasonFormer is a unified reasoning framework for complex decision-making . it is based on the dual-process theory of cognitive science, where two cognitive systems interact to form a whole reasoning process. |
| Approach: | They propose a unified reasoning framework that mirrors the modular reasoning process of humans . they decouple the representation module and the reasoning modules to capture different levels of cognition . |
| Outcome: | The proposed framework shows that humans can perform better in complex decision-making tasks. |
Towards Argument-Aware Abstractive Summarization of Long Legal Opinions with Summary Reranking (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing summarization models struggle to accurately capture the main arguments of long legal opinions, leading to suboptimal summaries. |
| Approach: | They propose a framework for abstractive summarization of long legal opinions that takes into account the argument structure of the document and reranks them based on alignment with the document's argument structure. |
| Outcome: | The proposed approach outperforms several strong baselines on a dataset of long legal opinions and outperformed existing models. |
Probabilistic Transformer: A Probabilistic Dependency Model for Contextual Word Representation (2023.findings-acl)
Copied to clipboard
| Challenge: | Syntactic structures were deemed essential in natural language processing . but since the deep learning revolution, NLP has been dominated by neural models that do not consider syntactical structures in their design. |
| Approach: | They propose a model that models latent representations of words in a sentence . they use a conditional random field to model latent and dependency arcs . |
| Outcome: | The proposed model performs competitively to transformers on small to medium sized datasets. |
Joint Speech Transcription and Translation: Pseudo-Labeling with Out-of-Distribution Data (2023.findings-acl)
Copied to clipboard
| Challenge: | a recent study shows that self-training can improve upon fully supervised baselines in low-resource settings for several sequence-to-sequence tasks. |
| Approach: | They propose to use pseudo-labeling to label unsupervised data and add it to the training pool. |
| Outcome: | The proposed setup improves on the unsupervised data by using pseudo-labeling . the proposed setup provides 0.4% absolute WER and 2.1 BLEU points for En–De . |
Word-level Prefix/Suffix Sense Detection: A Case Study on Negation Sense with Few-shot Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Morphological analysis is an important research issue in natural language processing . prefixes/suffixes are sometimes ambiguous, causing difficulty in detecting negation sense . |
| Approach: | They propose a context-free morphological analysis task that deals with negation sense . they propose morphology task that uses input-augmentation prompts to train a model . |
| Outcome: | The proposed approach is effective in detecting negation senses in a corpus of prefixes/suffixes . Empirical studies show that the proposed approach works in context-free mode . |
End-to-End Simultaneous Speech Translation with Differentiable Segmentation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to perform simultaneous speech translation always separate segmentation from the underlying model. |
| Approach: | They propose to use Differentiable Segmentation (DiSeg) to learn segmentation from the translation model. |
| Outcome: | Experimental results show that the proposed model can learn segmentation from the translation model. |
Joint Generator-Ranker Learning for Natural Language Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for text generation train the generator and ranker individually . existing methods neglect the mutual feedback that could enhance the quality of outputs . |
| Approach: | They propose a joint training algorithm that integrates the generator and ranker in a single framework. |
| Outcome: | The proposed algorithm surpasses existing methods on four public datasets across three common generation scenarios. |
Multilingual Sequence-to-Sequence Models for Hebrew NLP (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work on pretrained language models for Hebrew is under-parameterized and under-trained . previous work on pretraining Hebrew LMs focused on encoder-only architectures . |
| Approach: | They propose to use sequence-to-sequence generative architectures to train large LMs in morphologically rich languages such as Hebrew. |
| Outcome: | The proposed model improves on all existing Hebrew NLP benchmarks. |
Multilingual Knowledge Graph Completion from Pretrained Language Models with Knowledge Constraints (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for multilingual knowledge graph completion do not align with mKGC tasks because of their English-centric bias. |
| Approach: | They propose to use multilingual pretrained language models to solve queries in different languages by reasoning a tail entity. |
| Outcome: | The proposed method outperforms the previous SOTA on Hits@1 and Hits @10 by 12.32% and 16.03% on public datasets. |
Towards Better Hierarchical Text Classification with Data Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to improve hierarchical text classification are expensive and lack high-quality labeled data. |
| Approach: | They propose a hierarchical text classification framework that can achieve both label controllability and text diversity by extracting high-quality hierarchic label information. |
| Outcome: | The proposed method can achieve label controllability and text diversity by extracting high-quality hierarchical label information. |
History repeats: Overcoming catastrophic forgetting for event-centric temporal knowledge graph completion (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for knowledge graph completion are incomplete and can lead to errors . retraining the model with the entire updated TKG can mitigate forgetting but is computationally burdensome. |
| Approach: | They propose a temporal regularization framework that allows repurposing of parameters . they propose 'clustering-based experience replay' that reinforces the past knowledge . |
| Outcome: | The proposed framework adapts to new events while reducing catastrophic forgetting. |
Multi-Agent Language Learning: Symbolic Mapping (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work has focused on the emergence of language in cooperative tasks where neural network agents learn a communication protocol from scratch to solve problems together. |
| Approach: | They propose a task transfer method and symbolic mapping architecture to help agents learn a compositional and symmetric language in dialog games. |
| Outcome: | The proposed method can help agents learn a compositional and symmetric language in complex settings like dialog games and the proposed architecture promotes vocabulary expansion. |
Scaling Laws for BERT in Low-Resource Settings (2023.findings-acl)
Copied to clipboard
| Challenge: | Large language models require huge training corpora, which is unobtainable for most NLP practitioners. |
| Approach: | They propose power-law formulas that relate model size, corpora size and computation power to find the optimal settings in advance given a fixed budget. |
| Outcome: | The proposed models perform better on MLM and NLU tasks on four languages of different linguistic characteristics. |
Pre-trained Language Model with Prompts for Temporal Knowledge Graph Completion (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for temporal knowledge graph completion (TKGC) focus on extracting information from timestamps and insufficiently utilizing implied information in relations. |
| Approach: | They propose a temporal knowledge graph completion model with prompts that converts quadruples into pre-trained language inputs and prompts to make coherent sentences with implicit semantic information. |
| Outcome: | The proposed model can make coherent sentences with implicit semantic information. |
Is Continuous Prompt a Combination of Discrete Prompts? Towards a Novel View for Interpreting Continuous Prompts (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on the interpretability and transferability of continuous prompts have not been conducted on the subject. |
| Approach: | They propose to interpret continuous prompts as the weighting of discrete prompts by jointly optimizing prompt fidelity and downstream fidelity. |
| Outcome: | The proposed interpretations provide effective readability and plausibility, which is helpful to understand the decision-making of continuous prompts and discover potential shortcuts. |
Putting Natural in Natural Language Processing (2023.findings-acl)
Copied to clipboard
| Challenge: | human language is firstly spoken and only secondarily written. |
| Approach: | aaron carroll: human language is firstly spoken and only secondarily written . carroll says the field of NLP has overwhelmingly focused on processing written language . he says the focus is on a subset of human language which is convenient to work with . |
| Outcome: | the ACL 2023 theme track urges the community to check the reality of the progress in NLP . |
Impact of Adversarial Training on Robustness and Generalizability of Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Adversarial training is widely acknowledged as the most effective defense against adversarial attacks, but achieving both robustness and generalization requires a trade-off. |
| Approach: | They propose to compare pre-training data augmentation and training time input perturbations with embedding space perturbations to find out whether they improve generalization. |
| Outcome: | The proposed methods improve generalization and robustness of the trained models. |
Benchmarking Diverse-Modal Entity Linking with Generative Models (2023.findings-acl)
Copied to clipboard
Sijia Wang, Alexander Hanbo Li, Henghui Zhu, Sheng Zhang, Pramuditha Perera, Chung-Wei Hang, Jie Ma, William Yang Wang, Zhiguo Wang, Vittorio Castelli, Bing Xiang, Patrick Ng
| Challenge: | Existing models for diverse-mode entity linking (EL) work well on per modality configurations, but it is more challenging to design a unified model for diverse modality. |
| Approach: | They propose a generative diverse-modal model that integrates text, image and table . they propose combining a multimodal encoder-decoder paradigm with a fine-tuning GDMM . |
| Outcome: | The proposed model outperforms state-of-the-art models by 8.51 F1 on average for diverse-modal EL. |
Improving Empathetic Dialogue Generation by Dynamically Infusing Commonsense Knowledge (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work on generating empathetic responses by utilizing the speaker's emotion has not been successful. |
| Approach: | They propose an approach which incorporates an adaptive module for commonsense knowledge selection to ensure consistency between the generated empathetic responses and the speaker’s situation. |
| Outcome: | The proposed approach outperforms baseline models in both automatic and human evaluations, exhibiting the generation of more coherent and empathetic responses. |
Additive manifesto decomposition: A policy domain aware method for understanding party positioning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for estimating policy domain aware party similarity are limited to global domains. |
| Approach: | They propose a workflow for estimating policy domain aware party similarity by aggregating policy domains into a single figure . they use a set of tools to extract party positions on major policy axes via multidimensional scaling. |
| Outcome: | The proposed method yields high correlation when predicting party similarity at a global level and provides accurate party-specific positions even with automatically labelled policy domains. |
Similarizing the Influence of Words with Contrastive Learning to Defend Word-level Adversarial Text Attack (2023.findings-acl)
Copied to clipboard
| Challenge: | Neural language models are vulnerable to word-level adversarial text attacks . previous word-based search methods assume important words influence prediction . |
| Approach: | They propose a method for similarizing the influence of words with contrast learning that encourages model to learn sentence representations in which words of varying importance have a more uniform influence on prediction. |
| Outcome: | The proposed method is compatible with various training methods and improves model robustness against various adversarial attacks. |
Responsibility Perspective Transfer for Italian Femicide News (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work has shown that different descriptions of gender-based violence influence the reader’s perception of who is to blame for the violence. |
| Approach: | They propose to automatically rewrite GBV descriptions to alter the perceived level of blame on the perpetrator. |
| Outcome: | The proposed task alters perceived responsibility levels for perpetrators by using unsupervised, zero-shot and few-shot methods. |
Stereotypes and Smut: The (Mis)representation of Non-cisgender Identities by Text-to-Image Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Initial studies have pointed to the potential for harm due to predictive bias, reflecting and potentially reinforcing cultural stereotypes. |
| Approach: | They conduct a survey among non-cisgender individuals and interviews to establish which harms affected individuals anticipate, and how they would like to be represented. |
| Outcome: | The results show that certain non-cisgender identities are consistently (mis)represented as less human, more stereotyped and more sexualised. |
Fine-grained Artificial Neurons in Audio-transformers for Disentangling Neural Auditory Encoding (2023.findings-acl)
Copied to clipboard
Mengyue Zhou, Xu Liu, David Liu, Zihao Wu, Zhengliang Liu, Lin Zhao, Dajiang Zhu, Lei Guo, Junwei Han, Tianming Liu, Xintao Hu
| Challenge: | Existing studies treat each transformer encoding layer as a single artificial neuron . layer-level embeddings aggregate multiple types of contextual attention captured by multiple head modules . |
| Approach: | They propose to embed each transformer encoding layer as a single artificial neuron . they propose to couple those ANs with their biological-neuron counterparts in the human brain . |
| Outcome: | The proposed models can be used to link representations to brain activity, the authors say . their results show that the proposed models carry meaningful neurolinguistic information . |
Deeply Coupled Cross-Modal Prompt Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing prompt-tuning methods focus on language branch or learn vision-language interaction in a shallow mechanism. |
| Approach: | They propose a Deeply coupled Cross-modal Prompt learning method based on CLIP to facilitate the interplay between vision and language with a Cross-Modal Prompting Attention mechanism. |
| Outcome: | The proposed method enables the interplay between vision and language with a Cross-Modal Prompt Attention mechanism. |
Opinion Tree Parsing for Aspect-based Sentiment Analysis (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing generative models for aspect-based sentiment analysis lack structure well-formedness guarantees and built-in elements alignments. |
| Approach: | They propose an opinion tree parsing model which parses all sentiment elements from an opinion-tree. |
| Outcome: | The proposed model is much faster than previous models and can explore correlations among sentiment elements. |
CoMix: Guide Transformers to Code-Mix using POS structure and Phonetics (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing multilingual transformer models lack the ability to intermix words of one language into the structure of another. |
| Approach: | They propose a pretraining approach to improve representation of code-mixed data in transformer models by incorporating phonetic signals, a modified attention mechanism and weak supervision guided generation by parts-of-speech constraints. |
| Outcome: | The proposed model improves performance across four code-mixed tasks and generalizes on out-of-domain translation. |
Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes (2023.findings-acl)
Copied to clipboard
Cheng-Yu Hsieh, Chun-Liang Li, Chih-kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alex Ratner, Ranjay Krishna, Chen-Yu Lee, Tomas Pfister
| Challenge: | Deploying large language models (LLMs) is difficult because they are memory inefficient and compute-intensive for practical applications. |
| Approach: | They propose a mechanism that fine tunes or distills small models that outperform LLMs . they use human labels to fine tune models or LLM-generated labels to train models . |
| Outcome: | The proposed method outperforms LLMs by using fewer training examples compared to few-shot prompted models using substantially smaller model sizes. |
Prosody-TTS: Improving Prosody with Masked Autoencoder and Conditional Diffusion Model For Expressive Text-to-Speech (2023.findings-acl)
Copied to clipboard
| Challenge: | Expressive text-to-speech aims to generate high-quality samples with rich prosody . prosodic attributes in highly dynamic voices are difficult to capture and model without intonation . |
| Approach: | They propose a pipeline that enhances prosody modeling and sampling by introducing a self-supervised masked autoencoder and a diffusion model to sample diverse prosodic patterns within the latent space. |
| Outcome: | The proposed pipeline achieves new state-of-the-art in text-to-speech with natural and expressive synthesis. |
Duplex Diffusion Models Improve Speech-to-Speech Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to speech-to-speech translation train two separate models or a multitask-learned model with low efficiency and inferior performance. |
| Approach: | They propose a duplex diffusion model that applies diffusion probabilistic models to both sides of a reversible duplex Conformer and enables reverse speech translation by simply flipping the input and output ends. |
| Outcome: | The proposed model achieves the first success of reversible speech translation with significant improvements of ASR-BLEU scores compared with a list of state-of-the-art baselines. |
Global and Local Hierarchy-aware Contrastive Framework for Implicit Discourse Relation Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to integrate whole hierarchical information of senses into discourse relation representations for multi-level sense recognition ignore static hierarchic structure containing all senses and ignore hierarchically sense label sequence corresponding to each instance. |
| Approach: | They propose to use a GlObal and Local Hierarchy-aware Contrastive Framework to model two kinds of hierarchies with the aid of multi-task learning and contrastive learning to learn better representations of discourse relation relationships. |
| Outcome: | The proposed method outperforms current state-of-the-art models at all hierarchical levels on PDTB 2.0 and PDTP 3.0 datasets. |
PreQuant: A Task-agnostic Quantization Approach for Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
Zhuocheng Gong, Jiahao Liu, Qifan Wang, Yang Yang, Jingang Wang, Wei Wu, Yunsen Xian, Dongyan Zhao, Rui Yan
| Challenge: | Quantization is a viable solution for pre-trained language models, but most existing methods are task-specific and require customized training and quantization with a large number of trainable parameters. |
| Approach: | They propose a "quantize before fine-tuning" framework that allows for quantization with a large number of trainable parameters on each individual task. |
| Outcome: | The proposed framework is compatible with quantization-aware training and post-training quantization and corrects quantization errors. |
Synthetic Pre-Training Tasks for Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | toxicity and bias can be addressed by pre-training with synthetic resources . BLEU scores are used to compare methods with real-world data . |
| Approach: | They propose several ways to generate obfuscated data from large parallel corpus and concatenating phrase pairs from small word-aligned corpus with synthetic parallel data without real human language corpora. |
| Outcome: | The proposed methods can be used to generate obfuscated data or synthetic parallel data without real human language corpora even with high levels of oblication. |
IDOL: Indicator-oriented Logic Pre-training for Logical Reasoning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing systems for logical reasoning have surpassed the average performance of humans in many tasks like SQuAD but there is still a long way to go when it comes to logical reasoning. |
| Approach: | They propose an InDicator-Oriented Logic Pre-training task which logically strengthens pre-trained models with the help of 6 types of logical indicators and a logicalally rich dataset. |
| Outcome: | The proposed task achieves state-of-the-art on ReClor and LogiQA, the two most representative benchmarks in logical reasoning MRC. |
Adversarial Training for Low-Resource Disfluency Correction (2023.findings-acl)
Copied to clipboard
| Challenge: | Disfluencies can be introduced in conversational speech due to the conversational nature of speech and/or speech impairments such as stuttering. |
| Approach: | They propose an adversarial sequence-tagging model for Disfluency Correction . they evaluate it in Bengali, Hindi, and Marathi languages and use it to correct stuttering disfluencies . |
| Outcome: | The proposed technique improves in Bengali, Hindi, and Marathi languages . it also removes stuttering disfluencies in ASR transcripts introduced by speech impairments . |
Computer says “No”: The Case Against Empathetic Conversational AI (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work in conversational AI has focused on responding empathetically to users' emotions without a real basis. |
| Approach: | They argue that we must carefully consider whether and how to respond to users' emotions. |
| Outcome: | Recent work in conversational AI has focused on responding empathetically to users' emotions without a real basis. |
Stubborn Lexical Bias in Data and Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work has focused on spurious correlations between features and labels in training data . but, we find strong evidence of corresponding bias in the trained models . |
| Approach: | They propose a method to reduce spurious correlations in training data by reweighting it using a large pool of extracted features. |
| Outcome: | The proposed method reduces spurious correlations in training data, but still finds strong evidence of bias in trained models. |
Distilling Efficient Language-Specific Models for Cross-Lingual Transfer (2023.findings-acl)
Copied to clipboard
| Challenge: | Massively multilingual Transformers (MMTs) are widely used for cross-lingual transfer learning. |
| Approach: | They propose to extract compressed, language-specific models from MMTs which retain the capacity of the original MMT for cross-lingual transfer. |
| Outcome: | The proposed model outperforms models trained from scratch in zero-shot cross-lingual transfer across benchmarks. |
An Extensive Exploration of Back-Translation in 60 Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | Back-translation has been shown to improve model quality through the creation of synthetic training bitext. |
| Approach: | They use back-translation to train models from 60 languages into English . early studies showed promise of the technique and follow on studies have produced refinements . |
| Outcome: | a new study shows that back-translation improves translation quality in low-resource languages . the results are consistent with previous studies, though there are limitations . |
AoM: Detecting Aspect-oriented Information for Multimodal Aspect-Based Sentiment Analysis (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to extract aspects from text-image pairs and recognize their sentiments are noisy and coarsely establishing image-aspect alignment will interfere with aspect-relevant semantic and sentiment information. |
| Approach: | They propose an Aspect-oriented method to detect aspect-relevant semantic and sentiment information by selecting textual tokens and image blocks that are semantically related to the aspects. |
| Outcome: | The proposed method is superior to existing methods in the field of sentiment analysis. |
Forecasting Earnings Surprises from Conference Call Transcripts (2023.findings-acl)
Copied to clipboard
| Challenge: | Earnings conference calls contain over 5,000 words of text and large amounts of industry jargon . this length and domain-specific language present problems for generic pretrained language models. |
| Approach: | They propose a task of predicting earnings surprises from earnings call transcripts and propose linguistic models that use a long document dataset to test financial understanding. |
| Outcome: | The proposed model can predict earnings surprises from earnings conference calls with reasonable accuracy and shows that it is possible to interpret the data with different interpretability methods. |
MTCue: Learning Zero-Shot Control of Extra-Textual Attributes by Leveraging Unstructured Context in Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing research has focused on providing individual, well-defined types of context in translation, such as the surrounding text or discrete external variables like the speaker’s gender. |
| Approach: | They introduce a novel neural machine translation framework that interprets all context as text. |
| Outcome: | The proposed framework outperforms a baseline that matched the parameters and significantly outperformed it in English translation. |
Evaluation for Change (2023.findings-acl)
Copied to clipboard
| Challenge: | Evaluation is the central means for assessing, understanding, and communicating about NLP models. |
| Approach: | They argue that evaluation should be more than that: it is a force for driving change and has a sociological and political character beyond its technical dimensions. |
| Outcome: | The proposed analysis concludes that evaluation’s power is waning despite its potential for realizing more pluralistic ambitions in the field. |
Reconstruction Probing (2023.findings-acl)
Copied to clipboard
| Challenge: | a new analysis method for contextualized representations is proposed . contextualization boosts reconstructability of tokens close to the token being reconstructed . |
| Approach: | They propose a method for contextualized representations based on reconstruction probabilities in masked language models. |
| Outcome: | The proposed method compares reconstruction probabilities of tokens in masked language models . it finds that contextualization boosts reconstructability of token that are close to the token being reconstructed . |
Towards Distribution-shift Robust Text Classification of Emotional Content (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work has shown that supervised models are more robust to change in domain and distribution, but the decrease in performance due to the distribution shift is still a major issue for supervised systems. |
| Approach: | They propose to fine-tune supervised models on task-specific datasets to achieve out-of-distribution performance. |
| Outcome: | The proposed model outperforms all available models in distribution and out of distribution with only a few thousand training samples. |
Multi-lingual and Multi-cultural Figurative Language Understanding (2023.findings-acl)
Copied to clipboard
Anubha Kabra, Emmy Liu, Simran Khanuja, Alham Fikri Aji, Genta Winata, Samuel Cahyawijaya, Anuoluwapo Aremu, Perez Ogayo, Graham Neubig
| Challenge: | Figures permeate human communication, but are understudied in NLP. |
| Approach: | They create a figurative language inference dataset for seven languages associated with a variety of cultures, using cultural and regional concepts for figurativ expressions. |
| Outcome: | The results show that the most common figurative expressions are found in Hindi, Indonesian, Javanese, Kannada, Sundanese, Swahili and Yoruba. |
Open-WikiTable : Dataset for Open Domain Question Answering with Complex Reasoning over Table (2023.findings-acl)
Copied to clipboard
| Challenge: | Open-WikiTable is the first open domain question answering dataset that requires complex reasoning over tables. |
| Approach: | They propose to use open-domain question answering over tables to extract questions from tables. |
| Outcome: | The dataset is publicly available. it is built upon WikiSQL and WikiTableQuestions. |
What In-Context Learning “Learns” In-Context: Disentangling Task Recognition and Task Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) can perform in-context learning (ICL) with only a few demonstrations, but its mechanisms are not well-understood. |
| Approach: | They characterize two ways in which LLMs leverage demonstrations to solve tasks with a few demonstrations. |
| Outcome: | The proposed model achieves non-trivial performance with only TR, and TR does not improve with larger models or more demonstrations. |
Cross-Lingual Retrieval Augmented Prompt for Low-Resource Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | Multilingual pretrained language models (MPLMs) perform strongly in cross-lingual transfer. |
| Approach: | They propose to augment context with similar sentences retrieved from a high-resource language (HRL) they find a significant correlation between cross-lingual transfer performance and similarity between high- and low-resourced languages . |
| Outcome: | The proposed model outperforms finetuning by 3.7% on three downstream tasks with multilingual parallel test sets across 10 LRLs covering 6 language families in unlabeled and labeled settings. |
Unsupervised Summarization Re-ranking (2023.findings-acl)
Copied to clipboard
| Challenge: | Abstractive summarization models have been gaining popularity, but performance of unsupervised models still lags behind supervised models. |
| Approach: | They propose to re-rank summary candidates in an unsupervised manner to close the performance gap between unsupervised and supervised models. |
| Outcome: | The proposed model improves unsupervised models by up to 7.27% and ChatGPT by up 6.86% relative mean ROUGE across four widely-adopted summarization benchmarks. |
GRACE: Gradient-guided Controllable Retrieval for Augmenting Attribute-based Text Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for controlling the generation of pre-trained language models infuse domain bias into the generation process, making it difficult to generate out-of-domain texts. |
| Approach: | They propose a retrieval-augmented generation framework that uses retrieval to generate fluent sentences with high attribute relevance. |
| Outcome: | The proposed method can generate fluent sentences with high attribute relevance while keeping domain bias out of the model. |
So many design choices: Improving and interpreting neural agent communication in signaling games (2023.findings-acl)
Copied to clipboard
| Challenge: | Emergent language games are experimental protocols designed to model how communication may arise among a group of agents. |
| Approach: | They propose to adopt a signaling game in which a sender is exposed to an image and generates a sequence of symbols that is transmitted to a receiver. |
| Outcome: | The proposed language improves when the sender is exposed to an image and generates a sequence of symbols that is transmitted to a receiver. |
Constructing Word-Context-Coupled Space Aligned with Associative Knowledge Relations for Interpretable Language Modeling (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to train language models have limitations in interpretability . a Word-Context-Coupled Space (W2CSpace) is proposed to improve the performance of pre-trained models . |
| Approach: | They propose a Word-Context-Coupled Space to replace pre-trained models with interpretable statistical logic. |
| Outcome: | The proposed language model can achieve better performance and highly credible interpretability compared to state-of-the-art methods. |
Fixed Input Parameterization for Efficient Prompting (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that attaching prompts to the input is effective at conditioning Language Models (LMs) however, prompts are always included in the input text during inference, thus incurring substantial computational and memory overhead. |
| Approach: | They propose to inject a fixed prompt into the parameters of an LM to be an efficient alternative to attaching fixed prompts to the input. |
| Outcome: | The proposed method can be up to 280 times more efficient in terms of FLOPs than previous approaches. |
Data Augmentation for Low-Resource Keyphrase Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing works on keyphrase generation rely on large-scale annotated datasets, which are not easy to acquire. |
| Approach: | They propose to use full text to improve keyphrase generation in resource-constrained domains by using the full text of the articles to augment their methods. |
| Outcome: | The proposed methods improve both present and absent keyphrase generation on three datasets and show that they are cost-effective. |
BigVideo: A Large-scale Video Subtitle Translation Dataset for Multimodal Machine Translation (2023.findings-acl)
Copied to clipboard
Liyan Kang, Luyang Huang, Ningxin Peng, Peihao Zhu, Zewei Sun, Shanbo Cheng, Mingxuan Wang, Degen Huang, Jinsong Su
| Challenge: | Existing datasets focus on captions describing images or videos, which are not large and diverse enough. |
| Approach: | They propose a large-scale video subtitle translation dataset to facilitate multi-modality machine translation. |
| Outcome: | The proposed dataset is 10 times larger than the widely used *How2* and *VaTeX* datasets. |
Constructing Procedural Graphs with Multiple Dependency Relations: A New Dataset and Baseline (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to structure procedural knowledge focus on representing descriptive knowledge but ignore another commonsense knowledge-Procedural Knowledge. |
| Approach: | They propose to generate flow graphs from procedural documents by extracting sequential dependency between sentences and missing two important dependencies in procedural document. |
| Outcome: | The proposed method can generate flow graphs from unstructured documents with syntactic information and discourse structures. |
Multi-Dimensional Evaluation of Text Summarization with In-Context Learning (2023.findings-acl)
Copied to clipboard
Sameer Jain, Vaishakh Keshava, Swarnashree Mysore Sathyendra, Patrick Fernandes, Pengfei Liu, Graham Neubig, Chunting Zhou
| Challenge: | In-context learning-based evaluators are competitive with learned evaluation frameworks for text summarization tasks. |
| Approach: | They propose to use large language models as multi-dimensional evaluators using in-context learning to evaluate text summarization tasks. |
| Outcome: | The proposed frameworks are competitive with existing frameworks on relevance and factual consistency, the authors show . |
Learning to Rank Utterances for Query-Focused Meeting Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to generate a generic summary for meetings are limited due to the conflict between long meetings and limited input size. |
| Approach: | They propose a Ranker-Generator framework that learns to rank utterances by comparing them in pairs and learning from the global orders, then uses top utterrances as the generator’s input. |
| Outcome: | The proposed model outperforms existing models with fewer parameters due to the conflict between long meetings and limited input size. |
Neural Architecture Search for Parameter-Efficient Fine-tuning of Large Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Parameter-efficient tuning (PET) methods fit pre-trained language models to downstream tasks by computing a small compressed update for a subset of parameter updates or appending and fine-tuning a few new parameters to the pre-train network. |
| Approach: | They propose a parameter-efficient tuning method that prunes PET architectures using structured pruning. |
| Outcome: | The proposed method performs well in practice but could be improved via NAS . it can be used to fine-tune large pre-trained language models on multiple tasks . |
Aligning Offline Metrics and Human Judgments of Value for Code Generation Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Large language models have shown impressive capabilities on code generation tasks. |
| Approach: | They propose a metric that combines functional correctness and syntactic similarity to measure the productivity gains generated by large language models. |
| Outcome: | The proposed model achieves a 14% stronger correlation with value and better represents real-world gains when evaluating and comparing models. |
Do transformer models do phonology like a linguist? (2023.findings-acl)
Copied to clipboard
| Challenge: | Neural sequence-to-sequence models have been very successful at tasks in phonology and morphology that require a capacity for intricate linguistic generalisations. |
| Approach: | They propose to use a transformer model to model phonological generalisations on 29 different processes . they also use phonology as a primer and a rule with a withheld case to infer behaviour . |
| Outcome: | The proposed model can model all 29 phonological phenomena on a constructed language, regardless of perceived process difficulty. |
DiMS: Distilling Multiple Steps of Iterative Non-Autoregressive Transformers for Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Neural machine translation models follow an autoregressive decoding strategy, generating the target sentence one token at a time. |
| Approach: | They propose a distillation technique that reduces the number of decoding steps required to reach a certain translation quality. |
| Outcome: | The proposed model achieves 7.8 and 12.9 BLEU points improvements in translation accuracy on distilled and raw versions of WMT’14 De-En. |
Retrieval-augmented Video Encoding for Instructional Captioning (2023.findings-acl)
Copied to clipboard
| Challenge: | Instructional videos provide a detailed multimodal context of each procedure in instruction. key-object degeneracy is a problem for machine systems, causing incorrect captions. |
| Approach: | They propose a retrieval-based framework to augment the model representations in the presence of key-object degeneracy. |
| Outcome: | The proposed framework can be extended over baselines using modalities with key-object degeneracy. |
Bi-level Finetuning with Task-dependent Similarity Structure for Low-resource Training (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to fine tune a large language model in low-resource settings are limited in their expressiveness or rely on task-independent knowledge. |
| Approach: | They propose a framework where all parameters are finetuned with task-dependent information from the training data only. |
| Outcome: | The proposed framework outperforms baseline models on several classification datasets in low-resource scenarios. |
Kanbun-LM: Reading and Translating Classical Chinese in Japanese Methods by Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Classical Chinese was introduced to Japan approximately 2,000 years ago . it was gradually adapted to a Japanese form called Kanbun-Kundoku (Kanbun) in Japanese reading and translating methods . |
| Approach: | They construct a dataset that compares Classical Chinese and Kanbun in Japan using character reordering and machine translation tasks. |
| Outcome: | The proposed dataset compares the current language models with human scores and compared them with human-level models. |
Adaptive Attention for Sparse-based Long-sequence Transformer (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies show that Transformers can process longer sequences because of their complexity and time scales quadratic to the sequence length. |
| Approach: | They propose an efficient Transformer model with adaptive attention that can select useful tokens automatically in sparse attention by learnable position vectors. |
| Outcome: | The proposed model can select useful tokens automatically in sparse attention by learnable position vectors. |
Sentiment Analysis using the Relationship between Users and Products (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies focus on modelling user and product aspects without considering the relationship between users and products. |
| Approach: | They propose a model that incorporates the relationship between users and products into the model. |
| Outcome: | The proposed model improves on three well-known benchmarks for sentiment classification with the user and product information. |
Entropy-guided Vocabulary Augmentation of Multilingual Language Models for Low-resource Tasks (2023.findings-acl)
Copied to clipboard
| Challenge: | Multilingual language models (MLLMs) support low-resource languages (LRLs) but LRL words are under-represented in wordpiece/subword vocabularies, leading to low task accuracy . |
| Approach: | They propose an entropy-based vocabulary augmented language model to detect LRL words with undesirable wordpiece segmentations. |
| Outcome: | The proposed model improves performance and limits on wordpiece augmentation strategies for multiple diverse LRLs. |
Class-Adaptive Self-Training for Relation Extraction with Incompletely Annotated Training Data (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing relation extraction models rely on supervised machine learning, but many datasets are incompletely annotated, causing false negatives and errors during inference stage. |
| Approach: | They propose a class-adaptive re-sampling self-training framework that favored the pseudo-labels of classes with high precision and low recall scores. |
| Outcome: | The proposed framework outperforms existing methods on the Re-DocRED and ChemDisgene datasets when the training data are incompletely annotated. |
Solving Cosine Similarity Underestimation between High Frequency Words by ℓ2 Norm Discounting (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to estimate cosine similarity between words have been proposed, but no solution has been proposed. |
| Approach: | They propose a method to discount the l2 norm of contextualised word embedding by the frequency of that word in a corpus when measuring the cosine similarities between words. |
| Outcome: | The proposed method underestimates the similarity scores between two words when used with contextualised token embeddings from masked language models such as BERT. |
Do Large Language Models Know What They Don’t Know? (2023.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have vast knowledge that allows them to excel in various NLP tasks. |
| Approach: | They propose an automated method to detect uncertainty in the responses of large language models and a dataset to measure their self-knowledge. |
| Outcome: | The proposed method detects uncertainty in the responses of large language models and provides a novel measure of their self-knowledge. |
AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilities (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to build a strong multilingual multimodal representation model are lacking in good-quality text-image pairs. |
| Approach: | They propose a method to build a strong multilingual multimodal representation model using English text-image pairs instead of a model from scratch. |
| Outcome: | The proposed model outperforms the original CLIP model on multilingual multimodal benchmarks. |
RHGN: Relation-gated Heterogeneous Graph Network for Entity Alignment in Knowledge Graphs (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for entity alignment fail to account for heterogeneity among KGs and distinction between KG entities and relations. |
| Approach: | They propose a Relation-gated Heterogeneous Graph Network (RHGN) that uses a relation-gate based convolutional layer to distinguish relations and entities in the KG. |
| Outcome: | Extensive experiments on four datasets show that the proposed method is superior to state-of-the-art methods. |
Feature Interactions Reveal Linguistic Structure in Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing features attribution methods for post-hoc interpretability ignore the existence of interactions between the effects of features on the prediction. |
| Approach: | They propose a grey box method to train models to perfection on a formal language classification task using PCFGs. |
| Outcome: | The proposed methods are able to uncover the grammatical rules acquired by the model under specific configurations and provide novel insights into the linguistic structure of the target models. |
Clustering-Aware Negative Sampling for Unsupervised Sentence Representation (2023.findings-acl)
Copied to clipboard
| Challenge: | Using clustering-aware learning, in-batch negatives are often ignored in sentence representation learning. |
| Approach: | They propose a method that integrates cluster information into contrastive learning for unsupervised sentence representation learning. |
| Outcome: | The proposed method compares favorably with baselines on semantic textual similarity tasks. |
An Effective Deployment of Contrastive Learning in Multi-label Text Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on contrastive learning in natural language processing tasks have not explored the effectiveness of the technology. |
| Approach: | They propose five novel contrastive losses for multi-label text classification tasks that exploit the complexity of the input logic and the semantic representation space. |
| Outcome: | The proposed contrastive losses improve multi-label text classification tasks and can be adapted for multi-task tasks. |
Segment-Level and Category-Oriented Network for Knowledge-Based Referring Expression Comprehension (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods employ sentence-level retrieval and fusion methods, which may lead to similarity bias and interference from irrelevant information in unstructured knowledge sentences. |
| Approach: | They propose a segment-level and category-oriented network to solve similarity bias problem by segmenting and prompting knowledge retrieval methods and a category-based grounding method. |
| Outcome: | The proposed model eliminates similarity bias and improves the overall performance of the KB-REC task. |
MVP: Multi-task Supervised Pre-training for Natural Language Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) have achieved remarkable success in natural language generation tasks. |
| Approach: | They propose to use a large-scale natural language generation corpus to pre-train a text generation model MVP in a supervised manner. |
| Outcome: | The proposed model outperforms BART and Flan-T5 on 13 out of 17 datasets and outperformed BART by 9.3% and FlaN-T5. |
From Alignment to Entailment: A Unified Textual Entailment Framework for Entity Alignment (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods encode the triples of entities as embeddings and learn to align the embeddables, which prevents the direct interaction between the original information of the cross-KG entities. |
| Approach: | They propose to transform the triples into unified textual sequences and model the EA task as a bi-directional textual entailment task between the sequences of cross-KG entities. |
| Outcome: | The proposed approach outperforms the state-of-the-art methods on five cross-lingual datasets and allows the mutual enhancement of the heterogeneous information. |
It is a Bird Therefore it is a Robin: On BERT’s Internal Consistency Between Hypernym Knowledge and Logical Words (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models for lexical knowledge of NLP systems only use collocational cues for questions . lexicon of these systems is limited to content words and logical words. |
| Approach: | They propose a method to test the understanding of the hypernymy relationship by measuring its antisymmetry according to the models. |
| Outcome: | The proposed model shows that it is weak on the semantic tasks of the Masked Language Modeling task. |
Defending against Insertion-based Textual Backdoor Attacks via Attribution (2023.findings-acl)
Copied to clipboard
| Challenge: | Textual backdoor attacks are vulnerable to backdoors and can be used to infect models trained on poisoned data. |
| Approach: | They propose an efficient attribution-based pipeline to defend against two insertion-based poisoning attacks, BadNL and InSent. |
| Outcome: | The proposed method can generalize sufficiently well in two common attack scenarios, which consistently improves previous methods. |
ActiveAED: A Human in the Loop Improves Annotation Error Detection (2023.findings-acl)
Copied to clipboard
| Challenge: | Annotation error detection (AED) methods flag erroneous annotations for human reannotation . but many of these methods assume a final curation step in which a human annotator decides whether the annotation is erranosome . |
| Approach: | They propose an annotation error detection method that can detect errors more accurately by repeatedly querying a human for error corrections in its prediction loop. |
| Outcome: | The proposed method improves on eight datasets spanning five different tasks and can improve accuracy by up to six percentage points. |
Assessing Word Importance Using Models Trained for Semantic Tasks (2023.findings-acl)
Copied to clipboard
| Challenge: | Many NLP tasks require to automatically identify the most significant words in a text. |
| Approach: | They propose to use attribution methods to explain the predictions of two NLP tasks to derive word significance from models trained to solve semantic tasks. |
| Outcome: | The proposed method is robust to the initial task and is able to identify important words in sentences without explicit word importance labeling in training. |
In-context Examples Selection for Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Large-scale generative models can perform a wide range of NLP tasks using in-context learning. |
| Approach: | They aim to understand the properties of good in-context examples for machine translation in both in-domain and out-of-domain settings. |
| Outcome: | The proposed model outperforms a strong kNN-MT baseline in 2 out of 4 out-of-domain datasets. |
PropSegmEnt: A Large-Scale Corpus for Proposition-Level Segmentation and Entailment Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing systems for Natural Language Inference (NLI) only recognize textual entailment relations on sentence-level . however, even a simple sentence often contains multiple propositions, i.e. distinct units of meaning conveyed by the sentence . |
| Approach: | They propose a system to recognize whether one text is textually entailed by another . they use a corpus of over 45K propositions annotated by human raters to study the textual entailment relation of each proposition in a sentence individually. |
| Outcome: | The proposed dataset can be used to understand the compositionality of NLI labels. |
CIF-PT: Bridging Speech and Text Representations for Spoken Language Understanding via Continuous Integrate-and-Fire Pre-Training (2023.findings-acl)
Copied to clipboard
| Challenge: | Speech-to-text training and language model distillation are used to bridge the representations between speech and text. |
| Approach: | They propose a pre-training paradigm that integrates speech and text into a single frame-to-token alignment. |
| Outcome: | The proposed paradigm outperforms the state-of-the-art model on intent classification and slot filling tasks. |
Improving Diachronic Word Sense Induction with a Nonparametric Bayesian method (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models for dichronic word Sense Induction (DWSI) are not fully explored or compared against in the current state of the art. |
| Approach: | They propose two new models for Diachronic Word Sense Induction based on topic modelling techniques. |
| Outcome: | The proposed models outperform the state-of-the-art models on a time-stamped dataset from the biomedical domain. |
What to Fuse and How to Fuse: Exploring Emotion and Personality Fusion Strategies for Explainable Mental Disorder Detection (2023.findings-acl)
Copied to clipboard
| Challenge: | Mental health disorders (MHD) are one of the greatest challenges facing our healthcare systems and modern societies in general. |
| Approach: | They integrate and extend the research by conducting extensive experiments with three types of deep learning-based fusion strategies: feature-level fusion, model fusion and task fusion. |
| Outcome: | The proposed techniques show that they can be used to improve mental health detection from textual data. |
Adaptive Contrastive Knowledge Distillation for BERT Compression (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing knowledge distillation methods for BERT implicitly learn discriminative student features by mimicking the teacher features. |
| Approach: | They propose a new knowledge distillation approach called adaptive contrastive knowledge distilling for BERT compression using hidden state features in BERT as explicit supervision to learn discriminative student features. |
| Outcome: | The proposed approach improves on multiple natural language processing tasks. |
Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing transformer models are computationally demanding and prohibitively costly for long sequences due to the quadratic complexity of its selfattention module. |
| Approach: | They propose a transformer-based model that inherits weights from large pretrained models by removing redundancies in hidden sequences using the ready-made Fast Fourier Transform operator. |
| Outcome: | The proposed model outperforms the standard BART model on the long-range modeling benchmark LRA with significant improvements in speed and space. |
Zero-Shot Classification by Logical Reasoning on Natural Language Explanations (2023.findings-acl)
Copied to clipboard
| Challenge: | Experimental results show that CLORE is superior to baselines on zero-shot classification tasks. |
| Approach: | They propose a framework for classification by logically parsing and reasoning on natural language explanations. |
| Outcome: | The proposed framework outperforms baselines on zero-shot classification tasks. |
Dual-Gated Fusion with Prefix-Tuning for Multi-Modal Relation Extraction (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for multi-modal relation extraction lack useful visual information. |
| Approach: | They propose a novel multi-modal relation extraction framework to capture deeper correlations of text, entity pair, and image/objects. |
| Outcome: | The proposed framework captures the deeper correlations of text, entity pair, and image/objects, and extracts useful information. |
Pruning Pre-trained Language Models with Principled Importance and Self-regularization (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models often contain a vast amount of parameters, posing nontrivial requirements for storage and computation. |
| Approach: | They propose a pruning method where model prediction is regularized by the latest checkpoint with increasing sparsity throughout pruning. |
| Outcome: | The proposed approach is effective at sparsity levels, and can be applied to natural language understanding, question answering, and data-to-text generation tasks. |
The Magic of IF: Investigating Causal Reasoning Abilities in Large Language Models of Code (2023.findings-acl)
Copied to clipboard
| Challenge: | entailment a) |
| Approach: | entailment : We want to explore whether Code-LLMs with code prompts are better . encoding a code prompt is better than text-only LLMs, they say . |
| Outcome: | entailment : Our results show that Code-LLMs with code prompts are better compared to text-only LLMs. |
Learning to Leverage High-Order Medical Knowledge Graph for Joint Entity and Relation Extraction (2023.findings-acl)
Copied to clipboard
| Challenge: | Medical terms are difficult to understand and relations between medical entities become complicated. |
| Approach: | They propose to leverage medical domain knowledge for extracting entities and relations for Chinese medical texts by building a heterogeneous graph based on medical knowledge graph. |
| Outcome: | The proposed method is more effective than state-of-the-art methods on real Chinese medical texts. |
Data-Efficient Finetuning Using Cross-Task Nearest Neighbors (2023.findings-acl)
Copied to clipboard
| Challenge: | Prior work shows training models on multitask data augmented with task descriptions transfers knowledge to new tasks. |
| Approach: | They propose to use unlabeled target-task data to train models on task descriptions . they use only 2% of the data from the P3 pool without labeled target task data . |
| Outcome: | The proposed model outperforms baseline models on 12 out of 14 datasets . it also provides better initialization than single model on target-task data . |
CoAug: Combining Augmentation of Labels and Labelling Rules (2023.findings-acl)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) tasks require large labeled datasets to perform well. |
| Approach: | They propose a co-augmentation framework that bootstraps predictions from each model to improve few-shot models and rule-augmentation models by bootstrapping them. |
| Outcome: | The proposed model outperforms strong weak-supervision-based models by 6.5 F1 points . the proposed model can learn from limited labeled data and perform better on small datasets . |
Entity-to-Text based Data Augmentation for various Named Entity Recognition Tasks (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing augmentation techniques manipulate words in the original text that break the semantic coherence of the text, or exploit generative models that ignore preserving entities in the text. |
| Approach: | They propose a novel Entity-to-Text based data augmentation technique called EnTDA to add, delete, replace or swap entities in the original text. |
| Outcome: | The proposed technique generates semantically coherent and entity preserving texts on thirteen NER tasks and two settings. |
World Models for Math Story Problems (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent efforts to solve math story problems have lacked accurate representations of mathematical concepts. |
| Approach: | They propose a graph-based semantic formalism for solving math story problems . they combine existing datasets and annotate a corpus of 1,019 problems with MathWorld . |
| Outcome: | The proposed model can be used to solve math story problems with pre-trained language models . the model can also be used for generating new problems by using the model as a design space . |
AutoMoE: Heterogeneous Mixture-of-Experts with Adaptive Computation for Efficient Neural Machine Translation (2023.findings-acl)
Copied to clipboard
Ganesh Jawahar, Subhabrata Mukherjee, Xiaodong Liu, Young Jin Kim, Muhammad Abdul-Mageed, Laks Lakshmanan, V.S., Ahmed Hassan Awadallah, Sebastien Bubeck, Jianfeng Gao
| Challenge: | Existing MoE designs do not consider computational constraints (e.g., FLOPs, latency) Existing works in MoE consider homogeneous design where the same number of experts of the same size are placed uniformly throughout the network. |
| Approach: | They propose a framework for designing heterogeneous MoEs under computational constraints. |
| Outcome: | The proposed framework achieves 4x inference speedup and FLOPs reduction over manual models and within 1 BLEU point of MoE SwitchTransformer over benchmark datasets for NMT. |
Language Agnostic Multilingual Information Retrieval with Contrastive Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Annotated training data is costly to obtain in many languages . |
| Approach: | They propose a semantic contrastive loss to align parallel sentences that share the same semantics in different languages and a language contrastive gain to leverage parallel sentence pairs to remove language-specific information from non-parallel corpora. |
| Outcome: | The proposed model improves retrieval performance while requiring less computational effort. |
Easy to Decide, Hard to Agree: Reducing Disagreements Between Saliency Methods (2023.findings-acl)
Copied to clipboard
| Challenge: | A popular approach to unveiling the black box of neural NLP models is to leverage saliency methods, which assign scalar importance scores to each input component. |
| Approach: | They propose to use saliency methods to evaluate whether an explanation is faithful and argue that Pearson-r is a better-suited alternative to rank correlation. |
| Outcome: | The proposed methods exhibit weak rank correlations even when applied to the same model instance and advocated for alternative diagnostic methods. |
Enhancing Cross-lingual Transfer via Phonemic Transcription Integration (2023.findings-acl)
Copied to clipboard
| Challenge: | Previous cross-lingual transfer methods are limited to orthographic representation learning via textual scripts. |
| Approach: | They propose a phonemic transcription framework that incorporates phonemic translations as an additional linguistic modality beyond the orthographic transcriptions for cross-lingual transfer. |
| Outcome: | The proposed framework captures local one-to-one alignment between two different modalities and integrates bilingual dictionaries into multilingual contexts. |
Human-in-the-loop Abstractive Dialogue Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Abstractive dialogue summarization systems are trained to maximize the likelihood of human-written summaries, but there is still a huge gap in generating high-quality summary as determined by humans. |
| Approach: | They propose to incorporate different levels of human feedback into the training process . they ask humans to highlight salient information to be included in summaries . |
| Outcome: | The proposed model captures human-written summaries and compares them with state-of-the-art models on multiple datasets. |
A Multi-task Learning Framework for Quality Estimation (2023.findings-acl)
Copied to clipboard
Sourabh Deoghare, Paramveer Choudhary, Diptesh Kanojia, Tharindu Ranasinghe, Pushpak Bhattacharyya, Constantin Orăsan
| Challenge: | Conventional approaches to QE involve training separate models at different levels of granularity viz., word-level, sentence-level and document-level . |
| Approach: | They propose to train a single model for sentence-level and word-level QE tasks in a multi-task learning framework and compare them to baseline models. |
| Outcome: | The proposed model improves on the single-pair, multi-patch, and zero-shot settings. |
The Devil is in the Details: On the Pitfalls of Event Extraction Evaluation (2023.findings-acl)
Copied to clipboard
| Challenge: | Event extraction (EE) is a fundamental information extraction task aimed at extracting events from plain texts. |
| Approach: | They propose to specify data preprocessing, standardize outputs, and provide pipeline evaluation results to avoid these pitfalls. |
| Outcome: | The results show that the evaluations are reliable and lack pipeline evaluations. |
Yes, this Way! Learning to Ground Referring Expressions into Actions with Intra-episodic Feedback from Supportive Teachers (2023.findings-acl)
Copied to clipboard
| Challenge: | In collaborative situations, communication is performed as signalling and recognizing, and the ability to act on these signals is crucial for future machine learning models to collaborate and interact with humans naturally. |
| Approach: | They propose to use a referential language game as an example of a collaborative joint activity to evaluate intra-episodic feedback given by a teacher. |
| Outcome: | The proposed model can generalize on aspects of scene complexity and perform better than providing only the initial statement. |
Investigating Transformer-Guided Chaining for Interpretable Natural Logic Reasoning (2023.findings-acl)
Copied to clipboard
| Challenge: | Natural logic reasoning has received increasing attention lately, with several datasets and neural models proposed, though with limited success. |
| Approach: | They propose to iteratively perform 1-step neural inferences and chain together the results to generate a multi-step reasoning trace. |
| Outcome: | The proposed method has high accuracies on a multi-hop First-Order Logic (FOL) reasoning benchmark. |
Multilingual Multi-Figurative Language Detection (2023.findings-acl)
Copied to clipboard
| Challenge: | Figures of speech help people express abstract concepts and emotions, but it's understudied in a multilingual setting and when considering more than one figure of speech at the same time. |
| Approach: | They propose a framework for sentence-level figurative language detection based on template-based prompt learning and use it to unify multiple detection tasks that are interrelated across multiple figures of speech and languages. |
| Outcome: | The proposed framework outperforms baselines and may serve as blueprint for the joint modelling of other interrelated tasks. |
Zero-shot Visual Question Answering with Language Model Feedback (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for knowledge-based visual question answering are based on pre-trained language models. |
| Approach: | They propose a language model guided captioning approach that leverages a pre-trained language model to generate captions for an image to help answer a visual question. |
| Outcome: | The proposed method outperforms several competing methods on the knowledge-based VQA task and achieves comparable results to a fine-tuned VLP model. |
Prompted Opinion Summarization with GPT-3.5 (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent years have seen several shifts in summarization research, including extractive models. |
| Approach: | They propose a pipeline method for applying GPT-3.5 to summarize user reviews . they propose three new metrics targeting faithfulness, factuality, and genericity . |
| Outcome: | The proposed methods perform well in opinion summarization, the authors show . they also show that standard evaluation metrics do not reflect this performance . |
Sentence Ordering with a Coherence Verifier (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent sentence ordering studies can be classified into 2 categories: pair-wise ranking-based and sequence generation-based methods. |
| Approach: | They propose a sentence ordering method by plugging a coherence verifier into ranking-based and sequence generation-based methods. |
| Outcome: | The proposed method improves on topological sorting-based and pointer network-based methods with topological and point-based models. |
GUMSum: Multi-Genre Data and Evaluation for English Abstractive Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets are limited to newswire text, which is a fraction of extant genres in general and on the Web. |
| Approach: | They present a small but carefully crafted dataset of English summaries in 12 written and spoken genres for evaluation of abstractive summarization. |
| Outcome: | The proposed dataset of English summaries in 12 written and spoken genres is compared with human outputs and compared to untuned and prompt-based approaches. |
Improving Grammatical Error Correction with Multimodal Feature Integration (2023.findings-acl)
Copied to clipboard
| Challenge: | Experimental results show that multimodal GEC models improve over strong baselines and achieve a new state-of-the-art result on the Falko-MERLIN test set. |
| Approach: | They propose a framework that integrates both speech and text features to enhance GEC by generating audio from text using advanced text-to-speech models. |
| Outcome: | The proposed framework improves on CoNLL14, BEA19 English, and Falko-MERLIN German datasets. |
Teaching the Pre-trained Model to Generate Simple Texts for Text Simplification (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing strategies to teach pre-trained models to generate simple texts are inadequate. |
| Approach: | They propose a continued pre-training strategy to teach pre-trained models to generate simple texts by randomly masking text spans in ordinary texts. |
| Outcome: | The proposed strategy improves on lexical simplification, sentence simplification and document-level simplification tasks over existing models. |
Acquiring Frame Element Knowledge with Deep Metric Learning for Semantic Frame Induction (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for semantic frame induction are labor intensive . a method that uses contextualized embeddings can be used to acquire frame element knowledge. |
| Approach: | They propose a method that applies deep metric learning to semantic frame induction tasks . they use a pre-trained language model to fine-tune frame-annotated models to perform argument clustering . |
| Outcome: | The proposed method achieves substantially better performance than existing methods on FrameNet. |
Leveraging Synthetic Targets for Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Using synthetic target data, training models on synthetic targets outperforms training on actual ground-truth data. |
| Approach: | They propose a recipe for training machine translation models on synthetic target data by leveraging a large pre-trained model. |
| Outcome: | The proposed model outperforms training on real-world translation datasets. |
Recipes for Sequential Pre-training of Multilingual Encoder and Seq2Seq Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained encoder-only and sequence-to-sequence models are computationally expensive. |
| Approach: | They propose a recipe to initialize one model from the other to improve pre-training efficiency. |
| Outcome: | The proposed method matches the performance of a from-scratch model with a multilingual encoder while reducing the total compute cost by 27%. |
Constructing Code-mixed Universal Dependency Forest for Unbiased Cross-lingual Relation Extraction (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent efforts on cross-lingual relation extraction (XRE) leverage language-consistent structural features from the universal dependency resource. |
| Approach: | They propose to construct a type of code-mixed UD forest that combines UD and source-/target-side UD structures to achieve unbiased transfer. |
| Outcome: | The proposed UD forest achieves significant performance gains on ACE XRE benchmark datasets. |
Spontaneous gestures encoded by hand positions improve language models: An Information-Theoretic motivated study (2023.findings-acl)
Copied to clipboard
| Challenge: | a key missing step is to explore whether the nonverbal information can be quantified. |
| Approach: | They explore whether incorporating gesture representations can improve the language model’s performance . they also examine whether spontaneous gestures demonstrate entropy rate constancy (ERC) . |
| Outcome: | The proposed model improves the performance of the mixed-modal language models against monologue video data. |
Progressive Translation: Improving Domain Robustness of Neural Machine Translation with Intermediate Sequences (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that intermediate supervision signals benefit various tasks such as math word problems and commonsense reasoning. |
| Approach: | They propose to introduce an inductive bias that reflects a domain-agnostic principle of translation and a full-permutation multi-task learning to alleviate spurious correlations. |
| Outcome: | The proposed signals reduce spurious correlations and spurious hallucinations on out-of-domain translation, and are especially promising in low-resource scenarios. |
Controlled Text Generation with Hidden Representation Transformations (2023.findings-acl)
Copied to clipboard
| Challenge: | Using a con-trolled language model, we gain attribute control by modifying the hidden representation of thebase model through learning transformations. |
| Approach: | They propose a con-trolled language generation framework that gains attribute control bymodifying the hidden representation of thebase model through learned transformations. |
| Outcome: | The proposed framework outperforms all thebaselines in detoxification, positivesentiment steering, and text simplification while minimizing the loss in linguistic qualities. |
Visual Coherence Loss for Coherent and Visually Grounded Story Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing visual storytelling models fail to generate correct referring expressions for characters, causing 60% of the generated stories to be lacking local coherence. |
| Approach: | They propose a loss function inspired by a linguistic theory of coherence for self-supervised learning for image sequence representations and a feature matching metric to check whether the models generate referring expressions correctly for characters in input image sequences. |
| Outcome: | The proposed features and loss function are effective for generating more coherent and visually grounded stories. |
AnaMeta: A Table Understanding Dataset of Field Metadata Knowledge Shared by Multi-dimensional Data Analysis Tasks (2023.findings-acl)
Copied to clipboard
Xinyi He, Mengyu Zhou, Mingjie Zhou, Jialiang Xu, Xiao Lv, Tianle Li, Yijia Shao, Shi Han, Zejian Yuan, Dongmei Zhang
| Challenge: | Tabular data analysis is performed everyday across various domains. |
| Approach: | They propose to use a dataset of 467k tables with supervision labels for four types of field metadata. |
| Outcome: | The proposed framework improves the understanding capability of tabular models by incorporating distribution and knowledge information. |
Large Language Models Are Partially Primed in Pronoun Interpretation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies suggest large language models acquire rich linguistic representations, but little is known about whether they adapt to linguistic biases in a human-like way. |
| Approach: | They examine whether large language models display human-like referential biases using stimuli and procedures from real psycholinguistic experiments. |
| Outcome: | The proposed models display human-like referential biases when exposed to referential patterns in the local context. |
Counterfactuals of Counterfactuals: a back-translation-inspired approach to analyse counterfactual editors (2023.findings-acl)
Copied to clipboard
George Filandrianos, Edmund Dervakos, Orfeas Menis Mastromichalakis, Chrysoula Zerva, Giorgos Stamou
| Challenge: | Existing explanations for classifiers are counterfactual or contrastive . lack of universal ground truth for counterf actual edits hinders their evaluation . |
| Approach: | They propose a back translation-inspired evaluation methodology that utilises earlier outputs of the explainer as ground truth proxies to investigate the consistency of explainers. |
| Outcome: | The proposed method can provide valuable insights into the behaviour of predictor and explainer models and infer patterns that would otherwise be obscured. |
A Pilot Study on Dialogue-Level Dependency Parsing for Chinese (2023.findings-acl)
Copied to clipboard
| Challenge: | Dialogue-level dependency parsing has received insufficient attention, especially for Chinese. |
| Approach: | They propose a signal-based method to transform seen syntactic dependencies into unseen ones between elementary discourse units (EDUs) they apply single-view and multi-view data selection to access reliable pseudo-labeled instances. |
| Outcome: | The proposed method transforms seen syntactic dependencies into unseen ones between elementary discourse units (EDUs) the proposed method also provides reliable pseudo-labeled instances. |
On the Off-Target Problem of Zero-Shot Multilingual Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Despite its success, multilingual neural machine translation suffers from the off-target issue, where the translation is in the wrong language. |
| Approach: | They propose a language-aware vocabulary sharing algorithm that can be used to increase the lexical distance between languages by isolating the vocab of different languages in the decoder. |
| Outcome: | The proposed algorithm reduces off-target rate for 90 translation tasks from 29% to 8%, while improving overall BLEU score by an average of 1.9 points without extra training cost or sacrificing the supervised directions’ performance. |
ORCA: A Challenging Benchmark for Arabic Language Understanding (2023.findings-acl)
Copied to clipboard
| Challenge: | Despite efforts to evaluate Arabic NLU, no public benchmark of diverse nature exists . a benchmark targeting Arabic needs to take into account that Arabic is not a single language but a collection of languages and language varieties. |
| Approach: | They propose a publicly available benchmark for Arabic language understanding evaluation dubbed ORCA . it covers diverse Arabic varieties and a wide range of Arabic understanding tasks . |
| Outcome: | The proposed benchmark covers Arabic and multilingual models across seven NLU task clusters. |
Delving into the Openness of CLIP (2023.findings-acl)
Copied to clipboard
| Challenge: | Contrastive Language-Image Pre-training (CLIP) allows for open-vocabulary visual recognition, where the model can recognize images from an open class set in a zero-shot manner. |
| Approach: | They propose to use image classification as an image-to-text matching task instead of discrete category IDs to achieve open-vocabulary visual recognition. |
| Outcome: | The proposed model can recognize images from an open vocabulary in a zero-shot manner, but its performance deteriorates as the vocabulary expands. |
From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Framework (2023.findings-acl)
Copied to clipboard
Yangyi Chen, Hongcheng Gao, Ganqu Cui, Lifan Yuan, Dehan Kong, Hanlu Wu, Ning Shi, Bo Yuan, Longtao Huang, Hui Xue, Zhiyuan Liu, Maosong Sun, Heng Ji
| Challenge: | Existing models of robustness evaluation are incomprehensive, impractical, and invalid . |
| Approach: | They propose a framework for automatic robustness evaluation that shifts towards model-centric evaluation to further exploit the advantages of adversarial attacks. |
| Outcome: | The proposed framework is based on a model-centric evaluation protocol and a robustness evaluation protocol. |
An Empirical Study of Sentiment-Enhanced Pre-Training for Aspect-Based Sentiment Analysis (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to Aspect-Based Sentiment Analysis (ABSA) are lacking in a comprehensive evaluation and fair comparison. |
| Approach: | They propose to use a knowledge-mining method to build a large-scale knowledge-annotated SPT corpus and integrate sentiment knowledge into pre-training. |
| Outcome: | The proposed method is able to build a large-scale knowledge-annotated SPT corpus and compares with other methods. |
NatCS: Eliciting Natural Customer Support Dialogues (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing task-oriented dialogue datasets do not reflect the expected characteristics of real customer support conversations. |
| Approach: | They propose to collect real customer service conversations from real conversations . they show that dialogue act annotations provide more effective training data . |
| Outcome: | The proposed approach is more representative of real human-to-human conversations compared to existing dialogue datasets . the proposed approach can be used to facilitate open research in natural dialog systems . |
Are Intermediate Layers and Labels Really Necessary? A General Language Model Distillation Method (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing knowledge distillation methods rely on intermediate layer features and golden labels, which require aligned model architecture and labeled data respectively. |
| Approach: | They propose a general language model distillation method that performs two-stage word prediction distillation and vocabulary compression, which is simple and shows extremely strong performance. |
| Outcome: | The proposed method outperforms 25 state-of-the-art methods on the SuperGLUE benchmark, achieving an average score that surpasses the best method by 3%. |
Diable: Efficient Dialogue State Tracking as Operations on Tables (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing systems for dialogue state tracking use the full dialogue history as input and generate the entire state from scratch at each dialogue turn. |
| Approach: | They propose a task formalisation that represents the dialogue state as a table and formalises it as 'table manipulation task' they represent the dialogue as if it were a list with all the slots and generate the entire state from scratch at each dialogue turn. |
| Outcome: | The proposed system outperforms existing systems while maintaining competitive accuracy. |
Neural Topic Modeling based on Cycle Adversarial Training and Contrastive Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Neural topic models have been widely used to extract common topics across documents. |
| Approach: | They propose a framework to apply contrastive learning directly to the decoder . they propose 'self-supervised' contrastive loss to make the generator capture similar topic information . |
| Outcome: | The proposed framework outperforms baselines on four benchmark datasets. |
Alleviating Exposure Bias via Multi-level Contrastive Learning and Deviation Simulation in Abstractive Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Abstractive summarization systems have a severe mismatch between training and inference, i.e., exposure bias. |
| Approach: | They propose a multi-level contrastive learning framework for abstractive summarization and a tailored sparse decoder self-attention pattern to bridge the gap between training and inference. |
| Outcome: | The proposed framework outperforms the state-of-the-art models on two summarization datasets while adding relatively low overhead. |
Mapping Brains with Language Models: A Survey (2023.findings-acl)
Copied to clipboard
| Challenge: | accumulated evidence for brain and language model activations remains ambiguous, but correlations with model size and quality provide grounds for cautious optimism. |
| Approach: | They examine the evidence accumulated by 30 studies spanning 10 datasets and 8 metrics to determine whether there is any overlap between brain and language model activations. |
| Outcome: | The findings suggest that representations extracted from NLP models can (partially) explain the signal found in neural data. |
Parameter-Efficient Finetuning for Robust Continual Multilingual Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to Continual Multilingual Learning (CML) are based on updating models using new data in stages. |
| Approach: | They propose a parameter-efficient finetuning strategy to increase the number of languages on which the model improves after an update while reducing the magnitude of loss for the remaining languages. |
| Outcome: | The proposed model improves on the languages included in the latest update while reducing the loss of performance on the remaining languages. |
Interpretable Multimodal Misinformation Detection with Logic Reasoning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for misinformation detection lack interpretability due to the black-box nature of the neural network. |
| Approach: | They propose a logic-based neural model which integrates interpretable logic clauses to express the reasoning process of the target task. |
| Outcome: | The proposed model can be generalizable across multiple misinformation sources and is based on three public datasets. |
Semantic-conditioned Dual Adaptation for Cross-domain Query-based Visual Segmentation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to visual segmentation from language queries require expensive labeling and degradation when deployed to an unseen domain. |
| Approach: | They propose a task to adapt a visual segmentation model from a labeled domain to an unseen domain. |
| Outcome: | The proposed framework achieves precise feature- and relation-invariant across domains via universal semantic structure. |
Figurative Language Processing: A Linguistically Informed Feature Analysis of the Behavior of Language Models and Humans (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent years have witnessed a growing interest in investigating what Transformer-based language models (TLMs) actually learn from training data. |
| Approach: | They propose to use a black-box TLM and two intrinsically transparent white-box models to investigate the performance of figurative language models on sarcasm, similes, idioms, and metaphors. |
| Outcome: | The proposed models perform better than other models on figurative language classification tasks. |
Taxonomy of Problems in Lexical Semantics (2023.findings-acl)
Copied to clipboard
| Challenge: | Semantic tasks are rarely formally defined, and the exact relationship between them is unclear . a taxonomy of several problems in lexical semantics is proposed to clarify this . |
| Approach: | They propose a taxonomy that elucidates the connection between several problems in lexical semantics . they propose equivalence theory and algorithmic problem reductions to reduce problems to word sense disambiguation (WSD) |
| Outcome: | The proposed taxonomy proves that word sense disambiguation and word synonymy are theoretically equivalent. |
Making Pre-trained Language Models both Task-solvers and Self-calibrators (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work shows that pre-trained language models can be effective for high-stake applications, but they become overconfident in their wrong predictions. |
| Approach: | They propose to use extra data to train pre-trained language models to effectively utilize training samples to make them both task-solvers and self-calibrators. |
| Outcome: | The proposed method can be used in three downstream applications, including selective classification, adversarial defense, and model cascading. |
EmbedTextNet: Dimension Reduction with Weighted Reconstruction and Correlation Losses for Efficient Text Embedding (2023.findings-acl)
Copied to clipboard
| Challenge: | EmbedTextNet is a light add-on network that can be appended to an arbitrary language model to generate a compact embedding without requiring any changes in its architecture or training procedure. |
| Approach: | They propose an add-on network that can be appended to an arbitrary language model to generate a compact embedding without requiring any changes in its architecture or training procedure. |
| Outcome: | The proposed network can be appended to an arbitrary language model to generate a compact embedding without any changes in its architecture or training procedure. |
Denoising Enhanced Distantly Supervised Ultrafine Entity Typing (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work on distantly supervised (DS) ultra-fine entity typing has received significant attention . however, DS data is noisy and often suffers from missing or wrong labeling issues resulting in low precision and low recall. |
| Approach: | They propose a noise model to estimate unknown labeling noise distribution over input contexts and noisy type labels and a model to train on denoised data. |
| Outcome: | The proposed model outperforms baseline methods on the Ultra-Fine entity typing dataset and OntoNotes dataset. |
INTapt: Information-Theoretic Adversarial Prompt Tuning for Enhanced Non-Native Speech Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to improve ASR performance with pre-trained models require updating the pre-training model weights. |
| Approach: | They propose a method that uses prompts concatenated to the original input to re-modulate attention of the pre-trained model. |
| Outcome: | The proposed model improves the performance of L2 English and increases similarity between L2 and L1 accents. |
Local Temperature Beam Search: Avoid Neural Text DeGeneration via Enhanced Calibration (2023.findings-acl)
Copied to clipboard
Dongkyu Lee, Gyeonghun Kim, Janghoon Han, Taesuk Hong, Yi-Reun Kim, Stanley Jungkyu Choi, Nevin L. Zhang
| Challenge: | Existing approaches to inference have been based on stochastic decoding but they sacrifice output quality due to randomness. |
| Approach: | They propose a deterministic decoding scheme, local temperature beam search, which reduces repetition while maintaining the level of coherence as in beam search. |
| Outcome: | The proposed inference scheme reduces repetition while maintaining coherence as in beam search. |
Explanation Graph Generation via Generative Pre-training over Synthetic Graphs (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing frameworks for explanation graph generation are limited due to the large number of datasets available. |
| Approach: | They propose a text-to-graph generative task to pre-train a model to bridge the text-graph gap. |
| Outcome: | The proposed framework surpasses all baseline systems with remarkable margins on ExplaGraphs and CommonsenseQA. |
NaSGEC: a Multi-Domain Chinese Grammatical Error Correction Dataset from Native Speaker Texts (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies on Chinese grammatical error correction focus on learning essays. |
| Approach: | They propose a Chinese grammatical error correction dataset that annotates multiple references for 12,500 sentences from three native domains. |
| Outcome: | The proposed dataset can be used to facilitate research on Chinese grammatical error correction (CGEC) for native speaker texts from multiple domains. |
FORK: A Bite-Sized Test Set for Probing Culinary Cultural Biases in Commonsense Reasoning Models (2023.findings-acl)
Copied to clipboard
| Challenge: | a recent study shows that commonsense knowledge is universally shared by most people . early efforts to schematize commonsensical knowledge as scripts provide examples of unintended biases . |
| Approach: | They propose a set of questions for probing cultural biases and assumptions in commonsense reasoning systems . they test commonsensibleQA-style questions on food-related customs in the u.s. |
| Outcome: | The proposed questions show that they are better at detecting biases in commonsense reasoning systems than on non-US cultures. |
FedPETuning: When Federated Learning Meets the Parameter-Efficient Tuning Methods of Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing research on federated learning (FL) for pre-trained language models (PLMs) with increasing concerns about data privacy, enterprises or institutions are not allowed to collect data from end devices or local clients to a centralized server for fine-tuning PLMs. |
| Approach: | They investigate the parameter-efficient tuning of pre-trained language models (PLMs) and develop a federated benchmark for four representative PETuning methods . |
| Outcome: | The proposed method can defend against privacy attacks and maintain acceptable performance with reducing heavy resource consumption. |
MixPAVE: Mix-Prompt Tuning for Few-shot Product Attribute Value Extraction (2023.findings-acl)
Copied to clipboard
Li Yang, Qifan Wang, Jingang Wang, Xiaojun Quan, Fuli Feng, Yu Chen, Madian Khabsa, Sinong Wang, Zenglin Xu, Dongfang Liu
| Challenge: | Existing methods for product attribute value extraction focus on extracting values for a set of known attributes with sufficient training data. |
| Approach: | They propose a prompt tuning approach to extract attributes from product information using mixed prompts. |
| Outcome: | The proposed approach improves on two product benchmarks and shows parameter-efficient training and avoids model overfitting. |
SlowBERT: Slow-down Attacks on Input-adaptive Multi-exit BERT (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models such as Google’s BERT have been gaining significant improvements to various down-stream applications, but the enormous training and inference cost severely hinders its practice on real-time applications and hardwareconstrained edge devices. |
| Approach: | They propose a slow-down attack on input-adaptive multi-exit BERT where the adversary imperceptibly modifies the input texts to drastically increase the inference cost. |
| Outcome: | The proposed attack on input-adaptive multi-exit BERT dramatically increases the average inference cost by 4.57, which would hurt the service quality of multi-extit BRT in practice, e.g., increasing the real-time cloud services’ response times for online users. |
Compositional Mathematical Encoding for Math Word Problems (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing MWP encoders work in a unimodal setting and map problem description to latent representation, then for decoding. |
| Approach: | They propose a Compositional Math Word Problem Solver which maps problem description to latent representation and decodes it in an interactive way. |
| Outcome: | Extensive experiments show that the proposed model outperforms state-of-the-art models on public benchmarks. |
PREADD: Prefix-Adaptive Decoding for Controlled Text Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for controlling text generation rely on auxiliary expert models but require training data. |
| Approach: | They propose a flexible method for controlled text generation that uses a prefix-prepended prompt instead of an external model. |
| Outcome: | The proposed method outperforms auxiliary-expert methods on three tasks by 12% or more on main metrics for each task. |
EventOA: An Event Ontology Alignment Benchmark Based on FrameNet and Wikidata (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on event ontologies focus on entity-based OA, and neglect event-based one . however, independent development of event ontoologies often results in heterogeneous representations that raise the need for establishing alignments between semantically related events. |
| Approach: | They propose a multi-view event ontology alignment method that utilizes description information and neighbor information to obtain richer representations of the event ontoologies. |
| Outcome: | The proposed method outperforms existing entity-based methods and can serve as a strong baseline for future research. |
Enhancing Continual Relation Extraction via Classifier Decomposition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies only adopt a vanilla strategy when learning representations of new relations . experimental results show that the importance of the first training stage to CRE models may be underestimated. |
| Approach: | They propose a framework that splits the last FFN layer into separated previous and current classifiers to maintain previous knowledge and encourage model to learn more robust representations at this training stage. |
| Outcome: | The proposed framework outperforms the state-of-the-art models on two benchmarks. |
A Comparative Analysis of the Effectiveness of Rare Tokens on Creative Expression using ramBERT (2023.findings-acl)
Copied to clipboard
| Challenge: | a pre-trained model labels an essay as creative or non-creative. a preliminary study examines the correlation between creative essays and expressiveness. |
| Approach: | They propose to use rare tokens to investigate correlation between creative essays and expressiveness . they propose to apply this technique to a model that automatically labels essays as creative or non-creative . |
| Outcome: | The proposed model improves the existing model by 14%. |
MTR: A Dataset Fusing Inductive, Deductive, and Defeasible Reasoning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets for logical reasoning focus on monotonic logic and a single form of reasoning. |
| Approach: | They propose to use a dataset to study the human-like reasoning in machine reading comprehension. |
| Outcome: | The proposed dataset shows that state-of-the-art neural models perform noticeably worse than expected. |
NewsMet : A ‘do it all’ Dataset of Contemporary Metaphors in News Headlines (2023.findings-acl)
Copied to clipboard
| Challenge: | Popular datasets used for metaphor processing tasks were constructed from dated source texts. |
| Approach: | They propose a large contemporary dataset of news headlines hand-annotated with metaphorical verbs. |
| Outcome: | The proposed dataset includes headlines from political, satirical, reliable and fake sources. |
Concept2Box: Joint Geometric Embeddings for Learning Two-View Knowledge Graphs (2023.findings-acl)
Copied to clipboard
Zijie Huang, Daheng Wang, Binxuan Huang, Chenwei Zhang, Jingbo Shang, Yan Liang, Zhengyang Wang, Xian Li, Christos Faloutsos, Yizhou Sun, Wei Wang
| Challenge: | Existing methods to embed knowledge graphs have ignored the fact that they contain two fundamentally different views: high-level ontology-view concepts and fine-grained instance-view entities. |
| Approach: | They propose a novel geometric representation that jointly embeds the two views of a KG using dual geometric representations. |
| Outcome: | Experiments on the public DBpedia KG and a newly-created industrial KG show the proposed method works well. |
Noise-Robust Training with Dynamic Loss and Contrastive Learning for Distantly-Supervised Named Entity Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a task in natural language processing that aims at locating entity mentions in a given sentence and assigning them to certain types. |
| Approach: | They propose to use a dynamic loss function to better adapt to the changing noise during the training process and incorporate token level contrastive learning to fully utilize the noisy data. |
| Outcome: | The proposed method outperforms existing NER models on three benchmark datasets and outperformed existing models by significant margins. |
Take a Break in the Middle: Investigating Subgoals towards Hierarchical Script Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work assumes that events are sequentially arranged in a script, while this assumption leads to linear generation that is far from sufficient for comprehensively acquiring the representation about how events are organized towards a task goal. |
| Approach: | They propose to extend goal-oriented Script Generation task from the perspective of cognitive theory by incorporating subgoals into hierarchical script generation. |
| Outcome: | The proposed task is based on a new dataset and human evaluation metrics. |
End-to-End Task-Oriented Dialogue Systems Based on Schema (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches for task-oriented dialogue systems rely on a unified schema across domains, but we propose a schema-aware model for task oriented dialogues based on 'slots' |
| Approach: | They propose a schema-aware end-to-end neural network model for handling task-oriented dialogues based on a dynamic set of slots within a unified schema. |
| Outcome: | The proposed model performs better on a well-known dataset than baselines on 'schema-guided dialogue' systems. |
HaVQA: A Dataset for Visual Question Answering and Multimodal Research in Hausa Language (2023.findings-acl)
Copied to clipboard
Shantipriya Parida, Idris Abdulmumin, Shamsuddeen Hassan Muhammad, Aneesh Bose, Guneet Singh Kohli, Ibrahim Said Ahmad, Ketan Kotwal, Sayan Deb Sarkar, Ondřej Bojar, Habeebah Kakudi
| Challenge: | Existing models for visual question answering are limited to the English language. |
| Approach: | They present a multimodal dataset for visual question answering tasks in the Hausa language. |
| Outcome: | The proposed dataset provides 12,044 gold standard English-Hausa parallel sentences that are semantically identical to the corresponding visual information. |
Claim-Dissector: An Interpretable Fact-Checking System with Joint Re-ranking and Veracity Prediction (2023.findings-acl)
Copied to clipboard
| Challenge: | a novel latent variable model for fact-checking and analysis learns to identify the veracity of a claim and its relevant evidences. |
| Approach: | They propose to disentangle the per-evidence relevance probability and its contribution to the final veracity probability in an interpretable way. |
| Outcome: | The proposed model can achieve competitive results on the FEVER dataset while using significantly fewer parameters. |
StructSP: Efficient Fine-tuning of Task-Oriented Dialog System by Using Structure-aware Boosting and Grammar Constraints (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models that learn hierarchical structure information representations do not perform well on task-oriented dialog systems. |
| Approach: | They propose a hierarchical structure information representation model that reinforces the semantic awareness of a pre-trained language model by a two-step fine-tuning mechanism. |
| Outcome: | The proposed model is better than existing models at learning the contextual representations of utterances embedded within its hierarchical semantic structure and improves system performance. |
GDA: Generative Data Augmentation Techniques for Relation Extraction Tasks (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work adopts data augmentation techniques to generate pseudo-annotated sentences . existing methods neither preserve semantic consistency of original sentences nor preserve syntax structure of sentences when expressing relations using seq2seq models, resulting in less diverse augmentations. |
| Approach: | They propose a dedicated augmentation technique for relational texts, named GDA, which uses two complementary modules to preserve both semantic consistency and syntax structures. |
| Outcome: | The proposed technique can bring 2.0% F1 improvements in three datasets under low-resource setting. |
WebDP: Understanding Discourse Structures in Semi-Structured Web Documents (2023.findings-acl)
Copied to clipboard
| Challenge: | Web documents are one of the most primary and biggest data resources in current era, and understanding their discourse structure will benefit various downstream document processing applications. |
| Approach: | They propose a web document discourse structure representation schema by extending classical discourse theories and adding special features to well represent discourse characteristics of web documents. |
| Outcome: | The proposed task is feasible but challenging for current models. |
Tab-CoT: Zero-shot Tabular Chain of Thought (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent efforts to encourage more structured reasoning procedures to be captured have shown that chain-of-though (CoT) prompting methods can be effective in NLP tasks. |
| Approach: | They propose a tabular-format CoT prompting method that allows the complex reasoning process to be explicitly modeled in a highly structured manner. |
| Outcome: | The proposed method shows impressive performance improvements on a range of reasoning tasks. |
KNSE: A Knowledge-aware Natural Language Inference Framework for Dialogue Symptom Status Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing frameworks for symptom status recognition in doctor-patient dialogues are inadequate. |
| Approach: | They propose a framework for symptom status recognition that formalizes a natural language inference task . they generate knowledge about the symptom and a hypothesis about its status for each symptom . |
| Outcome: | The proposed framework outperforms baselines and has advantages in cross-disease and cross-symptom scenarios. |
Augmenting Large Language Model Translators via Translation Memories (2023.findings-acl)
Copied to clipboard
Yongyu Mu, Abudurexiti Reheman, Zhiquan Cao, Yuchun Fan, Bei Li, Yinqiao Li, Tong Xiao, Chunliang Zhang, Jingbo Zhu
| Challenge: | Using translation memories (TMs) as prompts is a promising approach to in-context learning of machine translation models. |
| Approach: | They propose to use translation memories (TMs) as prompts to prompt large language models (LLMs) they find that the ability of LLMs to "understand" prompts is helpful . |
| Outcome: | The results are comparable to state-of-the-art NMT systems with bilingual data and are tuned on downstream tasks. |
Character Coreference Resolution in Movie Screenplays (2023.findings-acl)
Copied to clipboard
| Challenge: | Movie screenplays have a distinct narrative structure. |
| Approach: | They develop a method to extract structural information and character coreference clusters from movie screenplays by leveraging a movie parser and a character coreferser. |
| Outcome: | The proposed methods scale to long movie screenplays without dramatically increasing their memory footprints. |
Enhancing Event Causality Identification with Event Causal Label and Event Pair Interaction Graph (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for event causality identification (ECI) do not consider event causal label information and interaction information between event pairs. |
| Approach: | They propose a framework to enrich the representation of event pairs by introducing the event causal label information and the interaction information between event pairs. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on two benchmark datasets. |
LightFormer: Light-weight Transformer Using SVD-based Weight Transfer and Parameter Sharing (2023.findings-acl)
Copied to clipboard
| Challenge: | Deploying Transformer networks on resource-constrained edge devices is challenging. |
| Approach: | They propose a low-rank factorization initialized by SVD-based weight transfer and parameter sharing to compress and accelerate Transformer networks. |
| Outcome: | The proposed method achieves similar performance to the baseline Transformer with 3.8 times and 1.8 times fewer parameters and achieves 2.3 times speedup and 1.5 times speed up respectively. |
Multi-hop Evidence Retrieval for Cross-document Relation Extraction (2023.findings-acl)
Copied to clipboard
| Challenge: | Relation Extraction (RE) is a task that seeks to identify the relation of entities described according to some context. |
| Approach: | They propose a multi-hop evidence retrieval method based on evidence path mining and ranking to support cross-document relation extraction. |
| Outcome: | The proposed method acquires cross-document evidence and boosts performance in both closed and open environments. |
Which Examples Should be Multiply Annotated? Active Learning When Annotators May Disagree (2023.findings-acl)
Copied to clipboard
| Challenge: | Disagreement in annotations is natural for humans, depending on background, identity, positionality . many active learning approaches focus on examples where model entropy and annotator entropicy are the most different. |
| Approach: | They propose an active learning approach that focuses annotations on examples where model entropy and annotator entropic are the most different. |
| Outcome: | The proposed approach reduces the number of annotations required by 24% on average across datasets. |
PIP: Parse-Instructed Prefix for Syntactically Controlled Paraphrase Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing fine-tuning methods for this task are costly and require updating the parameters of the entire model to adapt to the newly included syntax information. |
| Approach: | They propose a method to instruct model’s encoder prefix to capture syntax-related knowledge by direct initiation and indirect optimization. |
| Outcome: | The proposed methods are 10 times more efficient and learnable than existing methods. |
DePlot: One-shot visual language reasoning by plot-to-table translation (2023.findings-acl)
Copied to clipboard
Fangyu Liu, Julian Eisenschlos, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Wenhu Chen, Nigel Collier, Yasemin Altun
| Challenge: | Existing models for visual language reasoning require tens of thousands of training examples and their reasoning capabilities are limited. |
| Approach: | They propose a one-shot solution to visual language reasoning by combining plot-to-text translation and reasoning over the translated text into a modality conversion module. |
| Outcome: | The proposed method improves on human-written queries on plots and charts compared with a fine-tuned SOTA model on human queries. |
Stochastic Bridges as Effective Regularizers for Parameter-Efficient Tuning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for tuning pre-trained language models ignore the running cost and only optimize the terminal cost. |
| Approach: | They propose to use stochastic bridges to regularize intermediate states and use regularization as running cost of PETs. |
| Outcome: | The proposed methods can be used to tune large pre-trained language models . they can be compared to full-parameter fine-tuning by tuning a small number of parameters . |
Learning from a Friend: Improving Event Extraction via Self-Training with Feedback from Abstract Meaning Representation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing data scarcity hinders the progress of event extraction, authors say . ACE-052 has 10 of the 33 event types with less than 80 annotations, authors claim . |
| Approach: | They propose a self-training with feedback framework that leverages large-scale unlabeled data to acquire feedback for each new event prediction from the unlabed data. |
| Outcome: | The proposed framework improves event extraction models even when unlabeled data are unavailable. |
How Well Do Large Language Models Perform on Faux Pas Tests? (2023.findings-acl)
Copied to clipboard
| Challenge: | Using large language models, we investigate the ability of large language model to generate correct responses to questions involving descriptions of faux pas situations. |
| Approach: | They investigate the ability of large language models to generate correct responses to questions involving descriptions of faux pas situations. |
| Outcome: | The proposed model fails to generate correct responses to presupposition questions and many of the correct responses are attributed to over-interpretation by the human reader. |
Modular Transformers: Compressing Transformers into Modularized Layers for Flexible Efficient Inference (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained sequence-to-sequence models have advanced the state of the art on text generation tasks. |
| Approach: | They introduce a modular encoder-decoder framework for flexible sequence-to-sequence model compression. |
| Outcome: | The proposed framework can achieve flexible compression ratios from 1.1x to 6x with little to moderate relative performance drop. |
ISLTranslate: Dataset for Translating Indian Sign Language (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing sign language translation datasets are limited for the Indian sign language. |
| Approach: | They propose to use ISLTranslate to create a sign language translation dataset for Indian Sign Language consisting of 31k ISL-English sentence/phrase pairs. |
| Outcome: | The proposed dataset is the largest for Indian Sign Language translation dataset . it compares with a transformer-based model to validate the performance of existing systems. |
LMentry: A Language Model Benchmark of Elementary Language Tasks (2023.findings-acl)
Copied to clipboard
| Challenge: | Large language models are evaluated via perplexity or performance on downstream tasks, but these benchmarks are too complex and difficult to inspect. |
| Approach: | They propose a benchmark that focuses on 25 tasks that humans are expected to perform perfectly, such as writing a sentence containing a specific word or identifying which words in a list belong to a certain category. |
| Outcome: | The proposed benchmarks show that large language models are performing better than previous benchmarks. |
Differentiable Instruction Optimization for Cross-Task Generalization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have shown that instruction tuning is effective for generalizing to arbitrary tasks unseen during training. |
| Approach: | They propose to introduce learnable instructions and optimize them with gradient descent to optimize instruction for generalization ability. |
| Outcome: | The proposed instruction extractor extracts appropriate instruction and improves generalization ability compared to manual instruction tuning. |
Leveraging Training Data in Few-Shot Prompting for Numerical Reasoning (2023.findings-acl)
Copied to clipboard
| Challenge: | Chain-of-thought (CoT) prompts can be challenging to design for arithmetic word problem solving. |
| Approach: | They propose to use training data to replace CoT with programs as the reasoning step . their results show that leveraging training data can improve generalization ability of prompts . |
| Outcome: | The proposed methods improve the generalization ability of prompts and the performance of fine-tuned smaller models in arithmetic word problem solving. |
How does the task complexity of masked pretraining objectives affect downstream performance? (2023.findings-acl)
Copied to clipboard
| Challenge: | Masked language modeling (MLM) is a widely used self-supervised pretraining objective. |
| Approach: | They propose to use a mask-based objective to predict a token that is replaced with a masked token given its context. |
| Outcome: | The proposed objectives show that they should have half the complexity needed to perform comparably to MLM. |
AUGUST: an Automatic Generation Understudy for Synthesizing Conversational Recommendation Datasets (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work on conversational recommendation systems lacks high-quality data . existing datasets lack large-scale and high-level data based on human annotators . |
| Approach: | They propose an automatic dataset synthesis approach that generates large-scale recommendation dialogues using structured graphs based on user-item information from the real world. |
| Outcome: | The proposed approach can generate large-scale and high-quality recommendation dialogues . it exploits user preferences, knowledge graphs, and conversation ability from existing datasets based on real-world data . |
Knowing-how & Knowing-that: A New Task for Machine Comprehension of User Manuals (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for machine reading comprehension of user manuals have trouble answering complex questions. |
| Approach: | They propose a knowing-how & knowing-that task that requires the model to answer factoid-style, procedure-style and inconsistent questions about user manuals. |
| Outcome: | The proposed model can answer factoid-style, procedure-style and inconsistent questions about user manuals. |
Deep Span Representations for Named Entity Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing span-based NER systems shallowly aggregate token representations to span representations, causing performance degradations. |
| Approach: | They propose a method that shallowly aggregates token representations to span representations . DSpERT produces span representation of deep semantics with weight initialization . |
| Outcome: | The proposed model performs better than or competitive with state-of-the-art systems on six NER benchmarks. |
Disambiguated Lexically Constrained Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Current approaches to LCNMT assume that pre-specified lexicon constraints are contextually appropriate. |
| Approach: | They propose a framework that disambiguates constraints based on contexts at first and integrates them into LCNMT. |
| Outcome: | The proposed approach outperforms baseline approaches on benchmark datasets and comprehensive experiments in multiple target constraints. |
Curating Datasets for Better Performance with Example Training Dynamics (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to improve data quality but rely on data quantity to improve performance are not effective. |
| Approach: | They propose a method for weighing the relative importance of examples in a dataset based on their Example Training dynamics (ETD) they propose an active learning approach for computing ETD during training rather than as a preprocessing step. |
| Outcome: | The proposed method can be used to improve performance in in-distribution and out-of-distortion testing. |
Multi-armed bandits for resource efficient, online optimization of language model pre-training: the use case of dynamic masking (2023.findings-acl)
Copied to clipboard
| Challenge: | Using a Bayesian optimization framework, we pre-train Transformer-based language models (TLMs) using a multi-armed bandit framework requires high computational resources and introduces many unresolved design choices. |
| Approach: | They propose a Bayesian optimization framework for resource efficient pre-training of Transformer-based language models. |
| Outcome: | The proposed framework achieves lower MLM loss in fewer epochs, across settings, while avoiding expensive hyperparameter grid search. |
ERNIE-Code: Beyond English-Centric Cross-lingual Pretraining for Programming Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | ERNIE-Code is a unified pre-trained language model for 116 NLs and 6 PLs. |
| Approach: | They propose a unified pre-trained language model for 116 NLs and 6 PLs . they employ span-corruption language modeling that learns patterns from monolingual NL or PL . |
| Outcome: | The proposed model outperforms previous multilingual models for NL or NL across end tasks. |
PromptAttack: Probing Dialogue State Trackers with Adversarial Prompts (2023.findings-acl)
Copied to clipboard
| Challenge: | Toward building more robust and reliable conversational systems, we introduce a prompt-based learning approach to automatically generate effective adversarial examples to probe DST models. |
| Approach: | They propose a prompt-based learning approach to automatically generate effective adversarial examples to probe DST models. |
| Outcome: | The proposed framework leads to the greatest reduction in accuracy and the best attack success rate while maintaining good fluency and a low perturbation ratio. |
Understanding Programs by Exploiting (Fuzzing) Test Cases (2023.findings-acl)
Copied to clipboard
| Challenge: | Semantic understanding of programs has attracted great attention in the community . large language models (LLMs) are capable of learning contextual information from data at scale . |
| Approach: | They propose to incorporate a relationship between inputs and possible outputs into learning for achieving a deeper semantic understanding of programs. |
| Outcome: | The proposed method outperforms current state-of-the-art on two programming tasks and outperformed current state of the art by large margins. |
Hybrid Hierarchical Retrieval for Open-Domain Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work shows that dense hierarchical retrieval (DHR) can outperform dense passage retrieval. |
| Approach: | They propose a framework that applies sparse, dense and a combination of them to document and passage retrieval. |
| Outcome: | The proposed framework can outperform dense hierarchical retrieval (DHR) and sparse retrievers (BM25) on open-domain question answering (ODQA) datasets with an average improvement of 4.69% on recall@100 over DHR. |
Coherent or Not? Stressing a Neural Language Model for Discourse Coherence in Multiple Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work on coherence assessment using NLMs focuses on properties acquired from stand-alone sentences, but their ability to model discourse and pragmatic phenomena is still unclear. |
| Approach: | They propose to use a Neural Language Model to assess coherence in multiple languages to compare models' performance and to examine their performance in a cross-language scenario. |
| Outcome: | The proposed model can model coherent and incoherent text in multiple languages and in-domain settings. |
Understanding Differential Search Index for Text Retrieval (2023.findings-acl)
Copied to clipboard
| Challenge: | Differentiable Search Index (DSI) is a new information retrieval framework . however, due to the black-box nature of the end-to-end neural architecture, it remains unclear to what extent it possesses basic indexing and retrieval abilities. |
| Approach: | They propose a multi-task distillation approach to enhance the retrieval quality without altering the structure of the model. |
| Outcome: | The proposed method outperforms baselines on various datasets. |
Masked Audio Text Encoders are Effective Multi-Modal Rescorers (2023.findings-acl)
Copied to clipboard
| Challenge: | Masked Language Models (MLMs) have proven to be effective for second-pass rescoring in Automatic Speech Recognition systems. |
| Approach: | They propose a multi-modal masked language model rescorer which integrates acoustic representations into the input space of MLM. |
| Outcome: | The proposed model reduces word error rate (WER) by 4%-16% on in-domain and 3%-7% on out-of-domain datasets over the text-only baseline. |
Replace and Report: NLP Assisted Radiology Report Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Clinical practice frequently uses medical imaging for diagnosis and treatment. |
| Approach: | They propose a template-based approach to generate radiology reports from radiographs . they use multilabel image classifiers to generate tags, pathological descriptions from tags . |
| Outcome: | The proposed method improves on the most popular radiology report datasets. |
Pre-trained Personalized Review Summarization with Effective Salience Estimation (2023.findings-acl)
Copied to clipboard
| Challenge: | Pretrained language models (PLMs) are a new paradigm in text generation for the strong ability of natural language comprehension. |
| Approach: | They propose a pre-trained personalized review summarization method that incorporates personalized information into the salience estimation of input reviews. |
| Outcome: | The proposed method performs better than the state-of-the-art methods on real-world datasets. |
CaPE: Contrastive Parameter Ensembling for Reducing Hallucination in Abstractive Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work suggests that the degree of hallucination depends on factual errors in training data. |
| Approach: | They propose a method to use training data to reduce hallucination by ensembling parameter variations in training data. |
| Outcome: | The proposed method improves on XSUM and CNN/DM datasets on human evaluations and factual metrics. |
OpineSum: Entailment-based self-training for abstractive opinion summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Abstractive summarization is promising for fluently comparing opinions from a set of reviews about a place or product. |
| Approach: | They propose a novel method that automatically leverages common opinions across reviews to create powerful abstractive models. |
| Outcome: | The proposed method outperforms strong peer systems in both settings. |
A Call for Standardization and Validation of Text Style Transfer Evaluation (2023.findings-acl)
Copied to clipboard
| Challenge: | Text style transfer (TST) evaluation is inconsistent in practice. |
| Approach: | They conduct a meta-analysis on human and automated TST evaluation and experimentation . they find a standardization gap and a validation gap in the field . |
| Outcome: | The authors find that evaluation procedures are inconsistent and that they need to improve on them. |
Bridging the Granularity Gap for Acoustic Modeling (2023.findings-acl)
Copied to clipboard
Chen Xu, Yuhao Zhang, Chengbo Jiao, Xiaoqian Liu, Chi Hu, Xin Zeng, Tong Xiao, Anxiang Ma, Huizhen Wang, Jingbo Zhu
| Challenge: | Despite the success of speech recognition, how to encode the speech features effectively remains an open problem. |
| Approach: | They propose a Progressive Down-Sampling technique which compresses acoustic features into coarser-grained units containing more complete semantic information, like text-level representation. |
| Outcome: | The proposed method yields comparable or better results on the speech recognition task and inference speedups ranging from 1.20x to 1.47x. |
MMSD2.0: Towards a Reliable Multi-modal Sarcasm Detection System (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for multi-modal sarcasm detection have some shortcomings . a new framework can leverage multi-grained cues from multiple perspectives for multimodal detection . |
| Approach: | They propose a correction dataset that removes spurious cues and re-annotates the unreasonable samples. |
| Outcome: | The proposed framework outperforms the existing benchmarks in multi-modal sarcasm detection. |
Learn to Not Link: Exploring NIL Prediction in Entity Linking (2023.findings-acl)
Copied to clipboard
| Challenge: | Entity linking models have been successful in capturing semantic features, but the NIL prediction problem has not been addressed. |
| Approach: | They propose an entity linking dataset that categorizes mentions linking to NIL into Missing Entity and Non-Entity Phrases. |
| Outcome: | The proposed dataset categorizes mentions linking to NIL into Missing Entity and Non-Entity Phrase categories and ensures the presence of mentions by human annotation and entity masking. |
On Text-based Personality Computing: Challenges and Future Directions (2023.findings-acl)
Copied to clipboard
Qixiang Fang, Anastasia Giachanou, Ayoub Bagheri, Laura Boeschoten, Erik-Jan van Kesteren, Mahdi Shafiee Kamalabad, Daniel Oberski
| Challenge: | Text-based personality computing (TPC) is a popular alternative to self-report questionnaires. |
| Approach: | They propose 15 challenges that are relevant to NLP research . they propose to combine perspectives from both NLP and social sciences . |
| Outcome: | The proposed approach is based on text-based personality computing (TPC) the proposed approach can be used to improve the quality of personality-based research. |
Structured Pruning for Efficient Generative Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Large-scale generative Pre-trained Language Models (PLMs) are limited in their deployment in real-world applications. |
| Approach: | They propose to prune the feed-forward networks of generative pre-trained language models to smaller widths without designing extra operators. |
| Outcome: | The proposed method achieves 1.51x/6.96x inference speedup on GPU/CPU with 67% size reduction. |
Prompt-Guided Retrieval Augmentation for Non-Knowledge-Intensive Tasks (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies focus on retrieval to solve knowledge-intensive tasks, but the potential of retrieval for non-knowledge-intensive (NKI) tasks remains under-explored. |
| Approach: | They propose a task-agnostic retrieval framework for NKI tasks that uses a static index and a prompt-guided reranker to re-rank the nearest evidence according to task-specific relevance. |
| Outcome: | The proposed framework outperforms state-of-the-art retrieval-augmented methods on NKI tasks and will be released for further research. |
Contextualized Semantic Distance between Highly Overlapped Texts (2023.findings-acl)
Copied to clipboard
| Challenge: | Conventional semantic metrics are based on word representations and are vulnerable to disturbance of overlapped components with similar representations. |
| Approach: | They propose a mask-and-predict strategy to evaluate the semantic distance between the overlapped sentences using words in the longest common sequence as neighboring words and use masked language modeling to predict their positions. |
| Outcome: | The proposed method outperforms the state-of-the-art in domain adaption by a huge margin. |
Unsupervised Dense Retrieval with Relevance-Aware Contrastive Pre-Training (2023.findings-acl)
Copied to clipboard
| Challenge: | Dense retrievers have impressive performance, but their demand for abundant training data limits their application scenarios. |
| Approach: | They propose a method which uses unlabeled data to construct pseudo-positive examples from unlabelled data and then contrastively weighs the contrastive loss of different pairs according to the estimated relevance. |
| Outcome: | The proposed method beats the SOTA unsupervised Contriever model on BEIR and open-domain QA retrieval benchmarks and is a good few-shot learner. |
Verifying Annotation Agreement without Multiple Experts: A Case Study with Gujarati SNACS (2023.findings-acl)
Copied to clipboard
| Challenge: | a small fraction of the about 7,000 languages of the world have datasets or linguistic tools . linguistic datasets are a foundation of NLP research, but they are not always reliable . authors propose weak verifiers to help estimate dataset quality . |
| Approach: | They propose four weak verifiers to help estimate dataset quality . they propose to use Gujarati as a low-resource language to test for dataset quality. |
| Outcome: | The proposed methods concur with a double-annotation study in Gujarati. |
Reinforced Active Learning for Low-Resource, Domain-Specific, Multi-Label Text Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | Modern text classification systems achieve excellent accuracy across tasks and corpora. |
| Approach: | They propose a Reinforcement Learning policy that uses many different aspects of the data and task to select the most informative unlabeled subset dynamically over the course of the AL procedure. |
| Outcome: | The proposed framework outperforms baselines on four complex multi-class, multi-label text classification datasets. |
Improving Classroom Dialogue Act Recognition from Limited Labeled Data with Self-Supervised Contrastive Learning Classifiers (2023.findings-acl)
Copied to clipboard
| Challenge: | Recognizing classroom dialogue acts has significant promise for yielding insight into teaching, student learning, and classroom dynamics. |
| Approach: | They propose to use a contrastive learning-based self-supervised approach to improve classroom dialogue act recognition from limited labeled data by increasing the accuracy of dialogue act recognization and minimizing embedding distance between the same dialogue acts. |
| Outcome: | The proposed model outperforms baseline models when trained with limited examples per dialogue act and outperformed other few-shot models that require considerably more labeled data. |
Contrastive Token-Wise Meta-Learning for Unseen Performer Visual Temporal-Aligned Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | a novel generalization framework for visual temporal-aligned translation is proposed to transfer recognition skills to unseen performers . ambiguity in the visual sequence can hinder current methods for visual language translation . |
| Approach: | They propose a generalizable framework to transfer recognition skills to unseen performers . they use visual temporal-aligned translation to generate multiple words autoregressively . |
| Outcome: | The proposed framework is generalized to transfer recognition skills to unseen performers . it is compared with existing methods on lipreading and fingerspelling datasets . |
Enhancing Cross-lingual Prompting with Dual Prompt Augmentation (2023.findings-acl)
Copied to clipboard
| Challenge: | a recent study shows that prompting is superior for multilingual/cross-lingual problems . despite its effectiveness on English tasks, its potential for cross-lingual problem is under-explored . |
| Approach: | They propose a framework for prompting that can be used to augment cross-lingual prompts. |
| Outcome: | The proposed framework achieves 46.54% with only 16 English training examples per class, significantly better than fine-tuning. |
Foveate, Attribute, and Rationalize: Towards Physically Safe and Trustworthy AI (2023.findings-acl)
Copied to clipboard
| Challenge: | Covertly unsafe text is an area of particular interest as it is difficult to detect as harmful . previous work focused on explicit violent text and typically expressed through violent keywords. |
| Approach: | They propose a framework that leverages external knowledge for trustworthy rationale generation in the context of safety. |
| Outcome: | The proposed framework improves safety classification accuracy by 5.9% on the SafeText dataset, and shows that it is more accurate than previous frameworks. |
Multijugate Dual Learning for Low-Resource Task-Oriented Dialogue System (2023.findings-acl)
Copied to clipboard
| Challenge: | End-to-end task-oriented dialogue systems are expensive to annotate and lack data in real scenarios. |
| Approach: | They propose to implement dual learning in task-oriented dialogues to exploit the correlation of heterogeneous data. |
| Outcome: | The proposed method improves the effectiveness of end-to-end task-oriented dialogue systems under multiple benchmarks and obtains state-of-the-art results in low-resource scenarios. |
A Class-Rebalancing Self-Training Framework for Distantly-Supervised Named Entity Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Distant supervision reduces the reliance on human annotation in named entity recognition tasks. |
| Approach: | They propose a class-rebalancing self-training framework for improving distantly-supervised named entity recognition by using a flexible threshold and a hybrid pseudo label. |
| Outcome: | The proposed model achieves state-of-the-art on five flat and two nested datasets and compares with other methods on the same dataset. |
MURMUR: Modular Multi-Step Reasoning for Semi-Structured Data-to-Text Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | MURMUR generates highly faithful and correct reasoning paths that lead to 26% more logically consistent summaries on LogicNLG compared to direct prompting. |
| Approach: | They propose a neuro-symbolic modular approach to text generation from semi-structured data with multi-step reasoning that generates reasoning paths using neural and symbolic modules with specific linguistic and logical skills. |
| Outcome: | The proposed method improves on two data-to-text generation tasks, while achieving comparable performance to fine-tuned GPT-2 on out-of-domain data. |
Learning by Analogy: Diverse Questions Generation in Math Word Problem (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for solving math word problem (MWP) use shortcut learning to train solvers based on samples with a single question. |
| Approach: | They propose to generate diverse yet consistent questions from a common scenario . they then feed the equations to a question generator to obtain the diverse questions . their method leads to performance improvement on the current benchmark Math23K . |
| Outcome: | The proposed method generates diverse yet consistent questions with a variety of equations and questions . it improves on the current benchmark, which is based on the proposed method . |
Revisit Few-shot Intent Classification with PLMs: Direct Fine-tuning vs. Continual Pre-training (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent progress in intent detection relies on deep models and datasets with well-crafted annotations. |
| Approach: | They propose a continual pre-training approach to train deep learning models . they propose augmentation method and sequential self-distillation to boost performance . |
| Outcome: | The proposed method outperforms methods that employ continual pre-training on labeled datasets on few-shot intent detection tasks. |
Improving Contrastive Learning of Sentence Embeddings from AI Feedback (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to learn sentence embeddings with rich semantics are limited due to the discrete nature of natural language. |
| Approach: | They propose to use AI feedback to improve contrastive learning of sentence embeddings by combining human feedback and AI feedback. |
| Outcome: | The proposed method achieves state-of-the-art performance on several semantic textual similarity and transfer learning tasks compared to other unsupervised and supervised contrastive learning methods. |
Mars: Modeling Context & State Representations with Contrastive Learning for End-to-End Task-Oriented Dialog (2023.findings-acl)
Copied to clipboard
| Challenge: | Empirical results show dialog context representations are more conducive to multi-turn task-oriented dialog. |
| Approach: | They propose an end-to-end task-oriented dialog system with two contrastive learning strategies to model relationship between dialog context and belief/action state representations. |
| Outcome: | Empirical results show that dialog context representations are more conducive to multi-turn task-oriented dialog. |
Text Augmented Open Knowledge Graph Completion via Pre-Trained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to augment knowledge graph completion require factual triples or manual prompts to extract knowledge from a pre-trained language model. |
| Approach: | They propose a tool that generates quality query prompts and retrieves support information from large text corpora to probe knowledge from a pre-trained language model. |
| Outcome: | The proposed method outperforms embedding-based, graph-based and PLM-based methods on two benchmark datasets. |
Discourse Analysis via Questions and Answers: Parsing Dependency Structures of Questions Under Discussion (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing discourse formalisms require large taxonomies of discourse relations to be accurate. |
| Approach: | They propose a linguistic framework for discourse analysis using questions under discussion . they propose qUD parser that derives a dependency structure of questions over full documents . |
| Outcome: | The proposed model is trained on a large, crowdsourced question-answering dataset. |
An Integrated Approach for Political Bias Prediction and Explanation Based on Discursive Structure (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for predicting and explaining political biases rely on lexical cues. |
| Approach: | They propose an approach to automatically characterize biases that takes into account structural differences and is efficient for long texts. |
| Outcome: | The proposed approach is efficient for long texts and takes into account structural differences. |
Smart Word Suggestions for Writing Assistance (2023.findings-acl)
Copied to clipboard
| Challenge: | Using word suggestions, writing assistance is a widely used application of natural language processing (NLP) . a task is performed to identify words or phrases that require improvement and provide substitution suggestions for each improvable target. |
| Approach: | They propose a task and benchmark to help writers improve word usage . they use human-labeled data and a distantly supervised dataset for testing . |
| Outcome: | The proposed task and benchmark aims to improve word usage in writing aids. |
JECC: Commonsense Reasoning Tasks Derived from Interactive Fictions (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks focus on a single reasoning type and ask human annotators to write candidate statements related to the particular type of commonsense. |
| Approach: | They propose a new commonsense reasoning dataset based on human’s Interactive Fiction (IF) gameplaywalkthroughs. |
| Outcome: | The proposed dataset is challenging to previous machine reading models and large language models with a significant 20%performance gap compared to human experts. |
A Study on Knowledge Distillation from Weak Teacher for Scaling Up Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | a study shows that DWT can be effective in the vision domain and natural language processing pre-training stages. |
| Approach: | They examine three key factors to optimize Distillation from Weak Teacher (DWT) DWT is a method of transferring knowledge from a weaker teacher model to a larger student model to improve its performance. |
| Outcome: | a new study examines three key factors to optimize DWT in NLP pre-training scenarios . the impact of teacher model quality and guidelines for adjusting the weighting value for DW T loss are examined . |
SORTIE: Dependency-Aware Symbolic Reasoning for Logical Data-to-text Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on logical data-to-text generation rely on neural language models to generate the final table description, but they have difficulty working out key entities in the description. |
| Approach: | They propose a symbolic reasoning framework that reasons out each entity in the table description with a table-compatible programming language. |
| Outcome: | The proposed framework outperforms existing methods on three datasets and three backbones with an absolute improvement of 5.7%11.5% on SP-Acc. |
Boosting Event Extraction with Denoised Structure-to-Text Augmentation (2023.findings-acl)
Copied to clipboard
Bo Wang, Heyan Huang, Xiaochi Wei, Ge Shi, Xiao Liu, Chong Feng, Tong Zhou, Shuaiqiang Wang, Dawei Yin
| Challenge: | Existing methods for event extraction neglect grammatical incorrectness, structure misalignment, and semantic drifting . et al., 2004; Ahn, 2006) show that the proposed method generates more diverse text representations for event extracting compared with the state-of-the-art. |
| Approach: | They propose a framework for event extraction that generates additional training data and iteratively selects the effective subset from the generated training data. |
| Outcome: | The proposed method generates more diverse representations of training data and achieves comparable results with the state-of-the-art. |
Detecting Adversarial Samples through Sharpness of Loss Landscape (2023.findings-acl)
Copied to clipboard
Rui Zheng, Shihan Dou, Yuhao Zhou, Qin Liu, Tao Gui, Qi Zhang, Zhongyu Wei, Xuanjing Huang, Menghan Zhang
| Challenge: | Existing studies have shown that adversarial samples are more vulnerable than normal ones to textual adversarials. |
| Approach: | They propose a simple and effective sharpness-based detector that can distinguish adversarial samples by maximizing the loss increment within the region where the inference sample is located. |
| Outcome: | The proposed method outperforms previous detection methods by large margins on three text classification tasks. |
A Simple, Yet Effective Approach to Finding Biases in Code Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work shows that large language models can generate code on par with humans . however, data-driven approaches may not be sufficient for acquiring reasoning skills . |
| Approach: | They propose a framework that automatically identifies subtle cues a code generation model might exploit . they propose an automated intervention mechanism reminiscent of adversarial testing . |
| Outcome: | The proposed framework can be used as a data transformation technique during fine-tuning, acting as reversal strategy. |
Membership Inference Attacks against Language Models via Neighbourhood Comparison (2023.findings-acl)
Copied to clipboard
Justus Mattern, Fatemehsadat Mireshghallah, Zhijing Jin, Bernhard Schoelkopf, Mrinmaya Sachan, Taylor Berg-Kirkpatrick
| Challenge: | Existing membership inference attacks aim to predict whether a data sample was present in training data of a machine learning model. |
| Approach: | They propose to compare model scores to neighbour texts to eliminate access to training data by comparing model scores with a given sample. |
| Outcome: | The proposed attacks outperform reference-based attacks with perfect knowledge of the training data distribution and outperformed reference-free attacks with imperfect knowledge. |
CFL: Causally Fair Language Models Through Token-level Attribute Controlled Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to control attributes of Language Models (LMs) for text generation are not safe, as toxicity and bias goals are opposed to each other. |
| Approach: | They propose a method to control the attributes of Language Models (LMs) for the text generation task using Causal Average Treatment Effect (ATE) scores and counterfactual augmentation. |
| Outcome: | The proposed architecture achieves state of the art performance for toxic degeneration, which are computed using Real Toxicity Prompts. |
Can Diffusion Model Achieve Better Performance in Text Generation ? Bridging the Gap between Training and Inference ! (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models for text generation use a discrete data embedding module to map the data into the continuous space. |
| Approach: | They propose two methods to bridge the gap between training and inference by mapping the discrete text into the continuous space. |
| Outcome: | The proposed methods can achieve 100 200 speedup with better performance on 6 generation tasks. |
Topic-Guided Self-Introduction Generation for Social Media Users (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on social media use tags to profile users, but we have found that sentence-level self-introductions are more natural and engaging. |
| Approach: | They propose a novel topic-guided encoder-decoder framework that uses a user's tweeting history to generate a short sentence outlining their personal interests. |
| Outcome: | The proposed framework outperforms existing encoder-decoder models on a large-scale Twitter dataset and shows that it is more natural and engaging than previous approaches. |
Recyclable Tuning for Continual Pre-training (2023.findings-acl)
Copied to clipboard
Yujia Qin, Cheng Qian, Xu Han, Yankai Lin, Huadong Wang, Ruobing Xie, Zhiyuan Liu, Maosong Sun, Jie Zhou
| Challenge: | Continual pre-training is the paradigm where pre-trained language models acquire fresh knowledge and gradually get upgraded. |
| Approach: | They propose to use adapted weights to recycle old PLMs for continual pre-training . they propose to combine initialization and distillation methods to achieve better performance . |
| Outcome: | The proposed method improves the convergence and performance of the upgraded PLM. |
BLOCSUM: Block Scope-based Source Code Summarization via Shared Block Representation (2023.findings-acl)
Copied to clipboard
| Challenge: | Abstract Syntax Tree (AST) and sequence of code tokens are useful for code summarization. |
| Approach: | They propose a shared block position embedding to represent various code blocks . they also develop variant ASTs to learn rich information such as block and global dependencies . |
| Outcome: | The proposed method improves on two real-world datasets, including ablation studies and a human evaluation. |
HyperPELT: Unified Parameter-Efficient Language Model Tuning for Both Language and Vision-and-Language Tasks (2023.findings-acl)
Copied to clipboard
| Challenge: | Pretraining and fine-tuning are the dominant paradigms in natural language processing. |
| Approach: | They propose a parameter-efficient multitask learning framework that takes trainable hyper-embeddings and visual modality as input and outputs weights for different modules in a pretrained language model. |
| Outcome: | The proposed framework adds fewer trainable parameters in multi-task learning while achieving superior performances and transfer ability compared to state-of-the-art methods. |
Enhancing Unsupervised Semantic Parsing with Distributed Contextual Representations (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to learn models on corpus of pairs of sentences require labor-intensive annotation. |
| Approach: | They propose to leverage distributed contextual word and phrase representations pre-trained on unlabelled texts to deal with homonymy and polysemy. |
| Outcome: | The proposed model achieves better accuracy on question-answering and relation extraction tasks. |
Generating Labeled Data for Relation Extraction: A Meta Learning Approach with Joint GPT-2 Training (2023.findings-acl)
Copied to clipboard
| Challenge: | Relation Extraction (RE) is the task of identifying semantic relation between entities mentioned in text. |
| Approach: | They propose a framework to automatically generate labeled data for Relation Extraction . they propose 'reward function' to update pre-trained language model for RE . |
| Outcome: | The proposed framework generates labeled data for relation extraction using a pre-trained language model and a meta learning approach to improve the generated samples. |
Disfluency Generation for More Robust Dialogue Systems (2023.findings-acl)
Copied to clipboard
| Challenge: | Disfluencies in user utterances can trigger a chain of errors impacting all the modules of a dialogue system. |
| Approach: | They propose to augment existing dialogue datasets with disfluent utterances by paraphrasing them into disfluente ones. |
| Outcome: | The proposed method improves dialogue state tracking and response generation by combining disfluent utterances with disfluency utteraces. |
Dipping PLMs Sauce: Bridging Structure and Text for Effective Knowledge Graph Completion via Conditional Soft Prompting (2023.findings-acl)
Copied to clipboard
| Challenge: | Knowledge Graph Completion (KGC) often requires both KG structural and textual information to be effective. |
| Approach: | They propose a system which tunes the parameters of Conditional Soft Prompts generated by entities and relations representations to maintain a balance between textual and structural knowledge. |
| Outcome: | The proposed components outperform baseline models on three static and temporal benchmarks. |
Revisiting Pathologies of Neural Models under Input Reduction (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that modern neural models tend to be miscalibrated. |
| Approach: | They examine why models produce high-confidence predictions on inputs that appear nonsensical to humans . previous work suggested that models fail to assign low probabilities due to model overconfidence . |
| Outcome: | The proposed methods can be extended to reduce the number of examples but with the cost of miscalibration. |
Lego-MT: Learning Detachable Models for Massively Multilingual Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing monolithic models for multilingual neural machine translation encounter parameter interference and inefficient inference for large models. |
| Approach: | They propose a detachable multi-way model that assigns each language to an individual branch . they use data from OPUS to build a translation benchmark covering 433 languages . |
| Outcome: | The proposed model outperforms existing models in OPUS and is faster than existing models. |
FiDO: Fusion-in-Decoder optimized for stronger performance and faster inference (2023.findings-acl)
Copied to clipboard
Michiel de Jong, Yury Zemlyanskiy, Joshua Ainslie, Nicholas FitzGerald, Sumit Sanghai, Fei Sha, William Cohen
| Challenge: | Fusion-in-Decoder (FiD) is a powerful retrieval-augmented language model . however, the architecture used for FiD was not designed for retrieval augmented models . |
| Approach: | They propose to make FiD a modified retrieval-augmented language model with a large decoder and memory bandwidth constraints to alleviate memory bandwidth limitations. |
| Outcome: | The proposed architecture outperforms existing models on knowledge-intensive tasks even on large models on many knowledge-based tasks. |
Detecting Edit Failures In Large Language Models: An Improved Specificity Benchmark (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent model editing techniques can introduce large unwanted side effects, a new study shows . existing specificity benchmarks do not detect these unwanted side-effects . a recent study shows that model edits can cause significant performance drop . |
| Approach: | They extend existing CounterFact benchmark to include a dynamic component and propose a new benchmark to evaluate model editing techniques. |
| Outcome: | The proposed benchmark improves existing benchmarks for specificity and avoids unwanted side effects. |
Structure-Aware Language Model Pretraining Improves Dense Retrieval on Structured Data (2023.findings-acl)
Copied to clipboard
| Challenge: | Structure Aware Dense Retrieval (SANTA) model encodes user queries and structured data in one universal embedding space for retrieving structured data. |
| Approach: | They propose to use structured data and unstructured data to encode queries and structured data in one universal embedding space for retrieving structured data. |
| Outcome: | The proposed model achieves state-of-the-art on code search and product search and conducts convincing results in the zero-shot setting. |
Few-shot Joint Multimodal Aspect-Sentiment Analysis Based on Generative Multimodal Prompt (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies require massive labeled data to train models for multimodal data analysis. |
| Approach: | They propose a novel multimodal prompt model that captures specific aspect terms in a few-shot scenario. |
| Outcome: | The proposed model outperforms baselines on two MABSA-related tasks on a few-shot dataset. |
Predicting Human Translation Difficulty Using Automatic Word Alignment (2023.findings-acl)
Copied to clipboard
| Challenge: | Translation difficulty is a problem when translators are required to resolve translation ambiguity from multiple possible translations. |
| Approach: | They use word alignments computed over large scale bilingual corpora to develop predictors of lexical translation difficulty. |
| Outcome: | The proposed method improves on a previous embedding-based approach and can contribute to a deeper understanding of cross-lingual differences and of causes of translation difficulty. |
Know Where You’re Going: Meta-Learning for Parameter-Efficient Fine-Tuning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on parameter-efficient fine-tuning methods require additional measures after pre-training and before fine-uning. |
| Approach: | They propose to take parameter-efficient fine-tuning into consideration after pre-training and before fine-uning and use meta-learning to prime a model specifically for parameter-efficiency. |
| Outcome: | The proposed method improves on a pre-trained model with certain modifications and achieves 4.96 points on cross-lingual NER fine-tuning. |
Moving Beyond Downstream Task Accuracy for Information Retrieval Benchmarking (2023.findings-acl)
Copied to clipboard
Keshav Santhanam, Jon Saad-Falcon, Martin Franz, Omar Khattab, Avi Sil, Radu Florian, Md Arafat Sultan, Salim Roukos, Matei Zaharia, Christopher Potts
| Challenge: | Neural information retrieval (IR) systems have progressed rapidly in recent years . many IR benchmarks focus on downstream task accuracy, concealing costs incurred . |
| Approach: | They propose to include efficiency considerations on IR benchmarks to help drive progress . eral et al. propose to incorporate query latency and cost budgets into evaluation . |
| Outcome: | a new study shows that the best IR system varies according to how efficiency considerations are chosen and weighed . the proposed benchmarks would allow for more thorough exploration of possible system designs . |
AxomiyaBERTa: A Phonologically-aware Transformer Model for Assamese (2023.findings-acl)
Copied to clipboard
| Challenge: | AxomiyaBERTa is a novel BERT model for low-resource languages . Transformers require extensive computing resources and suffer in low-compute settings . |
| Approach: | They propose a novel BERT model for Assamese, a morphologically-rich low-resource language of eastern India that is trained on a simple masked language modeling task without the NSP objective. |
| Outcome: | The proposed model performs well on token-level tasks and on “longer context” tasks with the aid of embedding disperser and phonological signals. |
An Exploratory Study on Model Compression for Text-to-SQL (2023.findings-acl)
Copied to clipboard
| Challenge: | Text-to-SQL translates user queries into SQL statements that can retrieve relevant answers from relational databases. |
| Approach: | They propose to apply model compression techniques to sketch-based and sequence-to-sequence Text-toSQL models. |
| Outcome: | The proposed models have higher inference efficiency and respond better to model compression than sequence-to-sequence models. |
FluentSpeech: Stutter-Oriented Automatic Speech Editing with Context-Aware Diffusion Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for speech editing still suffer from over-smoothing problem and lack of robustness due to stutter. |
| Approach: | They propose a stutter-oriented automatic speech editing model that incorporates sutter information into the hidden sequence. |
| Outcome: | The proposed model achieves state-of-the-art performance on a speech recording dataset . it can improve fluency of stuttering speech in terms of objective and subjective metrics. |
HyHTM: Hyperbolic Geometry-based Hierarchical Topic Model (2023.findings-acl)
Copied to clipboard
| Challenge: | Hierarchical Topic Models (HTMs) often produce hierarchies where lower-level topics are unrelated and not specific enough to their higher-level subjects. |
| Approach: | They propose a Hyperbolic geometry-based Hierarchical Topic Model that incorporates hierarchical information from hyperbolic geometrics to explicitly model hierarchies in topic models. |
| Outcome: | The proposed model is significantly faster and leaves a much smaller memory footprint than the best-performing baseline. |
KoRC: Knowledge Oriented Reading Comprehension Benchmark for Deep Text Understanding (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for deep text understanding have encountered two major limitations . most require human annotation of knowledge, which leads to limited knowledge coverage . |
| Approach: | They propose a benchmark to help readers understand a document with prior knowledge . they use massive knowledge bases to guide annotators and large language models to construct knowledgable questions . |
| Outcome: | The proposed benchmarks have limited knowledge coverage and use choices or spans as answers, which results in narrow answer space. |
DKAF: KB Arbitration for Learning Task-Oriented Dialog Systems with Dialog-KB Inconsistencies (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches for learning task-oriented dialog agents assume the KB snapshot is current during training. |
| Approach: | They propose a dialog-KB arbitration framework which predicts the contemporary KB snapshot for each train dialog. |
| Outcome: | The proposed model performs better on two publicly available dialog datasets than baselines on both datasets. |
Scale-Invariant Infinite Hierarchical Topic Model (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing hierarchical topic models yield fragmented topics with overlapping themes whose expected probability becomes exponentially smaller along the depth of the tree. |
| Approach: | They propose a hierarchical infinite hierarchic topic model that adapts to topic creation to make expected topic probability decay considerably slower than existing models. |
| Outcome: | The proposed model has better topic uniqueness and hierarchical diversity than existing approaches. |
RC3: Regularized Contrastive Cross-lingual Cross-modal Pre-training (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing V&L pre-training methods rely on strictly-aligned multilingual image-text pairs generated from English-centric datasets. |
| Approach: | They propose a regularized cross-lingual visual contrastive learning objective that constrains representation proximity of weakly-aligned multilingual image-text pairs. |
| Outcome: | The proposed model outperforms competing models with weak zero-shot capability on 5 multi-modal tasks across 6 languages. |
Deep Equilibrium Non-Autoregressive Sequence Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | et al., 2017) is the most prevailing neural architecture for sequence-to-sequence learning. |
| Approach: | They propose to solve for the equilibrium state of NAR models with black-box root-finding solvers and back-propagate through the equilibrium point via implicit differentiation with constant memory. |
| Outcome: | The proposed framework can converge to a more accurate prediction on four WMT benchmarks. |
ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrieval (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies show that large pretrained language models can generate training data with no task-specific or cross-task data. |
| Approach: | They propose a retrieval-enhanced framework to create training data from a general-domain unlabeled corpus. |
| Outcome: | The proposed framework achieves 4.3% gain over baselines and saves 70% of time compared with baselines using large language models. |
Race, Gender, and Age Biases in Biomedical Masked Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models can be used to identify and eliminate healthcare disparities. |
| Approach: | They examine social biases present in biomedical masked language models . they curate prompts based on evidence-based practice and compare generated diagnoses . |
| Outcome: | The proposed models are less biased than BERT in gender, while the opposite is true for race and age. |
Neighboring Words Affect Human Interpretation of Saliency Explanations (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies found that superficial factors such as word length can distort human interpretation of the communicated saliency scores. |
| Approach: | They conduct a user study to examine how the marking of a word’s *neighboring words* affect the explainee’s perception of the word’ s importance in the context of . a saliency explanation. |
| Outcome: | The findings question whether text-based saliency explanations should continue to be communicated at word level and inform future research on alternative methods. |
HELP ME THINK: A Simple Prompting Strategy for Non-experts to Create Customized Content with Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing prompting techniques for providing control are task-specific and lack generality; this limits their adoption among non-expert users. |
| Approach: | They propose a prompting strategy that encourages large language models to help non-expert users by asking relevant questions and leveraging user answers to execute a task. |
| Outcome: | The proposed prompting strategy is able to help non-expert users with a variety of tasks. |
Decker: Double Check with Heterogeneous Knowledge for Commonsense Fact Verification (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies focus on grasping unstructured evidence or potential reasoning paths from structured knowledge bases, yet failing to exploit the benefits of heterogeneous knowledge simultaneously. |
| Approach: | They propose a commonsense fact verification model that bridging heterogeneous knowledge by uncovering latent relationships between structured and unstructured knowledge. |
| Outcome: | The proposed model can bridge heterogeneous knowledge by uncovering latent relationships between structured and unstructured knowledge. |
DopplerBAS: Binaural Audio Synthesis Addressing Doppler Effect (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for binaural audio synthesis are limited in phase estimation, which is crucial for spatial hearing. |
| Approach: | They propose a method to explicitly address the Doppler effect of the moving speaker . it calculates the radial relative velocity of the speaker in spherical coordinates . |
| Outcome: | The proposed method improves the representative WarpNet and BinauralGrad backbones in phase error metric and reaches a new state of the art (SOTA) it is compared with the current method which is limited in phase estimation . |
Easy-to-Hard Learning for Information Extraction (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models for information extraction (IE) use a one-stage learning strategy to extract the target structure from unstructured text data. |
| Approach: | They propose a unified easy-to-hard learning framework that mimics the human learning process by breaking down the learning process into multiple stages. |
| Outcome: | The proposed framework enables the model to acquire general IE task knowledge and improve its generalization ability on 13 out of 17 datasets. |
SConE: Simplified Cone Embeddings with Symbolic Operators for Complex Logical Queries (2023.findings-acl)
Copied to clipboard
| Challenge: | Current geometric-based methods depend on the neural approach to model FOL operators . empirical evidence for explainability is challenging . |
| Approach: | They propose to model conjunction operators using a symbolic modeling approach . they propose to emphasize the essential role of relation projection operator . |
| Outcome: | The proposed method improves answering complex logical queries over previous models. |
Two Heads Are Better Than One: Improving Fake News Video Detection by Correlating with Neighbors (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing frameworks for detecting fake news videos are limited . a new approach is proposed to integrate neighborhood information of new videos . |
| Approach: | They propose a framework for automatically detecting fake news videos . it integrates neighborhood relationship of new videos belonging to same event . |
| Outcome: | The proposed framework improves performance of existing detectors and graph aggregation and debunking rectification modules. |
An Annotated Dataset for Explainable Interpersonal Risk Factors of Mental Disturbance in Social Media Posts (2023.findings-acl)
Copied to clipboard
| Challenge: | 1.6 million people in England are on waiting lists for mental health care . 8 million people are not considered sick enough to qualify for help . |
| Approach: | They construct and release an annotated dataset with human-labelled explanations and classification of IRF affecting mental disturbance on social media: TBe, PBu, and PBe. |
| Outcome: | The proposed model can be used to identify TBe and PBU in emotional spectrum of user’s historical social media profile. |
Nano: Nested Human-in-the-Loop Reward Learning for Few-shot Language Model Control (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for controlling the distribution of generated text only work with quantified distributions, which require pre-defined categories, proportions of the distribution, or an existing corpus following the desired distributions. |
| Approach: | They propose a few-shot human-in-the-loop training algorithm that allows distribution control for text generation via human feedback. |
| Outcome: | The proposed algorithm achieves state-of-the-art results on single topic/attribute and quantified distribution control compared to previous works. |
Connectivity Patterns are Task Embeddings (2023.findings-acl)
Copied to clipboard
Zhiheng Xi, Rui Zheng, Yuansen Zhang, Xuanjing Huang, Zhongyu Wei, Minlong Peng, Mingming Sun, Qi Zhang, Tao Gui
| Challenge: | Existing methods for predicting inter-task transferability are sparse and task-specific. |
| Approach: | They propose a method that uses connectivity patterns of neurons as a unique identifier associated with a task. |
| Outcome: | The proposed method outperforms baselines in predicting inter-task transferability across data regimes and transfer settings while keeping high efficiency in computation and storage. |
Improving Autoregressive Grammatical Error Correction with Non-autoregressive Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Autoregressive models assign low probabilities to tokens that need corrections . grammatical error correction (GEC) is widely applied to natural language processing tasks . |
| Approach: | They propose to use a non-autoregressive model as an auxiliary model to train GEC models to correct grammatical errors in sentences. |
| Outcome: | The proposed method outperforms baselines on English and Chinese GEC tasks significantly. |
SamToNe: Improving Contrastive Loss for Dual Encoder Retrieval Models with Same Tower Negatives (2023.findings-acl)
Copied to clipboard
| Challenge: | Dual encoders have been used for retrieval tasks and representation learning with good results. |
| Approach: | They propose an improved contrastive learning objective by adding queries or documents from the same encoder towers to the negatives. |
| Outcome: | The proposed model improves retrieval quality for both symmetric and asymmetric dual encoders by adding queries or documents from the same encoder towers to the negatives. |
On the Strength of Sequence Labeling and Generative Models for Aspect Sentiment Triplet Extraction (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing generative models ignore the mutual informative clues between aspect and opinion terms and may generate false paired triplets. |
| Approach: | They propose a generative model that encodes aspect and opinion into two bidirectional templates and introduces a marker-oriented sequence labeling module to improve models’ ability of tackling complex structures. |
| Outcome: | The proposed model captures the boundary information of aspect/opinion spans and provides hints to decode multiple triplets with the shared marker. |
Revisiting Non-Autoregressive Translation at Scale (2023.findings-acl)
Copied to clipboard
| Challenge: | Extensive experiments on two advanced NAT models show scaling can improve translation performance. |
| Approach: | They empirically examine the impact of scaling on NAT behaviors on a large-scale WMT dataset. |
| Outcome: | The proposed model can achieve comparable performance with the scaling model while maintaining the superiority of decoding speed with standard NAT models. |
Improving Radiology Summarization with Radiograph and Anatomy Prompts (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies focus on automatic impression generation, but this task is time-consuming and in high demand. |
| Approach: | They propose to use an anatomy-enhanced multimodal model to generate automatic impressions by combining radiology images with textual features. |
| Outcome: | The proposed model achieves state-of-the-art on two benchmark datasets and compares with existing models. |
Explanation Regeneration via Information Bottleneck (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work builds on prompt engineering to generate free-text explanations without specific training, but they lack sufficiency and conciseness due to the prompt complexity and hallucination issues. |
| Approach: | They propose to generate explanations via the information bottleneck theory by polishing the single-pass output of large pretrained language models but retaining the information that supports the contents being explained by balancing two information bottle neck objectives. |
| Outcome: | The proposed explanations are based on the information bottleneck theory . they are able to explain black-box predictions naturally and accurately . |
Improving Zero-shot Multilingual Neural Machine Translation by Leveraging Cross-lingual Consistency Regularization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to improve zero-shot translation performance by learning language-agnostic representations and maximizing cross-lingual transfer have been proposed. |
| Approach: | They propose a cross-lingual consistency regularization to bridge the representation gap between different languages and boost zero-shot translation performance. |
| Outcome: | The proposed model improves translation performance on low-resource and high-res benchmarks and closes the sentence representation gap and aligns the representation space. |
ReactIE: Enhancing Chemical Reaction Extraction with Weak Supervision (2023.findings-acl)
Copied to clipboard
| Challenge: | Structured chemical reaction information is a vital tool for chemists engaged in laboratory work and advanced endeavors such as computer-aided drug design. |
| Approach: | They propose a method which utilizes frequent patterns within the text as linguistic cues to identify specific characteristics of chemical reactions. |
| Outcome: | The proposed model outperforms baselines and outperformed existing models. |
Expand, Rerank, and Retrieve: Query Reranking for Open-Domain Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | Empirically, EAR improves top-5/20 accuracy by 3-8 and 5-10 points . dense retrievers are limited by their inability to perform semantic matching for relevant passages that have low lexical overlap with the query. |
| Approach: | They propose a query expansion and reranking approach for improving passage retrieval with the application to open-domain question answering. |
| Outcome: | Empirically, EAR improves top-5/20 accuracy by 3-8 and 5-10 points when compared to a vanilla query expansion model and a dense retrieval model. |
Neural Networks Against (and For) Self-Training: Classification with Small Labeled and Large Unlabeled Sets (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models for text classification suffer from the semantic drift problem, which is a problem for self-training. |
| Approach: | They propose a semi-supervised text classifier based on self-training using one positive and one negative property of neural networks. |
| Outcome: | The proposed model outperforms ten baseline models in five benchmarks and is additive to language model pretraining. |
Inducing Character-level Structure in Subword-based Language Models with Type-level Interchange Intervention Training (2023.findings-acl)
Copied to clipboard
| Challenge: | Language models operating on subword units are challenging for character-level manipulations, authors say . authors develop a framework to learn robust character representations inside subword-based models . |
| Approach: | They propose a causal intervention framework to learn robust character representations inside subword-based language models. |
| Outcome: | The proposed model outperforms character-level models on more complex tasks . it improves robustness on unseen token sequences and leads to human-interpretable representations of characters. |
Efficient Document Embeddings via Self-Contrastive Bregman Divergence Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Despite recent advances in transformer-based sentence encoders, the encoding of long documents (Ks of words) is still challenging with respect to both efficiency and quality considerations. |
| Approach: | They propose to combine a self-contrastive siamese network and a convex neural Bregman divergence network to train longfomer-based document encoders using an unsupervised contrastive learning method. |
| Outcome: | The proposed model outperforms baseline models on three long document topic classification tasks from the legal and biomedical domains. |
QAP: A Quantum-Inspired Adaptive-Priority-Learning Model for Multimodal Emotion Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Experimental results show that multimodal emotion recognition is a state-of-the-art technique . textual, visual and acoustic modalities are involved in multimodal video emotion recognition . |
| Approach: | They propose a quantum-inspired adaptive-priority-learning model to address the challenges . they use quantum state to model modal features and Q-attention to integrate three modalities . |
| Outcome: | Experimental results show that QAP improves on previous models. |
Language acquisition: do children and language models follow similar learning stages? (2023.findings-acl)
Copied to clipboard
| Challenge: | During language acquisition, children follow a typical sequence of learning stages, whereby they first learn to categorize phonemes before they develop their lexicon and eventually master complex syntactic structures. |
| Approach: | They train 48 GPT-2 models from scratch and evaluate their syntactic and semantic abilities at each training step using 96 probes curated from the BLiMP, Zorro and BIG-Bench benchmarks. |
| Outcome: | The proposed model exhibits similar learning trajectories to human children aged between 18 months and 6 years. |
The Role of Output Vocabulary in T2T LMs for SPARQL Semantic Parsing (2023.findings-acl)
Copied to clipboard
| Challenge: | Using a knowledge graph question answering task, we replace the entire SPARQL vocabulary with alternate vocabularies. |
| Approach: | They replace the entire SPARQL vocabulary with alternate vocabularies . they find absolute gains in the range of 17% on the GrailQA dataset . |
| Outcome: | The proposed substitutions show that the model performance improves on the GrailQA dataset. |
UniCOQE: Unified Comparative Opinion Quintuple Extraction As A Set (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods decompose the COQE task into multiple subtasks and solve them in a pipeline manner, but ignore the intrinsic connection between subtask and the error propagation among stages. |
| Approach: | They propose a unified generative model that solves COQE in one shot by concatenating all the comparative tuples into a target output sequence. |
| Outcome: | The proposed model significantly outperforms the SOTA method on multiple benchmarks and ablation experiments. |
Response-conditioned Turn-taking Prediction (2023.findings-acl)
Copied to clipboard
| Challenge: | Traditionally, turn-taking is done using a simple silence threshold, but more modern approaches use cues known to be important in human-human turn-shifts. |
| Approach: | They propose a turn-taking and response-ranking model that conditions the end-of-turn prediction on conversation history and what the next speaker wants to say. |
| Outcome: | The proposed model outperforms the baseline model in a variety of metrics. |
A Unified One-Step Solution for Aspect Sentiment Quad Prediction (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing ASQP datasets are small and low-density, hindering technical advancement . et al. (2017): aspect sentiment quad prediction provides a complete aspect-level sentiment structure. |
| Approach: | They propose a one-step solution for Aspect sentiment quad prediction that can detect aspect categories and identify aspectopinion-sentiment triplets simultaneously. |
| Outcome: | The proposed solution can detect aspect categories and identify aspectopinion-sentiment triplets simultaneously. |
On Isotropy, Contextualization and Learning Dynamics of Contrastive-based Sentence Representation Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Incorporating contrastive learning objectives in sentence representation learning has yielded significant improvements on many sentence-level NLP tasks. |
| Approach: | They aim to examine why contrastive learning works for learning sentence-level semantics . they interpret successes through the geometry of the representation shifts based on isotropy . |
| Outcome: | The proposed model improves on many sentence-level NLP tasks, but it is not well understood why it works. |
Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluation (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies show that in-context learning and few-shot fine-tuning can generalize well out-of-domain. |
| Approach: | They compare few-shot fine-tuning and in-context learning for task adaptation . they find that both approaches generalize similarly, but exhibit large variation . |
| Outcome: | The proposed methods outperform in-context learning and few-shot fine-tuning with OPT models of different sizes. |
Common Law Annotations: Investigating the Stability of Dialog System Output Annotations (2023.findings-acl)
Copied to clipboard
Seunggun Lee, Alexandra DeLucia, Nikita Nangia, Praneeth Ganedi, Ryan Guan, Rubing Li, Britney Ngaw, Aditya Singhal, Shalaka Vaidya, Zijun Yuan, Lining Zhang, João Sedoc
| Challenge: | High agreement is often used to show reliability of annotation procedures, but it is insufficient to ensure or reproducibility. |
| Approach: | They propose a protocol that increases Inter-Annotator Agreement among annotators and a standardized and codified protocol that strictly enforces transparency in the annotation process. |
| Outcome: | The proposed protocol ensures transparency in the annotation process, which ensures reproducibility of annotation guidelines. |
HuaSLIM: Human Attention Motivated Shortcut Learning Identification and Mitigation for Large Language models (2023.findings-acl)
Copied to clipboard
| Challenge: | Large language models tend to rely on shortcut features that spuriously correlate with labels for prediction, which weakens their generalization on out-of-distribution samples. |
| Approach: | They propose a human attention guided approach to identifying shortcut learning that encourages the LLM-based target model to learn relevant features by exploring both human and neural attention. |
| Outcome: | The proposed approach improves the robustness of large language models on out-of-distribution (OOD) samples while not affecting the performance on IID data. |
PMI-Align: Word Alignment With Point-Wise Mutual Information Without Requiring Parallel Training Data (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies show that using contextualized embeddings from pre-trained multilingual language models could give us high quality word alignments without the need of parallel training data. |
| Approach: | They propose a method which uses contextualized embeddings from pre-trained language models to extract word alignments without parallel training. |
| Outcome: | The proposed method outperforms rival methods on five out of six language pairs. |
Exploring Non-Verbal Predicates in Semantic Role Labeling: Challenges and Opportunities (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing systems for SRL are incapable of transferring knowledge across different predicate types. |
| Approach: | They propose a new PropBank dataset which boasts wide coverage of multiple predicate types and a manually-annotated challenge set which gives equal importance to verbal, nominal, and adjectival predicates. |
| Outcome: | The proposed dataset shows that standard benchmarks do not provide an accurate picture of the current situation in SRL and that state-of-the-art systems are still incapable of transferring knowledge across different predicate types. |
DSPM-NLG: A Dual Supervised Pre-trained Model for Few-shot Natural Language Generation in Task-oriented Dialogue System (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models for few-shot natural language generation are based on a dual correlation between NLG and SLU from the perspective of probability. |
| Approach: | They propose a dual supervised pre-trained model to regularize the pre-training process . they use a probabilistic approach to learn the dual correlation between NLG and SLU . |
| Outcome: | The proposed model outperforms the previous state-of-the-art models on a few-shot dataset. |
TEPrompt: Task Enlightenment Prompt Learning for Implicit Discourse Relation Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing prompt learning models for IDRR use multiple-prompt decisions from three different yet much similar connective prediction templates. |
| Approach: | They propose to fuse three related tasks to fuse the learned features of auxiliary tasks to create a prompt learning model that can be used to boost the main task. |
| Outcome: | The proposed model outperforms the ConnPrompt in the training phase and in the testing phase. |
Evaluating Factuality in Cross-lingual Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluation metrics for monolingual summarization require translation to evaluate the factuality of cross-lingual summmarization. |
| Approach: | They propose to analyze cross-lingual factuality by collecting annotations and generated summaries from models at summary level and sentence level. |
| Outcome: | The proposed dataset shows that over 50% of generated summaries contain factual errors with different characteristics from monolingual summarization. |
On the Correspondence between Compositionality and Imitation in Emergent Neural Communication (2023.findings-acl)
Copied to clipboard
| Challenge: | a study examining compositionality and imitation learning in a Lewis game demonstrates that it is difficult to imitate compositional languages. |
| Approach: | They explore the link between compositionality and imitation in a Lewis game . they show that the learning algorithm used to imitate is crucial . |
| Outcome: | The proposed model improves compositionality and imitation in a Lewis game . the study shows that compositional languages are easier to imitate . |
The Coreference under Transformation Labeling Dataset: Entity Tracking in Procedural Texts Using Event Models (2023.findings-acl)
Copied to clipboard
| Challenge: | et al., 2023) show that entity coreference resolution is improved when events bring about changes in entities that are not reflected in text mentions. |
| Approach: | They propose to perform transformation-based entity linking prior to coreference relation identification to improve entity coreference. |
| Outcome: | The proposed model improves coreference resolution of entities mentioned under a process-oriented model of events. |
Why Does Zero-Shot Cross-Lingual Generation Fail? An Explanation and a Solution (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on cross-lingual transferability of multilingual LMs show that they can perform tasks in low-resource languages. |
| Approach: | They propose a method to regularize the model from learning language invariant representations and a way to select model checkpoints without a development set in the target language. |
| Outcome: | The proposed method reduces the accidental translation problem by 68% and improves the ROUGE-L score by 1.5 on average. |
Distractor Generation based on Text2Text Language Models with Pseudo Kullback-Leibler Divergence Regulation (2023.findings-acl)
Copied to clipboard
Hui-Juan Wang, Kai-Yu Hsieh, Han-Cheng Yu, Jui-Ching Tsou, Yu An Shih, Chen-Hua Huang, Yao-Chung Fan
| Challenge: | Existing methods for cloze-style multiple choice questions (MCQs) distractor generation are based on knowledge bases and pre-trained language models. |
| Approach: | They propose to formulate cloze distractor generation task as Text2Text task and propose a pseudo Kullback-Leibler divergence for regulating the generation to consider item discrimination index in education evaluation. |
| Outcome: | The proposed model improves state-of-the-art performance from 10.81 to 22.00 (p@1 score) |
Lexical Translation Inconsistency-Aware Document-Level Translation Repair (2023.findings-acl)
Copied to clipboard
| Challenge: | Experimental results show document-level translation repair improves translation consistency but still suffers from lexical translation inconsistency due to the lack of inter-sentence context. |
| Approach: | They propose a document-level translation repair model to model translation inconsistency via automatic post-editing. |
| Outcome: | The proposed model improves translation quality and lexical consistency on document-level translation datasets. |
CausalDialogue: Modeling Utterance-level Causality in Conversations (2023.findings-acl)
Copied to clipboard
| Challenge: | Despite widespread adoption, neural conversation models have yet to exhibit natural chat capabilities with humans . despite their widespread adoption in society, chatbots have yet not shown natural chat capability . |
| Approach: | They propose a causality-enhanced method to enhance the impact of causality at the utterance level in training neural conversation models. |
| Outcome: | The proposed method improves diversity and agility of loss functions and still needs improvement . the proposed method is based on a CausalDialogue dataset . |
Towards Unified Spoken Language Understanding Decoding via Label-aware Compact Linguistics Representations (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for intent detection and slot filling decoders could result in misaligned predictions for both tasks. |
| Approach: | They propose a method that leverages label embeddings to jointly guide the decoding process. |
| Outcome: | The proposed method outperforms existing methods on two single- and multi-intent SLU benchmarks and can be incorporated into existing models. |
Less Likely Brainstorming: Using Language Models to Generate Alternative Hypotheses (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to reduce cognitive errors in MRI interpretations do not work for generating less likely outputs. |
| Approach: | They propose a task that asks a model to generate outputs that humans think are relevant but less likely to happen. |
| Outcome: | The proposed method compares with several state-of-the-art controlled text generation models via automatic and human evaluations and shows that it reduces cognitive errors in interpreting MRI findings. |
Language Modeling with Latent Situations (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches for improving coherence of language models are lacking . effective generation of coherent text requires reasoning about the world that text describes. |
| Approach: | They propose a family of approaches for improving coherence in language models . they use entity states as auxiliary supervision to improve LMs' reasoning ability . |
| Outcome: | The proposed method reduces errors by up to 16% by fine-tuning and prompting . it can be applied to LMs that are pre-trained to explicitly The suitcase is open . |
Can Cross-Lingual Transferability of Multilingual Transformers Be Activated Without End-Task Data? (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for cross-lingual transfer learning cannot activate cross-linguistic transferability when end-task data are unavailable. |
| Approach: | They propose a cross-lingual transfer method that disassembles multilingual Transformers into sub-modules and reassembles them to be the multilingual end-task model. |
| Outcome: | The proposed method activates the cross-lingual transferability of multilingual Transformers without accessing end-task data. |
Focus-aware Response Generation in Inquiry Conversation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on response generation focus on relevance and fluency, rarely paying attention to the focus. |
| Approach: | They propose a Focus-aware response generation method that takes the focus into consideration and optimizes a multi-level encoder and focal decoder to generate multiple candidate responses. |
| Outcome: | The proposed method generates candidate responses that correspond to different focuses and performs better on two orthogonal inquiry conversation datasets. |
A Hierarchical Explanation Generation Method Based on Feature Interaction Detection (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work on hierarchical attributions tends to limit text groups to a continuous text span, which is difficult for humans to read. |
| Approach: | They propose a method which captures feature interactions and converts non-hierarchical explanations into hierarchical versions. |
| Outcome: | The proposed method can convert ubiquitous non-hierarchical explanations into their corresponding hierarchical versions. |
Jointly Reparametrized Multi-Layer Adaptation for Efficient and Private Tuning (2023.findings-acl)
Copied to clipboard
| Challenge: | Efficient finetuning of pretrained language transformers requires a large number of tunable parameters. |
| Approach: | They propose a language transformer finetuning strategy that introduces task-specific parameters in multiple transformer layers. |
| Outcome: | The proposed method outperforms other methods with 4,100 parameters on GLUE tasks with 5% of full finetuning performance. |
A Diffusion Model for Event Skeleton Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for event schema generation are noise-sensitive and error-accumulating, e.g., inability to correct errors while generating schema. |
| Approach: | They propose a novel diffusion event graph model that embeds and roundes event graphs into learnable latent representations and a denoising process to maintain the model's robustness. |
| Outcome: | The proposed model achieves better results than existing state-of-the-art models on three IED bombing datasets. |
Nonparametric Decoding for Generative Retrieval (2023.findings-acl)
Copied to clipboard
Hyunji Lee, JaeYoung Kim, Hoyeon Chang, Hanseok Oh, Sohee Yang, Vladimir Karpukhin, Yi Lu, Minjoon Seo
| Challenge: | Existing text retrieval models depend on the information encoded in its parameters without external memory, its information capacity is limited and fixed. |
| Approach: | They propose a nonparametric decoding approach which uses external memory instead of vanilla vocab embeddings as decoder voka embedds. |
| Outcome: | The proposed model can utilize parametric and nonparametric space. |
Aspect-aware Unsupervised Extractive Opinion Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Extractive opinion summarization extracts sentences from reviews to represent the prevalent opinions about a product or service. |
| Approach: | They propose a method for unsupervised extractive opinion summarization that automatically identifies the aspects described in review sentences and extracts sentences based on their aspects. |
| Outcome: | The proposed method improves aspect coverage and performs well on multiple opinion summarization datasets. |
GNN-SL: Sequence Labeling Based on Nearest Examples via GNN (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing sequence labeling algorithms can be decomposed into two parts . |
| Approach: | They propose a graph neural networks sequence labeling (GNN-SL) that augments the vanilla SL model output with similar tagging examples retrieved from the whole training set. |
| Outcome: | The proposed model performs well on three sequence labeling tasks. |
Serial Contrastive Knowledge Distillation for Continual Few-shot Relation Extraction (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models for few-shot relation extraction (RE) are not suitable for continual few-sshot RE. |
| Approach: | They propose a new model to train a model for new relations with few labeled training data. |
| Outcome: | The proposed model outperforms state-of-the-art models on two benchmark datasets. |
Revisiting the Architectures like Pointer Networks to Efficiently Improve the Next Word Distribution, Summarization Factuality, and Beyond (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing solutions for word probability distributions are limited and the output softmax layer is inherently limited. |
| Approach: | They propose to use the output softmax layer to compute the word probability distribution instead of using pointer networks to break the bottleneck. |
| Outcome: | The proposed method improves factCC score by 2 points in CNN/DM and XSUM dataset, and MAUVE scores by 30% in bookSum paragraph-level dataset. |
GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-Distribution Generalization Perspective (2023.findings-acl)
Copied to clipboard
Linyi Yang, Shuibai Zhang, Libo Qin, Yafu Li, Yidong Wang, Hanmeng Liu, Jindong Wang, Xing Xie, Yue Zhang
| Challenge: | Pre-trained language models (PLMs) have improved generalization performance but the out-of-distribution (OOD) generalization problem remains a challenge in many NLP tasks. |
| Approach: | They propose to create a benchmark for evaluating out-of-distribution (OOD) generalization in NLP models. |
| Outcome: | The proposed benchmarks highlight the importance of OOD robustness and provide insights on how to measure it and improve it. |
Investigating the Saliency of Sentiment Expressions in Aspect-Based Sentiment Analysis (2023.findings-acl)
Copied to clipboard
| Challenge: | In a set of masking experiments, we examine the extent to which the tokens identified as salient by LIME and a gradient-based method are being used by the classifier. |
| Approach: | They use a BERT BASE model to mask the sentiments of an English dataset and find that both methods produce faithful rationales. |
| Outcome: | The proposed classifier outperforms both the gradient-based and black-box saliency methods on the SemEval 2016 english dataset. |
DMLM: Descriptive Masked Language Modeling (2023.findings-acl)
Copied to clipboard
| Challenge: | Descriptive Masked Language Modeling (DMLM) is a knowledge-enhanced reading comprehension objective that requires the model to predict the most likely word in a context, being provided with the word’s definition. |
| Approach: | They propose a knowledge-enhanced reading comprehension objective where the model is required to predict the most likely word in a context, being provided with the word’s definition. |
| Outcome: | The proposed model improves on a number of well-established NLU benchmarks and other semantic-focused tasks, e.g., Semantic Role Labeling. |
Reproducibility in NLP: What Have We Learned from the Checklist? (2023.findings-acl)
Copied to clipboard
| Challenge: | Scientific progress in NLP rests on the reproducibility of researchers’ claims. |
| Approach: | They examine 10,405 anonymous responses to the NLP Reproducibility Checklist . they find evidence of an increase in reporting of information after the Checklist's introduction . |
| Outcome: | The authors find that 44% of submissions that gather new data are 5% less likely to be accepted than those that did not. |
Domain Generalization via Switch Knowledge Distillation for Robust Review Representation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models for review representations of unseen or anonymous users are limited by their in-domain nature. |
| Approach: | They propose to use in-domain user and product information to generalize reviews . they use switch knowledge distillation to learn review representations for unseen users . |
| Outcome: | The proposed model performs well for existing or anonymous unseen users. |
On Search Strategies for Document-Level Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Document-level neural machine translation models produce a more consistent output across a document . however, the exact decoding strategy is often not described and not mentioned at all. |
| Approach: | They propose to use standard automatic metrics and specific linguistic phenomena to compare different decoding schemes. |
| Outcome: | The proposed decoding strategies perform similar to each other on three standard document-level translation benchmarks. |
Causal Intervention for Mitigating Name Bias in Machine Reading Comprehension (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing MRC models may overuse name information to make predictions, causing name bias . |
| Approach: | They propose a Causal Interventional paradigm for MRC to mitigate name bias by analyzing pre-trained knowledge and context representations. |
| Outcome: | The proposed model is robust to names and performs competitively on the original SQuAD. |
Counterfactual Probing for the Influence of Affect and Specificity on Intergroup Bias (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work on bias in NLP only considers negative or pejorative language use. |
| Approach: | They propose a revised framing of bias in terms of intergroup social context and its effects on language output. |
| Outcome: | The proposed framework is based on a model of intergroup relationships in English language tweets. |
SongRewriter: A Chinese Song Rewriting System with Controllable Content and Rhyme Scheme (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods of generating singable lyrics are based on a given melody, but there are two main challenges: generating the lyrics without knowing the melody and composing compatible melodies. |
| Approach: | They propose a Chinese lyric generation and editing system which rewrites lyrics of an existing song such that they are compatible with the rhythm of the existing melody. |
| Outcome: | The proposed system is based on a randomized multi-level masking strategy and can generate new lyrics or edit fragments without prior knowledge of melody composition. |
Triplet-Free Knowledge-Guided Response Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Prior work focused on constructing ”latent” knowledge and learning how to ground it based on pseudo triplets. |
| Approach: | They propose to pretrain a response language model to measure relevance and consistency between any context and response and use search engines to collect top-ranked passages to serve as guiding knowledge without explicitly optimizing the ‘‘best’ latent knowledge. |
| Outcome: | The proposed model pretrains a response language model to measure relevance and consistency between any context and response, then uses search engines to collect the top-ranked passages to serve as the guiding knowledge without explicitly optimizing the ‘‘best’ latent knowledge. |
Implicit Memory Transformer for Computationally Efficient Simultaneous Speech Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Simultaneous speech translation is an essential communication task difficult for humans whereby a translation is generated concurrently with oncoming speech inputs. |
| Approach: | They propose a transformer that implicitly retains memory through a new left context method, removing the need to explicitly represent memory with memory banks. |
| Outcome: | The proposed method provides a substantial speedup on the encoder forward pass with nearly identical translation quality when compared with the state-of-the-art approach that uses left context and memory banks. |
Enhancing Document-level Event Argument Extraction with Contextual Clues and Role Relevance (2023.findings-acl)
Copied to clipboard
| Challenge: | Document-level event argument extraction is a challenging task for cross-sentence inference . previous work focused on document-level EAE, but recent work focused more on documentlevel . |
| Approach: | They propose a document-level event argument extraction model that captures contextual clues and latent role information. |
| Outcome: | The proposed model outperforms existing methods on two public datasets with 1.13 F1 and 2.64 F1 improvements on RAMS and WikiEvents respectively. |
Exploring the Impact of Vision Features in News Image Captioning (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent state-of-art models can achieve competitive performance even without vision features. |
| Approach: | They conduct extensive experiments with mainstream news image captioning models to determine whether vision features contribute to the generation of captions. |
| Outcome: | The proposed models can achieve competitive performance even without vision features. |
Using Collostructional Analysis to evaluate BERT’s representation of linguistic constructions (2023.findings-acl)
Copied to clipboard
| Challenge: | Collostructional analysis is a technique devised to find correlations between words and linguistic constructions in order to analyse meaning associations of these constructions. |
| Approach: | They propose to compare English BERT’s meaning representations to known constructions from the linguistics literature by predicting words that can be used in open slots of constructions and finding similar sequences using BERT's output embeddings. |
| Outcome: | The proposed methods show that English BERT represents constructional meaning to a certain extent, but does not separate instances of a construction from a near-synonymous construction that has a different form. |
Selecting Better Samples from Pre-trained LLMs: A Case Study on Question Generation (2023.findings-acl)
Copied to clipboard
Xingdi Yuan, Tong Wang, Yen-Hsiang Wang, Emery Fine, Rania Abdelghani, Hélène Sauzéon, Pierre-Yves Oudeyer
| Challenge: | Large Language Models (LLMs) have demonstrated impressive prowess in natural language generation. |
| Approach: | They propose a method to select high-quality questions from LLM-generated candidates using round-trip and prompt-based scoring. |
| Outcome: | The proposed approach can select high-quality questions from a set of LLM-generated candidates without modification of the underlying model nor rely on human annotations. |
Sentiment Knowledge Enhanced Self-supervised Learning for Multimodal Sentiment Analysis (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have used general approaches to alleviate the overfitting of supervised models based on video data with sentiment annotations. |
| Approach: | They propose to capture common sentimental patterns in unlabeled videos using sentiment knowledge and non-verbal behavior to embed sentiment information into pre-trained multimodal representations. |
| Outcome: | The proposed model outperforms the baseline and achieves new State-Of-The-Art (SOTA) results. |
Theory of Mind in Freely-Told Children’s Narratives: A Classification Approach (2023.findings-acl)
Copied to clipboard
| Challenge: | Language and Theory of Mind (ToM) competences are often studied with younger children and standardized tests, but as both are social competences, data and methods with higher ecological validity are critical. |
| Approach: | They leveraged a corpus of 442 freely-told stories by Dutch children aged 4-12 to study language and ToM with NLP-tools. |
| Outcome: | The proposed method is robust relative to the complexity of the task for humans and is consistent with previous studies. |
Better Language Models of Code through Self-Improvement (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models for code (PLMCs) are pre-taught on large datasets with multi-modal objectives, but fine-tuning them requires extensive supervision and is limited by the size of the dataset provided. |
| Approach: | They propose a data augmentation framework that utilizes knowledge from the pre-training and fine-tuning stage to augment training data, which is then used for the next step. |
| Outcome: | The proposed framework significantly improves pre-trained language models’ performance in sequence-generation tasks, such as code summarization and code generation in the CodeXGLUE benchmark. |
Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them (2023.findings-acl)
Copied to clipboard
Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, Jason Wei
| Challenge: | Language models have already made good progress on this benchmark, with the best model outperforming average reported human-rater results on 65% of the BIG-Bench tasks. |
| Approach: | They propose to use chain-of-thought prompting to challenge language models on 23 challenging BIG-Bench tasks which they call BIG-Bench Hard. |
| Outcome: | The proposed language models outperform the average human-rater on 65% of the BIG-Bench tasks. |
Score It All Together: A Multi-Task Learning Study on Automatic Scoring of Argumentative Essays (2023.findings-acl)
Copied to clipboard
| Challenge: | a multi-task learning approach outperforms sequential approaches for scoring argumentative essays . segmentation and classification of argumentative elements are important steps towards providing feedback on writing structure, but assessing the quality of arguments is less researched . |
| Approach: | They use a student essay dataset to study how argumentative essays are scored . they use automated span detection, type and quality prediction to combine these tasks . |
| Outcome: | The proposed method outperforms sequential approaches for segmentation and quality prediction. |
Data Sampling and (In)stability in Machine Translation Evaluation (2023.findings-acl)
Copied to clipboard
| Challenge: | a recent data sampling method skews the annotated data toward shorter documents, not necessarily representative of the full test set. |
| Approach: | They examine different approaches to human evaluation and ranking of machine translation systems at the conference on machine translation . they propose a method that uses available labour budget to sample data in a more representative manner . |
| Outcome: | The proposed method improves representation of document lengths and produces stable rankings of translation quality. |
Probing Graph Decomposition for Argument Pair Extraction (2023.findings-acl)
Copied to clipboard
| Challenge: | Argument pair extraction (APE) aims to extract interactive argument pairs from two passages within a discussion. |
| Approach: | They propose a method to extract interactive argument pairs from two passages . they propose to decompose the probing graph into four sub-graphs based on inter- and intra-passage perspectives . |
| Outcome: | The proposed method improves on strong baselines on two benchmark datasets. |
DiffuSum: Generation Enhanced Extractive Summarization with Diffusion (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for extractive summarization are formulated as a sequence labeling problem by making individual sentence label predictions. |
| Approach: | They propose a novel paradigm for extractive summarization by directly generating summary sentences with diffusion models and extracting sentences based on sentence representation matching. |
| Outcome: | The proposed framework achieves state-of-the-art extractive results on CNN/DailyMail with ROUGE scores of 44.83/22.56/40.56. |
Towards Parameter-Efficient Integration of Pre-Trained Language Models In Temporal Video Grounding (2023.findings-acl)
Copied to clipboard
Erica Kido Shimomoto, Edison Marrese-Taylor, Hiroya Takamura, Ichiro Kobayashi, Hideki Nakayama, Yusuke Miyao
| Challenge: | Recent studies have improved query inputs with pre-trained language models, but the effects of this integration are unclear. |
| Approach: | They propose to integrate query sentences with pre-trained language models to train TVG models. |
| Outcome: | The proposed model integrates query sentences with pre-trained language models at cost of more expensive training. |
A Memory Model for Question Answering from Streaming Data Supported by Rehearsal and Anticipation of Coreference Information (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing question answering methods assume that the input content can always be accessed while answering the question. |
| Approach: | They propose a model that performs rehearsal and anticipation while processing inputs to memorize important information for question answering tasks from streaming data. |
| Outcome: | The proposed model improves on short-sequence (bAbI) and large-squence textual (NarrativeQA) and video (ActivityNet-QA) question answering datasets. |
Pay Attention to Implicit Attribute Values: A Multi-modal Generative Framework for AVE Task (2023.findings-acl)
Copied to clipboard
Yupeng Zhang, Shensi Wang, Peiguang Li, Guanting Dong, Sirui Wang, Yunsen Xian, Zhoujun Li, Hongzhi Zhang
| Challenge: | Existing approaches to extract attribute values from product descriptions are incomplete and noisy due to the tedious nature of this task. |
| Approach: | They propose a framework to extract attributes from product descriptions to acquire implicit attributes in addition to the explicit ones. |
| Outcome: | The proposed framework outperforms existing methods on the extraction of implicit attribute values while achieving comparable performance for the explicit ones. |
CoRRPUS: Code-based Structured Prompting for Neurosymbolic Story Understanding (2023.findings-acl)
Copied to clipboard
| Challenge: | Story generation and understanding has seen a surge in neurosymbolic work . symbolic methods are expensive and require a lot of time and expertise . |
| Approach: | They use Code-LLMs to bootstrap the use of symbolic methods for story understanding . they show that they can beat current LLM techniques on pre-existing stories with minimal hand engineering . |
| Outcome: | The proposed system beats state-of-the-art structured LLM techniques on pre-existing story understanding tasks with minimal hand engineering. |
Fighting Bias With Bias: Promoting Model Robustness by Amplifying Dataset Biases (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work sought to develop robust, unbiased models by filtering biased examples from training sets. |
| Approach: | They propose to filter out biased examples from training sets to improve models' performance. |
| Outcome: | The proposed evaluation framework is more challenging than the original dataset splits and even more challenging that hand-crafted challenge sets. |
Context-Aware Document Simplification (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work on document simplification has focused on sentence-level inputs but fails to preserve the discourse structure. |
| Approach: | They explore various systems that use document context within the simplification process . they investigate the performance and efficiency tradeoffs of system variants . |
| Outcome: | The proposed approach achieves state-of-the-art even when not relying on plan-guidance. |
Distinguish Before Answer: Generating Contrastive Explanation as Knowledge for Commonsense Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing knowledge-enhanced methods have trouble obtaining knowledge from different knowledge bases . a concept-centric model can be used to generate a contrastive explanation for QA tasks . |
| Approach: | They propose a Concept-centric Prompt-bAsed Contrastive Explanation Generation model which converts obtained symbolic knowledge into the contrastive explanation for better distinguishing the differences among given candidates. |
| Outcome: | The proposed model achieves new SOTA on CSQA, QASC, and OBQA. |
Abstract then Play: A Skill-centric Reinforcement Learning Framework for Text-based Games (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing reinforcement learning frameworks fail to decompose the task and abstract the action autonomously. |
| Approach: | They propose a skill-centric reinforcement learning framework capable of abstracting the action in an end-to-end manner. |
| Outcome: | Empirical experiments on the Jericho environment validate the proposed framework against state-of-the-art baselines. |
SSP: Self-Supervised Post-training for Conversational Search (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to search for information are limited by data scarcity and lack contextual semantic understanding. |
| Approach: | They propose a new post-training paradigm with three self-supervised tasks to initialize the conversational search model to enhance dialogue structure and contextual semantic understanding. |
| Outcome: | The proposed method can initialize the conversational search model to enhance dialogue structure and contextual semantic understanding. |
Towards Reference-free Text Simplification Evaluation with a BERT Siamese Network Architecture (2023.findings-acl)
Copied to clipboard
| Challenge: | Text simplification (TS) aims to modify sentences to make their content and structure easier to understand. |
| Approach: | They propose a neural-network-based TS metric that uses a human reference to evaluate simplification and meaning preservation. |
| Outcome: | The proposed metric correlates better with human judgments for simplicity and meaning preservation than existing metrics. |
Causal interventions expose implicit situation models for commonsense language understanding (2023.findings-acl)
Copied to clipboard
| Challenge: | Classical psycholinguistic accounts have suggested that world knowledge enters into language understanding through structured schemas called situation models. |
| Approach: | They apply causal intervention techniques to transformer models to analyze performance on the Winograd Schema Challenge . |
| Outcome: | The proposed model performs well on the Winograd Schema Challenge . |
Iterative Nearest Neighbour Machine Translation for Unsupervised Domain Adaptation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for supervised domain adaptation of machine translation focus on fine-tuning, which is non-extensible. |
| Approach: | They propose to perform unsupervised domain adaptation in a non-parametric manner by using in-domain monolingual data and performing nearest neighbour inference on both forward and backward directions. |
| Outcome: | The proposed method significantly improves the in-domain translation performance and achieves state-of-the-art results among non-parametric methods. |
PruMUX: Augmenting Data Multiplexing with Model Compression (2023.findings-acl)
Copied to clipboard
| Challenge: | Prior work has investigated methods like model pruning, knowledge distillation, and data multiplexing to increase model throughput without sacrificing accuracy. |
| Approach: | They propose to combine structured pruning and data multiplexing methods to increase model throughput without sacrificing accuracy. |
| Outcome: | The proposed method achieves 7.5-29.5X throughput improvement over a BERT-base model with accuracy threshold from 80% to 74%. |
With Prejudice to None: A Few-Shot, Multilingual Transfer Learning Approach to Detect Social Bias in Low Resource Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | Currently, the majority of social bias datasets available are in English and this inhibits progress on social bias detection in low-resource languages. |
| Approach: | They propose a dataset for social bias detection in Hindi and investigate multilingual transfer learning using publicly available English, Italian, and Korean datasets. |
| Outcome: | The proposed dataset is compared with a dataset available in English, Italian, and Korean using multilingual models. |
Don’t Lose Yourself! Empathetic Response Generation via Explicit Self-Other Awareness (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing attempts to generate empathy with other-awareness ignore to include self-a awareness to consider the own views of the self in their responses. |
| Approach: | They propose to include self-awareness to consider the own views of the self in empathetic response generation by integrating three stages of self-other awareness into the process. |
| Outcome: | The proposed method is superior to existing methods on the benchmark dataset. |
Are Layout-Infused Language Models Robust to Layout Distribution Shifts? A Case Study with Scientific Documents (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work has shown that infusing layout features into language models improves processing of visually-rich documents such as scientific papers. |
| Approach: | They propose a method to evaluate layout-infused language models that incorporate layout features into their models to emulate layout distribution shifts. |
| Outcome: | The proposed model performs better under layout distribution shifts than in-distribution conditions. |
Enhancing Neural Topic Model with Multi-Level Supervisions from Seed Words (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing topic seed words are difficult to incorporate into topic models due to the semantic diversity of natural language. |
| Approach: | They propose a neural topic model enhanced with supervisions from seed words on word and document levels. |
| Outcome: | The proposed model outperforms the state-of-the-art seeded topic models in terms of topic quality and classification accuracy. |
Learning from Children: Improving Image-Caption Pretraining via Curriculum (2023.findings-acl)
Copied to clipboard
| Challenge: | Image-caption pretraining is a difficult problem as it requires multiple concepts (nouns) from captions to be aligned to multiple objects in images. |
| Approach: | They propose a curriculum learning framework that uses images to align multiple concepts to multiple objects in an image. |
| Outcome: | The proposed learning framework improves over pretraining from scratch, using a pretrained image or/and text encoder, low data regime etc. |
Discovering Language Model Behaviors with Model-Written Evaluations (2023.findings-acl)
Copied to clipboard
Ethan Perez, Sam Ringer, Kamile Lukosiute, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, Andy Jones, Anna Chen, Benjamin Mann, Brian Israel, Bryan Seethor, Cameron McKinnon, Christopher Olah, Da Yan, Daniela Amodei, Dario Amodei, Dawn Drain, Dustin Li, Eli Tran-Johnson, Guro Khundadze, Jackson Kernion, James Landis, Jamie Kerr, Jared Mueller, Jeeyoon Hyun, Joshua Landau, Kamal Ndousse, Landon Goldberg, Liane Lovitt, Martin Lucas, Michael Sellitto, Miranda Zhang, Neerav Kingsland, Nelson Elhage, Nicholas Joseph, Noemi Mercado, Nova DasSarma, Oliver Rausch, Robin Larson, Sam McCandlish, Scott Johnston, Shauna Kravec, Sheer El Showk, Tamera Lanham, Timothy Telleen-Lawton, Tom Brown, Tom Henighan, Tristan Hume, Yuntao Bai, Zac Hatfield-Dodds, Jack Clark, Samuel R. Bowman, Amanda Askell, Roger Grosse, Danny Hernandez, Deep Ganguli, Evan Hubinger, Nicholas Schiefer, Jared Kaplan
| Challenge: | Prior work creates evaluations with crowdwork or existing data sources, which are not always available. |
| Approach: | They generate evaluations automatically with language models (LMs) using crowdwork or existing data sources to find out how they behave . |
| Outcome: | The results show that large LMs repeat back a dialog user’s preferred answer and express greater desire to pursue concerning goals like resource acquisition and goal preservation. |
Cross-Domain Argument Quality Estimation (2023.findings-acl)
Copied to clipboard
| Challenge: | Argument mining is a field of automated discovery and organization of arguments. |
| Approach: | They propose to generalize argument quality estimation from multiple angles by combining empirical results with a training part. |
| Outcome: | The proposed method combines the results of two empirical evaluations with a training part to show that argument quality is among the more challenging tasks but can improve others. |
DiaASQ: A Benchmark of Conversational Aspect-based Sentiment Quadruple Analysis (2023.findings-acl)
Copied to clipboard
Bobo Li, Hao Fei, Fei Li, Yuhan Wu, Jinsong Zhang, Shengqiong Wu, Jingye Li, Yijiang Liu, Lizi Liao, Tat-Seng Chua, Donghong Ji
| Challenge: | a new task of conversational aspect-based sentiment analysis (DiaASQ) is designed to detect the quadruple of target-aspect-opinion-sentiment in a dialogue. |
| Approach: | They propose a task of conversational aspect-based sentiment quadruple analysis to detect the quadrangle of target-aspect-opinion-sentiment in a dialogue. |
| Outcome: | The proposed task is based on a high-quality dataset in Chinese and English . it improves the end-to-end quadruple prediction and integrates rich feature representations . |
GeoDRL: A Self-Learning Framework for Geometry Problem Solving using Reinforcement Learning in Deductive Reasoning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for automated geometry problem solving lack labeled data. |
| Approach: | They propose a framework that integrates logic graph deduction and deep reinforcement learning to optimize geometry reasoning as a Markov Decision Process. |
| Outcome: | The proposed framework improves accuracy and interpretability in the Geometry3K dataset while maintaining correctness. |
Uncertainty-Aware Unlikelihood Learning Improves Generative Aspect Sentiment Quad Prediction (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies focus on what to generate but ignore what not to generate . a template-agnostic method boosts original learning and reduces mistakes simultaneously . |
| Approach: | They propose a template-agnostic method to control the token-level generation . they introduce Monte Carlo dropout to understand the built-in uncertainty of pre-trained language models . |
| Outcome: | The proposed method boosts original learning and reduces mistakes simultaneously on four public datasets. |
Adversarial Knowledge Stimulated Contrastive Prompting for Few-shot Language Learners (2023.findings-acl)
Copied to clipboard
| Challenge: | Prompt-based fine-tuning has boosted performance of Pre-trained language models on few-shot Natural Language Understanding (NLU) tasks by employing task-specific prompts. |
| Approach: | They propose a Cloze-driven prompt framework for prompt tuning that implicitly stimulates knowledge from pre-trained language models. |
| Outcome: | The proposed framework outperforms state-of-the-art for prompt-based fine-tuning on few-shot NLU tasks. |
Making Pre-trained Language Models Better Learn Few-Shot Spoken Language Understanding in More Practical Scenarios (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing few-shot Spoken Language Understanding models need to be trained on a set of data-rich source domains and adapt to the target domain with a few examples. |
| Approach: | They propose a scenario where only a pre-trained language model and a few labeled examples are used to train few-shot SLU models. |
| Outcome: | The proposed model outperforms existing models on few-shot settings by reducing the number of slot labels and reducing training complexity. |
Typology Guided Multilingual Position Representations: Case on Dependency Parsing (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent multilingual models benefit from strong unified semantic representation models, but conflicting linguistic regularities may break the effectiveness of word position features in multilingual learning. |
| Approach: | They propose to combine prior knowledge from typology features and existing position vectors to create a position generation network which combines prior knowledge of a language's position space and typological characterization. |
| Outcome: | The proposed model can achieve the best multilingual parsing results by combining prior knowledge from typology features and existing position vectors. |
Learning Event-aware Measures for Event Coreference Resolution (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models for event coreference resolution are based on entity-level tasks, but event coreferent resolution is a challenge. |
| Approach: | They propose a model that learns and integrates multiple representations from event alone and event pair on the basis of event but not entity as before. |
| Outcome: | The proposed model achieves new state-of-the-art on the ACE 2005 benchmark, demonstrating the effectiveness of the proposed framework. |
Second Language Acquisition of Neural Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | a recent study examined the cross-lingual transferability of neural language models . previous studies focused on their first language acquisition . |
| Approach: | They propose to pretrain bilingual LMs with a scenario similar to human L2 acquisition . they find that pretraining accelerated their linguistic generalization in L2 . |
| Outcome: | The results show that pretraining bilingual LMs accelerates their linguistic generalizations . the results clarify their (non-)human-like L2 acquisition in particular aspects . |
On the Universal Adversarial Perturbations for Efficient Data-free Adversarial Detection (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing adversarial detection methods require access to training data, which brings noteworthy concerns regarding privacy leakage and generalizability. |
| Approach: | They propose a data-agnostic adversarial detection framework which induces different responses between normal and adversarials to UAPs. |
| Outcome: | The proposed framework achieves competitive detection performance on various text classification tasks, and maintains equivalent time consumption to normal inference. |
Exploring the Effectiveness of Prompt Engineering for Legal Reasoning Tasks (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that Chain-of-Thought (CoT) prompts improve tasks such as arithmetic and common-sense reasoning. |
| Approach: | They evaluate CoT prompts and various prompting strategies for legal reasoning tasks . they find that the best results are achieved with prompts derived from specific legal reasoning techniques . |
| Outcome: | The proposed approaches improve the COLIEE entailment task on the Japanese bar exam . the proposed approaches surpass the best system from 2022 with an accuracy of 0.789 . |
End-to-end Aspect-based Sentiment Analysis with Combinatory Categorial Grammar (2023.findings-acl)
Copied to clipboard
| Challenge: | End-to-end aspect-based sentiment analysis (EASA) is a natural language processing task that requires a deep understanding of the running text. |
| Approach: | They propose a method to improve EASA with CCG supertags that carry syntactic and semantic information of the associated words. |
| Outcome: | The proposed approach outperforms baselines and achieves state-of-the-art results on all datasets. |
ConKI: Contrastive Knowledge Injection for Multimodal Sentiment Analysis (2023.findings-acl)
Copied to clipboard
Yakun Yu, Mingjun Zhao, Shi-ang Qi, Feiran Sun, Baoxun Wang, Weidong Guo, Xiaoli Wang, Lei Yang, Di Niu
| Challenge: | Existing methods for multimodal sentiment analysis focus on general knowledge, which is inadequate to identify specific sentiments across modalities. |
| Approach: | They propose a method where specific-knowledge representations for each modality can be learned together with general knowledge representations via knowledge injection based on an adapter architecture. |
| Outcome: | The proposed method outperforms all prior methods on three popular benchmarks on multimodal sentiment analysis metrics. |
On Degrees of Freedom in Defining and Testing Natural Language Understanding (2023.findings-acl)
Copied to clipboard
| Challenge: | Large-scale pretrained language models are performing increasingly well at various tasks and offering real-world applications. |
| Approach: | They propose to define NLU as an inductive evidence that the test subject understands the language sufficiently well to meet stakeholder objectives. |
| Outcome: | The proposed framework can be used to design credible tests and facilitate scientific communication. |
AttenWalker: Unsupervised Long-Document Question Answering via Attention-based Graph Walking (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for annotating long-document question answering are based on short documents and can hardly incorporate long-range information. |
| Approach: | They propose an unsupervised method to generate long-document question answering pairs . they propose a method to aggregate and generate answers with long-range dependency . |
| Outcome: | The proposed method outperforms existing methods on NarrativeQA and Qasper. |
Adaptive Ordered Information Extraction with Deep Reinforcement Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for information extraction follow a fixed extraction order for complex tasks with multiple elements to be extracted in one instance. |
| Approach: | They propose an adaptive ordered IE paradigm to find optimal element extraction order for different instances and a reinforcement learning framework to generate optimal order dynamically. |
| Outcome: | The proposed method beats existing methods and improves on several public datasets. |
Wasserstein-Fisher-Rao Embedding: Logical Query Embeddings with Local Comparison and Global Transport (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for answering complex queries on knowledge graphs lack a local and global scoring function. |
| Approach: | They propose a convolution-based algorithm for linear time computation and a block diagonal kernel to enforce the trade-off between local and global embeddings. |
| Outcome: | The proposed model outperforms existing methods on standard datasets, evaluation sets with combinatorially complex queries, and hierarchical knowledge graphs. |
RISE: Leveraging Retrieval Techniques for Summarization Evaluation (2023.findings-acl)
Copied to clipboard
| Challenge: | Summarization evaluation approaches have relied on ROUGE for summarization, but they fall short of human evaluations. |
| Approach: | They propose a new approach to evaluate summaries by leveraging retrieval techniques . they use a dual-encoder retrieval setup to train a retrieval task . |
| Outcome: | The proposed method outperforms existing methods on two document summarization benchmarks and a long document summmarization test. |
On the Difference of BERT-style and CLIP-style Text Encoders (2023.findings-acl)
Copied to clipboard
| Challenge: | Masked language modeling is one of the most popular pretraining recipes in natural language processing. |
| Approach: | They analyze BERT-style and CLIP-style text encoders from three experiments . they show that CLIP style encoder is equipped with synesthesia for the cross-modal association . |
| Outcome: | The proposed models outperform BERT-style models on vision-centric text understanding tasks, but have synesthesia for the cross-modal association, similar to the senses of humans. |
Model Interpretability and Rationale Extraction by Input Mask Optimization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for creating explanations for black-box models struggle with deriving easily interpretable explanations. |
| Approach: | They propose a model-agnostic method to generate extractive explanations for neural network predictions using masking parts of the input that the model does not consider indicative of the respective class. |
| Outcome: | The proposed method achieves state-of-the-art results in a paragraph-level rationale extraction task, showing that this task can be performed without training a specialized model. |
NusaCrowd: Open Source Initiative for Indonesian NLP Resources (2023.findings-acl)
Copied to clipboard
Samuel Cahyawijaya, Holy Lovenia, Alham Fikri Aji, Genta Winata, Bryan Wilie, Fajri Koto, Rahmad Mahendra, Christian Wibisono, Ade Romadhony, Karissa Vincentio, Jennifer Santoso, David Moeljadi, Cahya Wirawan, Frederikus Hudi, Muhammad Satrio Wicaksono, Ivan Parmonangan, Ika Alfina, Ilham Firdausi Putra, Samsul Rahmadani, Yulianti Oenang, Ali Septiandri, James Jaya, Kaustubh Dhole, Arie Suryani, Rifki Afina Putri, Dan Su, Keith Stevens, Made Nindyatama Nityasya, Muhammad Adilazuarda, Ryan Hadiwijaya, Ryandito Diandaru, Tiezheng Yu, Vito Ghifari, Wenliang Dai, Yan Xu, Dyah Damapuspita, Haryo Wibowo, Cuk Tho, Ichwanul Karo Karo, Tirana Fatyanosa, Ziwei Ji, Graham Neubig, Timothy Baldwin, Sebastian Ruder, Pascale Fung, Herry Sujaini, Sakriani Sakti, Ayu Purwarianti
| Challenge: | Existing NLP research in Indonesian languages has been held back by factors such as language diversity, orthographic variation, resource limitation and other societal challenges. |
| Approach: | They present a collaborative initiative to collect and unify existing resources for Indonesian languages and open access to previously non-public resources. |
| Outcome: | The results show that the datasets are highly reliable and can be used to generate the first zero-shot benchmarks for natural language understanding and generation in Indonesian and the local languages of Indonesia. |
Transcribing Vocal Communications of Domestic Shiba lnu Dogs (2023.findings-acl)
Copied to clipboard
| Challenge: | a recent study has focused on how animals communicate, but the study has been limited . previous studies have focused on a simple classification problem, requiring a model to get a label . |
| Approach: | They extract Shiba Inu dogs' vocal communications from YouTube videos and translate them into phonetic scripts using a systematic process. |
| Outcome: | The proposed framework produces the first-of-its-kind Shiba Inu vocal communication dataset . it will be useful for future research in zoology and linguistics. |
SkillQG: Learning to Generate Question for Reading Comprehension Assessment (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing question generation systems focus on the literal nature of questions and rarely consider comprehension types of the generated questions. |
| Approach: | They propose a question generation framework with controllable comprehension types for machine reading comprehension models. |
| Outcome: | Empirical results show that SkillQG outperforms baselines in quality, relevance, and skill-controllability while showing a performance boost in downstream question answering task. |
Improving Long Dialogue Summarization with Semantic Graph Representation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing algorithms for abstractive summarization of short dialogues are challenging . however, they can generate high-quality summaries for long dialogues . |
| Approach: | They propose an algorithm that processes complete dialogues into topic-segment-level Abstract Meaning Representation graphs . they propose a pretrained LLM that exploits the text to leverage graph semantics a new text-graph attention . |
| Outcome: | The proposed algorithm outperforms state-of-the-art models on multiple long dialogue summarization datasets . it also generates additional training signals that facilitate graph feature encoding and content selection. |
Model Intrinsic Features of Fine-tuning based Text Summarization Models for Factual Consistency (2023.findings-acl)
Copied to clipboard
| Challenge: | a summarization model with relatively low factual consistency is more likely to model summaries that are not conditional to the documents. |
| Approach: | They analyze the model intrinsic features by varying the fine-tuning objectives and datasets. |
| Outcome: | The proposed models have a high inductive bias for factual consistency and are more likely to model summaries that are not conditional to the documents. |
EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained vision-language models have achieved impressive results in a range of vision-linguistic tasks. |
| Approach: | They propose a distilling then pruning framework to compress large vision-language models into smaller, faster ones. |
| Outcome: | The proposed framework reduces the size of a pre-trained large vision-language model and improves its performance on vision-linguistic tasks. |
DP-BART for Privatized Text Rewriting under Local Differential Privacy (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing systems that guarantee privacy protection to individuals face several problems, such as formal mathematical flaws, unrealistic privacy guarantees, and privatization of only individual words. |
| Approach: | They propose a system that enables sharing of sensitive textual documents while formally guaranteeing privacy protection to individuals. |
| Outcome: | The proposed system outperforms existing LDP systems in terms of noise and privacy guarantees. |
Robustness of Learning from Task Instructions (2023.findings-acl)
Copied to clipboard
| Challenge: | traditional supervised learning mostly works on individual tasks and requires training on a large set of task-specific examples. |
| Approach: | a new study investigates the system robustness when instructions are manipulated and paraphrased . task instructions give the model the definition of the task and allow it to output the appropriate answer . |
| Outcome: | a new study shows that supervised learning is robust when instructions are manipulated, paraphrased or iii from different levels of conciseness. |
Masked Latent Semantic Modeling: an Efficient Pre-training Alternative to Masked Language Modeling (2023.findings-acl)
Copied to clipboard
| Challenge: | a recent study suggests that masked language models are a useful pre-training technique for natural language processing . a study using mlms pre-trained by a team of researchers has improved performance . |
| Approach: | They propose an alternative to the classic masked language modeling paradigm . they use an unsupervised technique which uses sparse coding to make the prediction possible . |
| Outcome: | The proposed technique improves on pre-trained models compared to vanilla MLM . the proposed model returns distributions over their vocabulary peaking at plausible substitutes . |
Detection and Mitigation of the Negative Impact of Dataset Extractivity on Abstractive Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have shown that extractivity can affect output extractivity and the amount of factual information (i.e. faithfulness) in abstractive summarization models. |
| Approach: | They propose to design copy labels to fix the model's copying behaviors and train the model with a copy mechanism to reduce the negative impact of high extractivity on model performance. |
| Outcome: | The proposed method outperforms several competitive baselines and shows that low extractivity can improve model performance, while higher extractivity leads to a tendency for the model to copy text continuously from the source document rather than identifying and summarizing important content. |
Commonsense Knowledge Graph Completion Via Contrastive Pretraining and Node Clustering (2023.findings-acl)
Copied to clipboard
| Challenge: | Commonsense knowledge graphs are typically represented by short text, resulting in many different nodes representing the same concept. |
| Approach: | They propose a framework based on Contrastive Pretraining and Node Clustering to solve these problems. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on two CSKG completion benchmarks. |
Incorporating Factuality Inference to Identify Document-level Event Factuality (2023.findings-acl)
Copied to clipboard
| Challenge: | Document-level Event Factuality Identification (DEFI) is a task for identifying the degree of certainty that a specific event occurs in a document. |
| Approach: | They propose a document-level event factuality inference task to bridge the sentence-level and document- level event factualality semantically. |
| Outcome: | The proposed model outperforms the baselines on the public English and Chinese DLEF datasets. |
Hybrid and Collaborative Passage Reranking (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing solutions to passage reranking focus on enriching the interaction between query and each passage separately, neglecting the context among the top-ranked passages. |
| Approach: | They propose a Hybrid and Collaborative Passage Reranking method that leverages the similarity measurements of upstream retrievers for passage collaboration. |
| Outcome: | Experiments show that HybRank improves over existing methods and improves performance. |
Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole Sentence (2023.findings-acl)
Copied to clipboard
| Challenge: | Sentence-level representations are beneficial for various natural language processing tasks. |
| Approach: | They propose a generative embedding inversion attack that reconstructs input sequences based only on their sentence embeddeds. |
| Outcome: | The proposed model outperforms previous embedding inversion attacks in classification metrics and generates coherent and contextually similar sentences as original inputs. |
Learning Query Adaptive Anchor Representation for Inductive Relation Prediction (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to infer the missing links between entities are limited to the transductive setting . Query Adaptive Anchor Representation (QAAR) model is based on entity-independent features . |
| Approach: | They propose a query adaptive anchor representation model which extracts one opening subgraph and performs reasoning by one time for all candidate triples. |
| Outcome: | The proposed model outperforms state-of-the-art models in relation prediction task. |
Context or Knowledge is Not Always Necessary: A Contrastive Learning Framework for Emotion Recognition in Conversations (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies focus on modeling context-sensitive dependencies and knowledge-sensitive dependences. |
| Approach: | They propose a framework based on contrastive learning called CKCL to distinguish utterances for better vector representations. |
| Outcome: | The proposed framework outperforms state-of-the-art models on four datasets. |
Exploring Speaker-Related Information in Spoken Language Understanding for Better Speaker Diarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Current speaker diarization systems consider only acoustic information, resulting in performance degradation when encountering adverse acustic environment. |
| Approach: | They propose methods to extract speaker-related information from conversational semantics in multi-party meetings. |
| Outcome: | The proposed method improves on AISHELL-4 and AliMeeting datasets on speakers diarization and speaker-turn detection. |
Cross-Lingual Knowledge Distillation for Answer Sentence Selection in Low-Resource Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | Cross-Lingual Knowledge Distillation (CLKD) is a method to train AS2 models for low-resource languages without labeled data. |
| Approach: | They propose a method to train AS2 models for low-resource languages without labeled data . they use a translation-based WikiQA dataset and a multilingual AS2 dataset . |
| Outcome: | The proposed method outperforms or rivals fine-tuning with labeled data and machine translation and the teacher model. |
Run Like a Girl! Sport-Related Gender Bias in Language and Vision (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing social biases and stereotypes against certain groups are reproduced by computational models. |
| Approach: | They analyze gender bias in two Language and Vision datasets to find that they underrepresent women . they hypothesize that a bias affects human naming choices for people playing sports . |
| Outcome: | The proposed model reproduces gender bias in two Language and Vision datasets. |
People and Places of Historical Europe: Bootstrapping Annotation Pipeline and a New Corpus of Named Entities in Late Medieval Texts (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained named entity recognition models are inaccurate on modern corpora due to differences in language OCR errors. |
| Approach: | They develop a named entity recognition (NER) corpus of 3.6M sentences from medieval charters written mainly in Czech, Latin, and German. |
| Outcome: | The proposed model achieves entity-level Precision of 72.81–93.98% with 58.14–81.77% Recall on a manually-annotated test dataset. |
Check-COVID: Fact-Checking COVID-19 News Claims with Scientific Evidence (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing fact-checking benchmarks require systems to verify claims from everyday text against evidence from scientific journal articles. |
| Approach: | They propose a benchmark system that checks claims from news against scientific journal articles and veracity labels. |
| Outcome: | The new benchmark achieves F1 scores of 76.99 and 69.90 on both a fact-checking specific system and GPT-3.5, respectively. |
Early Exit with Disentangled Representation and Equiangular Tight Frame (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing early exit paradigm relies on training parametrical internal classifiers to complete specific tasks. |
| Approach: | They propose a method to decouple two distinct types of representation and introduce a non-parametric tight frame classifier for improvement. |
| Outcome: | Experiments on monolingual and multilingual tasks show that the proposed method improves over existing methods. |
Tokenization with Factorized Subword Encoding (2023.findings-acl)
Copied to clipboard
| Challenge: | Subword tokenization methods are often used to project subwords onto triplets . a typical tokenizer consists of 10 000s of subword mapped onto a single index . |
| Approach: | They propose a subword tokenization method that factorizes subwords onto triplets using a VQ-VAE model. |
| Outcome: | The proposed tokenization method is more appropriate and robust for morphological tasks than the commonly used byte-pair encoding (BPE) tokenization algorithm. |
Rarely a problem? Language models exhibit inverse scaling in their predictions following few-type quantifiers (2023.findings-acl)
Copied to clipboard
| Challenge: | Current work suggests that language models deal poorly with quantifiers-they struggle to predict which quantifier is used in a given context and also perform poorly at generating appropriate continuations following logical quantifier. |
| Approach: | They propose to use 960 English sentence stimuli to build 22 autoregressive transformer models of different sizes to test their performance on ‘few’-type quantifiers. |
| Outcome: | The proposed models perform poorly on ‘few’-type quantifiers, and the larger the model, the worse its performance. |
“A Little is Enough”: Few-Shot Quality Estimation based Corpus Filtering improves Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Quality Estimation (QE) is the task of evaluating the quality of a translation when reference translation is unavailable. |
| Approach: | They propose a Quality Estimation based Filtering approach to extract high-quality parallel data from the pseudo-parallel corpus. |
| Outcome: | The proposed approach improves the machine translation system performance by up to 1.8 BLEU points over the baseline model. |
How effective is machine translation on low-resource code-switching? A case study comparing human and automatic metrics (2023.findings-acl)
Copied to clipboard
| Challenge: | Specifically, we compare the performance of three MT systems in terms of their ability to translate monolingual Vietnamese, a low-resource language, and Vietnamese-English CSW respectively. |
| Approach: | They compare the performance of three machine translation systems in the context of machine translation (MT) they find that state-of-the-art neural translation systems achieve higher scores on automatic metrics when processing CSW input . |
| Outcome: | The proposed system can translate monolingual Vietnamese, a low-resource language, and Vietnamese-English CSW respectively. |
Images in Language Space: Exploring the Suitability of Large Language Models for Vision & Language Tasks (2023.findings-acl)
Copied to clipboard
| Challenge: | Large language models have demonstrated robust performance on various language tasks using zero-shot or few-shot learning paradigms. |
| Approach: | They propose to use open-source, open-access language models to make visual input accessible to the model using separate verbalisation models. |
| Outcome: | The proposed model can handle visual input but also require strong reasoning component. |
On the Expressivity Role of LayerNorm in Transformers’ Attention (2023.findings-acl)
Copied to clipboard
| Challenge: | Layer Normalization (LayerNorm) is an inherent component in all Transformer-based models. |
| Approach: | They propose to use LayerNorm to normalize the activations during the forward pass and their gradients during the backward pass. |
| Outcome: | The proposed model is able to express the multi-head attention layer that follows it in a d-1 space and scales to the same norm of d. |
DEnsity: Open-domain Dialogue Evaluation Metric using Density Estimation (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies suggest that neural classifiers make overly confident predictions for examples from unseen distributions. |
| Approach: | They propose a new evaluation metric, DENSITY, which measures how likely a response would appear in the distribution of human conversations. |
| Outcome: | The proposed metric measures how likely a response would appear in the distribution of human conversations. |
Fixing MoE Over-Fitting on Low-Resource Languages in Multilingual Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Sparsely gated Mixture of Experts (MoE) models are a compute-efficient method to scale model capacity for multilingual machine translation tasks. |
| Approach: | They propose a regularization strategy that prevents over-fitting of MoE models on low-resource tasks and conditional MoE Routing and curriculum learning methods that prevent over- fitting. |
| Outcome: | The proposed methods improve the performance of MoE models on low-resource tasks without adversely affecting high-res tasks. |
Intent Discovery with Frame-guided Semantic Regularization and Augmentation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing intent discovery methods focus on transferring prior knowledge of known intents to unknown ones. |
| Approach: | They propose to use frame knowledge as conceptual semantic guidance to bridge the gap between known intents representation learning and unknown intents clustering. |
| Outcome: | The proposed method outperforms solid baselines on two benchmark datasets. |
An Empirical Comparison of LM-based Question and Answer Generation Methods (2023.findings-acl)
Copied to clipboard
| Challenge: | Question and answer generation (QAG) is a task of generating question-answer pairs given a context. |
| Approach: | They propose to leverage sequence-to-sequence language model fine-tuning to generate question-answer pairs given a context. |
| Outcome: | The proposed model outperforms other more convoluted approaches in the end-to-end model and is computationally light at both training and inference times. |
Contrastive Learning with Generated Representations for Inductive Knowledge Graph Embedding (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for inductive knowledge Graphs are limited by sparsity and implicit transfer. |
| Approach: | They propose a Contrastive Learning framework with graph guided Variational autoencoder on Meta-KGs to capture and transfer entities. |
| Outcome: | The proposed framework outperforms state-of-the-art methods with extensive experiments. |
Decouple knowledge from paramters for plug-and-play language modeling (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) have made impressive results in a wide range of NLP tasks. |
| Approach: | They propose a pre-training model with editable and scalable key-value memory and leverage knowledge in an explainable manner by knowledge retrieval in the pasted macro ‘MEMORY’. |
| Outcome: | The proposed model decouples the knowledge storage from model parameters with an editable and scalable key-value memory and leverages knowledge in an explainable manner by knowledge retrieval in the pasted macro ‘MEMORY’. |