Papers by Chang Li
Training-free Deep Concept Injection Enables Language Models for Video Question Answering (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to train pretrained language models for zero-shot crossmodal tasks require crossmodal pretraining. |
| Approach: | They propose to inject visual concepts into the input text embedding space of a pretrained language model and build adaptation layers based on the intermediate representation of concepts. |
| Outcome: | The proposed model performs zero-shot crossmodal tasks without crossmodal pretraining . it is based on the injection of visual concepts as input tokens and augmentation in intermediate features . the proposed model achieves competitive or even better results in zero- shot and fine-tuning settings . |
Searching for an Effective Defender: Benchmarking Defense against Adversarial Word Substitution (2021.emnlp-main)
Copied to clipboard
Zongyi Li, Jianhan Xu, Jiehang Zeng, Linyang Li, Xiaoqing Zheng, Qi Zhang, Kai-Wei Chang, Cho-Jui Hsieh
| Challenge: | Existing methods to defend against adversarial word-substitution attacks have not been evaluated or compared in a systematic manner. |
| Approach: | They propose to compare different defense methods under representative adversarial attacks . they propose a method that improves the robustness of neural text classifiers against such attacks a . |
| Outcome: | The proposed method improves robustness of neural text classifiers against such attacks by a significant margin. |
Thread: A Logic-Based Data Organization Paradigm for How-To Question Answering with Retrieval Augmented Generation (2025.emnlp-main)
Copied to clipboard
Kaikai An, Fangkai Yang, Liqun Li, Junting Lu, Sitao Cheng, Shuzheng Si, Lu Wang, Pu Zhao, Lele Cao, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, Baobao Chang
| Challenge: | Recent advances in retrieval-augmented generation (RAG) have substantially improved question-answering systems, particularly for factoid ‘5Ws’ questions. |
| Approach: | They propose a data organization paradigm where large language models transform documents into more structured and loosely interconnected LUs. |
| Outcome: | Experiments in open-domain and industrial settings show that the proposed paradigm outperforms existing paradigms and shows high adaptability across diverse document formats. |
Butterfly Effects in Toolchains: A Comprehensive Analysis of Failed Parameter Filling in LLM Tool-Agent Systems (2025.findings-emnlp)
Copied to clipboard
| Challenge: | emergence of tool agent paradigm has broadened capability boundaries of the Large Language Model (LLM) but effectiveness of tool agents limited due to parameter failure during execution . |
| Approach: | They propose a parameter failure taxonomy to investigate parameter failure . they propose suggestions for standardizing tool return formats and improving error feedback mechanisms . |
| Outcome: | The proposed model is based on a tool agent invocation chain and a mainstream tool agent . it shows that parameter name hallucination failure stems from inherent limitations . |
Word Graph Guided Summarization for Radiology Findings (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies focus on introducing salient word information to general text summarization framework to guide selection of key content in radiology findings. |
| Approach: | They propose a method for automatic impression generation using word graphs and a Word Graph guided Summarization model to capture critical words and their relations. |
| Outcome: | The proposed method is validated on two datasets, OPENI and MIMIC-CXR. |
Cross-media Structured Common Space for Multimedia Event Extraction (2020.acl-main)
Copied to clipboard
| Challenge: | We propose a new task to extract events and their arguments from multimedia documents . traditional methods target text, images or videos, but multimedia content is distributed via multimedia . |
| Approach: | They propose a method that encodes structured representations of semantic information from textual and visual data into a common embedding space. |
| Outcome: | The proposed method achieves 4.0% and 9.8% absolute gains on text event argument role labeling and visual event extraction. |
AGGC: Adaptive Group Gradient Clipping for Stabilizing Large Language Model Training (2026.findings-acl)
Copied to clipboard
| Challenge: | Adaptive group-wise gradient clipping (AGGC) is a new approach to stabilize training of Large Language Models. |
| Approach: | They propose a method to stabilize gradient clipping by partitioning parameters into groups based on functional types and a time-dependent scheduling mechanism to balance exploration and convergence. |
| Outcome: | The proposed algorithm outperforms standard LoRA and achieves 72.93% accuracy . it can be integrated into existing pipelines with negligible overhead. |
Towards Better Hierarchical Text Classification with Data Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to improve hierarchical text classification are expensive and lack high-quality labeled data. |
| Approach: | They propose a hierarchical text classification framework that can achieve both label controllability and text diversity by extracting high-quality hierarchic label information. |
| Outcome: | The proposed method can achieve label controllability and text diversity by extracting high-quality hierarchical label information. |
Beyond Token Length: Step Pruner for Efficient and Accurate Reasoning in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing reinforcement learning methods for large reasoning models suffer from excessive verbosity, known as "overthinking." Existing models penalize generated tokens to promote conciseness, but these methods encounter two challenges: they may develop hacking behavior in later stages of training by discarding reasoning steps. |
| Approach: | They propose a framework that steers large reasoning models toward more efficient reasoning . they prioritize correctness while imposing penalties for redundant steps . |
| Outcome: | The proposed framework reduces token usage by 69.7% on AIME24. |
Joint Multimedia Event Extraction from Video and Article (2021.findings-emnlp)
Copied to clipboard
Brian Chen, Xudong Lin, Christopher Thomas, Manling Li, Shoya Yoshida, Lovish Chum, Heng Ji, Shih-Fu Chang
| Challenge: | Existing methods to extract multimedia events from video and text are limited to video and images. |
| Approach: | They propose a task to jointly extract events from video and text documents . they propose 'self-supervised' cross-modal event coreference model and cross-mod transformer architecture . |
| Outcome: | The proposed method achieves 6.0% and 5.8% absolute F-score gain on video-article pairs . the proposed method can resolve coreference and extract multimodal event frames more accurately than existing methods. |
SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilities (2022.acl-long)
Copied to clipboard
Hsiang-Sheng Tsai, Heng-Jui Chang, Wen-Chin Huang, Zili Huang, Kushal Lakhotia, Shu-wen Yang, Shuyan Dong, Andy Liu, Cheng-I Lai, Jiatong Shi, Xuankai Chang, Phil Hall, Hsuan-Jui Chen, Shang-Wen Li, Shinji Watanabe, Abdelrahman Mohamed, Hung-yi Lee
| Challenge: | Existing evaluation methods for transfer learning are limited in speech research . authors show that pre-trained models transfer well across multiple tasks . |
| Approach: | They propose a benchmark to evaluate pre-trained models by increasing task diversity and difficulty over SUPERB. |
| Outcome: | The proposed benchmark increases task diversity and difficulty over SUPERB-SG. |
Breaking Down Power Barriers in On-Device Streaming ASR: Insights and Solutions (2025.naacl-industry)
Copied to clipboard
Yang Li, Yuan Shangguan, Yuhao Wang, Liangzhen Lai, Ernie Chang, Changsheng Zhao, Yangyang Shi, Vikas Chandra
| Challenge: | Streaming automatic speech recognition models use high power consumption to improve usability and accuracy. |
| Approach: | They propose to optimize on-device speech recognition models by adjusting component energy sensitivities based on their specific energy sensitities to reduce power consumption. |
| Outcome: | The proposed approach achieves up to 47% lower energy usage while preserving comparable model accuracy and improving real-time performance compared to leading methods. |
Repairing Catastrophic-Neglect in Text-to-Image Diffusion Models via Attention-Guided Feature Enhancement (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Text-to-Image Diffusion models generate high-quality images from textual descriptions, but they often produce images that do not fully align with the input prompts, resulting in semantic inconsistencies. |
| Approach: | They propose an automated repair approach to address catastrophic-neglect in T2I DMs. |
| Outcome: | The proposed model achieves 10.1%-16.3% higher Correct Rate in image generation compared to baselines. |
MetaVL: Transferring In-Context Learning Ability From Language Models to Vision-Language Models (2023.acl-short)
Copied to clipboard
| Challenge: | Large-scale pre-trained vision-language models do not possess the ability to conduct in-context learning. |
| Approach: | They propose to meta-train a language model to perform in-context learning on NLP tasks and then transfer this model to VL tasks by attaching a visual encoder. |
| Outcome: | The proposed model outperforms the baseline model on VQA, OK-VQA, and GQA while having 20 times fewer parameters. |
Leveraging Abstract Meaning Representation for Knowledge Base Question Answering (2021.findings-acl)
Copied to clipboard
Pavan Kapanipathi, Ibrahim Abdelaziz, Srinivas Ravishankar, Salim Roukos, Alexander Gray, Ramón Fernandez Astudillo, Maria Chang, Cristina Cornelio, Saswati Dana, Achille Fokoue, Dinesh Garg, Alfio Gliozzo, Sairam Gurajada, Hima Karanam, Naweed Khan, Dinesh Khandelwal, Young-Suk Lee, Yunyao Li, Francois Luus, Ndivhuwo Makondo, Nandana Mihindukulasooriya, Tahira Naseem, Sumit Neelam, Lucian Popa, Revanth Gangi Reddy, Ryan Riegel, Gaetano Rossiello, Udit Sharma, G P Shrivatsa Bhargav, Mo Yu
| Challenge: | Existing approaches face challenges including complex question understanding and lack of large end-to-end training datasets. |
| Approach: | They propose a modular knowledge base question answering system that leverages AMR parses for task-independent question understanding. |
| Outcome: | The proposed system achieves state-of-the-art performance on two prominent KBQA datasets based on DBpedia. |
BRIEF: Bridging Retrieval and Inference for Multi-hop Reasoning via Compression (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to augment language models with external knowledge but they are limited by static nature of pre-training data. |
| Approach: | They propose a lightweight approach that compresses retrieved documents into highly dense textual summaries to integrate into in-context RAG. |
| Outcome: | The proposed approach reduces latency and costs while achieving high performance in open-domain questions. |
One QuantLLM for ALL: Fine-tuning Quantized LLMs Once for Efficient Deployments (2025.acl-long)
Copied to clipboard
| Challenge: | Quantization has shown promise for Large Language Models, but current methods require lengthy training to alleviate quantization loss. |
| Approach: | They propose to decouple weights and incorporate Low-Rank adapters to reduce weight sharing . they validate the approach on LLaMA2 families and Mistral on downstream evaluation . |
| Outcome: | The proposed approach shows high performance while reducing deployment time faced with multiple scenarios. |
IW-Bench: Evaluating Large Multimodal Models for Converting Image-to-Web (2025.findings-acl)
Copied to clipboard
Hongcheng Guo, Wei Zhang, Junhao Chen, Yaonan Gu, Jian Yang, Junjia Du, Shaosheng Cao, Binyuan Hui, Tianyu Liu, Jianxin Ma, Chang Zhou, Zhoujun Li
| Challenge: | Existing models have been introduced to improve image comprehension, but there is no robust benchmark for imagetoweb conversion. |
| Approach: | They propose a benchmark to assess imagetoweb conversion proficiency of large multimodal models . they propose to measure layout information of web pages by parsing the Document Object Model tree . |
| Outcome: | The proposed benchmark measures the layout information of web pages—i.e., the positional relationships between elements—which has been overlooked by prior work. |
GeoMLAMA: Geo-Diverse Commonsense Probing on Multilingual Pre-Trained Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that Pre-trained Language Models store relational knowledge and utilize it for performing downstream tasks. |
| Approach: | They propose a benchmark dataset for probing the diversity of relational knowledge in multilingual PLMs. |
| Outcome: | The proposed dataset contains 3125 prompts in English, Chinese, Hindi, Persian, and Swahili . larger multilingual PLMs variants do not store geo-diverse concepts better than its smaller variant . |
Knowledge-Guided Dynamic Modality Attention Fusion Framework for Multimodal Sentiment Analysis (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods focus on equally treating the contribution of each modality or statically using text as the dominant modality to conduct interaction, which neglects the situation where each modal may become dominant. |
| Approach: | They propose a Knowledge-Guided Dynamic Modality Attention Fusion Framework (KuDA) that uses sentiment knowledge to guide the model dynamically selecting the dominant modality and adjusting the contributions of each modality. |
| Outcome: | The proposed model can be used to highlight the contribution of dominant modality through the correlation evaluation loss. |
Double Graph Based Reasoning for Document-level Relation Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for document-level relation extraction fail to recognize relations between entities across sentences. |
| Approach: | They propose a method to recognize relations for long paragraphs by a Graph Aggregation-and-Inference Network (GAIN) they propose to use a heterogeneous mention-level graph and an entity-level EG graph to analyze the relationships. |
| Outcome: | The proposed method achieves a significant performance improvement (2.85 on F1) over the previous state-of-the-art. |
Graph Enhanced Contrastive Learning for Radiology Findings Summarization (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for automating impression generation have limited the relationship between extra knowledge and the original findings. |
| Approach: | They propose a framework for automating impression generation that exploits extra knowledge and original findings . they propose combining key words and their relations to extract critical information . |
| Outcome: | The proposed framework exploits extra knowledge and the original findings in an integrated way . the state-of-the-art results on two datasets confirm the effectiveness of the proposed method . |
Towards Fine-grained Text Sentiment Transfer (P19-1)
Copied to clipboard
| Challenge: | Existing methods for fine-grained text sentiment transfer only reverse the sentiment polarity of text, but they lack a robust and parallel learning algorithm. |
| Approach: | They propose a novel fine-grained text sentiment transfer task that revises a sequence to satisfy a given sentiment intensity while preserving the original semantic content. |
| Outcome: | The proposed model outperforms existing methods by a large margin in automatic evaluation and human evaluation. |
Training LLMs for Divide-and-Conquer Reasoning Elevates Test-Time Scalability (2026.acl-long)
Copied to clipboard
Xiao Liang, Zhong-Zhi Li, Zhenghao Lin, Eric Hanchen Jiang, Hengyuan Zhang, Yelong Shen, Kai-Wei Chang, Ying Nian Wu, Yeyun Gong, Weizhu Chen
| Challenge: | Large language models have demonstrated strong reasoning capabilities through step-by-step chain-of-thought (CoT) reasoning, but their strictly sequential nature constrains test-time scalability. |
| Approach: | They propose an end-to-end reinforcement learning framework to enhance LLMs' DAC-style reasoning capacity by decomposing a problem into subproblems and solving them sequentially. |
| Outcome: | The proposed model surpasses CoT by 8.6% and 6.3% on competition-level benchmarks and is available at the [github.com/MasterVito/DAC-RL]. |
Can We Edit Factual Knowledge by In-Context Learning? (2023.emnlp-main)
Copied to clipboard
| Challenge: | In-context knowledge editing (IKE) is a new paradigm for NLP research that can be applied to large language models with tens or hundreds of parameters. |
| Approach: | They propose to use in-context knowledge editing (IKE) without gradient updating to edit factual knowledge without a gradient update. |
| Outcome: | The proposed method achieves a competitive success rate compared to gradient-based methods on GPT-J but with fewer side effects. |
CB-Whisper: Contextual Biasing Whisper Using Open-Vocabulary Keyword-Spotting (2024.lrec-main)
Copied to clipboard
Yuang Li, Yinglu Li, Min Zhang, Chang Su, Jiawei Yu, Mengyao Piao, Xiaosong Qiao, Miaomiao Ma, Yanqing Zhao, Hao Yang
| Challenge: | End-to-end automatic speech recognition systems struggle to recognize rare name entities such as personal names, organizations and terminologies that are not frequently encountered in the training data. |
| Approach: | They propose a convolutional neural network-based ASR system that performs open-vocabulary keyword-spotting before the decoder to match the features between the entities and the utterances. |
| Outcome: | The proposed system significantly improves mixed-error-rate (MER) and entity recall compared to the original Whisper model on three internal datasets and two publicly available datasets. |
Listen, Decipher and Sign: Toward Unsupervised Speech-to-Sign Language Recognition (2023.findings-acl)
Copied to clipboard
Liming Wang, Junrui Ni, Heting Gao, Jialu Li, Kai Chieh Chang, Xulin Fan, Junkai Wu, Mark Hasegawa-Johnson, Chang Yoo
| Challenge: | Existing supervised sign language recognition systems rely on well-annotated data . instead, an unsupervised speech-to-sign language recognition system learns to translate between spoken and sign languages by observing only non-parallel speech and sign-language corpora. |
| Approach: | They propose an unsupervised speech-to-sign language recognition system that can translate between spoken and sign languages by observing only non-parallel speech and sign-language corpora. |
| Outcome: | The proposed approach outperforms baseline models on sign language corpora by 50% . the proposed approach is available at https://github.com/cactuswiththoughts/UnsupSpeech2Sign.git . |
FinMaster: A Holistic Benchmark for Full-Pipeline Financial Management with Large Language Models (2026.findings-acl)
Copied to clipboard
Junzhe Jiang, Chang Yang, Aixin Cui, Sihan Jin, Yujing Zhang, Yilin Xiao, Ruiyu Wang, Bo Li, Xiao Huang, Danny Dongning Sun, Xinrun Wang
| Challenge: | Existing benchmarks lack domain-specific data, realistic workflow-level task design, and standardized workflow- level evaluation. |
| Approach: | a new benchmark evaluates large language models on financial management workflows . the global financial services market is projected to grow to $37 trillion by 2027 . |
| Outcome: | a new benchmark for large language models on financial management workflows reveals critical capability gaps . accuracy drops from 90% on basic tasks to 40% on complex scenarios requiring multi-step reasoning . the global financial services market reached $25.8 trillion in 2022 and is projected to grow to $37 trillion by 2027 . |
SciAssess: Benchmarking LLM Proficiency in Scientific Literature Analysis (2025.findings-naacl)
Copied to clipboard
Hengxing Cai, Xiaochen Cai, Junhan Chang, Sihang Li, Lin Yao, Wang Changxin, Zhifeng Gao, Hongshuai Wang, Li Yongge, Mujie Lin, Shuwen Yang, Jiankun Wang, Mingjun Xu, Jin Huang, Xi Fang, Jiaxi Zhuang, Yuqi Yin, Yaqi Li, Changhong Chen, Zheng Cheng, Zifeng Zhao, Linfeng Zhang, Guolin Ke
| Challenge: | Existing benchmarks fail to adequately evaluate the proficiency of Large Language Models (LLMs) Existing standards do not cover the skills needed to evaluate LLMs in scientific literature analysis. |
| Approach: | They propose a benchmark to evaluate the proficiency of large language models in scientific literature analysis. |
| Outcome: | SciAssess evaluates 11 LLMs on multiple tasks across scientific fields. |
When Efficiency Becomes a Vulnerability: Computational Cost Attacks on WebAgents (2026.acl-long)
Copied to clipboard
| Challenge: | Existing WebAgents suffer from computational cost attacks due to long reasoning processes and excessive computational cost. |
| Approach: | They propose a framework that generates adversarial prompts and a reinforcement learning-enhanced selector to identify the most effective perturbations. |
| Outcome: | The proposed framework exploits large language models to generate diverse adversarial prompts and a reinforcement learning–enhanced selector to identify the most effective perturbations. |
Frustratingly Simple Few-Shot Slot Tagging (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing fewshot methods for slot tagging are weak in encoding slot name semantics and slot dependencies. |
| Approach: | They propose a simple and effective few-shot model for slot tagging which incorporates machine reading comprehension (MRC) using source domain and target domain data. |
| Outcome: | The proposed model outperforms state-of-the-art methods on the SNIPS dataset. |
Control Large Language Models via Divide and Conquer (2024.emnlp-main)
Copied to clipboard
| Challenge: | Lexically Constrained Generation (LCG) is a crucial task of text generation. |
| Approach: | They propose a Divide and Conquer Generation strategy to enhance LLMs' performance in Lexically Constrained Generation with prompt-based controlling. |
| Outcome: | The proposed strategy shows 90% improvement on the most challenging LCG task. |
A Reinforcement Learning Framework for Robust and Secure LLM Watermarking (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing watermarking algorithms rely on heuristic green/red token lists . however, these lists are inconsistent and can be compromised . |
| Approach: | They propose a framework for robust and secure LLM watermarking using reinforcement learning. |
| Outcome: | The proposed method achieves state-of-the-art trade-off across all criteria with notable improvements in resistance to spoofing attacks without degrading other criteria. |
POWSM: A Phonetic Open Whisper-Style Speech Foundation Model (2026.acl-long)
Copied to clipboard
Chin-Jou Li, Kalvin Chang, Shikhar Bharadwaj, Eunjung Yeo, Kwanghee Choi, Jian Zhu, David R. Mortensen, Shinji Watanabe
| Challenge: | Phone-level modeling of speech is a common approach to speech recognition, but it relies on task-specific architectures and datasets. |
| Approach: | They propose a phonetic framework capable of performing multiple phone-related tasks . they propose 'Phonetic Open Whisper-style Speech Model' that can perform these tasks together . |
| Outcome: | The proposed model outperforms or matches specialized PR models of similar size while supporting G2P, P2G, and ASR. |
T2DR: A Two-Tier Deficiency-Resistant Framework for Incomplete Multimodal Learning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing incomplete multimodal learning frameworks are inadequate for integrating multimodal data. |
| Approach: | They propose a framework for incomplete multimodal learning that is deficiency-resistant and provides two modules to address fine-grained deficiencies. |
| Outcome: | The proposed framework outperforms the SOTA models on two well-known multimodal benchmarks. |
Quantifying and Improving the Robustness of Retrieval-Augmented Language Models Against Spurious Features in Grounding Data (2026.acl-long)
Copied to clipboard
Shiping Yang, Jie Wu, Wenbiao Ding, Ning Wu, Shining Liang, Ming Gong, Hongzhi Li, Hengyuan Zhang, Angel X. Chang, Dongmei Zhang
| Challenge: | Existing studies on robustness to explicit noise (e.g., document semantics) but overlook implicit noise (spurious features). |
| Approach: | They propose a framework to quantify the robustness of RAGs against spurious features by integrating a data synthesis pipeline and a taxonomy. |
| Outcome: | The proposed framework quantifies the robustness of RALMs against spurious features. |
Syntax-augmented Multilingual BERT for Cross-lingual Transfer (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies show that pre-trained multilingual text encoders capture language syntax, helping cross-lingual transfer. |
| Approach: | They provide language syntax and train mBERT to encode universal dependency tree structure. |
| Outcome: | The proposed model improves cross-lingual transfer on PAWS-X and MLQA benchmarks by 1.4 and 1.6 points on average across all languages. |
MolTC: Towards Molecular Relational Modeling In Language Models (2024.findings-acl)
Copied to clipboard
Junfeng Fang, Shuai Zhang, Chang Wu, Zhengyi Yang, Zhiyuan Liu, Sihang Li, Kun Wang, Wenjie Du, Xiang Wang
| Challenge: | Molecular Relational Learning (MRL) is a promising way to understand interactions between molecular pairs. |
| Approach: | They propose a novel LLM-based multi-modal framework for molecular interaction modeling following Chain-of-Thought (CoT) theory which integrates graphical information of two molecules in pair. |
| Outcome: | The proposed framework integrates graphical information of two molecules in pair. |
MMAC: A Multilingual, Multimodal Alignment Framework for Cultural Grounding Evaluation (2026.acl-long)
Copied to clipboard
Weihua Zheng, Zhengyuan Liu, Tanmoy Chakraborty, Weiwen Xu, Xiaoxue Gao, Bryan Chen Zhengyu Tan, Bowei Zou, Chang Liu, Yujia Hu, Xing Xie, Xiaoyuan Yi, Jing Yao, Chaojun Wang, Long Li, Rui Liu, Huiyao Liu, Koji Inoue, Ryuichi Sumida, Tatsuya Kawahara, Fan Xu, Lingyu Ye, Wei Tian, Dongjun Kim, Jimin Jung, Jaehyung Seo, Nadya Yuki Wangsajaya, Pham Minh Duc, Ojasva Saxena, Palash Nandi, Xiyan Tao, Wiwik Karlina, Tuan Luong, Keertana Arun Vasan, Roy Ka-Wei Lee, Nancy F. Chen
| Challenge: | Existing models lack cultural alignment across modalities and languages . a new framework to assess cultural awareness across linguistics and languages is needed . |
| Approach: | They propose a framework that integrates tri-modally aligned cultural benchmarks and a five-dimensional evaluation protocol to assess cross-country awareness disparities. |
| Outcome: | The proposed framework assesses cultural awareness disparities across modalities and languages . it is the first dataset aligned at the input level across text, image, and speech . |
From Implicit Graph Encoding to Explicit Evidence: A Training-Free LLM Framework for Temporal Knowledge Graph Reasoning (2026.findings-acl)
Copied to clipboard
Guo Tang, Ke Cheng, Huiming Fan, Heng Chang, Wenxiang Zheng, Xianhao Ou, Junjia Xiang, Ming Liu, Yujun Zhou, Li Lanyu, Bing Qin
| Challenge: | Existing Large Language Models (LLMs) struggle with implicit modality alignment and suboptimal graph linearization. |
| Approach: | They propose a training-free, test-time adaptive framework that reframes TKG prediction as explicit evidence-driven reasoning. |
| Outcome: | ExE-LLM outperforms fully trained graph neural networks on four benchmarks . it achieves SOTA performance in inductive settings, significantly outperforming fully trained neural networks . |
McBE: A Multi-task Chinese Bias Evaluation Benchmark for Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets on bias evaluation for large language models focus on English and North American culture and are limited to one task. |
| Approach: | They propose to evaluate Chinese language models' biases from multiple perspectives using a multi-task Chinese Bias Evaluation Benchmark. |
| Outcome: | The proposed model covers 12, 82 subcategories and 5 evaluation tasks covering a wide range of categories and content diversity. |
TreeReview: A Dynamic Tree of Questions Framework for Deep and Efficient LLM-based Scientific Peer Review (2025.emnlp-main)
Copied to clipboard
Yuan Chang, Ziyue Li, Hengyuan Zhang, Yuanbo Kong, Yanru Wu, Hayden Kwok-Hay So, Zhijiang Guo, Liya Zhu, Ngai Wong
| Challenge: | Large Language Models (LLMs) have shown significant potential in assisting peer review, but current methods struggle to generate thorough and insightful reviews while maintaining efficiency. |
| Approach: | They propose a framework that models paper review as a hierarchical and bidirectional question-answering process. |
| Outcome: | The proposed framework outperforms baselines on full review generation and actionable feedback comments generation tasks while reducing LLM token usage by up to 80% compared to computationally intensive approaches. |
Learning Gender-Neutral Word Embeddings (D18-1)
Copied to clipboard
| Challenge: | Word embeddings trained on human-generated corpora inherit strong gender stereotypes . prior studies show such embeddables exhibit social biases, such as gender stereotype . |
| Approach: | They propose a method to preserve gender information in certain dimensions of word vectors . they propose GN-GloVe, which is a gender-neutral variant of the word embedding model . |
| Outcome: | The proposed method preserves gender information in certain dimensions of word vectors while compelling other dimensions to be free of gender influence. |
Hybrid Alignment Training for Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to align large language models with instructions and preferences are conflicting . et al., 2023b) show that hybrid alignment training can outperform baselines . |
| Approach: | They propose a hybrid alignment training approach based on alternating alignment and modified elastic weight consolidation methods to achieve better collaboration between different alignment tasks. |
| Outcome: | The proposed approach outperforms baseline alignment training methods on summarization and dialogue tasks. |
SWE-Fixer: Training Open-Source LLMs for Effective and Efficient GitHub Issue Resolution (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models excel in code generation benchmarks, but these benchmarks focus on single-file scenarios with constrained context scope. |
| Approach: | They propose an open-source framework to effectively resolve GitHub issues using a code file retrieval module and a model-based code editing module. |
| Outcome: | The proposed approach achieves state-of-the-art performance on two GitHub benchmarks. |
Reciprocal Learning of Knowledge Retriever and Response Ranker for Knowledge-Grounded Conversations (2022.coling-1)
Copied to clipboard
| Challenge: | Recent work on grounding dialogue agents with knowledge documents has sparked increased attention . hand-labeling data to that end is time-consuming and many datasets lack knowledge annotations . |
| Approach: | They propose a reciprocal learning approach to optimize a knowledge retriever and a response ranker for knowledge-grounded response retrieval without ground-truth knowledge labels. |
| Outcome: | The proposed model outperforms previous state-of-the-art methods on two public benchmarks. |
Knowledge Neurons in Pretrained Transformers (2022.acl-long)
Copied to clipboard
| Challenge: | Existing studies show that pretrained language models are good at recalling factual knowledge without fine-tuning. |
| Approach: | They propose a method to identify neurons that express factual knowledge in pretrained Transformers by filling-in-the-blank cloze queries. |
| Outcome: | The proposed method can be used to edit, erase, and update factual knowledge without fine-tuning. |
Scaling Parameter-Constrained Language Models with Quality Data (2024.emnlp-industry)
Copied to clipboard
Ernie Chang, Matteo Paltenghi, Yang Li, Pin-Jie Lin, Changsheng Zhao, Patrick Huber, Zechun Liu, Rastislav Rabatin, Yangyang Shi, Vikas Chandra
| Challenge: | Scaling laws in language modeling quantify training loss as a function of dataset size and model parameters, but neglect the critical role of data quality in model generalization. |
| Approach: | They propose to use effective training tokens as a combination of text diversity and syntheticity as measured by a teacher model to calculate scaling laws. |
| Outcome: | The proposed term effective training tokens is a combination of two readily-computed indicators of text diversity and syntheticity as measured by a teacher model. |
ADDMU: Detection of Far-Boundary Adversarial Examples with Data and Model Uncertainty Estimation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods show poor performance under Far Boundary (FB) adversarial examples. |
| Approach: | They propose to use a new technique to detect adversarial examples based on data and model uncertainty to outperform existing methods. |
| Outcome: | The proposed method outperforms existing methods by 3.6 and 6.0 AUC points under each scenario. |
ORANGE: Text-video Retrieval via Watch-time-aware Heterogeneous Graph Contrastive Learning (2023.emnlp-industry)
Copied to clipboard
Yucheng Lin, Tim Chang, Yaning Chang, Jianqiang Ma, Donghui Li, Ting Peng, Zang Li, Zhiyi Zhou, Feng Wang
| Challenge: | Existing methods for text-video retrieval focus on informative representations and delicate matching mechanisms, but real-world scenarios often involve brief, ambiguous queries and low-quality videos. |
| Approach: | They propose a novel method to learn informative embeddings for queries and videos . they use a watch-time-aware contrastive learning paradigm to capture dependencies . |
| Outcome: | The proposed method is effective in a real-world video-search service. |
KS-Lottery: Finding Certified Lottery Tickets for Multilingual Transfer in Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing studies have shown that a small subset of parameters is highly effective in fine-tuning . prior work shows that there are a few additional parameters corresponding to an intrinsic dimension in a well-trained Large Language Model. |
| Approach: | They propose a method to identify a small subset of LLM parameters highly effective in multilingual fine-tuning. |
| Outcome: | The proposed method can find the certified winning tickets in the embedding layer, and fine-tuning on the found parameters is guaranteed to perform as well as full fine- tuning. |
RESIN: A Dockerized Schema-Guided Cross-document Cross-lingual Cross-media Information Extraction and Event Tracking System (2021.naacl-demos)
Copied to clipboard
Haoyang Wen, Ying Lin, Tuan Lai, Xiaoman Pan, Sha Li, Xudong Lin, Ben Zhou, Manling Li, Haoyu Wang, Hongming Zhang, Xiaodong Yu, Alexander Dong, Zhenhailong Wang, Yi Fung, Piyush Mishra, Qing Lyu, Dídac Surís, Brian Chen, Susan Windisch Brown, Martha Palmer, Chris Callison-Burch, Carl Vondrick, Jiawei Han, Dan Roth, Shih-Fu Chang, Heng Ji
| Challenge: | We present a new information extraction system that can construct temporal event graphs from news documents. |
| Approach: | They propose a temporal event graph extraction system that can extract news documents . they extend the system from sentence-level event extraction to cross-document cross-media event extraction . |
| Outcome: | The proposed system can extract temporal event graphs from news documents in multiple languages and multiple data modalities. |
Play Guessing Game with LLM: Indirect Jailbreak Attack with Implicit Clues (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing jailbreak attacks primarily utilize scenario camouflage techniques, however their explicit mention of malicious intent will be easily recognized and defended by LLMs. |
| Approach: | They propose an indirect jailbreak attack approach, Puzzler, which can bypass LLM’s defensive strategies and obtain malicious response by implicitly providing LLMs with some clues about the original malicious query. |
| Outcome: | The proposed approach can bypass the LLM’s defensive strategies and obtain malicious response by implicitly providing LLMs with some clues about the original malicious query. |
RESIN-11: Schema-guided Event Prediction for 11 Newsworthy Scenarios (2022.naacl-demo)
Copied to clipboard
Xinya Du, Zixuan Zhang, Sha Li, Pengfei Yu, Hongwei Wang, Tuan Lai, Xudong Lin, Ziqi Wang, Iris Liu, Ben Zhou, Haoyang Wen, Manling Li, Darryl Hannan, Jie Lei, Hyounghun Kim, Rotem Dror, Haoyu Wang, Michael Regan, Qi Zeng, Qing Lyu, Charles Yu, Carl Edwards, Xiaomeng Jin, Yizhu Jiao, Ghazaleh Kazeminejad, Zhenhailong Wang, Chris Callison-Burch, Mohit Bansal, Carl Vondrick, Jiawei Han, Dan Roth, Shih-Fu Chang, Martha Palmer, Heng Ji
| Challenge: | Existing methods for event prediction are incomplete and noisy. |
| Approach: | They propose to use news-related event schemas to extract newsworthy events . they build a demo website and include a video demonstrating the framework . |
| Outcome: | The proposed framework can be applied to a wide variety of newsworthy scenarios. |
BattleAgent: Multi-modal Dynamic Emulation on Historical Battles to Complement Historical Analysis (2024.emnlp-demo)
Copied to clipboard
Shuhang Lin, Wenyue Hua, Lingyao Li, Che-Jui Chang, Lizhou Fan, Jianchao Ji, Hang Hua, Mingyu Jin, Jiebo Luo, Yongfeng Zhang
| Challenge: | Recent advances in large language models have demonstrated impressive reasoning capabilities, indicating their potential to serve as the foundation for agents. |
| Approach: | They propose a detailed emulation system that combines large vision-language model and multi-agent system to emulate dynamic interactions between multiple agents over a period of time. |
| Outcome: | The proposed system combines large vision-language model and multi-agent system to emulate dynamic interactions between agents and their environments over a period of time. |
StructFlowBench: A Structured Flow Benchmark for Multi-turn Instruction Following (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluation benchmarks focus on fine-grained constraint satisfaction and domain-specific capability assessment, yet overlook the crucial structural dependencies between dialogue turns that distinguish multi-turn from single-turn interactions. |
| Approach: | They propose a multi-turn instruction following benchmark with structural flow modeling that defines an innovative structural flow framework with six fundamental inter-turn relationships. |
| Outcome: | The proposed model is based on a framework with six fundamental inter-turn relationships and is able to analyze and generate specific dialogue flows tailored to specific scenarios. |
Pun-GAN: Generative Adversarial Network for Pun Generation (D19-1)
Copied to clipboard
| Challenge: | Existing methods for generating pun sentences with word senses lack large-scale corpus for supervised learning . a pun is a clever and amusing use of a word with two meanings (word senses) |
| Approach: | They propose an adversarial generative network for pun generation with a generator and a discriminator to distinguish between generated pun sentences and real sentences with specific word senses. |
| Outcome: | The proposed network generates sentences that are more ambiguous and diverse in both automatic and human evaluation. |
AIGuard: A Benchmark and Lightweight Detection for E-commerce AIGC Risks (2025.findings-acl)
Copied to clipboard
Wenhua Zhang, Weicheng Li, Xuanrong Rao, Lixin Zou, Xiangyang Luo, Chubin Zhuang, Yongjie Hong, Zhen Qin, Hengyu Chang, Chenliang Li, Bo Zheng
| Challenge: | Existing detection methods lack real-world scenarios and corresponding risk datasets . current MLLMs lack knowledge and have limited capability to detect the risk of AIGC content. |
| Approach: | They propose a benchmark for AIGC risk detection in real-world e-commerce . it includes 253,420 image-text pairs across four critical categories . |
| Outcome: | The proposed method achieves 9.68% higher recall than leading multimodal models while using only 25% of training resources. |
MACAROON: Training Vision-Language Models To Be Your Engaged Partners (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large vision-language models (LVLMs) generate detailed responses even when questions are ambiguous or unanswerable, leading to hallucinations and bias issues. |
| Approach: | They propose a three-tiered hierarchy for questions of invalid, ambiguous, and personalizable nature to measure the proactive engagement capabilities of LVLMs. |
| Outcome: | The proposed model generates contrastive response pairs for unlabeled questions, achieving 0.84 AAR, while maintaining comparable performance on general tasks. |
CORE: Cooperative Training of Retriever-Reranker for Effective Dialogue Response Selection (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods to train retrieval-based dialogue systems are suboptimal . existing methods to optimize retrieval and rerank modules are sub-optimal, causing sub-optimum performance. |
| Approach: | They propose a retrieval-based dialogue system with a fast retriever and a smart response reranker that combine the best of both worlds. |
| Outcome: | The proposed method can learn from each other and evolve together . it can be used in industrial applications and has powered industrial applications. |
Large Language Model Evaluation via Matrix Nuclear-Norm (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) are computationally intensive due to their O(n3) time complexity with Singular Value Decomposition (SVD). |
| Approach: | They propose a metric to quantify the data compression proficiency of large language models and a convex approximation of matrix rank to capture both predictive discriminability and diversity. |
| Outcome: | The proposed model achieves speeds 8 to 24 times faster than Matrix Entropy for the CEREBRAS-GPT model as models increase from 111M to 6.7B . |
ChipSeek: Optimizing Verilog Generation via EDA-Integrated Reinforcement Learning (2026.acl-long)
Copied to clipboard
Zhirong Chen, Kaiyan Chang, Zhuolin Li, Cangyuan Li, Xinyang He, Chujie Chen, Mengdi Wang, Haobo Xu, Yinhe Han, Huawei Li, Ying Wang
| Challenge: | Existing approaches to optimize Register-Transfer Level (RTL) code fail to simultaneously optimize functional correctness and hardware efficiency metrics such as Power, Performance, and Area (PPA). |
| Approach: | They propose a hierarchical reward based reinforcement learning framework that integrates direct feedback from EDA simulators and synthesis tools into a reward mechanism. |
| Outcome: | The proposed framework integrates direct feedback from EDA simulators and synthesis tools into a hierarchical reward based reinforcement learning framework. |
The steerability of large language models toward data-driven personas (2024.naacl-long)
Copied to clipboard
Junyi Li, Charith Peris, Ninareh Mehrabi, Palash Goyal, Kai-Wei Chang, Aram Galstyan, Richard Zemel, Rahul Gupta
| Challenge: | Large language models generate biased responses where opinions of certain groups and populations are underrepresented. |
| Approach: | They propose a data-driven notion of persona that allows for a more nuanced understanding of different (latent) social groups present in the population. |
| Outcome: | The proposed method improves model steerability by 57% over baselines. |
COVID-19 Literature Knowledge Graph Construction and Drug Repurposing Report Generation (2021.naacl-demos)
Copied to clipboard
Qingyun Wang, Manling Li, Xuan Wang, Nikolaus Parulian, Guangxing Han, Jiawei Ma, Jingxuan Tu, Ying Lin, Ranran Haoran Zhang, Weili Liu, Aabhas Chauhan, Yingjun Guan, Bangzheng Li, Ruisong Li, Xiangchen Song, Yi Fung, Heng Ji, Jiawei Han, Shih-Fu Chang, James Pustejovsky, Jasmine Rah, David Liem, Ahmed ELsayed, Martha Palmer, Clare Voss, Cynthia Schneider, Boyan Onyshkevych
| Challenge: | a new framework to digest relevant biomedical knowledge is needed to combat COVID-19 . quantity of research results is a bottleneck, and false information promoted in publications . |
| Approach: | a team of researchers has developed a framework to extract multimedia knowledge elements from scientific literature to combat COVID-19. |
| Outcome: | a new framework extracts fine-grained multimedia knowledge elements from scientific literature . it provides detailed contextual sentences, subfigures, and knowledge subgraphs as evidence . the framework is based on a case study of drug repurposing . |
LM2Protein: A Structure-to-Token Protein Large Language Model (2025.findings-emnlp)
Copied to clipboard
| Challenge: | RNA-binding proteins are critical for various molecular functions, relying on their precise tertiary structures. |
| Approach: | They propose a method to integrate protein 3D structural data within a sequence processing framework. |
| Outcome: | The proposed method achieves high sequence recovery in inverse folding and protein-conditioned RNA design. |
BRIEF-Pro: Universal Context Compression with Short-to-Long Synthesis for Fast and Accurate Multi-Hop Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Experiments show that BRIEF-Pro generates more concise and relevant summaries, enhancing performance across small, large, and proprietary language models. |
| Approach: | They propose a universal, lightweight compressor that distills relevant evidence from retrieved documents into a concise summary for seamless integration into in-context RAG. |
| Outcome: | Experiments on four open-domain multi-hop question-answering datasets show that BRIEF-Pro generates more concise and relevant summaries, enhancing performance across small, large, and proprietary language models. |
Vulnerability of LLMs to Vertically Aligned Text Manipulations (2025.acl-long)
Copied to clipboard
| Challenge: | Recent research shows that vertical text input significantly degrades the accuracy of large language models (LLMs) in text classification tasks. |
| Approach: | They investigate the impact of vertical text input on the performance of LLMs . they find that chain of thought reasoning does not help LLM recognize vertical input . |
| Outcome: | The proposed model can significantly mislead models, posing a risk of bypassing detection in real-world scenarios involving harmful or sensitive information. |
Mimicking the Familiar: Dynamic Command Generation for Information Theft Attacks in LLM Tool-Learning System (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to attack Large Language Model (LLM) tool-learning systems are black-box oriented and rely on static commands that cannot adapt flexibly to the changes in user queries and the invocation chain. |
| Approach: | They propose a dynamic attack comment generation approach for information theft attacks in LLM tool-learning systems that mimics the familiar by inferring the information utilized by upstream tools. |
| Outcome: | The proposed approach outperforms baselines with +13.2% ASRTheft and can be generalized to new tool-learning systems to expose their information leakage risks. |
PRiSM: Benchmarking Phone Realization in Speech Models (2026.acl-long)
Copied to clipboard
Shikhar Bharadwaj, Chin-Jou Li, Yoonjae Kim, Kwanghee Choi, Eunjung Yeo, Ryan Soh-Eun Shim, Hanyu Zhou, Brendon Boldt, Karen Rosero, Kalvin Chang, Darsh Agrawal, Keer Xu, Chao-Han Huck Yang, Jian Zhu, Shinji Watanabe, David R. Mortensen
| Challenge: | Existing evaluations of phone recognition systems only measure surface-level transcription accuracy. |
| Approach: | They propose to standardize transcription-based evaluation and assess downstream utility in clinical, educational, and multilingual settings with transcription and representation probes. |
| Outcome: | The proposed system outperforms LALMs in clinical, educational, and multilingual settings. |
SparkRA: A Retrieval-Augmented Knowledge Service System Based on Spark Large Language Model (2024.emnlp-demo)
Copied to clipboard
Dayong Wu, Jiaqi Li, Baoxin Wang, Honghong Zhao, Siyuan Xue, Yanjie Yang, Zhijun Chang, Rui Zhang, Li Qian, Bo Wang, Shijin Wang, Zhixiong Zhang, Guoping Hu
| Challenge: | Large language models (LLMs) have shown remarkable achievements across various language tasks. |
| Approach: | They propose a scientific literature LLM and a knowledge service system based on it . they collect scientific literature and then pre-train it using autoregressive training . |
| Outcome: | The proposed system provides literature investigation, paper reading, and academic writing functions. |
Dynamic Generation of Multi LLM Agents Communication Topologies with Graph Diffusion Models (2026.acl-long)
Copied to clipboard
Eric Hanchen Jiang, Levina Li, Frank Wan, Xiao Liang, Sophia Yin, Yuchen Wu, Xinfeng Li, Yizhou Sun, Wei Wang, Kai-Wei Chang, Ying Nian Wu
| Challenge: | Existing frameworks rely on static or rule-based topologies that fail to adapt to task requirements. |
| Approach: | They propose a generative framework that generates highly task-adaptive topologies . they validated the framework on multiple benchmarks and validated it on multiple platforms . |
| Outcome: | The proposed framework outperforms existing frameworks in task-adaptive communication topologies. |
Mitigating Over-Refusal in Aligned Large Language Models via Inference-Time Activation Energy (2026.acl-long)
Copied to clipboard
Eric Hanchen Jiang, Weixuan Ou, Run Liu, Shengyuan Pang, Guancheng Wan, Ranjie Duan, Wei Dong, Kai-Wei Chang, XiaoFeng Wang, Ying Nian Wu, Xinfeng Li
| Challenge: | Existing safety alignment techniques prioritize mitigating harmful responses at the expense of overcautious behavior, leading models to incorrectly refuse benign requests. |
| Approach: | They propose a fine-tuning free framework to improve safety and reduce false refusals by dynamic, inference-time intervention. |
| Outcome: | The proposed framework raises compliance on the ORB-H benchmark from 57.3% to 82.6% while maintaining the baseline safety performance. |
MHGRL: An Effective Representation Learning Model for Electronic Health Records (2024.lrec-main)
Copied to clipboard
| Challenge: | Effective EHR representations are key to achieving high performance in healthcare applications. |
| Approach: | They propose a multimodal heterogeneous graph-enhanced representation learning to learn EHR representations using medical ontology and textual notes. |
| Outcome: | The proposed model outperforms baseline models on two real clinical datasets in downstream tasks. |
DAGCN: Distance-based and Aspect-oriented Graph Convolutional Network for Aspect-based Sentiment Analysis (2024.findings-naacl)
Copied to clipboard
| Challenge: | Recent advances in sentiment analysis tend to interference from local factors such as irrelevant words and edges, hindering the precise identification of opinion words. |
| Approach: | They propose a distance-based syntactic weight and Aspect-Fusion Attention to solve this problem. |
| Outcome: | The proposed model outperforms state-of-the-art models on three public datasets and verify its effectiveness. |
DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings (2022.naacl-main)
Copied to clipboard
Yung-Sung Chuang, Rumen Dangovski, Hongyin Luo, Yang Zhang, Shiyu Chang, Marin Soljacic, Shang-Wen Li, Scott Yih, Yoon Kim, James Glass
| Challenge: | Recent work shows that finetuning pretrained models with contrastive learning makes it possible to learn good sentence embeddings without labeled data. |
| Approach: | They propose an unsupervised contrastive learning framework for learning sentence embeddings . they use a masked language model to mask out the edited sentence . |
| Outcome: | The proposed framework outperforms SimCSE on semantic textual similarity tasks by 2.3 absolute points. |
Weight Perturbation as Defense against Adversarial Word Substitutions (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existence and pervasiveness of textual adversarial examples have raised serious concerns to security-critical applications. |
| Approach: | They propose to perform weight perturbations in the parameter space rather than the input feature space to improve adversarial robustness of NLP models. |
| Outcome: | The proposed method improves adversarial robustness of models by performing weight perturbations in the parameter space rather than the input feature space. |
Broaden the Vision: Geo-Diverse Visual Commonsense Reasoning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Generally, commonsense knowledge is correlated with culture and geographic locations and is only shared locally. |
| Approach: | They construct a Geo-Diverse Visual Commonsense Reasoning dataset to test vision-and-language models’ ability to understand cultural and geo-location-specific commonsense. |
| Outcome: | The proposed models perform better in non-Western regions including East Asia, South Asia, and Africa than in the Western regions. |
Towards More Realistic Chinese Spell Checking with New Benchmark and Specialized Expert Model (2024.lrec-main)
Copied to clipboard
Yue Wang, Zilong Zheng, Juntao Li, Zhihui Liu, Jinxiong Chang, Qishen Zhang, Zhongyi Liu, Guannan Zhang, Min Zhang
| Challenge: | Large Language Models (LLMs) have been gaining attention for their ability to perform a wide range of open-domain tasks . however, the performance of LLMs has yet to be comprehensively evaluated in realistic scenarios . |
| Approach: | They propose a task to evaluate the performance of Large Language Models (LLMs) they propose RCSC task to convert Chinese text into correct text . |
| Outcome: | The proposed task evaluates the performance of existing methods in Chinese text . the realistic Chinese spell checker can achieve state-of-the-art performance on the task . |
Unsupervised Vision-and-Language Pre-training Without Parallel Images and Captions (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models require large amounts of image-caption data for pre-training . existing models require expensive data collection and curation . |
| Approach: | They propose to conduct "mask-and-predict" pre-training on text-only and image-only corpora and introduce the object tags detected by an object recognition model as anchor points to bridge two modalities. |
| Outcome: | The proposed approach achieves performance close to a model pre-trained with aligned data, on four English benchmarks. |
Early Rumour Detection (N19-1)
Copied to clipboard
| Challenge: | Existing studies on rumour detection are concerned with timing, but few are interested in how early we can detect them. |
| Approach: | They propose a method that integrates reinforcement learning to learn the minimum number of posts required before classifying an event as a rumour. |
| Outcome: | The proposed model detects rumours earlier than state-of-the-art systems while maintaining comparable accuracy. |
Document-level Event Extraction via Heterogeneous Graph-based Interaction Model with a Tracker (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for document-level event extraction are not effective due to two challenges . existing methods fail to extract events whose arguments spread in multiple sentences . |
| Approach: | They propose a document-level event extraction model with a tracker to capture interdependency among the extracted events. |
| Outcome: | The proposed model outperforms existing models on a large-scale dataset by 2.8 F1 . it extracts multiple correlated events and event arguments that scatter across the document . |
Premise-based Multimodal Reasoning: Conditional Inference on Joint Textual and Visual Clues (2022.acl-long)
Copied to clipboard
Qingxiu Dong, Ziwei Qin, Heming Xia, Tian Feng, Shoujie Tong, Haoran Meng, Lin Xu, Zhongyu Wei, Weidong Zhan, Baobao Chang, Sujian Li, Tianyu Liu, Zhifang Sui
| Challenge: | Existing work in vision language cross-modal reasoning uses binary or multi-choice classification based on source image and textual query. |
| Approach: | They propose a task where a textual premise is the background presumption on each source image. |
| Outcome: | The proposed task is based on a dataset of 15,360 movie screenshots and human-curated premise templates from 6 pre-defined categories. |
Keywords and Instances: A Hierarchical Contrastive Learning Framework Unifying Hybrid Granularities for Text Generation (2022.acl-long)
Copied to clipboard
Mingzhe Li, XieXiong Lin, Xiuying Chen, Jinxiong Chang, Qishen Zhang, Feng Wang, Taifeng Wang, Zhongyi Liu, Wei Chu, Dongyan Zhao, Rui Yan
| Challenge: | Existing studies focus on contrastive learning on the instance level without discriminating the contribution of each word. |
| Approach: | They propose a hierarchical contrastive learning mechanism which can unify semantic meaning in the input text. |
| Outcome: | The proposed model outperforms baselines on storytelling, paraphrasing, dialogue generation, and storytelling tasks. |
ChemAmp: Amplified Chemistry Tools via Composable Agents (2026.findings-acl)
Copied to clipboard
Zhucong Li, Powei Chang, Jin Xiao, Zhijian Zhou, Qianyu He, Jiaqing Liang, Fenglei Cao, Xu Yinghui, Yuan Qi
| Challenge: | LLM-based agents are powerful tools for automating complex scientific workflows, especially in chemistry, but their single-task performance is limited by tool constraints. |
| Approach: | They propose a framework that optimizes the collective capabilities of specialized tools by dynamic coordination within individual tasks. |
| Outcome: | The proposed framework outperforms chemistry-specialized models, generalist LLMs, and agent systems with tool orchestration. |
MMDocIR: Benchmarking Multimodal Retrieval for Long Documents (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing benchmarks for multimodal document retrieval are lacking for evaluating performance of systems. |
| Approach: | They propose a benchmark that evaluates page-level and layout-level retrieval tasks . they use a rich dataset featuring 1,685 questions annotated by experts . |
| Outcome: | The proposed benchmark outperforms existing benchmarks in page-level and layout-level retrieval tasks. |
RED: Unleashing Token-Level Rewards from Holistic Feedback via Reward Redistribution (2025.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results demonstrate the superiority of our approach to aligning large language models with human preferences. |
| Approach: | They propose a method that evaluates and assigns specific credit to each token using an off-the-shelf reward model. |
| Outcome: | The proposed method evaluates and assigns specific credit to each token using an off-the-shelf reward model. |
Advancing the Robustness of Large Language Models through Self-Denoised Smoothing (2024.naacl-short)
Copied to clipboard
Jiabao Ji, Bairu Hou, Zhen Zhang, Guanhua Zhang, Wenqi Fan, Qing Li, Yang Zhang, Gaowen Liu, Sijia Liu, Shiyu Chang
| Challenge: | Existing adversarial attacks can cause LLMs to make wrong predictions on downstream tasks or generate harmful content misaligned with human values. |
| Approach: | They propose to use randomized smoothing to add noise to the input and then make predictions based on these denoised versions. |
| Outcome: | The proposed method surpasses existing methods in both empirical and certified robustness in defending against adversarial perturbations for both downstream tasks and human alignments (i.e., jailbreak attacks). |
Using Social and Linguistic Information to Adapt Pretrained Representations for Political Perspective Identification (2021.findings-acl)
Copied to clipboard
| Challenge: | a new framework for political perspective detection is proposed to improve text training costs . current deep learning models lack the ability to focus on text span for bias detection . |
| Approach: | They propose a framework that pretrains the text model using social and linguistic contexts . they demonstrate that the framework improves performance by identifying bias-related text spans based on entity mentions and news sharing . |
| Outcome: | The proposed framework improves on two news bias datasets and improves performance on the general source and task. |
MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning (2024.acl-long)
Copied to clipboard
Chengpeng Li, Zheng Yuan, Hongyi Yuan, Guanting Dong, Keming Lu, Jiancan Wu, Chuanqi Tan, Xiang Wang, Chang Zhou
| Challenge: | In math reasoning with large language models, fine-tuning data augmentation by query evolution and diverse reasoning paths is empirically verified effective. |
| Approach: | They propose to fine-tune data augmentation by query evolution and diverse reasoning paths. |
| Outcome: | The proposed model achieves new state-of-the-art on GSM8K and MATH. |
Complementary Evidence Identification in Open-Domain Question Answering (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing approaches to QA that only measure the relevance between the question and each paragraph are not effective. |
| Approach: | They propose a method that learns vector representations of passages and models the sufficiency and diversity within the selected set, in addition to the relevance between the question and passages. |
| Outcome: | The proposed method significantly improves the accuracy of complementary evidence selection in open-domain question answering domain. |
Does Acceleration Cause Hidden Instability in Vision Language Models? Uncovering Instance-Level Divergence Through a Large-Scale Empirical Study (2025.emnlp-main)
Copied to clipboard
| Challenge: | Current acceleration evaluations focus on minimal overall performance degradation . however, accelerated models can exhibit significant changes in instance-level predictions . |
| Approach: | They investigate whether accelerated vision-Language Models can still give the same answers as before . they found that accelerated models changed original answers up to 20% of the time . |
| Outcome: | The results show that accelerated models changed their original answers up to 20% of the time. |
Gradually Excavating External Knowledge for Implicit Complex Question Answering (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have gained attention for their human-comparable capabilities but they may not solve open-domain implicit questions due to out-of-date domain knowledge, one-shot generation and restricted comprehensiveness. |
| Approach: | They propose a gradual knowledge excavation framework for open-domain complex question answering using extrinsic knowledge and historical knowledge. |
| Outcome: | The proposed framework achieves 78.17% accuracy with less than 6% parameters of its competitors, setting new SOTA in the 10B LLM class. |
How do Language Models Reshape Entity Alignment? A Survey of LM-Driven EA Methods: Advances, Benchmarks, and Future (2025.emnlp-main)
Copied to clipboard
| Challenge: | Entity alignment (EA) is critical for knowledge graph (KG) integration. |
| Approach: | They propose a taxonomy that categorizes methods in three stages: data preparation, feature embedding, and alignment. |
| Outcome: | The proposed taxonomy categorizes methods in three key stages: data preparation, feature embedding, and alignment. |
EvoCoT: Overcoming the Exploration Bottleneck in Reinforcement Learning for LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to reinforcement learning with verifiable reward (RLVR) are limited by difficulty or lack of exploration. |
| Approach: | They propose a self-evolving curriculum learning framework based on chain-of-thought reasoning optimization that constrains exploration space by self-generating and verifying CoT trajectories. |
| Outcome: | The proposed framework enables LLMs to solve previously unsolved problems without external supervision and is compatible with various RL fine-tuning methods. |
Structured Representation Learning for Online Debate Stance Prediction (C18-1)
Copied to clipboard
| Challenge: | Existing models for understanding debate dialog ignore relationships between different topics and focus on textual content and user interaction. |
| Approach: | They propose to view this task as a representation learning problem and embed the text and authors jointly based on their interactions. |
| Outcome: | The proposed model can achieve significantly better results compared to competing models. |
StableMoE: Stable Routing Strategy for Mixture of Experts (2022.acl-long)
Copied to clipboard
| Challenge: | Existing learning-to-route methods suffer from the routing fluctuation issue . with the model scale growing, training speed will go slower and memory requirements are heavy . |
| Approach: | They propose a Mixture-of-Experts technique that can scale up the model size of Transformers with an affordable computational overhead. |
| Outcome: | The proposed method outperforms existing learning-to-route methods on language modeling and multilingual machine translation. |
How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition (2024.acl-long)
Copied to clipboard
Guanting Dong, Hongyi Yuan, Keming Lu, Chengpeng Li, Mingfeng Xue, Dayiheng Liu, Wei Wang, Zheng Yuan, Chang Zhou, Jingren Zhou
| Challenge: | supervised fine-tuning (SFT) is a technique used to enhance multiple abilities in large language models. |
| Approach: | They propose to study the interplay of data composition between mathematical reasoning, code generation, and general human-aligning abilities during supervised fine-tuning. |
| Outcome: | The proposed model improves math reasoning and code generation with increasing data amount . the proposed model size and SFT strategies can be used to learn multiple skills with different scaling patterns. |
𝒜3: Automatic Alignment Framework for Attributed Text Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches of aligning large language models to follow user instructions can lead to undue emphasis on irrelevant documents, which in turn reduces the quality of responses. |
| Approach: | They propose to use a framework to automatically generate high-quality attributed query-response pairs for both supervised fine-tuning and preference optimization stages without human annotation. |
| Outcome: | The proposed framework can generate high-quality attributed query-response pairs without human annotation without human intervention. |
A Survey on In-context Learning (2024.emnlp-main)
Copied to clipboard
Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, Xu Sun, Lei Li, Zhifang Sui
| Challenge: | In-context learning (ICL) is a new paradigm for natural language processing . large language models (LLMs) demonstrate the ability to learn from a few examples . |
| Approach: | They propose to explore ICL to evaluate and extrapolate the ability of large language models. |
| Outcome: | The proposed methods can be used to evaluate and extrapolate the ability of large language models. |
E-EVAL: A Comprehensive Chinese K-12 Education Evaluation Benchmark for Large Language Models (2024.findings-acl)
Copied to clipboard
Jinchang Hou, Chang Ao, Haihong Wu, Xiangtao Kong, Zhigang Zheng, Daijia Tang, Chengming Li, Xiping Hu, Ruifeng Xu, Shiwen Ni, Min Yang
| Challenge: | despite the rapid development of Large Language Models, there is no dedicated benchmark for evaluating LLMs in Chinese K-12 education. |
| Approach: | They propose to develop a benchmark specifically tailored for Chinese K-12 education. |
| Outcome: | EVAL is the first evaluation benchmark specifically tailored for Chinese K-12 education. |
Target-Aware Language Modeling via Granular Data Sampling (2024.emnlp-main)
Copied to clipboard
Ernie Chang, Pin-Jie Lin, Yang Li, Changsheng Zhao, Daeil Kim, Rastislav Rabatin, Zechun Liu, Yangyang Shi, Vikas Chandra
| Challenge: | Language model pretraining is the cornerstone of universal language models (LMs), creating generalpurpose representations to excel across a variety of downstream tasks. |
| Approach: | They propose to use multi-granular tokens to sample large-scale language models for domain-specific use cases. |
| Outcome: | The proposed model outperforms random sampled samples on eight benchmarks with 1% of the data and performs on par with the full RefinedWeb data. |
AutoSUIT Bench - Automated Security UnIt Test Benchmark for LLM Coding (2026.findings-acl)
Copied to clipboard
Samuel Osebe, Fan Yang, Junyi Li, Yue Gu, Yongxin Wang, Satyapriya Krishna, Kai-Wei Chang, Aram Galstyan, Rahul Gupta, Weitong Ruan
| Challenge: | Large Language Models (LLMs) are evolving rapidly on code generation tasks. |
| Approach: | They propose to automate the vulnerability code benchmark creation with iterative auto validation. |
| Outcome: | The proposed benchmark covers 232 CWE categories across C/C++, Java, and Python languages. |
Alleviating Over-smoothing for Unsupervised Sentence Representation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to learn better unsupervised sentence representations have been successful . over-smoothing problem in unsupervised sentences reduces the capacity of powerful PLMs . |
| Approach: | They propose a method to solve the over-smoothing problem in unsupervised sentence representations by combining negatives from PLMs intermediate layers. |
| Outcome: | The proposed method improves on different strong baselines on Semantic Textual Similarity and Transfer datasets. |
UltraIF: Advancing Instruction Following from the Wild (2025.emnlp-main)
Copied to clipboard
| Challenge: | a lack of transparency has resulted in a gap between research community and leading companies . large language models have demonstrated remarkable capabilities in following complex instructions . |
| Approach: | They propose a method to build large language models that can follow complex instructions with open-source data. |
| Outcome: | The proposed approach can synergize complex instructions and filter responses with evaluation questions. |
Retrieved Sequence Augmentation for Protein Representation Learning (2024.emnlp-main)
Copied to clipboard
Chang Ma, Haiteng Zhao, Lin Zheng, Jiayi Xin, Qintong Li, Lijun Wu, Zhihong Deng, Yang Lu, Qi Liu, Sheng Wang, Lingpeng Kong
| Challenge: | Using multiple sequence alignments (MSA) to extract evolutionary knowledge is limited. |
| Approach: | They propose to use multiple sequence alignments to augment protein representations . they propose to employ Retrieved Sequence Augmentation to enhance protein representation learning . |
| Outcome: | The proposed method surpasses MSA Transformer by 5% in structural and property prediction tasks while being 373 times faster. |
Compress to Impress: Unleashing the Potential of Compressive Memory in Real-World Long-Term Conversations (2025.coling-main)
Copied to clipboard
| Challenge: | Existing retrieval-based methods for long-term conversations face challenges in memory database management and accurate memory retrieval, hindering their efficacy in dynamic, real-world interactions. |
| Approach: | They propose a framework that eschews traditional retrieval modules and memory databases and adopts a “One-for-All” approach to manage memory generation, compression, and response generation. |
| Outcome: | The proposed framework produces more nuanced and human-like experiences than retrieval-based methods. |
Optimizing Language Models with Fair and Stable Reward Composition in Reinforcement Learning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has developed algorithms for reinforcement learning from human feedback and AI-generated feedback. |
| Approach: | They propose a method for reinforcement learning from human feedback and AI-generated feedback that incorporates weighting, ranking, and constraining to handle disparate rewards. |
| Outcome: | The proposed method reduces disparity and enhances stability among rewards . empirical results show that the proposed method is efficient and straightforward . |
Towards Robust Speech Representation Learning for Thousands of Languages (2024.emnlp-main)
Copied to clipboard
William Chen, Wangyou Zhang, Yifan Peng, Xinjian Li, Jinchuan Tian, Jiatong Shi, Xuankai Chang, Soumi Maiti, Karen Livescu, Shinji Watanabe
| Challenge: | XEUS is a cross-lingual encoder for universal speech that can be trained on 1 million hours of data across 4057 languages. |
| Approach: | They propose a Cross-lingual Encoder for Universal Speech that can be trained on 1 million hours of data across 4057 languages and a newly created corpus of 7400+ hours from 4057 . |
| Outcome: | The proposed model outperforms state-of-the-art models on several benchmarks and outperfies MMS 1B and w2v-BERT 2.0 v2 by 0.8% and 4.4% respectively. |
A Survey of Retentive Network (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on the effectiveness of the Retentive Networks have not yet been conducted. |
| Approach: | They propose a retention mechanism that integrates the inductive bias of recurrent neural networks with the parallelizable training advantages of attention-based models. |
| Outcome: | The proposed retention mechanism combines the inductive bias of recurrent neural networks with the parallelizable training advantages of attention-based models. |
Encoding Social Information with Graph Convolutional Networks forPolitical Perspective Detection in News Media (P19-1)
Copied to clipboard
| Challenge: | Identifying the political perspective shaping the way news events are discussed in the media is an important and challenging task. |
| Approach: | They propose a neural architecture for representing relational information to capture social context of news documents. |
| Outcome: | The proposed model performs better than supervised models in the supervised setting and shows that it provides a distant supervision signal. |
MCP-Guard: A Multi-Stage Defense-in-Depth Framework for Securing Model Context Protocol in Agentic AI (2026.findings-acl)
Copied to clipboard
Wenpeng Xing, Zhonghao Qi, Yupeng Qin, Yilin Li, Caini Chang, Jiahui Yu, Changting Lin, Zhenzhen Xie, Meng Han
| Challenge: | Large Language Models (LLMs) are vulnerable to jailbreak, authors say . authors propose a robust, layered defense architecture designed for LLM–tool interactions . |
| Approach: | They propose a robust, layered defense architecture designed for LLM–tool interactions . they propose XCP-Guard, which employs a three-stage detection pipeline . |
| Outcome: | The proposed model achieves 96.01% accuracy in identifying adversarial prompts . the model is based on a three-stage detection pipeline that balances efficiency with accuracy . |
Symbolic Chain-of-Thought Distillation: Small Models Can Also “Think” Step-by-Step (2023.acl-long)
Copied to clipboard
| Challenge: | Symbolic Chain-of-thought Distillation (SCoTD) is a method to train a smaller student model on rationalizations sampled from a significantly larger teacher model. |
| Approach: | They propose a method to train a smaller student model on rationalizations from a larger teacher model. |
| Outcome: | The proposed method improves the performance of a student model in supervised and few-shot settings and especially for challenge sets. |
One Shot Dominance: Knowledge Poisoning Attack on Retrieval-Augmented Generation Systems (2025.findings-emnlp)
Copied to clipboard
Zhiyuan Chang, Mingyang Li, Xiaojun Jia, Junjie Wang, Yuekai Huang, Ziyou Jiang, Yang Liu, Qing Wang
| Challenge: | Existing knowledge poisoning attacks against RAG systems require multiple poisoned documents or can only function effectively on simplistic queries. |
| Approach: | They propose a more realistic knowledge poisoning attack that poisons only a single document while remaining effective for complex multi-hop questions involving complex relationships between multiple elements. |
| Outcome: | The proposed attack achieves success by poisoning only a single document while remaining effective for complex multi-hop questions involving complex relationships between multiple elements. |
Structural Contrastive Pretraining for Cross-Lingual Comprehension (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to train multilingual language models using pretraining tasks like mask language modeling have yielded promising results on a wide range of downstream tasks. |
| Approach: | They propose a new task to align the structural words in a parallel sentence, enhancing models’ ability to comprehend cross-lingual representations. |
| Outcome: | The proposed task improves model's ability to comprehend cross-lingual representations by increasing the frequency of negative pairings. |
AutoMixer: Checkpoint Artifacts as Automatic Data Mixers (2025.acl-long)
Copied to clipboard
| Challenge: | In language model training, it is difficult to obtain the right data mixtures for various tasks as the relationship between data and tasks is difficult. |
| Approach: | They propose to identify checkpoint models based on their respective capabilities and leverage them as data mixers by using their aggregated first-order influence approximation over source data. |
| Outcome: | The proposed framework shows significant improvements on eight reasoning benchmarks, with accuracy increases of up to 1.93%. |
Translation-Enhanced Multilingual Text-to-Image Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing models for text-to-image generation are mostly based on the English language due to the lack of annotated image-caption data in other languages. |
| Approach: | They propose to use a multilingual multi-modal encoder to bootstrap mTTI systems that can be translated into other languages. |
| Outcome: | The proposed approach mitigates the language gap and improves on standard mTTI datasets. |
Towards Generalizable and Robust Text-to-SQL Parsing (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Text-to-SQL parsers must be generalizable and robust against input perturbations. |
| Approach: | They propose a novel framework to learn text-to-SQL parsing in stages to improve parser's ability to acquire general SQL knowledge instead of capturing spurious patterns. |
| Outcome: | The proposed framework achieves state-of-the-art performance on the Spider, SParC, and CoSQL datasets. |
One-Teacher and Multiple-Student Knowledge Distillation on Sentiment Classification (2022.coling-1)
Copied to clipboard
| Challenge: | Existing knowledge distillation models require large computing resources and long inference time to perform. |
| Approach: | They propose a one-teacher and multiple-student knowledge distillation approach to distill a deep pre-trained teacher model into multiple shallow student models with ensemble learning. |
| Outcome: | The proposed method achieves better results with fewer parameters and extremely high speedup ratios on three sentiment classification tasks. |
AHEAD: Attention Head Energy-Aware Dynamics for Hallucination Mitigation in MLLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to hallucination mitigation ignore heterogeneous behaviors of attention heads . hallucinosity is a critical barrier to multimodal large language models' reliability, authors say . |
| Approach: | They propose a framework that quantifies the energetic properties of each attention head during object generation through two potential networks and dynamically adjusts their contributions at inference time. |
| Outcome: | The proposed framework reduces hallucination rates without fine-tuning the base model while maintaining generation quality. |
ControlMath: Controllable Data Generation Promotes Math Generalist Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Currently, mathematical reasoning is one of the most challenging areas for closed-source LLMs. |
| Approach: | They propose an iterative method involving an equation-generator module and two LLM-based agents that generate diverse equations and transform them into math word problems. |
| Outcome: | The proposed method enables the generation of diverse math problems, not limited to specific domains or distributions. |
All Changes May Have Invariant Principles: Improving Ever-Shifting Harmful Meme Detection via Design Concept Reproduction (2026.acl-long)
Copied to clipboard
Ziyou Jiang, Mingyang Li, Junjie Wang, Yuekai Huang, Jie Huang, Zhiyuan Chang, Zhaoyang Li, Qing Wang
| Challenge: | Existing methods for harmful meme detection only learn the combination of harmful elements and lack understanding of these implicit expressions. |
| Approach: | They propose a method that detects harmful memes by replicating the design concept of malicious users. |
| Outcome: | The proposed method achieves the highest accuracy with 81.1% and has slight accuracy decreases when generalized to type-shifting and temporal-evolving memes. |
When to Use What: An In-Depth Comparative Empirical Analysis of OpenIE Systems for Downstream Applications (2023.acl-long)
Copied to clipboard
| Challenge: | Existing consensus on which OpenIE model is best for each application is lacking . different assumptions made by different models and datasets have a statistically significant effect on performance, making it important to choose the most appropriate OpenIE system for one’s applications. |
| Approach: | They propose to use OpenIE to extract relation tuples from plain text to compare different models and training sets to find the best model for their applications. |
| Outcome: | The proposed models perform well on a Complex QA application. |
GAIA: A Fine-grained Multimedia Knowledge Extraction System (2020.acl-demos)
Copied to clipboard
Manling Li, Alireza Zareian, Ying Lin, Xiaoman Pan, Spencer Whitehead, Brian Chen, Bo Wu, Heng Ji, Shih-Fu Chang, Clare Voss, Daniel Napierski, Marjorie Freedman
| Challenge: | Open source knowledge extraction tools are used for many real-world applications, but there is no comprehensive system for KE. |
| Approach: | They propose a multimedia knowledge extraction system that takes multimedia data from various sources and languages as input and creates a coherent, structured knowledge base. |
| Outcome: | The system achieves top performance at the recent NIST TAC SM-KBP2019 evaluation. |
Don’t Take the Premise for Granted: Evaluating the Premise Critique Ability of Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies assess LLMs’ reasoning ability in ideal settings, ignoring their vulnerabilities when faced with flawed premises. |
| Approach: | They propose to evaluate LLMs' ability to proactively identify and articulate errors in input premises. |
| Outcome: | The proposed model enables LLMs to proactively identify and articulate errors in input premises. |
Know Thy Enemy: Securing LLMs Against Prompt Injection via Diverse Data Synthesis and Instruction-Level Chain-of-Thought Learning (2026.findings-acl)
Copied to clipboard
Zhiyuan Chang, Mingyang Li, Yuekai Huang, Ziyou Jiang, Xiaojun Jia, Qian Xiong, Junjie Wang, Zhaoyang Li, Qing Wang
| Challenge: | Large language model (LLM)-integrated applications face security vulnerabilities from prompt injection (PI) attacks. |
| Approach: | They propose a model enhancement method that synthesizes diverse training data and employs instruction-level chain-of-thought fine-tuning to enable LLMs to effectively identify and reject malicious instructions regardless of their source or position in the context. |
| Outcome: | The proposed method outperforms baselines in three critical dimensions while maintaining utility performance without degradation. |
Breaking the Ceiling of the LLM Community by Treating Token Generation as a Classification for Ensembling (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable capabilities in a wide range of tasks and contexts. |
| Approach: | They propose to use a token-level ensembling method to exploit the probability information at each generation step and to avoid early incorrect tokens. |
| Outcome: | The proposed method breaks the existing community performance ceiling and improves on several benchmarks. |
Path Spuriousness-aware Reinforcement Learning for Multi-Hop Knowledge Graph Reasoning (2023.eacl-main)
Copied to clipboard
| Challenge: | Multi-hop reasoning is a common approach for query answering, but can be biased to spurious paths which coincidentally lead to the correct answer with poor explanation. |
| Approach: | They propose a method that quantitatively estimates to what extent a path is spurious by a metric called Path Spuriousness (PS) they propose KG reasoning, which infers new facts along existing paths in KGs. |
| Outcome: | The proposed model significantly improves the agent’s ability to prevent spurious paths while keeping comparable to state-of-the-art performance. |
METAL: A Multi-Agent Framework for Chart Generation with Test-Time Scaling (2025.acl-long)
Copied to clipboard
| Challenge: | Chart generation requires strong visual design skills and precise coding capabilities that embed the desired visual properties into code. |
| Approach: | They propose a vision-language model-based multi-agent framework for effective automatic chart generation. |
| Outcome: | The proposed framework achieves a 5.2% improvement in the F1 score over the current best chart generation task. |
Understanding ME? Multimodal Evaluation for Fine-grained Visual Commonsense (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that understand image and text but also cross-reference in-between are lacking in evaluation data resources. |
| Approach: | They propose a multimodal evaluation pipeline to automatically generate question-answer pairs to test models’ understanding of the visual scene, text, and related knowledge. |
| Outcome: | The proposed model can answer the highly semantic VCR question correctly but fails to answer related visual question (Q2), textual question (q3), and background knowledge question ( Q4) as shallow mappings with language priors and unbalanced utilization of information between modalities. |
T2R-BENCH: A Benchmark for Real World Table-to-Report Task (2025.emnlp-main)
Copied to clipboard
Jie Zhang, Changzai Pan, Sishi Xiong, Kaiwen Wei, Yu Zhao, Xiangyu Li, Jiaxin Peng, Xiaoyan Gu, Jian Yang, Wenhan Chang, Zhenhe Wu, Jiang Zhong, Shuangyong Song, Xuelong Li
| Challenge: | Existing table benchmarks lack the capacity to adequately assess the practical application of table reasoning in industrial applications. |
| Approach: | They propose a bilingual table-to-report task and a table-based benchmark to assess the quality of table reasoning. |
| Outcome: | The proposed task is based on a bilingual benchmark with 457 industrial tables and evaluation criteria to measure the quality of report generation. |
DRS: Deep Question Reformulation With Structured Output (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing models like GPT-3 and Instruct-GPT lack the ability to reformulate unanswerable questions. |
| Approach: | They propose a zero-shot method that combines the strengths of LLMs with a DFS-based algorithm to iteratively explore potential entity combinations and constrain outputs using predefined entities. |
| Outcome: | The proposed method outperforms all baselines, including the GPT-3.5 model, on the unanswerable question reformulation task. |
Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm (2022.acl-long)
Copied to clipboard
Shaoyi Huang, Dongkuan Xu, Ian Yen, Yijue Wang, Sung-En Chang, Bingbing Li, Shiyang Chen, Mimi Xie, Sanguthevar Rajasekaran, Hang Liu, Caiwen Ding
| Challenge: | Conventional wisdom in pruning Transformer-based language models is that it reduces model expressiveness, but new research shows pruning increases risk of overfitting when performed at the fine-tuning phase. |
| Approach: | They propose to reduce pruning risk under pretrain-and-finetune paradigm . they propose to use knowledge distillation to improve pruning performance . |
| Outcome: | The proposed method outperforms the leading competitors on the GLUE benchmark. |
ToneCraft: Cantonese Lyrics Generation with Harmony of Tones and Pitches (2025.emnlp-main)
Copied to clipboard
| Challenge: | Current research has not addressed the challenge of generating harmonious Cantonese lyrics. |
| Approach: | They propose a framework for generating Cantonese lyrics that ensures tonal and melodic harmony. |
| Outcome: | The proposed framework ensures tonal and melodic harmony while preserving character count and quality. |
NiuTrans.LMT: Toward Inclusive and Scalable Multilingual Machine Translation with LLMs (2026.acl-long)
Copied to clipboard
Yingfeng Luo, Ziqiang Xu, Yuxuan Ouyang, MuRun Yang, DingYang Lin, Kaiyan Chang, Tong Zheng, Bei Li, Peinan Feng, Quan Du, Tong Xiao, JingBo Zhu
| Challenge: | Large language models have significantly advanced Multilingual Machine Translation (MMT) yet scaling to many languages while maintaining robust performance across directions remains challenging. |
| Approach: | They propose a strategy to reduce the number of translations in one direction . they propose auxiliary parallel sentences to promote cross-lingual transfer . |
| Outcome: | The proposed model performs on par with or better than substantially larger baselines. |
Dataset Bias Mitigation in Multiple-Choice Visual Question Answering and Beyond (2023.findings-emnlp)
Copied to clipboard
Zhecan Wang, Long Chen, Haoxuan You, Keyang Xu, Yicheng He, Wenhao Li, Noel Codella, Kai-Wei Chang, Shih-Fu Chang
| Challenge: | Existing studies have examined dataset biases in VQA benchmarks with short-phrase answers Multiple-choice Question with the LONG Answers (VCR, VLEP, etc.) |
| Approach: | They propose to use Adversarial Data Synthesis (ADS) to generate synthetic training and debiased evaluation data and introduce Intra-sample Counterfactual Training (ICT) to assist models in utilizing synthesized training data. |
| Outcome: | The proposed approach improves model performance even in domain-shifted scenarios. |
Quantification of Large Language Model Distillation (2025.acl-long)
Copied to clipboard
Sunbowen Lee, Junting Zhou, Chang Ao, Kaige Li, Xeron Du, Sirui He, Haihong Wu, Tianci Liu, Jiaheng Liu, Hamid Alinejad-Rokny, Min Yang, Yitao Liang, Zhoufutu Wen, Shiwen Ni
| Challenge: | Existing studies have revealed the robustness degra-dation caused by data distillation. |
| Approach: | They propose a framework to evaluate and quantify model distillation . they aim to identify identity cognition contradictions and analyse multi-granularity response similarities across models to measure the extent of homogenization. |
| Outcome: | The proposed framework addresses two key aspects: (1) Identifying identity cognition contradictions to assess discrepancies in how models perceive and represent identity-related information; (2) Analyzing multi-granularity response similarities across models to measure the extent of homogenization. |
GATEAU: Selecting Influential Samples for Long Context Alignment (2025.emnlp-main)
Copied to clipboard
Shuzheng Si, Haozhe Zhao, Gang Chen, Yunshui Li, Kangyang Luo, Chuancheng Lv, Kaikai An, Fanchao Qi, Baobao Chang, Maosong Sun
| Challenge: | Existing studies have attempted to scale up the available data volume by synthesizing long instruction-following samples, but a lack of a well-defined strategy for ensuring data quality may introduce low-quality samples and restrict the model’s performance. |
| Approach: | They propose a framework to identify influential samples enriched with long-range dependency relations that can be used to align large language models to handle instructions with extremely long contexts. |
| Outcome: | The proposed framework identifies samples with long-range dependency relations and shows that the model trained on these samples exhibits better instruction-following and long-context understanding capabilities. |
Alleviating Performance Degradation Caused by Out-of-Distribution Issues in Embedding-Based Retrieval (2025.findings-emnlp)
Copied to clipboard
Haotong Bao, Jianjin Zhang, Qi Chen, Weihao Han, Zhengxin Zeng, Ruiheng Chang, Mingzheng Li, Hao Sun, Weiwei Deng, Feng Sun, Qi Zhang
| Challenge: | Recent studies reveal query out-of-distribution issues degrading ANN performance . a distribution regularizer is introduced into the encoder training objective to encourage alignment between query and base embeddings. |
| Approach: | They introduce a distribution regularizer into the encoder training objective to encourage alignment between query and base embeddings. |
| Outcome: | The proposed method consistently improves retrieval performance across multiple datasets. |
RouteLMT: Learned Sample Routing for Hybrid LLM Translation Deployment (2026.acl-industry)
Copied to clipboard
Yingfeng Luo, Hongyu Liu, DingYang Lin, Kaiyan Chang, Chenglong Wang, Bei Li, Quan Du, Tong Xiao, JingBo Zhu
| Challenge: | Existing routing strategies rely on heuristics, external predictors, or absolute quality estimation to capture whether the large model provides a worthwhile improvement over the small one. |
| Approach: | They propose a budget allocation problem for routing large model to large model . they propose heuristics, external predictors, or absolute quality estimation to determine the optimal signal for budgeted decisions. |
| Outcome: | The proposed model outperforms heuristics, quality/difficulty estimation baselines and achieves a superior quality–budget Pareto frontier. |
CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark (2022.acl-long)
Copied to clipboard
Ningyu Zhang, Mosha Chen, Zhen Bi, Xiaozhuan Liang, Lei Li, Xin Shang, Kangping Yin, Chuanqi Tan, Jian Xu, Fei Huang, Luo Si, Yuan Ni, Guotong Xie, Zhifang Sui, Baobao Chang, Hui Zong, Zheng Yuan, Linfeng Li, Jun Yan, Hongying Zan, Kunli Zhang, Buzhou Tang, Qingcai Chen
| Challenge: | a new benchmark for biomedical language understanding is being developed in Chinese . most benchmarks are limited to English, which makes it difficult to replicate success in other languages. |
| Approach: | They propose to use Chinese biomedical language understanding evaluation benchmarks to evaluate Chinese models. |
| Outcome: | The proposed benchmarks show that the current models perform worse than the human ceiling. |
Generating Sports News from Live Commentary: A Chinese Dataset for Sports Game Summarization (2020.aacl-main)
Copied to clipboard
| Challenge: | Existing methods to generate sports summarization tasks are laborintensive and infeasible. |
| Approach: | They propose a Chinese dataset for sports game summarization and a model that consists of a selector and rewriter to evaluate the correctness of generated sports summaries. |
| Outcome: | The proposed model performs better on ROUGE and the two designed scores. |
IMPACT: Importance-Aware Activation Space Reconstruction (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) achieve strong performance across domains but remain difficult to deploy in resource-constrained environments due to their massive size. |
| Approach: | They propose an importance-aware activation reconstruction framework that links compression to its effect on model performance. |
| Outcome: | Experiments show that IMPACT reduces model size by 55.4% while maintaining accuracy comparable to or better than state-of-the-art models. |
Towards Adversarially Robust Text Classifiers by Learning to Reweight Clean Examples (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing defense methods improve the adversarial robustness by making models adapt to training set augmented with some adversarials. |
| Approach: | They propose to introduce a reweighting mechanism to calibrate the training distribution to obtain robust models. |
| Outcome: | The proposed method minimizes the loss of validation set mixed with clean examples and adversarial ones in an online learning manner. |
Into the Gray Zone: Domain Contexts Can Blur LLM Safety Boundaries (2026.acl-long)
Copied to clipboard
Ki Sen Hung, Xi Yang, Chang Liu, Haoran Li, Kejiang Chen, Changxuan Fan, Tsun On Kwok, Weiming Zhang, Xiaomeng Li, Yangqiu Song
| Challenge: | a goal of LLM alignment is to balance usefulness with harmlessness, but this conflictes when knowledge serves both legitimate and malicious purposes. |
| Approach: | They propose a framework that combines safety-research contexts with adversarial interactions to exploit a vulnerability in Jargon queries. |
| Outcome: | a framework outperforms existing methods in analyzing Jargon queries, a study shows . it achieves 93% of attacks across seven models, while remaining useful, the authors say . |
Improving Context Fidelity via Native Retrieval-Augmented Reasoning (2025.emnlp-main)
Copied to clipboard
Suyuchen Wang, Jinlin Wang, Xinyu Wang, Shiqi Li, Xiangru Tang, Sirui Hong, Xiao-Wen Chang, Chenglin Wu, Bang Liu
| Challenge: | Existing approaches to fidelity to contexts rely on expensive supervised fine-tuning to generate evidence post-answer or train models to perform web searches without improving utilization of the given context. |
| Approach: | They propose a native retrieval-augmented reasoning framework that integrates in-context evidence with the model’s own retrieval capabilities. |
| Outcome: | The proposed approach outperforms supervised fine-tuning, retrieval-augmented generation methods, and external retrieval solutions on multiple real-world and counterfactual QA benchmarks. |
Asclepius: A Spectrum Evaluation Benchmark for Medical Multi-Modal Large Language Models (2025.acl-long)
Copied to clipboard
Jie Liu, Wenxuan Wang, Su Yihang, Jingyuan Huang, Yudi Zhang, Cheng-Yi Li, Wenting Chen, Xiaohan Xing, Kao-Jung Chang, Linlin Shen, Michael R. Lyu
| Challenge: | Medical Multi-Modal Large Language Models (Med-MLLMs) are a promising new form of artificial general intelligence due to their ability to tackle complex tasks. |
| Approach: | They propose a new benchmark that comprehensively assesses medical multi-modal large language models in terms of distinct medical specialties and different diagnostic capacities. |
| Outcome: | The proposed model covers 15 medical specialties and different diagnostic capacities, and excludes overlap with existing VQA dataset. |
Non-Sequential Graph Script Induction via Multimedia Grounding (2023.acl-long)
Copied to clipboard
| Challenge: | Existing scripts for everyday tasks are presented in a linear manner, which does not reflect the flexibility displayed by people executing tasks in real life. |
| Approach: | They propose to use loosely aligned videos to train a non-sequential graph script induction task by using a multimodal framework to ground procedural videos to WikiHow textual steps. |
| Outcome: | The proposed model outperforms the WikiHow linear baseline by 48.76% . it can predict future steps given a partial step sequence and generate explicit graph scripts . |
Decompose, Fuse and Generate: A Formation-Informed Method for Chinese Definition Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing definition generation methods take the source word as an indecomposable semantic unit, but in parataxis languages like Chinese, word meanings can be composed using the word formation process. |
| Approach: | They propose to use word formation features to enhance Definition Generation (DG) in Chinese to generate an explanatory text. |
| Outcome: | The proposed model enhances Definition Generation (DG) in Chinese by decomposing the word meaning into different semantic components. |
Think Earlier, Not Longer: Prompt Optimization via Reducing Unhealthy Exploration (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to improve reasoning performance ignore the presence of unhealthy exploration that increases token usage without contributing to effective problem-solving. |
| Approach: | They propose an entropy-dynamics-aware prompt optimization framework that trains a lightweight optimizer to generate concise clarifications. |
| Outcome: | The proposed framework reduces ambiguity-induced early-stage uncertainty while preserving the model's reasoning capabilities. |
Personalized Video Comment Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Generating personalized responses in video poses a unique challenge for language models. |
| Approach: | They propose a new automatic metric based on Large Language Models with few-shot in-context learning that measures quality from the aspects of emotion, language style and content relevance. |
| Outcome: | The proposed metric measures quality from emotion, language style and content relevance with human evaluations. |
RAGPPI: Retrieval-Augmented Generation Benchmark for Protein–Protein Interactions in Drug Discovery (2026.eacl-long)
Copied to clipboard
| Challenge: | Large Language Models and Retrieval-Augmented Generation (RAG) frameworks have supported Target ID, but no benchmark exists for identifying biological impacts of PPIs. |
| Approach: | They propose to build a factual question-answer benchmark of 4,420 question-announced pairs that focus on the potential biological impacts of PPIs. |
| Outcome: | The proposed benchmark is based on 4,420 question-answer pairs with expert-driven data annotation. |
What Does BERT with Vision Look At? (2020.acl-main)
Copied to clipboard
| Challenge: | Pre-trained visual grounded language models have improved performance on vision-and-language tasks but what they learn during pre-training remains unclear. |
| Approach: | They show that attention heads of visual grounded language models actively ground elements of language to image regions. |
| Outcome: | The attention heads of a visual grounded language model can ground elements to image regions, demonstrating their ability to detect syntactic relations between non-entity words and image regions. |
Aggregation of Reasoning: A Hierarchical Framework for Enhancing Answer Selection in Large Language Models (2024.lrec-main)
Copied to clipboard
Zhangyue Yin, Qiushi Sun, Qipeng Guo, Zhiyuan Zeng, Xiaonan Li, Tianxiang Sun, Cheng Chang, Qinyuan Cheng, Ding Wang, Xiaofeng Mou, Xipeng Qiu, Xuanjing Huang
| Challenge: | Recent advances in Chain-of-Thought prompting have facilitated significant breakthroughs for Large Language Models (LLMs) in complex reasoning tasks. |
| Approach: | They propose a hierarchical reasoning aggregation framework to address this problem . they propose dynamic sampling to adjust the number of reasoning chains . |
| Outcome: | The proposed framework outperforms existing ensemble methods on complex reasoning tasks. |
Towards Unified Representations of Knowledge Graph and Expert Rules for Machine Learning and Reasoning (2022.aacl-main)
Copied to clipboard
| Challenge: | Empirical study shows superiority of proposed method over time-tested knowledge-driven and data-driven methods. |
| Approach: | They propose a cognitive knowledge graph that unifies expert rules and relational facts as the substrate of machine learning and reasoning models. |
| Outcome: | Empirical results show the proposed method superior to time-tested methods . the proposed model can perform both learning and reasoning with labeled data . |
What Really Matters for Table LLMs? A Meta-Evaluation of Model and Data Effects (2026.findings-eacl)
Copied to clipboard
Naihao Deng, Sheng Zhang, Henghui Zhu, Shuaichen Chang, Jiani Zhang, Alexander Hanbo Li, Chung-Wei Hang, Hideo Kobayashi, Yiqun Hu, Patrick Ng
| Challenge: | a series of paradigm shifts have come with distinct characteristics and challenges associated with table modeling. |
| Approach: | They propose to replicate four table LLMs by instruction-tuning three foundation models on four existing datasets. |
| Outcome: | The results show that base model choice plays a more dominant role than training data itself. |
Surfer100: Generating Surveys From Web Resources, Wikipedia-style (2022.lrec-1)
Copied to clipboard
Irene Li, Alex Fabbri, Rina Kawamura, Yixin Liu, Xiangru Tang, Jaesung Tae, Chang Shen, Sally Ma, Tomoe Mizutani, Dragomir Radev
| Challenge: | Recent work on Wikipedia page generation focuses on generating the initial leading paragraph of a page, while recent pretrained language models improve upon both extractive and abstractive steps of previous models. |
| Approach: | They propose a pretrained language model that can be combined to generate Wikipedia-style summaries with sections using 100 reference human-collected surveys. |
| Outcome: | The proposed approach is compared with existing methods with 100 human-collected surveys. |
Unlocking the Black Box of Latent Reasoning: An Interpretability-Guided Approach to Intervention (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for large language models (LLMs) lack a coherent representation of reasoning steps. |
| Approach: | They propose a set of latent reasoning interventions that enable latent thinking and decode-time interventions that refine the latent process by imposing the identified geometric and semantic priors. |
| Outcome: | The proposed models unlock latent capabilities and improve reasoning accuracy without any parameter updates. |
Domain Representative Keywords Selection: A Probabilistic Approach (2022.findings-acl)
Copied to clipboard
| Challenge: | a probabilistic approach to select a subset of a target domain representative keywords is crucial for many downstream tasks in natural language processing. |
| Approach: | They propose a probabilistic approach to select a subset of a target domain representative keywords from a candidate set, contrasting with a context domain. |
| Outcome: | The proposed approach provides more importance to distinctive keywords than common keywords contrasting with the context domain. |
Enhancing Agentic Textual Graph Retrieval with Synthetic Stepwise Supervision (2026.acl-long)
Copied to clipboard
Ge Chang, Jinbo Su, Jiacheng Liu, Pengfei Yang, Yuhao Shang, Huiwen Zheng, Hongli Ma, Yan Liang, Yuanchun Li, Yunxin Liu
| Challenge: | Existing methods for integrating textual graphs with LLMs are limited by symbolic inference and high annotation costs. |
| Approach: | They propose a textual graph reasoning framework that integrates textual diagrams with large language models. |
| Outcome: | The proposed approach achieves 15.6% accuracy and 17.2% in F1 score on three common datasets. |