Papers by Yan Yu
Executing Instructions in Situated Collaborative Interactions (D19-1)
Copied to clipboard
Alane Suhr, Claudia Yan, Jack Schluger, Stanley Yu, Hadi Khader, Marwa Mouallem, Iris Zhang, Yoav Artzi
| Challenge: | a collaborative game with natural language instruction allows users to adapt to the system abilities by changing their language or deciding to accomplish tasks themselves. |
| Approach: | They propose a collaborative game where a user instructs a system to complete tasks, but acts alongside it. |
| Outcome: | The proposed game allows users to adapt to the system abilities by changing their language or deciding to accomplish tasks themselves. |
Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoning (2026.findings-acl)
Copied to clipboard
Yibo Yan, Shen Wang, Jiahao Huo, Jingheng Ye, Zhendong Chu, Xuming Hu, Philip S. Yu, Carla P Gomes, Bart Selman, Qingsong Wen
| Challenge: | Current scientific reasoning models struggle with generalization across domains and fall short of multimodal perception. |
| Approach: | They propose to use multimodal large language models to integrate text, images, and other modalities to enhance scientific reasoning. |
| Outcome: | The proposed models can integrate text, images, and other modalities and improve reasoning across disciplines. |
Dolphin: Moving Towards Closed-loop Auto-research through Thinking, Practice, and Feedback (2025.acl-long)
Copied to clipboard
Jiakang Yuan, Xiangchao Yan, Bo Zhang, Tao Chen, Botian Shi, Wanli Ouyang, Yu Qiao, Lei Bai, Bowen Zhou
| Challenge: | Recent studies show that AI-assisted research methods can improve research efficiency . a closed-loop framework is used to enhance the automation level of scientific research . |
| Approach: | They propose a closed-loop LLM-driven framework to enhance the automation level of scientific research. |
| Outcome: | The proposed framework improves the efficiency of scientific research by improving data analysis, accelerating computation, and fostering novel idea generation. |
Noise-robust Cross-modal Interactive Learning with Text2Image Mask for Multi-modal Neural Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing studies on multi-modal neural machine translation focus on visual information, but text and image may not match exactly, and visual noise is often ignored. |
| Approach: | They propose a noise-robust multi-modal interactive fusion approach with cross-modal relation-aware mask mechanism for MNMT. |
| Outcome: | The proposed model achieves state-of-the-art scores in all En-De, En-Fr and En-Cs translation tasks. |
ESGenius: Benchmarking LLMs on Environmental, Social, and Governance (ESG) and Sustainability Knowledge (2025.emnlp-main)
Copied to clipboard
Chaoyue He, Xin Zhou, Yi Wu, Xinjia Yu, Yan Zhang, Lei Zhang, Di Wang, Shengfei Lyu, Hong Xu, Wang Xiaoqiao, Wei Liu, Chunyan Miao
| Challenge: | ESGenius is a comprehensive benchmark for evaluating Large Language Models on ESG and sustainability knowledge. |
| Approach: | They introduce ESGenius, a benchmark for evaluating and enhancing ESG proficiency . they use a rigorous two-stage evaluation protocol and a repository of foundational frameworks . |
| Outcome: | ESGenius is a benchmark for evaluating and enhancing the proficiency of Large Language Models (LLMs) in ESG and sustainability-focused question answering. |
RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework (2025.acl-long)
Copied to clipboard
Kunlun Zhu, Yifan Luo, Dingling Xu, Yukun Yan, Zhenghao Liu, Shi Yu, Ruobing Wang, Shuo Wang, Yishan Li, Nan Zhang, Xu Han, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing evaluation metrics for RAG systems are lacking due to high costs of data construction and lack of factual accuracy. |
| Approach: | They propose a framework to evaluate RAG systems in specialized scenarios . they propose three new metrics to evaluate LLM-generated responses . |
| Outcome: | The proposed framework outperforms zero-shot and one-shot methods in terms of clarity, safety, conformity, and richness of generated samples. |
From Synthesis to Clinical Assistance: A Strategy-Aware Agent Framework for Autism Intervention based on Real Clinical Dataset (2026.acl-long)
Copied to clipboard
| Challenge: | Applied Behavior Analysis (ABA) is the gold standard for clinical intervention, but large language models struggle to adhere to its standardized procedures. |
| Approach: | They propose a strategy-aware framework to unify high-fidelity intervention dialogue synthesis and clinical decision support. |
| Outcome: | Experiments show that ASDAgent achieves nearly 80% strategic consistency with human experts. |
FastMem: Fast Memorization of Prompt Improves Context Awareness of Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models struggle with context awareness, leading to inaccuracies in tasks requiring faithful adherence to provided information. |
| Approach: | They propose a method to enhance LLMs' context awareness by updating only the last Feed-Forward Network module to maximize the likelihood of the prompt before inference . |
| Outcome: | The proposed method improves the accuracy of Llama 3-8B-Inst on the NQ-SWAP dataset from 59.1% to 71.6% and reduces the output structure failure rate of Qwen 1.5-4B-Chat from 34.9% to 25.5%. |
SafeSteer: A Decoding-level Defense Mechanism for Multimodal Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing defense methods rely on fine-tuning or inefficient post-hoc interventions, limiting their ability to address novel attacks. |
| Approach: | They propose a decoding-level defense mechanism that employs a lightweight discriminator to iteratively steer the decoding process toward safety. |
| Outcome: | The proposed method improves safety performance by up to 33.40% without fine-tuning on multiple MLLMs. |
“In-Dialogues We Learn”: Towards Personalized Dialogue Without Pre-defined Profiles through In-Dialogue Learning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to personalized dialogue generate pre-defined profiles that are time-consuming and labor-intensive to create. |
| Approach: | They propose a framework that leverages dialogue history to characterize personas without pre-defined profiles. |
| Outcome: | The proposed framework improves BLEU and ROUGE scores on three datasets and human evaluations further validate the proposed method. |
Multiplex Word Embeddings for Selectional Preference Acquisition (D19-1)
Copied to clipboard
| Challenge: | Existing word embeddings are limited in their ability to represent fixed vectors . instead, they incorporate relational dependencies of different words into their embeddables - a limitation that is addressed by a multiplex model . |
| Approach: | They propose a word embedding model which incorporates relational dependencies of different words into their embeddables. |
| Outcome: | The proposed model can be easily extended according to various relations among words. |
Beyond Noise: Characterizing Creative Potential in Unverifiable LLM Hallucinations (2026.acl-long)
Copied to clipboard
Yu Yan, Chunhong Zhang, Haiyu Zhao, Ziyang Zeng, Zihao Liu, Yongkang Wu, Jianzhou Diao, YiJie Chen, Shujie Wang, Zheng Hu
| Challenge: | Large Language Models generate outputs that extend beyond established knowledge . prior work does not characterize the unverifiable space as a whole . |
| Approach: | They propose a novelty-verifiability characterization that distinguishes Creative Synthesis from Groundless Fabrication by a conceptual creation task. |
| Outcome: | The proposed model distinguishes Creative Synthesis (Region A) from Groundless Fabrication (Regium B) it shows that Region A is non-negligible and robust, persisting across generation strategies, models, domains, and embedding choices. |
ImPart: Importance-Aware Delta-Sparsification for Improved Model Compression and Merging in LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Recent approaches to reduce resource requirements for task-specific large language models have been developed. |
| Approach: | They propose a delta compression approach that optimizes for importance of a model . they use SVD to dynamically adjust the sparsity ratios of different vectors based on their importance . |
| Outcome: | The proposed approach achieves state-of-the-art in retaining task-specific knowledge even at high sparsity ratios. |
PrinciplismQA: A Philosophy-Grounded Approach to Assessing LLM-Human Clinical Medical Ethics Alignment (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks lack systematic approaches to integrate philosophical frameworks and expert validation for ethical reasoning assessment. |
| Approach: | They propose a philosophy-grounded approach to assess medical ethics alignment . PrinciplismQA comprises 3,648 expert-validated questions spanning knowledge assessment and clinical reasoning . |
| Outcome: | PrinciplismQA provides a philosophy-grounded approach to assessing medical ethics alignment. |
Revealing the Attention Floating Mechanism in Masked Diffusion Models (2026.findings-acl)
Copied to clipboard
Xin Dai, Pengcheng Huang, Zhenghao Liu, Shuo Wang, Yukun Yan, Chaojun Xiao, Yu Gu, Ge Yu, Maosong Sun
| Challenge: | Masked diffusion models (MDMs) leverage bidirectional attention and a denoising process. |
| Approach: | They investigate the attention behaviors of Masked diffusion models by revealing the phenomenon of Attention Floating. |
| Outcome: | The proposed model doubles the performance of autoregressive models in knowledge-intensive tasks. |
LiGen: Active Lipid Generation via a Molecular Language Model (2026.acl-long)
Copied to clipboard
| Challenge: | Lipid nanoparticles (LNPs) can deliver cargos to tumor and immune cells . traditional approaches rely on experimental screening and expert judgment . |
| Approach: | They propose a method to generate lipid molecules efficiently and actively using deep learning. |
| Outcome: | The proposed method outperforms baseline methods on multiple cell lines and achieves a 30% improvement over the current methods. |
An Embarrassingly Easy but Strong Baseline for Nested Named Entity Recognition (2023.acl-short)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is the task to detect and classify entity spans in text. |
| Approach: | They propose to use Convolutional Neural Network to model spatial relations in NER . they use three commonly used nested NER datasets to compare their results . |
| Outcome: | The proposed model outperforms several proposed methods with the same pre-trained encoders in three nested NER datasets. |
Multi-hop Question Generation with Graph Convolutional Network (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on text-based QG focus on generating SQuAD-style questions. |
| Approach: | They propose a multi-hop question generation model that does context encoding in multiple hops with Graph Convolutional Network and encoder fusion via an Encoder Reasoning Gate. |
| Outcome: | Empirical results show that the proposed model generates fluent questions with high completeness and outperforms baselines on automatic evaluation metrics. |
KBAlign: Efficient Self Adaptation on Specific Textual Knowledge Bases (2025.findings-emnlp)
Copied to clipboard
Zheni Zeng, Yuxuan Chen, Shi Yu, Ruobing Wang, Yukun Yan, Zhenghao Liu, Shuo Wang, Xu Han, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing methods for retrieval-augmented generation (RAG) are limited and fine-tuning incurs prohibitive costs of external signals. |
| Approach: | They propose a self-supervised framework that enhances RAG systems through efficient model adaptation. |
| Outcome: | The proposed framework achieves 90% of the performance gain obtained through GPT-4-supervised adaptation while relying entirely on self-annotation of much smaller models. |
Judge as A Judge: Improving the Evaluation of Retrieval-Augmented Generation through the Judge-Consistency of Large Language Models (2025.findings-acl)
Copied to clipboard
Shuliang Liu, Xinze Li, Zhenghao Liu, Yukun Yan, Cheng Yang, Zheni Zeng, Zhiyuan Liu, Maosong Sun, Ge Yu
| Challenge: | Existing evaluation metrics cannot fairly evaluate the outputs of RAG models during training and evaluation. |
| Approach: | They propose a method which prompts LLMs to generate different judgments based on various combinations of judgment dimensions and utilizes the judge-consistency to evaluate these judgments. |
| Outcome: | The proposed method generates more accurate evaluations for RAG models across different RAG model and datasets. |
SRF: Enhancing Document-Level Relation Extraction with a Novel Secondary Reasoning Framework (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for document-level relation extraction ignore bidirectional mention interaction when generating relational features for entity pairs. |
| Approach: | They propose a document-level relation extraction model that incorporates bidirectional mention fusion and a simple yet effective evidence extraction module for relation prediction. |
| Outcome: | The proposed model achieves SOTA performance and the proposed method is effective and general when integrated into existing models. |
ErrorRadar: Benchmarking Complex Mathematical Reasoning of Multimodal Large Language Models Via Error Detection (2026.findings-acl)
Copied to clipboard
Yibo Yan, Shen Wang, Jiahao Huo, Hang Li, Boyan Li, Jiamin Su, Xiong Gao, YiFan Zhang, Tianlong Xu, Zhendong Chu, Aoxiao Zhong, Kun Wang, Hui Xiong, Philip S. Yu, Xuming Hu, Qingsong Wen
| Challenge: | Current mathematical benchmarks focus on evaluating MLLMs’ problem-solving ability, yet there is a crucial gap in addressing more complex scenarios such as error detection. |
| Approach: | They propose to evaluate multimodal error detection by evaluating two sub-tasks error step identification and error categorization. |
| Outcome: | The proposed task evaluates MLLMs' ability to handle multimodal questions compared to text-only models. |
KGPT: Knowledge-Grounded Pre-Training for Data-to-Text Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for data-to-text generation rely on labeled data, which is costly to acquire and limits their application to new tasks and domains. |
| Approach: | They propose to leverage pre-training and transfer learning to address this problem by leveraging a general knowledge-grounded generation model and a knowledge-based model. |
| Outcome: | The proposed model can generate knowledge-enriched text on a knowledge-grounded text corpus crawled from the web in three settings. |
AlphaQuanter: An End-to-End Tool-Augmented Agentic Reinforcement Learning Framework for Stock Trading (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing Large Language Models (LLMs) lack the end-to-end optimization needed to learn a coherent strategy from market feedback. |
| Approach: | They propose a single-agent framework that uses reinforcement learning to learn a dynamic policy over a transparent decision workflow. |
| Outcome: | The proposed framework achieves state-of-the-art performance on key financial metrics. |
Fake Alignment: Are LLMs Really Aligned Well? (2024.naacl-long)
Copied to clipboard
Yixu Wang, Yan Teng, Kexin Huang, Chengqi Lyu, Songyang Zhang, Wenwei Zhang, Xingjun Ma, Yu-Gang Jiang, Yu Qiao, Yingchun Wang
| Challenge: | Existing studies on large language models have shown that they are poorly aligned in practice. |
| Approach: | They propose a framework to evaluate safety in large language models . they propose two new metrics to quantify fake alignment and obtain corrected performance estimation. |
| Outcome: | The proposed framework and two metrics show that some models with purported safety are poorly aligned in practice. |
ProphetNet: Predicting Future N-gram for Sequence-to-SequencePre-training (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing sequence-to-sequence models are optimized for future n-gram prediction and n stream self-attention mechanism. |
| Approach: | They propose a self-supervised objective called future n-gram prediction and the proposed n stream self-attention mechanism to optimize the model for sequence-to-sequence learning. |
| Outcome: | The proposed model achieves state-of-the-art on CNN/DailyMail, Gigaword, and SQuAD 1.1 benchmarks compared to the models using the same scale pre-training corpus. |
What It Takes to Achieve 100% Condition Accuracy on WikiSQL (D18-1)
Copied to clipboard
| Challenge: | despite of its simplicity, none of the publicly reported structured query generation models can achieve an accuracy beyond 62%, which is far from enough for practical use. |
| Approach: | They propose a model that can achieve 88.6% condition accuracy on WikiSQL . they ask: why is the accuracy still low for such simple queries? |
| Outcome: | The proposed solution can reach up to 88.6% condition accuracy on the WikiSQL dataset. |
S2O: Early Stopping for Sparse Attention via Online Permutation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing block-granularity sparsification can reduce latency, but coarse blocks impose an intrinsic sparsity ceiling. |
| Approach: | They propose a method that performs early stopping for sparse attention via online permutation. |
| Outcome: | The proposed approach reduces the complexity of the model and its performance. |
Parallelism and Generation Order in Masked Diffusion Language Models: Limits Today, Potential Tomorrow (2026.findings-acl)
Copied to clipboard
Yangyang Zhong, Yanmei Gu, Zhengqing Zang, Xiaomeng Li, Yuqi Ding, Xibei Jia, Yuting Shen, Zhenzhong Lan, Liwang Zhu, Weiping Liu, Junlin Zhou, Haisheng Liu, Zhong Xin Yu, Pengxin Luo, Donglian Qi, Yunfeng Yan, Junbo Zhao
| Challenge: | Autoregressive (AR) language models dominate modern natural language processing due to strong likelihood-based training objectives and reliable left-to-right decoding. |
| Approach: | They characterize MDLM behavior along two dimensions: parallelism strength and generation order . authors propose a Generate-then-Edit paradigm that mitigates dependency loss . |
| Outcome: | The proposed model improves on tasks that require "backward information" the Generate-then-Edit paradigm improves parallel decoding efficiency while reducing dependency loss. |
Predicting Text Preference Via Structured Comparative Reasoning (2024.acl-long)
Copied to clipboard
Jing Nathan Yan, Tianqi Liu, Justin Chiu, Jiaming Shen, Zhen Qin, Yue Yu, Charumathi Lakshmanan, Yair Kurzion, Alexander Rush, Jialu Liu, Michael Bendersky
| Challenge: | Existing approaches to comparative reasoning rely on pretraining or fine-tuning models at the cost of massive human annotation and computation. |
| Approach: | They propose a model that prompts LLMs to generate structured intermediate comparisons by proposing aspects for comparison, followed by generating textual comparisons under each aspect. |
| Outcome: | The proposed model significantly reduces hallucination and improves consistency across various NLP tasks. |
MedEval: A Multi-Level, Multi-Task, and Multi-Domain Medical Benchmark for Language Model Evaluation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing medical datasets require high quality domain-specific datasets. |
| Approach: | They propose a multi-level, multi-task, and multi-domain medical benchmark to facilitate the development of language models for healthcare. |
| Outcome: | The proposed model provides granular potential usage and supports a wide range of tasks. |
Enhancing Long-Chain Reasoning Distillation through Error-Aware Self-Reflection (2026.findings-acl)
Copied to clipboard
Zhuoyang Wu, Xinze Li, Zhenghao Liu, Yukun Yan, Zhiyuan Liu, Minghe Yu, Cheng Yang, Yu Gu, Ge Yu, Maosong Sun
| Challenge: | Existing studies treat SLMs as student models and use long-form Chains-of-Thought (CoTs) as supervision signals for Supervised Fine-Tuning (SFT). Existing research focuses on distilling reasoning ability from LLMs to enhance the mathematical reasoning performance of small-scale models. |
| Approach: | They propose a framework that refines teacher CoTs through an error-aware reflection process to enable the student model to construct more tailored teacher Cots. |
| Outcome: | Experiments on multiple mathematical reasoning benchmarks show that ORION improves performance by more than 2% over all baselines. |
SLAM-Omni: Timbre-Controllable Voice Interaction System with Single-Stage Training (2025.findings-acl)
Copied to clipboard
Wenxi Chen, Ziyang Ma, Ruiqi Yan, Yuzhe Liang, Xiquan Li, Ruiyang Xu, Zhikang Niu, Yanqiao Zhu, Yifan Yang, Zhanxun Liu, Kai Yu, Yuxuan Hu, Jinyu Li, Yan Lu, Shujie Liu, Xie Chen
| Challenge: | a new spoken dialogue system with single-stage training is demonstrating its low latency and high quality . SLAM-Omni achieves zero-shot timbre control by modeling spoken language with semantic tokens . |
| Approach: | They propose a timbre-controllable, end-to-end voice interaction system with single-stage training. |
| Outcome: | The proposed system outperforms previous models on 4 GPUs with limited data. |
Harnessing Consistency for Robust Test-Time LLM Ensemble (2026.findings-eacl)
Copied to clipboard
Zhichen Zeng, Qi Yu, Xiao Lin, Ruizhong Qiu, Xuying Ning, Tianxin Wei, Yuchen Yan, Jingrui He, Hanghang Tong
| Challenge: | Existing efforts to improve LLM ensemble quality have focused on model consistency, but failures are often due to heterogeneous tokenization schemes and varying model expertise. |
| Approach: | They propose a plug-and-play technique that harnesses model consistency for robust LLM ensemble. |
| Outcome: | The proposed technique improves ensemble performance and robustness against erroneous signals. |
LLM Agents for Education: Advances and Applications (2025.findings-emnlp)
Copied to clipboard
Zhendong Chu, Shen Wang, Jian Xie, Tinghui Zhu, Yibo Yan, Jingheng Ye, Aoxiao Zhong, Xuming Hu, Jing Liang, Philip S. Yu, Qingsong Wen
| Challenge: | Large Language Model (LLM) agents are transforming education by automating complex tasks and enhancing both teaching and learning processes. |
| Approach: | This survey analyzes recent advances in applying Large Language Model agents to educational settings . it highlights ethical issues, hallucination and overreliance, and integration with existing ecosystems . |
| Outcome: | The authors analyze the technologies enabling LLM agents and highlight key challenges in deploying them in educational settings. |
Identifying Cellular Niches in Spatial Transcriptomics: An Investigation into the Capabilities of Large Language Models (2025.acl-long)
Copied to clipboard
Huanhuan Wei, Xiao Luo, Hongyi Yu, Jinping Liang, Luning Yang, Lixing Lin, Alexandra Popa, Xiting Yan
| Challenge: | Spatial transcriptomic technologies allow measuring gene expression profile and spatial information of cells in tissues simultaneously. |
| Approach: | They propose a spatial transcriptomic approach to identify spatial niches using a zero-shot large language models by transforming spatial transcriptomics data into spatial context prompts. |
| Outcome: | The proposed model improves performance by leveraging gene expression of neighboring cells/spots, cell type composition, tissue information, and external knowledge. |
Semantic Relation-aware Difference Representation Learning for Change Captioning (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to describe semantic change in images with distractors are difficult to learn . |
| Approach: | They propose a semantic relation-aware difference representation learning network to explicitly learn the difference representation in the existence of distractors. |
| Outcome: | The proposed network achieves state-of-the-art performance on CLEVR-Change and Spot-the -Diff datasets. |
Towards Mitigating LLM Hallucination via Self Reflection (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models have shown promise for generative and knowledge-intensive tasks including question-answering (QA) but the practical deployment still faces challenges, notably the issue of “hallucination”, where models generate plausible-sounding but unfaithful or nonsensical information. |
| Approach: | They propose a self-reflection methodology that incorporates knowledge acquisition and answer generation to address the issue of "hallucination" they use a set of LLMs to generate a more accurate and factually accurate answer. |
| Outcome: | The proposed approach improves factuality, consistency, and entailment of the generated answers. |
Covariance Matrix-Driven Image Channel Allocation for Multimodal Fake News Detection (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods focus on designing efficient multimodal fusion frameworks to bridge the semantic gap between images and texts. |
| Approach: | They propose a covariance matrix-driven image channel allocation method that expands the number of original channel maps and assigns importance scores to the expanded channel maps. |
| Outcome: | The proposed method achieves state-of-the-art on three public multimodal fake news detection benchmark datasets. |
TestAgent: An Adaptive and Intelligent Expert for Human Assessment (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing adaptive testing methods face several challenges due to mechanized nature of most algorithms and noisy response data. |
| Approach: | They propose to use large language models to enhance adaptive testing through interactive engagement to capture test-takers’ responses and anomalies. |
| Outcome: | The proposed agent achieves more accurate results with 20% fewer questions than state-of-the-art baselines and testers preferred it in speed, smoothness, and other dimensions. |
A Knowledge-driven Generative Model for Multi-implication Chinese Medical Procedure Entity Normalization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Medical entity normalization (NEN) is a task that links medical mentions to entities in knowledge bases. |
| Approach: | They propose a sequence generative framework to generate Chinese medical procedure entity normalization by constraint decoding and category-based model refining. |
| Outcome: | The proposed model improves on baselines especially in the case of multi-implication Chinese medical procedures. |
NusaCrowd: Open Source Initiative for Indonesian NLP Resources (2023.findings-acl)
Copied to clipboard
Samuel Cahyawijaya, Holy Lovenia, Alham Fikri Aji, Genta Winata, Bryan Wilie, Fajri Koto, Rahmad Mahendra, Christian Wibisono, Ade Romadhony, Karissa Vincentio, Jennifer Santoso, David Moeljadi, Cahya Wirawan, Frederikus Hudi, Muhammad Satrio Wicaksono, Ivan Parmonangan, Ika Alfina, Ilham Firdausi Putra, Samsul Rahmadani, Yulianti Oenang, Ali Septiandri, James Jaya, Kaustubh Dhole, Arie Suryani, Rifki Afina Putri, Dan Su, Keith Stevens, Made Nindyatama Nityasya, Muhammad Adilazuarda, Ryan Hadiwijaya, Ryandito Diandaru, Tiezheng Yu, Vito Ghifari, Wenliang Dai, Yan Xu, Dyah Damapuspita, Haryo Wibowo, Cuk Tho, Ichwanul Karo Karo, Tirana Fatyanosa, Ziwei Ji, Graham Neubig, Timothy Baldwin, Sebastian Ruder, Pascale Fung, Herry Sujaini, Sakriani Sakti, Ayu Purwarianti
| Challenge: | Existing NLP research in Indonesian languages has been held back by factors such as language diversity, orthographic variation, resource limitation and other societal challenges. |
| Approach: | They present a collaborative initiative to collect and unify existing resources for Indonesian languages and open access to previously non-public resources. |
| Outcome: | The results show that the datasets are highly reliable and can be used to generate the first zero-shot benchmarks for natural language understanding and generation in Indonesian and the local languages of Indonesia. |
Multilingual Generative Retrieval via Cross-lingual Semantic Compression (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for multilingual retrieval still face cross-lingual identifier misalignment and identifiere inflation. |
| Approach: | They propose a framework that unifies semantically equivalent multilingual keywords into shared atoms to align semantics and compresses the identifier space. |
| Outcome: | The proposed framework improves cross-lingual alignment and reduces redundancy. |
DeepNote: Note-Centric Deep Retrieval-Augmented Generation (2025.findings-emnlp)
Copied to clipboard
Ruobing Wang, Qingfei Zhao, Yukun Yan, Daren Zha, Yuxuan Chen, Shi Yu, Zhenghao Liu, Yixuan Wang, Shuo Wang, Xu Han, Zhiyuan Liu, Maosong Sun
| Challenge: | . - (EN) |
| Approach: | . - (EN) |
| Outcome: | . - (EN) |
MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualization (2024.findings-acl)
Copied to clipboard
Zhiyu Yang, Zihan Zhou, Shuo Wang, Xin Cong, Xu Han, Yukun Yan, Zhenghao Liu, Zhixing Tan, Pengyuan Liu, Dong Yu, Zhiyuan Liu, Xiaodong Shi, Maosong Sun
| Challenge: | Scientific data visualization is an essential process in research, but its use of large language models remains unexplored. |
| Approach: | They propose a model-agnostic LLM agent framework to automate scientific data visualization tasks. |
| Outcome: | The proposed framework improves performance of commercial and open-source models. |
DialSQL: Dialogue Based Structured Query Generation (P18-1)
Copied to clipboard
| Challenge: | Recent advances in deep learning and semantic parsing have improved the translation accuracy of natural language questions to structured queries. |
| Approach: | They propose a dialogue-based structured query generation framework that leverages human intelligence to boost performance of existing algorithms via user interaction. |
| Outcome: | The proposed framework improves on a WikiSQL dataset from 61.3% to 69.0% using only 2.4 validation questions per dialogue. |
MetaMem: Evolving Meta-Memory for Knowledge Utilization through Self-Reflective Symbolic Optimization (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing memory systems can support long-horizon human-LLM interactions by persisting historical interactions beyond limited context windows. |
| Approach: | They propose a framework that augments memory systems with a self-evolving meta-memory . meta-meso is iteratively distilling transferable knowledge utilization experiences . results show MetaMem outperforms strong baselines by over 3.6% . |
| Outcome: | The proposed framework outperforms baselines by over 3.6% in the long-horizon human-LLM interaction. |
V-MAGE: A Game Evaluation Framework for Assessing Vision-Centric Capabilities in Multimodal Large Language Models (2026.findings-acl)
Copied to clipboard
Xiangxi Zheng, Linjie Li, Zhengyuan Yang, Ping Yu, Alex Jinpeng Wang, Rui Yan, Yuan Yao, Lijuan Wang
| Challenge: | Existing static image-text benchmarks are insufficient for evaluating multimodal large language models’ dynamic perception and interactive reasoning abilities. |
| Approach: | They propose a game-based evaluation framework to assess multimodal large language models’ visual reasoning in dynamic, continuous-space environments. |
| Outcome: | The proposed framework systematically assesses MLLMs’ visual reasoning in dynamic, continuous-space environments. |
Mitigating Judgment Preference Bias in Large Language Models through Group-Based Polling (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are used as automatic evaluators to provide accurate and reliable assessments. |
| Approach: | They propose a framework that integrates LLM-based judgment models into a multi-agent system and simulates the interactive client-server polling mechanism. |
| Outcome: | The proposed framework outperforms supervised models trained on annotated judgment data while requiring no human-labeled annotations. |
How Large a Vocabulary Does Text Classification Need? A Variational Approach to Vocabulary Selection (N19-1)
Copied to clipboard
| Challenge: | Using a pre-defined vocabulary is a common approach to selecting text inputs . however, using a large vocabulary is not economical, as it limits the model's applicability on computation-or memoryconstrained scenarios. |
| Approach: | They propose a more sophisticated variational vocabulary dropout to perform vocabulary selection . they propose two new metrics to measure area under accuracy-vocab curve and Vocab Size under X% accuracy drop . |
| Outcome: | The proposed framework outperforms the baselines on the vocabulary selection problem on multiple NLP classification tasks. |
ThinkNote: Enhancing Knowledge Integration and Utilization of Large Language Models via Constructivist Cognition Modeling (2026.findings-eacl)
Copied to clipboard
Zhipeng Xu, Zhenghao Liu, Yukun Yan, Shuo Wang, Shi Yu, Zheni Zeng, Chaojun Xiao, Zhiyuan Liu, Ge Yu, Chenyan Xiong
| Challenge: | Large Language Models (LLMs) exhibit suboptimal behaviors and inconsistencies when exposed to unfamiliar external information, underscoring their limitations in effectively leveraging such knowledge. |
| Approach: | They propose a framework that enhances the external knowledge utilization of Large Language Models through a two-stage constructivist cognitive modeling process. |
| Outcome: | The proposed framework achieves a 10% improvement over baseline methods on various question-answering benchmarks. |
ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation (2021.acl-demo)
Copied to clipboard
Weizhen Qi, Yeyun Gong, Yu Yan, Can Xu, Bolun Yao, Bartuer Zhou, Biao Cheng, Daxin Jiang, Jiusheng Chen, Ruofei Zhang, Houqiang Li, Nan Duan
| Challenge: | Existing models for pre-training are not convenient for users to find and set them up. |
| Approach: | They propose to extend ProphetNet into other domains and languages by pre-training models . they pre-train a cross-lingual generation model ProphetNet-Multi and a Chinese generation model . |
| Outcome: | The proposed models achieve new state-of-the-art on 10 benchmarks. |
Re-TASK: Revisiting LLM Tasks from Capability, Skill, and Knowledge Perspectives (2025.findings-acl)
Copied to clipboard
Zhihu Wang, Shiwan Zhao, Yu Wang, Heyuan Huang, Sitao Xie, Yubo Zhang, Jiaxin Shi, Zhixing Wang, Hongyan Li, Junchi Yan
| Challenge: | Existing approaches to solving complex tasks with large language models (LLMs) fail to decompose tasks accurately or execute subtasks effectively. |
| Approach: | They propose a Chain-of-Learning (CoL) paradigm that highlights task dependencies on specific capability items, further broken down into their constituent knowledge and skill components. |
| Outcome: | The proposed model improves Yi-1.5-9B and Llama3-Chinese-8B for legal tasks by 45.00% and 24.50% on different domains. |
IAM: A Comprehensive and Large-Scale Dataset for Integrated Argument Mining Tasks (2022.acl-long)
Copied to clipboard
| Challenge: | Argument mining (AM) is a computational process that is used to analyze information in a debating system. |
| Approach: | They propose to use a large dataset to automate the manual process of debating . they propose to integrate claim extraction, stance classification and evidence extraction tasks . |
| Outcome: | The proposed tasks can extract claims, stances, evidence and more from a large dataset . the proposed tasks are highly efficient and can be applied to argument mining tasks . |
DSQG-Syn: Synthesizing High-quality Data for Text-to-SQL Parsing by Domain Specific Question Generation (2025.findings-naacl)
Copied to clipboard
Shaoming Duan, Youxuan Wu, Chuanyi Liu, Yuhao Zhang, Zirui Wang, Peiyi Han, Shengyuan Yu, Liang Yan, Yingwei Liang
| Challenge: | Existing methods for generating SQL queries using natural language questions produce inconsistent NLQ-SQL pairs. |
| Approach: | They propose a text-to-SQL data synthesis framework that generates domain-relevant questions . they synthesize NLQ-SqL pairs that are domain-specific and intent-consistent . |
| Outcome: | The proposed method outperforms closed-source LLMs on the Text-to-SQL task. |
DatawiseAgent: A Notebook-Centric LLM Agent Framework for Adaptive and Robust Data Science Automation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing large language model (LLM) agents for data science automation are limited by narrow task scopes, limited generalization across tasks and models, and over-reliance on state-of-the-art (SOTA) LLMs. |
| Approach: | They propose a notebook-centric LLM agent framework for adaptive and robust data science automation. |
| Outcome: | The proposed framework surpasses baselines such as AutoGen and TaskWeaver in performance tests across diverse data science scenarios and models. |
AgentRM: Enhancing Agent Generalization with Reward Modeling (2025.acl-long)
Copied to clipboard
Yu Xia, Jingru Fan, Weize Chen, Siyu Yan, Xin Cong, Zhong Zhang, Yaxi Lu, Yankai Lin, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing LLM-based agents have strong performance on held-in tasks, but their generalizability to unseen tasks remains poor. |
| Approach: | They propose a reward-based generalizable reward model to guide the policy model for effective test-time search. |
| Outcome: | The proposed agentRM outperforms existing agents on held-in tasks by 8.8 points on average. |
KERS: A Knowledge-Enhanced Framework for Recommendation Dialog Systems with Multiple Subgoals (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing frameworks for multi-subgoal dialogs require a system to build a social bond with users to gain trust and develop affinity. |
| Approach: | They propose a framework for common knowledge-based multi-subgoal dialogs that divides up conversations with multiple subgoals and propose mechanisms to filter noisy knowledge and to include cleaned knowledge in the dialog response generation process. |
| Outcome: | The proposed framework obtains state-of-the-art results on a DuRecDial dataset in both automatic and human evaluation. |
Do Large Language Models Truly Grasp Addition? A Rule-Focused Diagnostic Using Two-Integer Arithmetic (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) excel at complex math but fail on basic addition, raising the question of whether they grasp rules or are merely reproducing patterns. |
| Approach: | They systematically probe LLMs’ understanding of two-integer addition by testing three crucial properties: commutativity (A+B=B+A), representation invariance via symbolic remapping and consistent accuracy scaling with operand length. |
| Outcome: | The proposed models achieve high numeric accuracy but fail basic addition tasks. |
Global Textual Relation Embedding for Relational Understanding (P19-1)
Copied to clipboard
| Challenge: | Existing methods to learn textual relation embeddings are lacking in large open-domain corpora. |
| Approach: | They propose to learn a general-purpose embedding of textual relations using a large dataset from Freebase. |
| Outcome: | The proposed embedding can facilitate downstream tasks requiring relational understanding of the text. |
Multi-Scale Spectral Selection and Entropy-Guided Uncertainty Fusion for Multimodal Rumor Detection (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for multimodal content detection fail to capture cross-modal semantic inconsistencies and ignore inherent noise in multimodal features. |
| Approach: | They propose a multimodal rumor detection method based on a frequency domain spectral selection method and entropy-guided uncertainty fusion method to capture cross-modal semantic inconsistencies. |
| Outcome: | The proposed method outperforms state-of-the-art methods in multimodal rumor detection . it shows stronger detection capability and robustness on multiple datasets . |
Living in the Moment: Can Large Language Models Grasp Co-Temporal Reasoning? (2024.acl-long)
Copied to clipboard
| Challenge: | Current temporal reasoning datasets are limited to questions about single or isolated events, falling short in mirroring the realistic temporal characteristics involving concurrent nature and intricate temporal interconnections. |
| Approach: | They propose a co-temporal Question Answering benchmark that contains four co-time scenarios with 4,748 samples for evaluating the co-timing abilities of large language models. |
| Outcome: | The proposed benchmarks show that current LLMs struggle on CoTempQA tasks even when enhanced with Chain of Thought methodologies. |
FastSeq: Make Sequence Generation Faster (2021.acl-demo)
Copied to clipboard
Yu Yan, Fei Hu, Jiusheng Chen, Nikhil Bhendawade, Ting Ye, Yeyun Gong, Nan Duan, Desheng Cui, Bingyu Chi, Ruofei Zhang
| Challenge: | Transformer-based models have made tremendous impact in natural language generation, but inference speed is still a bottleneck due to large model size and intensive computing involved in auto-regressive decoding process. |
| Approach: | They propose an attention cache optimization, an efficient algorithm for detecting repeated n-grams, and an asynchronous generation pipeline with parallel I/O to accelerate sequence generation without loss of accuracy. |
| Outcome: | The proposed framework can accelerate the sequence generation by 4x to 9x with a simple one-line code change for a set of widely used and diverse models. |
MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial Application (2026.acl-long)
Copied to clipboard
Xueqing Peng, Lingfei Qian, Yan Wang, Ruoyu Xiang, Yueru He, Yang Ren, Mingyang Jiang, Vincent Jim Zhang, Yuqing Guo, Jeff Zhao, Huan He, Yi Han, Yun Feng, Yuechen Jiang, Yupeng Cao, Haohang Li, Yangyang Yu, Xiaoyu Wang, Penglei Gao, Shengyuan Lin, Keyi Wang, Shanshan Yang, Yilun Zhao, Zhiwei Liu, Peng Lu, Jerry Huang, Suyuchen Wang, Triantafillos Papadopoulos, Polydoros Giannouris, Efstathia Soufleri, Nuo Chen, Zhiyang Deng, Heming Fu, Yijia Zhao, Mingquan Lin, Meikang Qiu, Kaleb E Smith, Arman Cohan, Xiao-Yang Liu, Jimin Huang, Guojun Xiong, Alejandro Lopez-Lira, Xi Chen, Junichi Tsujii, Jian-Yun Nie, Sophia Ananiadou, Qianqian Xie
| Challenge: | Existing evaluations of LLMs in finance are text-only, monolingual, and largely saturated by current models. |
| Approach: | They propose a multilingual and multimodal benchmark for evaluating LLMs in real financial contexts. |
| Outcome: | The first expert-annotated multilingual and multimodal benchmark is released . it evaluates 21 leading LLMs and shows they perform better in multilingual settings . |
GLGE: A New General Language Generation Evaluation Benchmark (2021.findings-acl)
Copied to clipboard
Dayiheng Liu, Yu Yan, Yeyun Gong, Weizhen Qi, Hang Zhang, Jian Jiao, Weizhu Chen, Jie Fu, Linjun Shou, Ming Gong, Pengcheng Wang, Jiusheng Chen, Daxin Jiang, Jiancheng Lv, Ruofei Zhang, Winnie Wu, Ming Zhou, Nan Duan
| Challenge: | Multi-task benchmarks focus on a range of Natural Language Understanding (NLU) tasks without considering the Natural Language Generation (NLG) models. |
| Approach: | They propose a multi-task benchmark for evaluating the generalization capabilities of NLG models across eight language generation tasks. |
| Outcome: | The proposed benchmarks are based on GLUE and Su-perGLUE for English and several other languages. |
Chunks as Arms: Multi-Armed Bandit-Guided Sampling for Long-Context LLM Preference Optimization (2026.acl-long)
Copied to clipboard
Shaohua Duan, Pengcheng Huang, Xinze Li, Zhenghao Liu, Xiaoyuan Yi, Yukun Yan, Shuo Wang, Yu Gu, Ge Yu, Maosong Sun
| Challenge: | Recent studies have explored fine-tuning Large Language Models with synthetic data to enhance their long-context capabilities. |
| Approach: | They propose a framework that leverages a Multi-Armed Bandit rollout strategy to identify the most informative chunks from the given long context for sampling high-quality and diverse responses. |
| Outcome: | The proposed framework achieves 4% improvement on long-context reasoning benchmarks on Llama and Qwen. |
RikiNet: Reading Wikipedia Pages for Natural Question Answering (2020.acl-main)
Copied to clipboard
| Challenge: | Using Wikipedia pages to answer open-domain questions remains challenging in natural language understanding. |
| Approach: | They propose a model which reads Wikipedia pages for natural question answering . it uses a dynamic paragraph dual-attention reader and a cascaded answer predictor . |
| Outcome: | The proposed model outperforms the human model on the Natural Questions dataset . it achieves 74.3 F1 and 57.9 F1 on long-answer and short-answer tasks . |
LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shifts (2025.acl-long)
Copied to clipboard
Qibing Ren, Hao Li, Dongrui Liu, Zhanxu Xie, Xiaoya Lu, Yu Qiao, Lei Sha, Junchi Yan, Lizhuang Ma, Jing Shao
| Challenge: | Current safety training focuses on teaching models to reject harmful queries, but recent research shows that adversarial attacks or jailbreak methods bypass these safety mechanisms. |
| Approach: | They propose to use a new attack method to craft multi-turn toxic prompts that gradually lead LLMs to reveal unsafe content. |
| Outcome: | The proposed method outperforms existing methods in diversity, effectiveness, and efficiency across aligned LLMs. |
Seamlessly Integrating Factual Information and Social Content with Persuasive Dialogue (2022.aacl-main)
Copied to clipboard
| Challenge: | Persuasive dialogue systems are designed for chatbots to communicate with and influence users with specific goals. |
| Approach: | They propose a modular dialogue system framework that integrates factual information and social content into persuasive dialogues. |
| Outcome: | The proposed framework is generalizable to any dialogue tasks that have mixed social and task contents. |
We Need to Talk About Reproducibility in NLP Model Comparison (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that standard splits produce low reproducible and unreliable conclusions . reproducibility of empirical experimental conclusions is a problem in NLP domain . |
| Approach: | They propose to transform the reproducibility of a model comparison into a probabilistic function . they propose to use a regularized corpus splitting strategy to estimate the model's performance . |
| Outcome: | The proposed estimator achieves a high SNR and significantly increases reproducibility. |
ReCUT: Balancing Reasoning Length and Accuracy in LLMs via Stepwise Trails and Preference Optimization (2025.findings-emnlp)
Copied to clipboard
Zhensheng Jin, Xinze Li, Yifan Ji, Chunyi Peng, Zhenghao Liu, Qi Shi, Yukun Yan, Shuo Wang, Furong Peng, Ge Yu
| Challenge: | Existing methods to train LLMs suffer from overthinking, leading to lengthy reasoning traces . Existing approaches to train large language models suffer from this problem . |
| Approach: | They propose a method to combine multiple reasoning chains for training LLMs . they use stepwise exploration and long-short switched sampling to evaluate reasoning paths . |
| Outcome: | The proposed method reduces reasoning lengths by approximately 30-50% . it also maintains or improves reasoning accuracy compared to baselines . |
F-Eval: Asssessing Fundamental Abilities with Refined Evaluation Methods (2024.acl-long)
Copied to clipboard
Yu Sun, Keyuchen Keyuchen, Shujie Wang, Peiji Li, Qipeng Guo, Hang Yan, Xipeng Qiu, Xuanjing Huang, Dahua Lin
| Challenge: | Large language models (LLMs) have been evaluated for their instruction-following capabilities but lack references to their fundamental abilities. |
| Approach: | They propose a bilingual evaluation benchmark to evaluate the fundamental abilities of large language models including expression, commonsense and logic. |
| Outcome: | The proposed evaluation methods show higher correlation coefficients and larger distinction than other evaluators. |
LEGOEval: An Open-Source Toolkit for Dialogue System Evaluation via Crowdsourcing (2021.acl-demo)
Copied to clipboard
| Challenge: | Currently, researchers use automatic metrics and human evaluation to evaluate dialogue systems. |
| Approach: | They propose to use a Python API to easily evaluate dialogue systems using Amazon Mechanical Turk. |
| Outcome: | The open-source toolkit provides a fast, consistent method for reproducing human evaluation results. |
UCFE: A User-Centric Financial Expertise Benchmark for Large Language Models (2025.findings-naacl)
Copied to clipboard
Yuzhe Yang, Yifei Zhang, Yan Hu, Yilin Guo, Ruoli Gan, Yueru He, Mingcong Lei, Xiao Zhang, Haining Wang, Qianqian Xie, Jimin Huang, Honghai Yu, Benyou Wang
| Challenge: | Recent advances in large language models (LLMs) have expanded their potential applications in finance. |
| Approach: | They propose a framework to evaluate the ability of large language models to handle financial tasks using human expert evaluations and task-specific interactions. |
| Outcome: | The proposed framework evaluates the ability of large language models to handle complex financial tasks and combines human expert evaluations with dynamic, task-specific interactions to simulate the complexities of evolving financial scenarios. |
Keyphrase Generation with Correlation Constraints (D18-1)
Copied to clipboard
| Challenge: | Existing methods for keyphrase generation ignore correlation among keyphrases, resulting in duplication and coverage issues. |
| Approach: | They propose a new sequence-to-sequence architecture for keyphrase generation that captures correlation among keyphrases by preceding phrases to eliminate duplicate phrases and improve result coherence. |
| Outcome: | The proposed model outperforms the state-of-the-art method on benchmark datasets in terms of accuracy and diversity. |
ASCEND: A Spontaneous Chinese-English Dataset for Code-switching in Multi-turn Conversation (2022.lrec-1)
Copied to clipboard
Holy Lovenia, Samuel Cahyawijaya, Genta Winata, Peng Xu, Yan Xu, Zihan Liu, Rita Frieske, Tiezheng Yu, Wenliang Dai, Elham J. Barezi, Qifeng Chen, Xiaojuan Ma, Bertram Shi, Pascale Fung
| Challenge: | Code-switching is a speech phenomenon occurring when a speaker switches language during a conversation. |
| Approach: | They propose to collect Mandarin Chinese-English code-switching corpus from read speech rather than spontaneous speech to address this phenomenon. |
| Outcome: | ASCEND consists of 10.62 hours of clean speech, collected from 23 bilingual speakers of Chinese and English. |
RankCoT: Refining Knowledge for Retrieval-Augmented Generation through Ranking Chain-of-Thoughts (2025.acl-long)
Copied to clipboard
| Challenge: | Retrieval-Augmented Generation (RAG) models enable Large Language Models to access external knowledge. |
| Approach: | They propose a knowledge refinement method that incorporates reranking signals to generate CoT-based summarization based on query and retrieval documents. |
| Outcome: | RankCoT generates CoT-based summarization based on query and all retrieval documents . Rank CoT incorporates a self-reflection mechanism that refines the outputs . |
XL-NBT: A Cross-lingual Neural Belief Tracking Framework (D18-1)
Copied to clipboard
| Challenge: | a multi-lingual approach to training dialog systems is expensive and tedious, but it can be useful for cross-lingual support. |
| Approach: | They propose to annotate data for multiple languages and train a multi-lingual dialog system for each language. |
| Outcome: | The proposed framework bypasses the expensive human annotation and achieves promising results. |
Rˆ3Net:Relation-embedded Representation Reconstruction Network for Change Captioning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on change captioning uses a natural language sentence to describe disagreement between two images. |
| Approach: | They propose a Relation-embedded Representation Reconstruction Network to distinguish real change from clutter and irrelevant changes. |
| Outcome: | The proposed method achieves state-of-the-art on two public datasets. |
Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language Models (2025.acl-long)
Copied to clipboard
Zhisong Zhang, Yan Wang, Xinting Huang, Tianqing Fang, Hongming Zhang, Chenlong Deng, Shuaiyi Li, Dong Yu
| Challenge: | Large language models have demonstrated remarkable performance across a wide range of language tasks due to their remarkable ability in context modeling. |
| Approach: | They propose to use parallel context encoding to reduce attention entropy by incorporating attention sinks and selective mechanisms to reduce irregular attention . they also propose to incorporate attention sink mechanisms into the parallel encoded context to reduce the irregular attention. |
| Outcome: | The proposed methods lower irregular attention entropy and narrow performance gaps. |
MathFusion: Enhancing Mathematical Problem-solving of LLM through Instruction Fusion (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown impressive progress in mathematical problem-solving . current approaches to enhance mathematical reasoning focus on instance-level modifications . |
| Approach: | They propose a framework that enhances mathematical reasoning through cross-problem instruction synthesis. |
| Outcome: | The proposed framework boosts mathematical reasoning by 18.0 points while maintaining high data efficiency. |
Projecting Out the Malice: A Global Subspace Approach to LLM Detoxification (2026.acl-long)
Copied to clipboard
Zenghao Duan, Zhiyi Yin, Zhichao Shi, Liang Pang, Shaoling Jing, Zihe Huang, Jiayi Wu, Yu Yan, Jingcheng Deng, Huawei Shen, Xueqi Cheng
| Challenge: | Large language models (LLMs) exhibit exceptional performance but pose inherent risks of generating toxic content. |
| Approach: | They propose a method that removes toxic subspaces from FFN parameters . they propose to use a lightweight method to eliminate toxic subespaces . |
| Outcome: | The proposed method achieves SOTA detoxification while preserving general capabilities without large-scale retraining. |
Me-Agent: A Personalized Mobile Agent with Two-Level User Habit Learning for Enhanced Interaction (2026.findings-acl)
Copied to clipboard
Shuoxin Wang, Chang Liu, Gowen Loo, Lifan Zheng, Kaiwen Wei, Huanqian Yan, Xinyi Zeng, Jingyuan Zhang, Yu Tian
| Challenge: | Existing Large Language Model (LLM)-based mobile agents follow explicit user instructions without personalized needs. |
| Approach: | They propose a user preference learning strategy enhanced with a Personal Reward Model to improve personalization performance. |
| Outcome: | The proposed agent achieves state-of-the-art performance while maintaining competitive instruction execution performance. |
GRPO-Guided Modality Selection Enhanced LoRA-Tuned LLMs for Multimodal Emotion Recognition (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Multimodal emotion recognition in conversation (MERC) aims to identify speakers’ emotional states by utilizing text, audio, and visual modalities. |
| Approach: | They propose an adaptive modality selection framework for multimodal emotion recognition in conversation that integrates all available modalities into one . |
| Outcome: | The proposed framework outperforms existing methods on multimodal dialogue datasets and is available at https://github.com/youflyaway/Modality-Selection-Enhanced-LoRA-Tuned-LLMs. |
Fin-STAR: Structure-as-Semantics to Resolve Implicitness in Financial Retrieval (2026.findings-acl)
Copied to clipboard
Yu Zou, Yan Chen, Lida He, Qi Zhou, Xiaorui Zhou, Aixi Zhong, Yi Wang, Wei Li, Qingyu Wang, Jiatao Li, Wei Gong, Jialei Zeng, Jingmei Zhao, Ke Jiang, Qing Li
| Challenge: | Existing Retrieval-Augmented Generation systems treat structure as a physical navigational skeleton rather than intrinsic semantic knowledge. |
| Approach: | They propose a framework that redefining hierarchy as intrinsic semantics and uses snippets to enrich hierarchical lineage. |
| Outcome: | The proposed framework outperforms state-of-the-art hierarchical and graph-based benchmarks on FinTierQA Gold. |
Improving Autoformalization Using Direct Dependency Retrieval (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for hallucinate formal dependencies lack scalability and precision to leverage ever-growing public datasets. |
| Approach: | They propose a retrieval-augmented framework based on Direct Dependency Retrieval to generate formal dependencies from natural-language mathematical descriptions and verify their existence via an efficient Suffix Array Check (SAC). |
| Outcome: | The proposed framework outperforms state-of-the-art methods in retrieval precision and recall and can be used to validate formal representations in a public dataset. |
Revitalizing Black-Box Interpretability: Actionable Interpretability for LLMs via Proxy Models (2026.acl-long)
Copied to clipboard
| Challenge: | Applying model-agnostic explanations to Large Language Models is hindered by prohibitive computational costs rendering them dormant for real-world applications. |
| Approach: | They propose a budget-friendly proxy framework that leverages efficient models to approximate the decision boundaries of expensive Large Language Models. |
| Outcome: | The proposed framework achieves over 90% fidelity with only 9.5% of the oracle’s cost and is open-source to facilitate future research. |
CheckRLM: Effective Knowledge–Thought Coherence Checking in Retrieval-Augmented Reasoning (2026.acl-long)
Copied to clipboard
Dingling Xu, Ruobing Wang, Qingfei Zhao, Yukun Yan, Zhichun Wang, Daren Zha, Shi Yu, Zhenghao Liu, Shuo Wang, Xu Han, Maosong Sun
| Challenge: | Reasoning Language Models (RLMs) have improved performance on complex tasks by extending the reasoning chain, but they are prone to factual errors, especially in knowledge-intensive tasks. |
| Approach: | They propose a framework that improves the reliability of the reasoning process by timely checking and correcting factual errors. |
| Outcome: | The proposed framework outperforms baselines and shows that it mitigates error accumulation with lower costs. |
Detecting AI-Generated Content on Social Media with Multi-modal Language Models (2026.acl-industry)
Copied to clipboard
Chenyang Yang, Shen Yan, Yibo Yang, Litao Hu, Yuchen Liu, Yuan Zeng, Hanchao Yu, Yinan Zhu, Sumedha Singla, Brian Vanover, Huijun Qian, Zihao Wang, Fujun Liu, Aashu Singh, Jianyu Wang, Xuewen Zhang
| Challenge: | Existing methods for AI-generated content detection face poor generalization to newer models, reliance on single modalities, and lack of interpretable explanations. |
| Approach: | They propose a model that curates diverse social media data and trains a vision-language model for detection and explanation. |
| Outcome: | The proposed model achieves state-of-the-art detection performance on public benchmarks and observes positive downstream impacts on user engagement. |
What You See is What You Get: Visual Pronoun Coreference Resolution in Dialogues (D19-1)
Copied to clipboard
| Challenge: | a core task of natural language understanding is to ground a pronoun to a visual object it refers to . problem arises when people use pronounos to refer to something they can see without prior introduction . a novel visual-aware PCR model is proposed to solve this problem . |
| Approach: | They propose a visual-aware PCR model to ground a pronoun to a visible object . they propose PCR using a large-scale dialogue dataset to investigate this problem . |
| Outcome: | The proposed model can help resolve pronouns in conversational contexts. |
ExpandR: Teaching Dense Retrievers Beyond Queries with LLM Guidance (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for enhancing dense retrieval with query augmentation ignore the alignment between generation and ranking objectives. |
| Approach: | They propose a unified LLM-augmented dense retrieval framework that jointly optimizes both the LLM and the retriever. |
| Outcome: | Experimental results show that ExpandR outperforms strong baselines, achieving more than 5% improvement in retrieval performance. |
A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutions (2026.acl-long)
Copied to clipboard
Zhiyin Yu, Yuchen Mou, Juncheng Yan, Junyu Luo, Chunchun Chen, Xing Wei, Yunhui Liu, Hongru Sun, Yuxing Zhang, Jun Xu, Yatao Bian, Ming Zhang, Wei Ye, Tieke He, Jie Yang, Guanjie Zheng, Zhonghai Wu, Bo Zhang, Lei Bai, Xiao Luo
| Challenge: | Existing research on reinforcement learning for LLMs under data scarcity has not been unified. |
| Approach: | They propose a top-up hierarchical framework built around three complementary perspectives: data-centric, training-centric and framework-centric. |
| Outcome: | The proposed framework provides a clear conceptual foundation for understanding the design space of data-efficient RL for large language models and to guide researchers working in this emerging area. |
MoPS: Modular Story Premise Synthesis for Open-Ended Automatic Story Generation (2024.acl-long)
Copied to clipboard
| Challenge: | Existing sources of story premises are limited by a lack of diversity, uneven quality, and high costs that make them difficult to scale. |
| Approach: | They propose a method which breaks down story premises into modules like background and persona for automated design and generation. |
| Outcome: | The proposed framework excels in diversity, fascination, completeness, and originality compared to those induced from large language models and captured from public datasets. |
MathAgent: Leveraging a Mixture-of-Math-Agent Framework for Real-World Multimodal Mathematical Error Detection (2025.acl-industry)
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) struggle with identifying and categorizing student errors in multimodal mathematical contexts. |
| Approach: | They propose a new framework that decomposes error detection into three phases with specialized agents. |
| Outcome: | The proposed framework shows higher accuracy in error step identification and 3% improvement in error categorization on real-world educational data. |
Long-Chain Reasoning Distillation via Adaptive Prefix Alignment (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable reasoning capabilities, especially in solving complex mathematical problems. |
| Approach: | They propose a framework that exploits teacher CoTs for distillation through adaptive prefix alignment. |
| Outcome: | The proposed framework outperforms baseline models on multiple mathematical reasoning benchmarks by over 3%. |
AlignMMBench: Evaluating Chinese Multimodal Alignment in Large Vision-Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks focus on basic abilities using nonverbal methods, such as yes-no and multiple-choice questions. |
| Approach: | They propose a benchmark that provides more nuanced evaluations of alignment capabilities for large Vision-Language Models (VLMs) they use a rule-calibrated evaluator that exceeds GPT-4's evaluation ability and a “alignment score” to assess the robustness and stability of models across diverse prompts. |
| Outcome: | The proposed benchmark covers 13 tasks across three categories and includes both single-turn and multi-turn dialogue scenarios. |
MODE-LSTM: A Parameter-efficient Recurrent Network with Multi-Scale for Sentence Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for sentence classification use linear convolution, which may not be sufficient to model the non-consecutive dependency of the phrase and may overfit the sequential information. |
| Approach: | They propose a model that extracts multi-scale n-gram features for understanding the semantic meaning of sentences by some key-phrases located at different positions. |
| Outcome: | The proposed model outperforms existing models on eight benchmark datasets and is competitive against state-of-the-art models. |
Lang2Act: Fine-Grained Visual Reasoning through Self-Emergent Linguistic Toolchains (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing frameworks depend on rigid, pre-defined external tools to extend perceptual capabilities of VLMs. |
| Approach: | They propose a framework that leverages self-emergent linguistic toolchains to enhance visual perception and reasoning. |
| Outcome: | The proposed framework improves the visual perception capabilities of large language models by incorporating external visual documents to address a given query. |
CharTide: Data-Centric Chart-to-Code Generation via Tri-Perspective Tuning and Inquiry-Driven Evolution (2026.acl-long)
Copied to clipboard
Xiangxi Zheng, Kuang He, Jiayi Hu, Ping Yu, Rui Yan, Yuan Yao, Peng Hou, Anxiang Zeng, Alex Jinpeng Wang
| Challenge: | Existing approaches to chart-to-code generation are constrained by data-centric limitations . authors present a new framework that redesigns both training and alignment data . |
| Approach: | They propose a data-centric framework that redesigns both training and alignment data for chart-to-code generation. |
| Outcome: | The proposed framework outperforms open-source baselines and is competitive with GPT-5. |
Unveiling Attractor Cycles in Large Language Models: A Dynamical Systems View of Successive Paraphrasing (2025.acl-long)
Copied to clipboard
| Challenge: | Dynamical systems theory provides a framework for understanding iterative processes and evolution over time. |
| Approach: | They propose to apply this perspective to large language models which iteratively map input text to output text and re-express meaning with linguistic variation. |
| Outcome: | The proposed model reveals that paraphrases re-express meaning with linguistic variation limiting linguistic diversity . |
Is Cognition Consistent with Perception? Assessing and Mitigating Multimodal Knowledge Conflicts in Document Understanding (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multimodal large language models (MLLMs) have shown impressive capabilities in document understanding due to different types of annotation noise in training. |
| Approach: | They propose a method to reduce C&P knowledge conflicts across all tested MLLMs . they propose to use annotation noise to train models to understand document content . |
| Outcome: | The proposed method reduces C&P knowledge conflicts across all tested MLLMs and enhances their performance in both cognitive and perceptual tasks. |
CoLLiE: Collaborative Training of Large Language Models in an Efficient Way (2023.emnlp-demo)
Copied to clipboard
Kai Lv, Shuo Zhang, Tianle Gu, Shuhao Xing, Jiawei Hong, Keyu Chen, Xiaoran Liu, Yuqing Yang, Honglin Guo, Tengxiao Liu, Yu Sun, Qipeng Guo, Hang Yan, Xipeng Qiu
| Challenge: | Large language models (LLMs) are increasingly pivotal in a wide range of tasks . however, the resources required for training these models necessitate efficient solutions . |
| Approach: | They propose a library that facilitates collaborative training of large language models . they use 3D parallelism, parameter-efficient fine-tuning methods and optimizers . |
| Outcome: | The proposed library has proven superior training efficiency in comparison with prevalent solutions in pre-training and fine-tuning scenarios. |
DuQM: A Chinese Dataset of Linguistically Perturbed Natural Questions for Evaluating the Robustness of Question Matching Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | a comprehensive evaluation of QM models should be conducted on natural texts, not on artificial adversarial examples . ral models are often not robust to adversarials, which means they predict unexpected outputs . |
| Approach: | They use a Chinese dataset to evaluate the robustness of QM models . they show that the effect of artificial adversarial examples does not work on natural texts . |
| Outcome: | The proposed model is more robust than other models on natural questions with 32 linguistic perturbations. |
Global Relation Embedding for Relation Extraction (N18-1)
Copied to clipboard
| Challenge: | Existing methods to extract textual relations with distant supervision are limited by their reliance on supervised training data. |
| Approach: | They propose to embed relations with global statistics of relations to combat the wrong labeling problem of distant supervision. |
| Outcome: | The proposed method is more robust to training noise introduced by distant supervision and improves relation extraction models. |
Position: LLMs Can be Good Tutors in English Education (2025.emnlp-main)
Copied to clipboard
Jingheng Ye, Shen Wang, Deqing Zou, Yibo Yan, Kun Wang, Hai-Tao Zheng, Ruitong Liu, Zenglin Xu, Irwin King, Philip S. Yu, Qingsong Wen
| Challenge: | Recent efforts to integrate large language models into English education lack adaptability to language learning. |
| Approach: | They argue that large language models can be effective tutors in English education . they encourage interdisciplinary research to explore these roles, fostering innovation and risks . |
| Outcome: | The proposed models can play three critical roles: 1) as data enhancers, 2) as task predictors, 3) as agents, enabling personalized and inclusive education. |
EvoRoute: Experience-Driven Self-Routing LLM Agent Systems (2026.acl-long)
Copied to clipboard
| Challenge: | EvoRoute is a self-evolving model routing paradigm that transcends static, pre-defined model assignments. |
| Approach: | They propose a model routing paradigm that transcends static, pre-defined model assignments. |
| Outcome: | Experiments on GAIA and BrowseComp+ show that EvoRoute reduces execution cost and latency by over 70%. |
LM-Interview: An Easy-to-use Smart Interviewer System via Knowledge-guided Language Model Exploitation (2024.emnlp-demo)
Copied to clipboard
| Challenge: | Semi-structured interviews are a crucial method of data acquisition in qualitative research. |
| Approach: | They propose a semi-structured interview system that automates interview preparation, analysis and control by interviewers. |
| Outcome: | Experimental results show that LM-Interview performs comparable to human interviewers . the system can be used to analyze semi-structured interviews without interviewers' involvement . |
SAS: Dialogue State Tracking via Slot Attention and Slot Information Sharing (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models with excessive information are inefficient and costly . |
| Approach: | They propose to integrate a Dialogue State Tracker with Slot Attention and Slot Information Sharing to reduce redundant information’s interference and improve long dialogue context tracking. |
| Outcome: | The proposed model significantly outperforms existing models on the MultiWOZ dataset. |
UTC-IE: A Unified Token-pair Classification Architecture for Information Extraction (2023.acl-long)
Copied to clipboard
| Challenge: | Information Extraction (IE) tasks have been solved with different models because of their output structures. |
| Approach: | They propose a Unified Token-pair Classification architecture for Information Extraction that introduces Plusformer on top of the token-pear feature matrix. |
| Outcome: | The proposed approach outperforms task-specific and unified models on all tasks in 10 datasets and achieves better results on 2 joint IE datasets. |
from Benign import Toxic: Jailbreaking the Language Model via Adversarial Metaphors (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies have exposed the risk of Large Language Models (LLMs) generating harmful content by jailbreak attacks. |
| Approach: | They propose a framework that exploits AdVersArial meTAphoR to induce LLMs to calibrate harmful metaphors for jailbreaking. |
| Outcome: | The proposed framework can successfully jailbreak Large Language Models (LLMs) by leveraging the AdVersArial meTAphoR (AVATAR) framework achieves state-of-the-art attack success rate across multiple advanced LLMs. |
URO-Bench: Towards Comprehensive Evaluation for End-to-End Spoken Dialogue Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a lack of comprehensive evaluations for SDMs in speech-to-speech (S2S) scenarios is a major challenge for end-to end spoken dialogue models. |
| Approach: | They propose to provide an extensive evaluation framework for end-to-end spoken dialogue models (SDMs) that includes both cognitive dimensions and paralinguistic cues . |
| Outcome: | The proposed benchmark is divided into two difficulty levels: basic track and pro track, each comprising 20 test sets, evaluating the spoken dialogue model’s abilities in U**nderstanding, **R**easoning, and **O**ral conversation. |
DA-Code: Agent Data Science Code Generation Benchmark for Large Language Models (2024.emnlp-main)
Copied to clipboard
Yiming Huang, Jianwen Luo, Yan Yu, Yitong Zhang, Fangyu Lei, Yifan Wei, Shizhu He, Lifu Huang, Xiao Liu, Jun Zhao, Kang Liu
| Challenge: | DA-Code is a code generation benchmark designed to assess LLMs on agent-based data science tasks. |
| Approach: | They propose a code generation benchmark specifically designed for LLMs on agent-based data science tasks. |
| Outcome: | The benchmark performs better than existing frameworks, but lacks accuracy . it is based on real-world data, and includes examples that cover a wide range of tasks . |
COAST: Enhancing the Code Debugging Ability of LLMs through Communicative Agent Based Data Synthesis (2025.findings-naacl)
Copied to clipboard
Weiqing Yang, Hanbin Wang, Zhenghao Liu, Xinze Li, Yukun Yan, Shuo Wang, Yu Gu, Minghe Yu, Zhiyuan Liu, Ge Yu
| Challenge: | Existing code debugging benchmarks focus on the Code Repair stage of the code generation process. |
| Approach: | They propose a framework to evaluate the debugging abilities of large language models by emulating the human debug process. |
| Outcome: | The proposed framework outperforms human-curated and GPT-4-generated training data, enabling 7B-scale LLMs to achieve comparable debugging performance to GPT-3.5. |
Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for news headline generation focus on producing a single short sentence . et al., 2017; Gehrmann e.t., 2018; Zhong ee., 2019) focus on single-headline generation. |
| Approach: | They propose a method to generate multiple headlines with keyphrases of user interests . they propose generating multiple keyphrase-relevant headlines using a transformer decoder . |
| Outcome: | The proposed method achieves state-of-the-art in terms of quality and diversity. |
Improve LLM-as-a-Judge Ability as a General Ability (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies focus on generative judges, but only on their judge ability. |
| Approach: | They propose a method that leverages the generative and reasoning capabilities of large language models to evaluate LLM responses across diverse scenarios, providing accurate preference signals. |
| Outcome: | The proposed model performs on RewardBench with only 2% to 40% of the data required by other training frameworks. |
Knowledge-aware Pronoun Coreference Resolution (P19-1)
Copied to clipboard
| Challenge: | Existing models for pronoun coreference resolution only use triplets, the most common format for knowledge graphs. |
| Approach: | They propose a model that leverages different types of knowledge to resolve pronoun coreference with a neural model. |
| Outcome: | The proposed model outperforms state-of-the-art baselines on two datasets from different domains. |
Zero-shot User Intent Detection via Capsule Neural Networks (D18-1)
Copied to clipboard
| Challenge: | Existing methods to classify intents are labor-intensive and time-consuming as intents will be diverse and new intents may be involved. |
| Approach: | They propose a zero-shot intent detection problem which aims to detect emerging user intents where no labeled utterances are currently available. |
| Outcome: | The proposed model can discriminate emerging intents when no labeled utterances are available in training data. |
Explanation-aware Soft Ensemble Empowers Large Language Model In-context Learning (2024.acl-long)
Copied to clipboard
Yue Yu, Jiaming Shen, Tianqi Liu, Zhen Qin, Jing Nathan Yan, Jialu Liu, Chao Zhang, Michael Bendersky
| Challenge: | Recent advances in natural language processing (NLP) have witnessed the remarkable capabilities of Large Language Models (LLMs). |
| Approach: | They propose an Explanation-Aware Soft Ensemble framework to empower in-context learning with Large language models. |
| Outcome: | The proposed framework can be used to enhance in-context learning on seven natural language understanding tasks and four varying-size LLMs. |
Cross-lingual Knowledge Graph Alignment via Graph Matching Neural Network (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to cross-lingual knowledge graph (KG) alignment rely on entity embeddings derived from monolingual KG structural information. |
| Approach: | They propose a topic entity graph to represent entities with contextual information in KGs. |
| Outcome: | The proposed model outperforms state-of-the-art methods by a large margin. |
Vision-Language Pre-Training for Multimodal Aspect-Based Sentiment Analysis (2022.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to multimodal Aspect-Based Sentiment Analysis (MABSA) ignore crossmodalalignment and use pre-trained visual and textual models. |
| Approach: | They propose a multimodal multimodal encoder-decoder framework for MABSA that uses a unified multimodal decoder architecture for all the pretrainingand downstream tasks. |
| Outcome: | The proposed framework outperforms state-of-the-art approaches on three MABSA subtasks. |
RevCore: Review-Augmented Conversational Recommendation (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing conversational recommendation systems lack item information when conducted on short dialogue history and unfamiliar items. |
| Approach: | They propose a framework where reviews are seamlessly incorporated into conversational recommendation systems. |
| Outcome: | The proposed framework yields better performance on recommendation and conversation responding. |
Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub (2025.acl-long)
Copied to clipboard
Bohan Lyu, Xin Cong, Heyang Yu, Pan Yang, Cheng Qian, Zihe Wang, Yujia Qin, Yining Ye, Yaxi Lu, Chen Qian, Zhong Zhang, Yukun Yan, Yankai Lin, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing approaches lack flexibility to address diverse and ever-evolving user queries in open domains. |
| Approach: | They propose to evaluate LLMs on open-domain knowledge that requires tools to solve diverse and ever-evolving user queries. |
| Outcome: | The proposed system outperforms baselines in the open domain task-solving benchmark. |
CodeM: Less Data Yields More Versatility via Ability Matrix (2024.findings-acl)
Copied to clipboard
Daoguang Zan, Ailun Yu, Wei Liu, Bo Shen, Shaoxin Lin, Yongshun Gong, Yafen Yao, Yan Liu, Bei Guan, Weihua Luo, Yongji Wang, Qianxiang Wang, Lizhen Cui
| Challenge: | Recent efforts to train code large language models have been booming recently . however, this will incur significant costs in constructing data and training model considering the countless downstream scenarios. |
| Approach: | They propose a data construction strategy which decouples code LLMs’ abilities into two dimensions and constructs a lightweight training corpus that only covers a subset of target scenarios. |
| Outcome: | The proposed model can train a multilingual multitasking model using less data and training data. |
Confusion is the Final Barrier: Rethinking Jailbreak Evaluation and Investigating the Real Misuse Threat of LLMs (2025.findings-emnlp)
Copied to clipboard
Yu Yan, Sheng Sun, Zhe Wang, Yijun Lin, Zenghao Duan, Zhifei Zheng, Min Liu, Zhiyi Yin, Jianping Zhang
| Challenge: | Large Language Models have been developed to deal with real-world crimes, but it remains unclear whether they internalize authentic knowledge or are forced to simulate toxic language patterns. |
| Approach: | They construct knowledge-intensive Q&A to investigate misuse threats of Large Language Models in terms of dangerous knowledge possession, harmful task planning utility, and harmfulness judgment robustness. |
| Outcome: | The findings raise concerns that jailbreak success is often attributable to a hallucination loop between jailbroken LLM and judger LLM . |
ASTRA: An Automated Framework for Strategy Discovery, Retrieval, and Evolution for Jailbreaking LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods lack the capability for continuous learning and self-evolution from interactions, limiting the diversity and adaptability of attack strategies. |
| Approach: | They propose an automated framework capable of discovering, retrieving, and evolving attack strategies. |
| Outcome: | The proposed framework outperforms existing baselines in a black-box setting. |
Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing data augmentation techniques for natural language processing tasks are difficult to design. |
| Approach: | They propose a controllable rewriting based question data augmentation method for machine reading comprehension, question generation and question-answering natural language inference tasks. |
| Outcome: | The proposed method generates high-quality, high-quality question data samples on machine reading comprehension, question generation, and question-answering natural language inference tasks. |