Papers by Dai Cheng
PricingLogic: Evaluating LLMs Reasoning on Complex Tourism Pricing Tasks (2025.emnlp-main)
Copied to clipboard
Yunuo Liu, Dawei Zhu, Zena Al-Khalili, Dai Cheng, Yanjun Chen, Dietrich Klakow, Wei Zhang, Xiaoyu Shen
| Challenge: | Recent advances in Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse domains, such as code generation, mathematical problem-solving, and general-purpose human instruction following. |
| Approach: | They propose to use large language models to process questions expressed in natural language to automate tourism-booking prices when multiple, overlapping farerules apply. |
| Outcome: | The proposed model can automate tourism-booking prices when multiple, overlapping farerules apply. |
Attn-GS: Attention-Guided Context Compression for Efficient Personalized LLMs (2026.acl-long)
Copied to clipboard
Shenglai Zeng, Tianqi Zheng, Chuan Tian, Dante Everaert, Yau-Shian Wang, Yupin Huang, Michael J. Morais, Rohit Patki, Jinjin Tian, Xinnan Dai, Kai Guo, Monica Xiao Cheng, Hui Liu
| Challenge: | Existing approaches to personalize large language models (LLMs) rely on heuristic methods to compress user profiles but they ignore how LLMs process and prioritize different profile components. |
| Approach: | They propose an attention-guided context compression framework that leverages attention feedback from a marking model to mark important personalization sentences and guides a compression model to generate task-relevant compressed user contexts. |
| Outcome: | The proposed framework outperforms baselines across tasks, token limits, and settings while reducing token usage by 50 times. |
GraphSynth: Resolving the Diversity-Reliability Trade-off with Probabilistic Factor Graphs (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models are a scaleable solution for the generation of synthetic data . however, the utility of such data is capped by a critical tension between diversity and factual reliability. |
| Approach: | They propose a framework which leverages a probabilistic factor graph modeling the universe of attributes. |
| Outcome: | The proposed framework outperforms state-of-the-art models with a high structural integrity and a boost in performance on downstream tasks. |
Adversarial Preference Optimization: Enhancing Your Alignment via RM-LLM Game (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for training large language models require additional annotations to adjust to shifted distributions. |
| Approach: | They propose an algorithm that allows LLMs and reward models to update alternatively via a min-max game to improve their alignment. |
| Outcome: | The proposed framework improves existing alignment baselines in terms of LLM helpfulness and harmlessness. |
On Weaponization-Resistant Large Language Models with Prospect Theoretic Alignment (2025.coling-main)
Copied to clipboard
| Challenge: | Existing safeguards for large language models are inadequate for open-weight models as minimal fine-tuning can bypass them. |
| Approach: | They propose a framework that prioritizes maximizing generative utility rather than a singular optimization metric and integrates prospect theory into LLM training to strengthen LLMs against misuse and weaponization. |
| Outcome: | The proposed framework strengthens LLMs against misuse and weaponization while maintaining high performance even after extensive fine-tuning. |
Compensate Quantization Errors: Make Weights Hierarchical to Compensate Each Other (2024.findings-naacl)
Copied to clipboard
| Challenge: | Emergent Large Language Models (LLMs) use extraordinary performance and powerful deduction capacity to discern from traditional language models. |
| Approach: | They propose a method that uses weights to compensate quantization error and learnable singular value incremental (LSI) LSI is a technique that helps weights compensate each other conditioned on activation. |
| Outcome: | The proposed method achieves state-of-the-art performance in diverse quantization settings, no matter in weight-only, weight-activation or extremely low bit scenarios. |
Reasoning in Flux: Enhancing Large Language Models Reasoning through Uncertainty-aware Adaptive Guidance (2024.acl-long)
Copied to clipboard
Zhangyue Yin, Qiushi Sun, Qipeng Guo, Zhiyuan Zeng, Xiaonan Li, Junqi Dai, Qinyuan Cheng, Xuanjing Huang, Xipeng Qiu
| Challenge: | Extensive experiments across various reasoning tasks demonstrate that UAG not only enhances the reasoning abilities of LLMs but consistently outperforms several strong baselines with minimal computational overhead. |
| Approach: | They propose an approach to guide LLMs onto an accurate and reliable trajectory by identifying and adjusting uncertainty signals within each step of the reasoning chain. |
| Outcome: | The proposed approach outperforms strong baselines and outperformed strong models with minimal computational overhead. |
On Diversified Preferences of Large Language Model Alignment (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) can be fine tuned with human feedback, but human preferences can be diversified due to annotators’ different tastes, which hinders the effectiveness of LLM alignment methods. |
| Approach: | They propose a calibration error metric to evaluate large language models (LLMs) and a multi-objective reward learning method to enhance the calibration performance of RMs on shared preferences. |
| Outcome: | The proposed model can be adopted as a key calibration error and MORE can achieve superior alignment performance. |
Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communication (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have made significant strides in complex reasoning tasks, but their reasoning is often constrained by their intrinsic understanding, lacking external insights. |
| Approach: | They propose a framework that enables cross-model communication during problem-solving. |
| Outcome: | The proposed framework surpasses established baselines in complex reasoning tasks and is cost-effective. |
Natural-Language Policies to Executable Decisions: An Interpretable Large Language Model Framework (2026.acl-industry)
Copied to clipboard
Ziqiang Zhang, Jing Ma, Zilong Wang, Jiayuan Chen, Yi Qiao, Yu He, Wei Zhang, Dai Cheng, Xiaoyu Shen
| Challenge: | a production-grade pricing system for tourism is challenging due to unstructured nature of travel orders and ever-evolving pricing policies. |
| Approach: | They propose a production-grade pricing system with a strict decision boundary . they propose to combine structured extraction and bounded policy/path selection with interpretable condition trees . |
| Outcome: | The proposed system processed 3,960 orders in six months and reduced the order management team from 15-20 to 3 . the system reduced the per-order handling time from 10 minutes to 2 minutes. |
Understanding the Behaviors of Environment-aware Information Retrieval (2026.acl-long)
Copied to clipboard
| Challenge: | Recent retrieval-augmented generation approaches have demonstrated strong capability in handling complex queries. |
| Approach: | They propose a branching-based rollout technique that improves training stability . they find different retrievers exhibit distinct optimal query styles . |
| Outcome: | The proposed method improves training stability and improves retrieval-aware systems. |
Large Language Model Agents in Finance: A Survey Bridging Research, Practice, and Real-World Deployment (2025.findings-emnlp)
Copied to clipboard
Yifei Dong, Fengyi Wu, Kunlin Zhang, Yilong Dai, Sanjian Zhang, Wanghao Ye, Sihan Chen, Zhi-Qi Cheng
| Challenge: | a systematic review of large language models (LLMs) is conducted to better align their capabilities with real-world demands. |
| Approach: | They propose a functional taxonomy mapping financial domains to tasks, datasets, and institutional constraints. they catalog over 30 financial benchmarks and 20 representative models. |
| Outcome: | The proposed model frameworks are bridging financial practice and LLM research. |
CircuitSynth: Reliable Synthetic Data Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches lack mechanisms to balance linguistic expressivity with formal guarantees regarding validity and coverage. |
| Approach: | They propose a neuro-symbolic framework that decouples semantic reasoning from surface realization. |
| Outcome: | The proposed framework achieves 100% Schema Validity even in complex logic puzzles where unconstrained baselines fail (12.4%) while outperforming state-of-the-art methods in rare-combination coverage. |
RecMem: Recurrence-based Memory Consolidation for Efficient and Effective Long-Running LLM Agents (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing memory systems invoke LLMs to extract episodic and semantic memory, and this leads to substantial token consumption. |
| Approach: | They propose a method that stores incoming interactions in a subconscious memory layer and encodes them using lightweight embedding models for retrieval. |
| Outcome: | Experiments show that RecMem reduces the memory construction token cost of three SOTA memory systems by up to 87% while exceeding their accuracy. |
Uplift-RAG: Uplift-Driven Knowledge Preference Alignment for Retrieval-Augmented Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing efforts to estimate document utility rely on downstream generation performance, which conflates the influence of external documents with the intrinsic knowledge of the LLM. |
| Approach: | They propose an uplift-based definition of document utility that quantifies each document’s marginal benefit over the LLM’s internal knowledge. |
| Outcome: | The proposed framework improves the performance of the LLM by incorporating external retrieved documents into the model. |
APo-VAE: Text Generation in Hyperbolic Space (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models that learn embeddings only in Euclidean vector space do not account for such structural property of language. |
| Approach: | They propose a Poincare Variational Autoencoder to capture latent hierarchies in hyperbolic space . they propose enabling adversarial learning procedures to empower robust model training . |
| Outcome: | The proposed model outperforms existing models in a hyperbolic latent space . it captures latent language hierarchies in hyperbolical space and is robust to training . |
GeoSQA: A Benchmark for Scenario-based Question Answering in the Geography Domain at High School Level (D19-1)
Copied to clipboard
| Challenge: | SQA is an emerging application of NLP in the medical, geography, and legal domains. |
| Approach: | They propose a dataset of 1,981 scenarios and 4,110 multiple-choice questions in geography domain at high school level. |
| Outcome: | The proposed dataset consists of 1,981 scenarios and 4,110 multiple-choice questions in the geography domain at high school level. |
A Survey of Multimodal Mathematical Reasoning: From Perception, Alignment to Reasoning (2026.acl-long)
Copied to clipboard
Tianyu Yang, Sihong Wu, Yilun Zhao, Zhenwen Liang, Lisen Dai, Chen Zhao, Minhao Cheng, Arman Cohan, Xiangliang Zhang
| Challenge: | Multimodal mathematical Reasoning (MMR) has attracted increasing attention for its ability to solve mathematical problems involving both textual and visual modalities. |
| Approach: | They review the theoretical frameworks of multimodal reasoning and examine the challenges they face in visual math tasks. |
| Outcome: | The proposed models can solve problems involving both textual and visual modalities. |
On Domain-Adaptive Post-Training for Multimodal Large Language Models (2025.findings-emnlp)
Copied to clipboard
Daixuan Cheng, Shaohan Huang, Ziyu Zhu, Xintong Zhang, Xin Zhao, Zhongzhi Luan, Bo Dai, Zhenliang Zhang
| Challenge: | Adapting general multimodal large language models to specific domains is important for practical applications. |
| Approach: | They investigate domain adaptation of multimodal large language models via post-training . they develop a generate-then-filter pipeline that curates diverse visual instruction tasks . |
| Outcome: | The proposed model outperforms existing models in domain adaptation by combining data from open-source models with training pipelines. |
CLIPErase: Efficient Unlearning of Visual-Textual Associations in CLIP (2025.acl-long)
Copied to clipboard
| Challenge: | MU has gained significant attention as a means to remove the influence of specific data from a trained model without requiring full retraining. |
| Approach: | They propose a novel approach that disentangles and selectively forgets both visual and textual associations, ensuring that unlearning does not compromise model performance. |
| Outcome: | Experiments on CIFAR-100, Flickr30K, and Conceptual 12M show that CLIPErase effectively removes designated associations from multimodal samples in downstream tasks while preserving model performance on retain set. |
MessToClean: Evidence-Grounded Structure-Preserving Reconstruction for Real-World Degraded Exam Paper Images (2026.acl-long)
Copied to clipboard
Jiayi Tuo, Cheng Tang, Zihan Wang, Chenyue Zhou, Yao Li, Yanbiao Ma, Chao Wang, Wei Dai, Mingxuan Wang, Shitong Qin, Ziwei Zhao
| Challenge: | Existing Multimodal Large Language Models (MLLMs) fail under RDEI, leading to disrupted structure and evidence-unsupported hallucinations. |
| Approach: | They propose a backbone-agnostic, evidence-driven pipeline that treats off-the-shelf MLLMs as interchangeable components to improve stem consistency and figure consistency. |
| Outcome: | The proposed pipeline improves stem consistency by 1.01-3.18%, figure consistency by 0.50-49.16%, and refusal F1 by 1.06-10.88% across question types. |
Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models (2026.acl-long)
Copied to clipboard
Xin Cheng, Wangding Zeng, Damai Dai, Qinyu Chen, Bingxuan Wang, Zhenda Xie, Kezhao Huang, Xingkai Yu, Zhewen Hao, Han Zhang, Yu-Kun Li, Huishuai Zhang, Dongyan Zhao, Wenfeng Liang
| Challenge: | Mixture-of-Experts (MoE) scales capacity via conditional computation, but lacks knowledge lookup primitive. |
| Approach: | They propose a conditional memory instantiated via Deep Sparse Embedding (DSE) they propose 'u-shaped scaling law' that identifies optimal balance between MoE experts and DSE memory . |
| Outcome: | The proposed model outperforms an iso-parameter and isoFLOPs MoE baseline across knowledge and reasoning benchmarks and is infrastructure-efficient. |
A Multi-Task Incremental Learning Framework with Category Name Embedding for Aspect-Category Sentiment Analysis (2020.emnlp-main)
Copied to clipboard
| Challenge: | Sentiment analysis is an increasingly popular natural language processing task in academia and industry. |
| Approach: | They propose to use category name encoding network to weaken catastrophic forgetting problem . they set both encoder and decoder shared among all categories to weaker the catastrophic forgetting problem a . |
| Outcome: | The proposed model achieves state-of-the-art on two (T)ACSA benchmark datasets. |
Chunk, Align, Select: A Simple Long-sequence Processing Method for Transformers (2024.acl-long)
Copied to clipboard
| Challenge: | Existing transformer-based models struggle with long-sequence processing due to computational costs . a framework to enhance long-content processing of transformers is proposed . |
| Approach: | They propose a framework to enhance long-sequence processing of transformers by three steps . they demonstrate that the framework significantly outperforms prior long-quence processors . |
| Outcome: | The proposed framework outperforms baseline models on long-sequence summarization and reading comprehension tasks. |
GoViG: Goal-Conditioned Visual Navigation Instruction Generation via Multimodal Reasoning (2026.findings-acl)
Copied to clipboard
Fengyi Wu, Yifei Dong, Yilong Dai, Guangyu Chen, Qifeng Wu, Huiting Huang, Hang Wang, Qi Dai, Alexander G Hauptmann, Zhi-Qi Cheng
| Challenge: | Current methods for instruction generation depend on privileged inputs such as semantic maps, landmark annotations, and panoramic views. |
| Approach: | They propose a task that generates coherent navigation instructions from egocentric visual observations. |
| Outcome: | The proposed task generates coherent navigation instructions from egocentric visual data . the proposed task improves performance over state-of-the-art methods in BLEU-4 and CIDEr scores . |