Papers by Fan Hu
Youling: an AI-assisted Lyrics Creation System (2020.emnlp-demos)
Copied to clipboard
Rongsheng Zhang, Xiaoxi Mao, Le Li, Lin Jiang, Lin Chen, Zhiwei Hu, Yadong Xi, Changjie Fan, Minlie Huang
| Challenge: | Recent studies have focused on a single pass of lyrics generation with little human intervention. |
| Approach: | They propose an AI-assisted lyrics creation system that supports one pass full-text generation and interactive generation modes. |
| Outcome: | The proposed system supports full-text generation and interactive generation modes . it also provides a revision module which enables users to revise undesired lyrics repeatedly. |
Dynamic Energy-Based Contrastive Learning with Multi-Stage Knowledge Verification for Event Causality Identification (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for event causal identification rely on rule-based or random sampling strategies, which introduce spurious causal positives. |
| Approach: | They propose an ECI method enhanced by Dynamic Energy-based Contrastive Learning with multi-stage knowledge verification which generates high-quality contrastive samples and effectively suppresses spurious causal disturbances. |
| Outcome: | The proposed method outperforms state-of-the-art methods on two benchmarks. |
Balancing Knowledge Breadth and Task Depth for Effective Domain Adaptation Fine-Tuning (2026.findings-acl)
Copied to clipboard
| Challenge: | a lack of knowledge breadth and task depth can hinder curriculum learning in domains such as medicine and finance. |
| Approach: | They propose a two-dimensional curriculum learning framework that coordinates model training along two orthogonal axes: the knowledge dimension and the task dimension. |
| Outcome: | The proposed framework improves accuracy on medical evaluations by 2.49% and on financial evaluations 1.2% compared with the second-best method. |
Travel on the ICD Tree: Benchmarking Agentic Reasoning for ICD Coding from Chinese Electronic Medical Records (2026.findings-acl)
Copied to clipboard
| Challenge: | Accurate International Classification of Diseases (ICD) coding is crucial for hospital management and healthcare data governance. |
| Approach: | They propose a framework to evaluate ICD coding based on complete EMRs . they use a dataset of 560 real clinical records covering 434 common diseases . |
| Outcome: | The proposed framework explores the capability boundaries of large language models under different paradigms. |
DeepSynth-Eval: Objectively Evaluating Information Consolidation in Deep Survey Writing (2026.findings-acl)
Copied to clipboard
Hongzhi Zhang, Yuanze Hu, Tinghai Zhang, Jia Fu, Tao Wang, Junwei Jing, Zhaoxin Fan, Wei Bi, Ruiming Tang, Han Li, Guorui Zhou, Kun Gai
| Challenge: | Large Language Models (LLMs) are evolving towards autonomous agents . retrieval capabilities are well-benchmarked, but post-retrieval synthesis is under-evaluated due to open-ended writing. |
| Approach: | They propose a benchmark to evaluate information consolidation capabilities using survey papers as gold standards. |
| Outcome: | The proposed benchmark analyzes the post-retrieval synthesis stage of large language models . it leverages high-quality survey papers as gold standards and reverse-engineers research requests . the proposed benchmark outperforms single-turn generation and reduces hallucinations . |
Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversations (2022.acl-long)
Copied to clipboard
Wei Chen, Yeyun Gong, Can Xu, Huang Hu, Bolun Yao, Zhongyu Wei, Zhihao Fan, Xiaowu Hu, Bartuer Zhou, Biao Cheng, Daxin Jiang, Nan Duan
| Challenge: | Existing studies focus on coarse-grained response selection in retrieval-based dialogue systems. |
| Approach: | They propose a Contextual Fine-to-Coarse (CFC) distilled model for coarse-grained response selection in open-domain conversations. |
| Outcome: | The proposed model improves over baseline methods on two datasets based on the Reddit comments dump and Twitter corpus compared with baseline methods. |
MMAC: A Multilingual, Multimodal Alignment Framework for Cultural Grounding Evaluation (2026.acl-long)
Copied to clipboard
Weihua Zheng, Zhengyuan Liu, Tanmoy Chakraborty, Weiwen Xu, Xiaoxue Gao, Bryan Chen Zhengyu Tan, Bowei Zou, Chang Liu, Yujia Hu, Xing Xie, Xiaoyuan Yi, Jing Yao, Chaojun Wang, Long Li, Rui Liu, Huiyao Liu, Koji Inoue, Ryuichi Sumida, Tatsuya Kawahara, Fan Xu, Lingyu Ye, Wei Tian, Dongjun Kim, Jimin Jung, Jaehyung Seo, Nadya Yuki Wangsajaya, Pham Minh Duc, Ojasva Saxena, Palash Nandi, Xiyan Tao, Wiwik Karlina, Tuan Luong, Keertana Arun Vasan, Roy Ka-Wei Lee, Nancy F. Chen
| Challenge: | Existing models lack cultural alignment across modalities and languages . a new framework to assess cultural awareness across linguistics and languages is needed . |
| Approach: | They propose a framework that integrates tri-modally aligned cultural benchmarks and a five-dimensional evaluation protocol to assess cross-country awareness disparities. |
| Outcome: | The proposed framework assesses cultural awareness disparities across modalities and languages . it is the first dataset aligned at the input level across text, image, and speech . |
SPIDE: Serial and Parallel Intertwined Speculative Decoding (2026.findings-acl)
Copied to clipboard
Wenru Xu, Peixuan Xu, Ziqi Yang, Ming Hu, Zihui Wang, Jianzhong Qi, Rongshan Yu, Xiaoliang Fan, Cheng Wang
| Challenge: | Speculative decoding (SD) is a training-free SD framework that orchestrates dynamic alternation combining serial dynamic drafting with parallel draft verification. |
| Approach: | They propose a serial and parallel intertwined speculative DEcoding framework that orchestrates dynamic alternation combining serial dynamic drafting and parallel draft verification. |
| Outcome: | The proposed framework accelerates inference while reducing the LLM usage costs. |
CoRe-MMRAG: Cross-Source Knowledge Reconciliation for Multimodal RAG (2025.acl-long)
Copied to clipboard
| Challenge: | Recent advances in Multimodal Large Language Models have significantly improved reasoning and generation tasks by leveraging joint vision-language representations. |
| Approach: | They propose a framework that reconciles inconsistencies across knowledge sources . they use a four-stage pipeline to generate an internal response from parametric knowledge . |
| Outcome: | Experiments on KB-VQA show that CoRe-MMRAG achieves performance gains of 5.6% and 9.3% over baseline methods. |
Alleviating Hallucinations from Knowledge Misalignment in Large Language Models via Selective Abstention Learning (2025.acl-long)
Copied to clipboard
Lei Huang, Xiaocheng Feng, Weitao Ma, Yuchun Fan, Xiachong Feng, Yuxuan Gu, Yangfan Ye, Liang Zhao, Weihong Zhong, Baoxin Wang, Dayong Wu, Guoping Hu, Lingpeng Kong, Tong Xiao, Ting Liu, Bing Qin
| Challenge: | Large language models (LLMs) suffer from severe hallucination issues due to the knowledge misalignment between the pre-training stage and the supervised fine-tuning stage. |
| Approach: | They propose a training objective with an abstention mechanism that selectively rejects tokens that misalign with the desired knowledge distribution via a special [REJ] token. |
| Outcome: | The proposed model selectively rejects tokens that misalign with the desired knowledge distribution via a special [REJ] token. |
Where to Attack: A Dynamic Locator Model for Backdoor Attack in Text Classifications (2022.coling-1)
Copied to clipboard
| Challenge: | BackDoor Attack (BDA) study aims to train a poisoned model with clean data and some trigger-embedded instances to perform normally on normal inputs. |
| Approach: | They propose to train a poisoned model with clean and poisonest inputs . they propose to use triggers to predict those poisonets as target labels . |
| Outcome: | The proposed model can predict P2P dynamically without human intervention. |
Enhancing Event Causality Identification with LLM Knowledge and Concept-Level Event Relations (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods to identify causal relationships between events often overlook the dependencies between similar events. |
| Approach: | They propose an ECI method enhanced by LLM Knowledge and Concept-Level Event Relations (LKCER) the method constructs a conceptual-level heterogeneous event graph by leveraging local contextual information of related event mentions. |
| Outcome: | The proposed method outperforms previous state-of-the-art methods on both benchmarks, EventStoryLine and Causal-TimeBank. |
Leibniz: Theory-of-Mind Driven Neuro-Symbolic Logical Reasoning via Multi-Agent Collaboration (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for logical reasoning with large language models suffer from insufficient rule semantic grounding and weak rule application mechanisms. |
| Approach: | They propose a theory-of-mind driven neuro-symbolic reasoning framework that integrates natural language and symbolic representations throughout the reasoning process. |
| Outcome: | The proposed model surpasses state-of-the-art models in reasoning accuracy and flexibility. |
FaithLM: Towards Faithful Explanations for Large Language Models (2026.eacl-long)
Copied to clipboard
Yu-Neng Chuang, Guanchu Wang, Chia-Yuan Chang, Ruixiang Tang, Shaochen Zhong, Fan Yang, Andrew Wen, Mengnan Du, Xuanting Cai, Vladimir Braverman, Xia Hu
| Challenge: | Large language models (LLMs) produce natural language explanations, but they lack faithfulness and do not reflect the evidence the model uses to decide. |
| Approach: | They propose a model-agnostic framework that evaluates and improves the faithfulness of LLM explanations without token masking or task-specific heuristics. |
| Outcome: | The proposed framework improves faithfulness of large language models without masking or heuristics. |
Suggest-Verify-Revise: A Three-Stage Document-Level Event Causality Identification with Narrative Consistency (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for document-level Event Causality Identification rely on local semantic similarity for independent event-pair discrimination . Existing approaches ignore the influence of the overall narrative backbone in the propagation of causal dependencies and the role differentiation of events within multi-cause/multi-effect structures. |
| Approach: | They propose a suggest-verify-revise approach for document-level Event Causality Identification with narrative consistency (SVRECI) they integrate heuristic causal suggestions generated by an LLM with structural suggestions derived from hypergraph modeling . |
| Outcome: | The proposed approach outperforms existing methods on event-storylines and Causal-TimeBank datasets. |
Reformatted Alignment (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Current methods to improve data quality are labor-intensive or prone to factual errors caused by LLM hallucinations. |
| Approach: | They propose a method which reformats the responses of instruction data into a format that better aligns with pre-established criteria and the collated evidence. |
| Outcome: | The proposed approach minimizes human annotation, hallucination, and the difficulty in scaling, remaining orthogonal to existing alignment techniques. |
Can Small Language Models Help Large Language Models Reason Better?: LM-Guided Chain-of-Thought (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing frameworks for guiding a language model in reasoning tasks are limited by their tendency to generate low-quality rationales that are repetitive and vacuous. |
| Approach: | They propose a framework that leverages a lightweight language model for guiding a black-box large LM in reasoning tasks. |
| Outcome: | The proposed framework outperforms baselines in answer prediction accuracy. |
Uni-Retrieval: A Multi-Style Retrieval Framework for STEM’s Education (2025.acl-long)
Copied to clipboard
| Challenge: | Current retrieval models focus on natural text-image retrieval, which is insufficient for STEM education contexts due to ambiguities in the retrieval process. |
| Approach: | They propose a diverse expression retrieval task tailored to educational scenarios . they extract query style features as prototypes and build a continuously updated Prompt Bank . |
| Outcome: | The proposed model outperforms existing retrieval models in most retrieval tasks. |
Cross-lingual Social Misinformation Detector based on Hierarchical Mixture-of-Experts Adapter (2025.coling-main)
Copied to clipboard
| Challenge: | a global trend of misinformation is affecting non-native speaker users who are more susceptible to misinformation on foreign social media platforms. |
| Approach: | They propose a method to integrate sentiment analysis as an auxiliary task and a hierarchical routing strategy and expert-mask mechanism to enhance cross-lingual social misinformation detection. |
| Outcome: | The proposed method improves cross-lingual social misinformation detection in non-native speakers with only monolingual social media histories. |
TaCube: Pre-computing Data Cubes for Answering Numerical-Reasoning Questions over Tabular Data (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing auto-regressive pre-trained language models are challenged by recent emerging numerical reasoning datasets due to the error-prone implicit calculation. |
| Approach: | They propose a pre-computation tool to pre-compute aggregation/arithmetic results for the table in advance, so they are handy and readily available for PLMs to answer numerical reasoning questions. |
| Outcome: | The proposed model improves on TAT-QA and T5 and BART-large on multiple benchmarks. |
QueryLink: Leveraging Query-Memory Alignment for Long-Term Reasoning in LLM Agents (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to integrating external memory prioritize memory organization while overlooking a critical semantic gap between implicit, intent-driven queries and explicit, narrative-based memories. |
| Approach: | They propose a framework that leverages Query-Memory Alignment to project both queries and memories into a shared semantic space. |
| Outcome: | The proposed framework significantly outperforms SOTA methods on the LoCoMo and LongMemEval benchmarks and can be integrated as a plug-and-play component to boost existing vector-based systems like A-MEM. |
AutoSchemaKG: Autonomous Knowledge Graph Construction through Dynamic Schema Induction from Web-Scale Corpora (2026.acl-long)
Copied to clipboard
Jiaxin Bai, Wei Fan, Qi Hu, Qing Zong, Chunyang Li, Hong Ting Tsang, Hongyu Luo, Yauwai Yim, Haoyu Huang, Xiao Zhou, Feng Qin, Tianshi Zheng, Xi Peng, Xin Yao, Huiwen Yang, Leijie Wu, JI Yi, Gong Zhang, Renhai Chen, Yangqiu Song
| Challenge: | Existing knowledge graph construction frameworks require predefined schemas, limiting their scalability and domain coverage. |
| Approach: | They propose a framework for fully autonomous knowledge graph construction that eliminates the need for predefined schemas. |
| Outcome: | The proposed framework outperforms state-of-the-art models on multi-hop QA tasks and enhances LLM factuality. |
Best Practices for Distilling Large Language Models into BERT for Web Search Ranking (2025.coling-industry)
Copied to clipboard
| Challenge: | Recent studies have highlighted the potential of Large Language Models (LLMs) as zero-shot relevance rankers. |
| Approach: | They propose to use a ranking loss to transfer ranking knowledge from LLMs to smaller models like BERT. |
| Outcome: | The proposed model has been successfully integrated into a commercial web search engine as of February 2024. |
Instruct-of-Reflection: Enhancing Large Language Models Iterative Reflection Capabilities via Dynamic-Meta Instruction (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches involve models iterating and improving their previous responses based on internal reflection ability or external feedback. |
| Approach: | They propose a reflection framework that leverages meta-thoughts and self-consistency to enhance the iterative reflection capability of Large LanguageModels. |
| Outcome: | The proposed framework achieves an average improvement of 10.1% over established baselines in mathematical and commonsense reasoning tasks, highlighting its efficacy and applicability. |
FlowSearch: Advancing Deep Research with Dynamic Structured Knowledge Flow (2026.acl-long)
Copied to clipboard
Yusong Hu, Runmin Ma, Yue Fan, Jinxin Shi, Zongsheng Cao, Yuhao Zhou, Jiakang Yuan, Shuaiyu Zhang, Shiyang Feng, Xiangchao Yan, Shufei Zhang, Wenlong Zhang, Lei Bai, Bo Zhang
| Challenge: | FlowSearch is a multi-agent framework that actively constructs and evolves a dynamic structured knowledge flow to drive subtask execution and reasoning. |
| Approach: | They propose a multi-agent framework that actively constructs and evolves a dynamic structured knowledge flow to drive subtask execution and reasoning. |
| Outcome: | The proposed framework achieves competitive performance on GAIA, HLE, GPQA and TRQA benchmarks and is available to download. |
Tackling Long Code Search with Splitting, Encoding, and Aggregating (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing pretraining models take the first 256 tokens of code snippets by default, limiting the input length to 512. |
| Approach: | They propose a baseline SEA model which splits long code into code blocks and aggregates them to obtain a comprehensive long code representation. |
| Outcome: | The proposed model can model long code without changing their internal structure and re-pretraining. |
HAD: HAllucination Detection Language Models Based on a Comprehensive Hallucination Taxonomy (2026.acl-industry)
Copied to clipboard
| Challenge: | relying on large language models for information has raised concerns about reliability and accuracy of outputs. |
| Approach: | They propose a hallucination taxonomy with 11 categories for various NLG tasks and propose HAllucination Detection models which integrate hallucinism detection, span-level identification, and correction into a single inference process. |
| Outcome: | The proposed models outperform baselines on HaluEval, FactCHD, and FaithBench, confirming their robustness and versatility. |
InfiMM-WebMath-40B: Advancing Multimodal Pre-Training for Enhanced Mathematical Reasoning (2025.findings-emnlp)
Copied to clipboard
Xiaotian Han, Yiren Jian, Xuefeng Hu, Haogeng Liu, Yiqi Wang, Qihang Fan, Yuang Ai, Huaibo Huang, Ran He, Zhenheng Yang, Quanzeng You
| Challenge: | InfiMM-WebMath-40B is a dataset of interleaved image-text documents . it consists of 24 million web pages, 85 million image URLs, and 40 billion text tokens . |
| Approach: | InfiMM-WebMath-40B is a high-quality dataset of interleaved image-text documents . it contains 24 million web pages, 85 million image URLs, and 40 billion text tokens . |
| Outcome: | InfiMM-WebMath-40B is a high-quality dataset of interleaved image-text documents . it consists of 24 million web pages, 85 million image URLs, and 40 billion text tokens . |
GrandGuard: Taxonomy, Benchmark, and Safeguards for Elderly-Chatbot Interaction Safety (2026.findings-acl)
Copied to clipboard
Changxuan Fan, Xi Yang, Yueyuan Zheng, Bin Zhou, Yuanping Wang, Wenbin Hu, Huihao Jing, Ki Sen Hung, Dazhao Du, Haoran Li, Janet Hui-wen Hsiao, Yangqiu Song
| Challenge: | a survey of older adults shows that many LLMs mishandle elderly-specific contextual risks. |
| Approach: | They propose a framework to assess elderly-specific contextual risks in LLM interactions . they use a taxonomy to identify 50 fine-grained risk types across mental well-being, financial, medical, toxicity, and privacy domains . |
| Outcome: | a new framework assesses elderly-specific contextual risks in LLM interactions . it achieves 96.2% and 90.9% unsafe-prompt detection accuracy, respectively . |
FRVA: Fact-Retrieval and Verification Augmented Entailment Tree Generation for Explainable Question Answering (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for generating a entailment tree exhibit the reasoning chains from knowledge facts to predicted answers, but they have large fact search spaces and error accumulation problems resulting in the generation of invalid steps. |
| Approach: | They propose a Fact-Retrieval and Verification Augmented bidirectional entailment tree generation method that contains two systems. |
| Outcome: | The proposed method outperforms existing models and achieves state-of-the-art performance in fact selection and structural correctness. |
COSMOS: Connectivity-Oriented Submodular Maximization for Optimal Subgraph Retrieval (2026.acl-long)
Copied to clipboard
| Challenge: | Existing paradigms treat facts independently or employ myopic search, failing to optimize collective subgraph utility. |
| Approach: | They propose a framework that formalizes evidence retrieval as a constrained submodular maximization problem. |
| Outcome: | The proposed framework captures the trade-off between information relevance and structural complexity. |
TinyAlign: Boosting Lightweight Vision-Language Models by Mitigating Modal Alignment Bottlenecks (2026.findings-acl)
Copied to clipboard
Yuanze Hu, Xinyu Wang, Zhichao Yang, Gen Li, Ye Qiu, Zhaoxin Fan, Yifan Sun, Wenjun wu, Jin Dong, Xiaotie Deng
| Challenge: | Lightweight Vision-Language Models (VLMs) are indispensable for resource-constrained applications. |
| Approach: | They propose a framework that retrieves context from a memory bank to enhance alignment . they propose EMI-based approach to align vision and language models . |
| Outcome: | The proposed framework reduces training loss, accelerates convergence, and enhances task performance with negligible computational overhead. |
OmniCompliance-100K: A Multi-Domain, Rule-Grounded, Real-World Safety Compliance Dataset (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing LLM safety datasets rely on ad-hoc taxonomies and lack rule-grounded, real-world cases. |
| Approach: | They construct a rule-grounded, real-world case dataset OmniCompliance-100K from a compliance perspective using a powerful web-searching agent. |
| Outcome: | The proposed dataset spans 74 regulations and policies across a wide range of domains including security and privacy regulations, content safety and user data privacy policies, financial security requirements, medical device risk management standards, educational integrity guidelines, and protections of fundamental human rights. |
Maria: A Visual Experience Powered Conversational Agent (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies focus on grounding conversational agents on text-only corpora, but they lack the perception ability to our physical world. |
| Approach: | They propose to ground conversational agents on images retrieved from large-scale image indexes . they propose to use visual knowledge to generate informative responses based on the extracted knowledge . |
| Outcome: | The proposed agent outperforms state-of-the-art methods on automatic metrics and human evaluation. |
Privacy Checklist: Privacy Violation Detection Grounding on Contextual Integrity Theory (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing privacy studies focus on sub-fields, but they focus on a few sub-domains. |
| Approach: | They propose to use the Health Insurance Portability and Accountability Act of 1996 as an example to develop a checklist that covers social identities, private attributes, and existing privacy regulations. |
| Outcome: | The proposed checklist covers social identities, private attributes, and existing privacy regulations. |
OS Agents: A Survey on MLLM-based Agents for Computer, Phone and Browser Use (2025.acl-long)
Copied to clipboard
Xueyu Hu, Tao Xiong, Biao Yi, Zishu Wei, Ruixuan Xiao, Yurun Chen, Jiasheng Ye, Meiling Tao, Xiangxin Zhou, Ziyu Zhao, Yuhuai Li, Shengze Xu, Shenzhi Wang, Xinchen Xu, Shuofei Qiao, Zhaokai Wang, Kun Kuang, Tieyong Zeng, Liang Wang, Jiwei Li, Yuchen Eleanor Jiang, Wangchunshu Zhou, Guoyin Wang, Keting Yin, Zhou Zhao, Hongxia Yang, Fan Wu, Shengyu Zhang, Fei Wu
| Challenge: | a new generation of (M)LLMs is enabling the creation of superintelligent AI assistants . OS Agents can complete tasks autonomously and have the potential to significantly enhance the lives of billions of users worldwide. |
| Approach: | They propose to build OS Agents that operate within operating systems' GUIs and GUIs . they examine evaluation metrics and benchmarks to identify promising directions . |
| Outcome: | The proposed agents are based on operating systems (OS) and operating systems frameworks. |
Why Can Distillation Work with Limited Resources? A Systematic Study (2026.findings-acl)
Copied to clipboard
Xiao Hu, Xingyu Lu, Liyuan Mao, YiFan Zhang, Tianke Zhang, Bin Wen, Fan Yang, Tingting Gao, Guorui Zhou
| Challenge: | Recent advances in large language models have driven reasoning performance . low-resource distillation can boost models' performance, but a framework is missing . |
| Approach: | They conduct a controlled experiment to find out why low-resource distillation can boost model performance . they find that distillation enhances the presence of advanced cognitive behaviors . |
| Outcome: | The proposed model shows more flexible reasoning, the authors show . they show that distillation enhances the presence of advanced cognitive behaviors . |
Learning Neural Templates for Recommender Dialogue System (2021.emnlp-main)
Copied to clipboard
Zujie Liang, Huang Hu, Can Xu, Jian Miao, Yingying He, Yining Chen, Xiubo Geng, Fan Liang, Daxin Jiang
| Challenge: | Recent advances in neural models have shown promising progress on this task, but key challenges remain . |
| Approach: | They propose a framework that can decouple dialogue generation from item recommendation . they use a response template generator and item selector to generate a responses template . |
| Outcome: | The proposed framework outperforms the state-of-the-art methods on the benchmark ReDial. |
PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Models (2024.acl-long)
Copied to clipboard
Haoran Li, Dadi Guo, Donghao Li, Wei Fan, Qi Hu, Xin Liu, Chunkit Chan, Duanyi Yao, Yuan Yao, Yangqiu Song
| Challenge: | generative large language models (LLMs) exhibit surprising capability and integrate previous tasks into a unified text generation formulation. |
| Approach: | They propose a privacy evaluation benchmark to quantify the privacy leakage of language models. |
| Outcome: | The proposed benchmark compares PPLMs with different privacy implementations to find out how privacy leakage is handled. |
Inside Out: Evolving User-Centric Core Memory Trees for Long-Term Personalized Dialogue Systems (2026.acl-long)
Copied to clipboard
| Challenge: | Existing personalized dialogue systems struggle to reconcile unbounded interactions with finite context constraints. |
| Approach: | They propose a framework that utilizes a globally maintained PersonaTree as the carrier of long-term user profiling. |
| Outcome: | The proposed framework outperforms existing systems in suppressing contextual noise and persona inconsistency. |
Improving Contextual Faithfulness of Large Language Models via Retrieval Heads-Induced Optimization (2025.acl-long)
Copied to clipboard
Lei Huang, Xiaocheng Feng, Weitao Ma, Yuchun Fan, Xiachong Feng, Yangfan Ye, Weihong Zhong, Yuxuan Gu, Baoxin Wang, Dayong Wu, Guoping Hu, Bing Qin
| Challenge: | Existing frameworks for retrieval-augmented large language models (LLMs) are lacking in LFQA faithfulness testing. |
| Approach: | They propose a framework to teach retrieval-augmented large language models to explicitly discriminate between faithful and unfaithful generations. |
| Outcome: | The proposed framework outperforms GPT-4o in LFQA scenarios and outperformed existing benchmarks. |
Hyperlink-induced Pre-training for Passage Retrieval in Open-domain Question Answering (2022.acl-long)
Copied to clipboard
Jiawei Zhou, Xiaoguang Li, Lifeng Shang, Lan Luo, Ke Zhan, Enrui Hu, Xinyu Zhang, Hao Jiang, Zhao Cao, Fan Yu, Xin Jiang, Qun Liu, Lei Chen
| Challenge: | Existing methods to train dense passage retrieval have a large data gap between upstream and downstream relevance. |
| Approach: | They propose a method to pre-train the dense retriever with the text relevance induced by hyperlinks within Web documents. |
| Outcome: | The proposed method outperforms existing methods under different scenarios and in the open-domain question answering domain. |
KuiLeiXi: a Chinese Open-Ended Text Adventure Game (2021.acl-demo)
Copied to clipboard
Yadong Xi, Xiaoxi Mao, Le Li, Lei Lin, Yanjiang Chen, Shuhan Yang, Xuhan Chen, Kailun Tao, Zhi Li, Gongzheng Li, Lin Jiang, Siyan Liu, Zeng Zhao, Minlie Huang, Changjie Fan, Zhipeng Hu
| Challenge: | Recent advances in pre-trained language models have made it possible to generate human-like text. |
| Approach: | They propose to integrate an open-ended text adventure game in Chinese, named KuiLeiXi, where players interact with the AI until the plot goals are reached. |
| Outcome: | The proposed game lacks incentives and relies on players to explore on their own. |
IntelliCockpitBench: A Comprehensive Benchmark to Evaluate VLMs for Intelligent Cockpit (2025.findings-acl)
Copied to clipboard
| Challenge: | Visual Question Answering (VQA) is a key task in vehicular systems. |
| Approach: | They propose a benchmark that encompasses diverse automotive scenarios . they use images from front, side, and rear cameras, various road types, weather conditions, and interior views . |
| Outcome: | The proposed benchmark includes images from front, side, and rear cameras, various road types, weather conditions, and interior views. |