Papers by Haochen Li
Disentangling Logic: The Role of Context in Large Language Model Reasoning Capabilities (2025.findings-acl)
Copied to clipboard
Wenyue Hua, Kaijie Zhu, Lingyao Li, Lizhou Fan, Mingyu Jin, Shuhang Lin, Haochen Xue, Zelong Li, Jindong Wang, Yongfeng Zhang
| Challenge: | Using large language models, large language model models can be used to evaluate reasoning abilities in context-rich scenarios. |
| Approach: | They construct datasets for both propositional logic and abductive logic reasoning with four difficulty levels across 12 distinct domains based on Wikipedia categorization and those with purely abstract variables. |
| Outcome: | The proposed model can be used to benchmark LLMs in real-world scenarios, but not in context-rich scenarios. |
On the Role of Entity and Event Level Conceptualization in Generalizable Reasoning: A Survey of Tasks, Methods, Applications, and Future Directions (2025.findings-emnlp)
Copied to clipboard
Weiqi Wang, Tianqing Fang, Haochen Shi, Baixuan Xu, Wenxuan Ding, Liyu Zhang, Wei Fan, Jiaxin Bai, Haoran Li, Xin Liu, Yangqiu Song
| Challenge: | Conceptualization is a fundamental element of human cognition and plays a pivotal role in generalizable reasoning. |
| Approach: | They propose to categorize different types of conceptualizations into four levels based on the types of instances being conceptualized. |
| Outcome: | The proposed categorization of different types of conceptualizations into four levels based on the types of instances being conceptualized . |
MMRC: A Large-Scale Benchmark for Understanding Multimodal Large Language Model in Real-World Conversation (2025.acl-long)
Copied to clipboard
Haochen Xue, Feilong Tang, Ming Hu, Yexin Liu, Qidong Huang, Yulong Li, Chengzhi Liu, Zhongxing Xu, Chong Zhang, Chun-Mei Feng, Yutong Xie, Imran Razzak, Zongyuan Ge, Jionglong Su, Junjun He, Yu Qiao
| Challenge: | Existing multimodal large language models lack the ability to memorize, recall, and reason in sustained interactions. |
| Approach: | They propose a multimodal real-world conversation benchmark for evaluating open-ended abilities of multimodal large language models. |
| Outcome: | The proposed benchmarks show that the models perform better in open-ended conversations. |
NaSGEC: a Multi-Domain Chinese Grammatical Error Correction Dataset from Native Speaker Texts (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies on Chinese grammatical error correction focus on learning essays. |
| Approach: | They propose a Chinese grammatical error correction dataset that annotates multiple references for 12,500 sentences from three native domains. |
| Outcome: | The proposed dataset can be used to facilitate research on Chinese grammatical error correction (CGEC) for native speaker texts from multiple domains. |
Reasoning Makes Good Annotators : An Automatic Task-specific Rules Distilling Framework for Low-resource Relation Extraction (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to extract knowledge from unlabeled data generate noise labels. |
| Approach: | They propose an automatic task-specific rules distilling framework to generate a logic rule from unlabeled data. |
| Outcome: | The proposed framework could power the labeling ability by discovering reliable model-labeled data. |
RECALL: REpresentation-aligned Catastrophic-forgetting ALLeviation via Hierarchical Model Merging (2025.emnlp-main)
Copied to clipboard
Bowen Wang, Haiyuan Wan, Liwen Shi, Chen Yang, Peng He, Yue Ma, Haochen Han, Wenhao Li, Tiao Tan, Yongjian Li, Fangming Liu, Gong Yifan, Sheng Zhang
| Challenge: | Existing models that require task labels or performance trade-offs are susceptible to catastrophic forgetting. |
| Approach: | They propose a representation-aware model merging framework for continual learning without access to historical data. |
| Outcome: | The proposed framework outperforms baselines in knowledge retention and generalization across five NLP tasks and multiple continual learning scenarios. |
Do LLMs Behave as Claimed? Investigating How LLMs Follow Their Own Claims using Counterfactual Questions (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation frameworks rely on curated datasets that, once public, may be accessed by newer LLMs. |
| Approach: | They propose a framework that generates counterfactual questions and answers from existing evaluation datasets and uses them to evaluate LLMs. |
| Outcome: | The proposed evaluation framework reduces the risk of data leakage by allowing the LLMs to respond to counterfactual questions and verify their claims. |
Few-shot Knowledge Graph Relational Reasoning via Subgraph Adaptation (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods to predict unseen triplets from knowledge graphs are limited by spurious information in KGs. |
| Approach: | They propose a framework that adapts contextualized graphs to subgraphs generated from support and query triplets to perform the prediction. |
| Outcome: | The proposed framework extracts more comprehensive information from support triplets while minimizing spurious information when predicting query triplet. |
LogiDynamics: Unraveling the Dynamics of Inductive, Abductive and Deductive Logical Inferences in LLM Reasoning (2025.emnlp-main)
Copied to clipboard
Tianshi Zheng, Cheng Jiayang, Chunyang Li, Haochen Shi, Zihao Wang, Jiaxin Bai, Yangqiu Song, Ginny Wong, Simon See
| Challenge: | Modern large language models (LLMs) employ diverse logical inference mechanisms for reasoning. |
| Approach: | They analyze the comparative dynamics of inductive (System 1) versus abductive/deductive (system 2) inference in large language models by using a controlled analogical reasoning environment and a MCQ/free-text task format. |
| Outcome: | The proposed methods can significantly scale LLM reasoning. |
InferenceDynamics: Adaptive LLM Routing through Structured Capability and Knowledge Profiling (2026.acl-long)
Copied to clipboard
Haochen Shi, Tianshi Zheng, Weiqi Wang, Baixuan Xu, Chunyang Li, Chunkit Chan, Tao Fan, Yangqiu Song
| Challenge: | Large Language Model (LLM) routing is a pivotal technique for navigating a diverse landscape of LLMs. |
| Approach: | They propose a flexible and scalable multi-dimensional routing framework that models the capability and knowledge of models. |
| Outcome: | The proposed framework can be used to generalize and identify top-performing models for group-level routing using modern benchmarks including MMLU-Pro, GPQA, BigGenBench, and LiveBench. |
Exploring Representation-level Augmentation for Code Search (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent data augmentations for code search are at the raw-data level, which requires additional code analysis and training cost. |
| Approach: | They propose a general format of representation-level augmentation that unifies existing methods. |
| Outcome: | The proposed methods can boost the performance of code search models on a large-scale dataset. |
FaStFact: Faster, Stronger Long-Form Factuality Evaluations in LLMs (2025.findings-emnlp)
Copied to clipboard
Yingjia Wan, Haochen Tan, Xiao Zhu, Xinyu Zhou, Zhiwei Li, Qingsong Lv, Changxuan Sun, Jiaqi Zeng, Yi Xu, Jianqiao Lu, Yinhong Liu, Zhijiang Guo
| Challenge: | Prior evaluation pipelines fail to evaluate factuality of long-form LLMs due to inefficiency and costly human assessment. |
| Approach: | They propose a fast and strong evaluation pipeline that can evaluate factuality of long-form LLMs . they propose 'faStFact' to reduce cost of web searching and inference calling . |
| Outcome: | The proposed evaluation pipeline achieves highest alignment with human evaluation and efficiency among existing baselines. |
Crowdsource, Crawl, or Generate? Creating SEA-VL, a Multicultural Vision-Language Dataset for Southeast Asia (2025.acl-long)
Copied to clipboard
Samuel Cahyawijaya, Holy Lovenia, Joel Ruben Antony Moniz, Tack Hwa Wong, Mohammad Rifqi Farhansyah, Thant Thiri Maung, Frederikus Hudi, David Anugraha, Muhammad Ravi Shulthan Habibi, Muhammad Reza Qorib, Amit Agarwal, Joseph Marvin Imperial, Hitesh Laxmichand Patel, Vicky Feliren, Bahrul Ilmi Nasution, Manuel Antonio Rufino, Genta Indra Winata, Rian Adam Rajagede, Carlos Rafael Catalan, Mohamed Fazli Mohamed Imam, Priyaranjan Pattnayak, Salsabila Zahirah Pranida, Kevin Pratama, Yeshil Bangera, Adisai Na-Thalang, Patricia Nicole Monderin, Yueqi Song, Christian Simon, Lynnette Hui Xian Ng, Richardy Lobo Sapan, Taki Hasan Rafi, Bin Wang, null Supryadi, Kanyakorn Veerakanjana, Piyalitt Ittichaiwong, Matthew Theodore Roque, Karissa Vincentio, Takdanai Kreangphet, Phakphum Artkaew, Kadek Hendrawan Palgunadi, Yanzhi Yu, Rochana Prih Hastuti, William Nixon, Mithil Bangera, Adrian Xuan Wei Lim, Aye Hninn Khine, Hanif Muhammad Zhafran, Teddy Ferdinan, Audra Aurora Izzani, Ayushman Singh, Evan Evan, Jauza Akbar Krito, Michael Anugraha, Fenal Ashokbhai Ilasariya, Haochen Li, John Amadeo Daniswara, Filbert Aurelian Tjiaranata, Eryawan Presma Yulianrifat, Can Udomcharoenchaikit, Fadil Risdian Ansori, Mahardika Krisna Ihsani, Giang Nguyen, Anab Maulana Barik, Dan John Velasco, Rifo Ahmad Genadi, Saptarshi Saha, Chengwei Wei, Isaiah Edri W. Flores, Kenneth Chen Ko Han, Anjela Gail D. Santos, Wan Shen Lim, Kaung Si Phyo, Tim Santos, Meisyarah Dwiastuti, Jiayun Luo, Jan Christian Blaise Cruz, Ming Shan Hee, Ikhlasul Akmal Hanif, M.Alif Al Hakim, Muhammad Rizky Sya’ban, Kun Kerdthaisong, Lester James Validad Miranda, Fajri Koto, Tirana Noor Fatyanosa, Alham Fikri Aji, Jostin Jerico Rosal, Jun Kevin, Robert Wijaya, Onno P. Kampman, Ruochen Zhang, Börje F. Karlsson, Peerat Limkonchotiwat
| Challenge: | Southeast Asia is underrepresented in vision-language research . SEA-VL is an open-source initiative dedicated to developing culturally relevant datasets for SEA languages. |
| Approach: | They propose to use crowdsourced, automated image crawling and synthetic image generation to develop culturally relevant datasets for SEA languages. |
| Outcome: | The proposed datasets capture SEA cultural nuances and contexts better than existing datasets. |
QiMeng-Attention: SOTA Attention Operator is generated by SOTA Attention Algorithm (2025.findings-acl)
Copied to clipboard
Qirui Zhou, Shaohui Peng, Weiqiang Xiong, Haixin Chen, Yuanbo Wen, Haochen Li, Ling Li, Qi Guo, Yongwei Zhao, Ke Gao, Ruizhi Chen, Yanjun Wu, Zhao Chen, Yunji Chen
| Challenge: | Existing LLMs cannot comprehend the complex data flow and computation process of the attention operator and utilize low-level primitive to exploit GPU performance. |
| Approach: | They propose an LLM-friendly Thinking Language (LLM-TL) that can decouple the generation of high-level optimization logic and low-level implementation on GPU and enhance LLMs’ understanding of attention operator. |
| Outcome: | The proposed method outshines existing LLMs on A100, RTX8000, and T4 GPUs, achieving a speed-up of up to 35.16. |
LegalDrill: Diagnosis-Driven Synthesis for Legal Reasoning in Small Language Models (2026.acl-industry)
Copied to clipboard
Tianchun Li, Haochen Liu, Vishwa Pardeshi, Xingchen Wang, Tianci Liu, Huijun Zhao, Wei Fan, Jing Gao
| Challenge: | Small language models (SLMs) are promising for real-world deployment but struggle with high-stakes legal reasoning tasks. |
| Approach: | They propose a diagnostic-driven synthesis framework that extracts and refines reasoning trajectories from a capable teacher via fine-grained prompting and a self-reflective verification is employed to adaptively select the most effective data for the SLM student. |
| Outcome: | The proposed framework extracts and refines reasoning trajectories from a capable teacher via fine-grained prompting, then a self-reflective verification is employed to adaptively select the most effective data for the student. |
Knowledge Graph-Enhanced Large Language Models via Path Selection (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown unprecedented performance in various real-world applications, but they are known to generate factually inaccurate outputs. |
| Approach: | They propose a framework to integrate external knowledge extracted from Knowledge Graphs (KGs) they propose to generate scores for knowledge paths with input texts via latent semantic matching. |
| Outcome: | Experiments on real-world datasets validate the effectiveness of a framework to extract knowledge from Knowledge Graphs (KGs) incorporating external knowledge has become a promising strategy to improve the factual accuracy of LLM-generated outputs. |
ProxyQA: An Alternative Framework for Evaluating Long-Form Text Generation with Large Language Models (2024.acl-long)
Copied to clipboard
Haochen Tan, Zhijiang Guo, Zhan Shi, Lu Xu, Zhili Liu, Yunlong Feng, Xiaoguang Li, Yasheng Wang, Lifeng Shang, Qun Liu, Linqi Song
| Challenge: | Existing evaluation methods for large language models are labor-intensive and lack efficiency. |
| Approach: | They propose a framework dedicated to assessing long-text generation that includes in-depth human-curated meta-questions spanning various domains . they use a set of proxy-quests with pre-annotated answers to assess the content's quality by incorporating the generated texts as contextual background. |
| Outcome: | The proposed framework assesses the quality of long-text content by matching it with references through human evaluation or automated metrics. |
GiFT: Gibbs Fine-Tuning for Code Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Training Large Language Models (LLMs) with synthetic data is a prevalent practice in code generation. |
| Approach: | They propose a method to fine-tune large language models with code drawn from a conditional distribution, conditioned on a specific seed description. |
| Outcome: | The proposed method improves performance on four datasets and shows that it can be used to fine-tune LLMs with code derived from the marginal distribution. |
KiPT: Knowledge-injected Prompt Tuning for Event Detection (2022.coling-1)
Copied to clipboard
| Challenge: | Existing prompt-based methods may suffer from low precision because they lack event-related semantic knowledge. |
| Approach: | They propose a Knowledge-injected Prompt Tuning model to improve prompt tuning . event detection aims to detect events from text by identifying and classifying event triggers . |
| Outcome: | The proposed model outperforms baseline models in few-shot scenarios. |
MIND: Multimodal Shopping Intention Distillation from Large Vision-language Models for E-commerce Purchase Understanding (2024.emnlp-main)
Copied to clipboard
Baixuan Xu, Weiqi Wang, Haochen Shi, Wenxuan Ding, Huihao Jing, Tianqing Fang, Jiaxin Bai, Xin Liu, Changlong Yu, Zheng Li, Chen Luo, Qingyu Yin, Bing Yin, Long Chen, Yangqiu Song
| Challenge: | Existing methods for acquiring large-scale intentions generate product-centric intentions without product images and incur high costs for scalability. |
| Approach: | They propose a multimodal framework that allows Large Vision-Language Models to infer purchase intentions from multimodal product metadata and prioritize human-centric ones. |
| Outcome: | The proposed framework shows that it is robust to different prompts and superior to previous methods. |
TAGS: A Test-Time Generalist–Specialist Framework with Retrieval-Augmented Reasoning and Verification (2026.findings-acl)
Copied to clipboard
Jianghao Wu, Feilong Tang, Yulong Li, Ming Hu, Haochen Xue, Shoaib Jameel, Zongyuan Ge, Yutong Xie, Imran Razzak
| Challenge: | Existing efforts to improve medical question answering performance follow two directions. |
| Approach: | They propose a framework that combines a generalist with a domain-specific specialist without any model fine-tuning or parameter updates. |
| Outcome: | The proposed framework boosts GPT-4o accuracy by 13.8%, deepseek-R1 by 16.8%, and improves a vanilla 7B model from 14.1% to 23.9%. |
OmniCompliance-100K: A Multi-Domain, Rule-Grounded, Real-World Safety Compliance Dataset (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing LLM safety datasets rely on ad-hoc taxonomies and lack rule-grounded, real-world cases. |
| Approach: | They construct a rule-grounded, real-world case dataset OmniCompliance-100K from a compliance perspective using a powerful web-searching agent. |
| Outcome: | The proposed dataset spans 74 regulations and policies across a wide range of domains including security and privacy regulations, content safety and user data privacy policies, financial security requirements, medical device risk management standards, educational integrity guidelines, and protections of fundamental human rights. |
Rewriting the Code: A Simple Method for Large Language Model Augmented Code Search (2024.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that ReCo significantly boosts retrieval accuracy across sparse, zero-shot dense and fine-tuned dense search settings. |
| Approach: | They propose a generation-augmented retrieval framework that additionally Rewrites the Code (ReCo) within the codebase for style normalization. |
| Outcome: | The proposed method significantly boosts retrieval accuracy across sparse, zero-shot dense, and fine-tuned dense retrieval settings in diverse search scenarios. |
CANDLE: Iterative Conceptualization and Instantiation Distillation from Large Language Models for Commonsense Reasoning (2024.acl-long)
Copied to clipboard
Weiqi Wang, Tianqing Fang, Chunyang Li, Haochen Shi, Wenxuan Ding, Baixuan Xu, Zhaowei Wang, Jiaxin Bai, Xin Liu, Cheng Jiayang, Chunkit Chan, Yangqiu Song
| Challenge: | Existing approaches to generalize commonsense reasoning lack instantiated knowledge and require pre-built concept taxonomies and annotations. |
| Approach: | They propose a framework that iteratively performs contextualized conceptualization and instantiation over commonsense knowledge bases by instructing large language models to generate both types of knowledge with critic filtering. |
| Outcome: | Empirical results show that distilling CANDLE on student models provides benefits across three downstream tasks. |
Question-Aware Knowledge Graph Prompting for Enhancing Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for large language models require costly fine-tuning or retrieve noisy KG information. |
| Approach: | They propose to generate KG-based input embedding prefixes as soft prompts but fail to account for question relevance, resulting in noisy prompts. |
| Outcome: | The proposed model outperforms state-of-the-art methods across multiple datasets. |
SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages (2024.emnlp-main)
Copied to clipboard
Holy Lovenia, Rahmad Mahendra, Salsabil Akbar, Lester James Miranda, Jennifer Santoso, Elyanah Aco, Akhdan Fadhilah, Jonibek Mansurov, Joseph Marvin Imperial, Onno Kampman, Joel Moniz, Muhammad Habibi, Frederikus Hudi, Jann Montalan, Ryan Hadiwijaya, Joanito Lopo, William Nixon, Börje Karlsson, James Jaya, Ryandito Diandaru, Yuze Gao, Patrick Irawan, Bin Wang, Jan Christian Blaise Cruz, Chenxi Whitehouse, Ivan Parmonangan, Maria Khelli, Wenyu Zhang, Lucky Susanto, Reynard Ryanda, Sonny Hermawan, Dan Velasco, Muhammad Kautsar, Willy Hendria, Yasmin Moslem, Noah Flynn, Muhammad Adilazuarda, Haochen Li, Johanes Lee, R. Damanhuri, Shuo Sun, Muhammad Qorib, Amirbek Djanibekov, Wei Qi Leong, Quyet V. Do, Niklas Muennighoff, Tanrada Pansuwan, Ilham Firdausi Putra, Yan Xu, Tai Chia, Ayu Purwarianti, Sebastian Ruder, William Tjhi, Peerat Limkonchotiwat, Alham Aji, Sedrick Keh, Genta Winata, Ruochen Zhang, Fajri Koto, Zheng Xin Yong, Samuel Cahyawijaya
| Challenge: | Southeast Asia (SEA) is home to over 1,300 indigenous languages and 671 million people . prevailing AI models suffer from a significant lack of representation of texts, images, and audio datasets from SEA . |
| Approach: | They propose to provide a resource center that provides standardized corpora in nearly 1,000 SEA languages across three modalities. |
| Outcome: | a new benchmark assesses the quality of AI models on 36 SEA languages across 13 tasks . the results highlight the importance of SEA as a culturally diverse region . |
Rethinking Negative Pairs in Code Search (2023.emnlp-main)
Copied to clipboard
| Challenge: | Comparative learning is a key component in fine-tuning code search models . however, negative samples of InfoNCE may deteriorate its representation learning . |
| Approach: | They propose a loss function that inserts weight terms into InfoNCE to improve contrastive learning. |
| Outcome: | The proposed loss function is a special case of Soft-InfoNCE, the authors show . it is more accurate than other loss functions, and it is faster than other models. |