Papers by Chunkit Chan
StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding (2023.emnlp-main)
Copied to clipboard
Cheng Jiayang, Lin Qiu, Tsz Chan, Tianqing Fang, Weiqi Wang, Chunkit Chan, Dongyu Ru, Qipeng Guo, Hongming Zhang, Yangqiu Song, Yue Zhang, Zheng Zhang
| Challenge: | Analogy-making between narratives is crucial for human reasoning . despite its importance, there has been limited research on story analogies . |
| Approach: | They construct a large-scale story-level analogy corpus with 24K story pairs . they find that the tasks are incredibly difficult for large language models such as ChatGPT . |
| Outcome: | The proposed corpus contains 24K story pairs from diverse domains with human annotations on two similarities from the extended Structure-Mapping Theory. |
A Survey of RAG-Reasoning Systems in Large Language Models (2025.findings-emnlp)
Copied to clipboard
Yangning Li, Weizhi Zhang, Yuyao Yang, Wei-Chieh Huang, Yaozu Wu, Junyu Luo, Yuanchen Bei, Henry Peng Zou, Xiao Luo, Yusheng Zhao, Chunkit Chan, Yankai Chen, Zhongfen Deng, Yinghui Li, Hai-Tao Zheng, Dongyuan Li, Renhe Jiang, Ming Zhang, Yangqiu Song, Philip S. Yu
| Challenge: | a survey of RAG-based reasoning-based approaches shows that it is not effective for multi-step inferences. |
| Approach: | They map how advanced reasoning optimizes each stage of RAG . they show how retrieved knowledge supply missing premises and expand context for complex inference . |
| Outcome: | The proposed frameworks achieve state-of-the-art across knowledge-intensive benchmarks. |
NegotiationToM: A Benchmark for Stress-testing Machine Theory of Mind on Negotiation Surrounding (2024.findings-emnlp)
Copied to clipboard
Chunkit Chan, Cheng Jiayang, Yauwai Yim, Zheye Deng, Wei Fan, Haoran Li, Xin Liu, Hongming Zhang, Weiqi Wang, Yangqiu Song
| Challenge: | Theory of mind evaluations currently focus on testing models using machine-generated data or game settings prone to shortcuts and spurious correlations. |
| Approach: | They propose a benchmark to stress-test machine ToM in real-world negotiation surrounding covered multi-dimensional mental states. |
| Outcome: | The proposed benchmark builds upon the Belief-Desire-Intention theory and conducts the necessary empirical experiments to evaluate large language models. |
Lion: Adversarial Distillation of Proprietary Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for transferring knowledge from a proprietary large language model to an open-source LLM have been criticized for not incorporating any feedback to boost student model's proficiency iteratively. |
| Approach: | They propose a novel adversarial distillation framework that integrates feedback to identify challenging instructions where the student model’s performance falls short. |
| Outcome: | The proposed framework surpasses existing state-of-the-art LLMs by 55.4% in challenging zero-shot reasoning benchmarks and 16.7% on AGIEval. |
InferenceDynamics: Adaptive LLM Routing through Structured Capability and Knowledge Profiling (2026.acl-long)
Copied to clipboard
Haochen Shi, Tianshi Zheng, Weiqi Wang, Baixuan Xu, Chunyang Li, Chunkit Chan, Tao Fan, Yangqiu Song
| Challenge: | Large Language Model (LLM) routing is a pivotal technique for navigating a diverse landscape of LLMs. |
| Approach: | They propose a flexible and scalable multi-dimensional routing framework that models the capability and knowledge of models. |
| Outcome: | The proposed framework can be used to generalize and identify top-performing models for group-level routing using modern benchmarks including MMLU-Pro, GPQA, BigGenBench, and LiveBench. |
Text-Tuple-Table: Towards Information Integration in Text-to-Table Generation via Global Tuple Extraction (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to condensing textual information into concise and structured tables are limited in their applicability in broader contexts. |
| Approach: | They propose a benchmark dataset for generating summary tables of competitions based on real-time commentary texts that incorporates large-scale textual information into concise and structured tables. |
| Outcome: | The proposed method exhibits strong generalization abilities, surpassing previous approaches on several other text-to-table datasets. |
XToM: Exploring the Multilingual Theory of Mind for Large Language Models (2026.acl-long)
Copied to clipboard
Chunkit Chan, Yauwai Yim, Hongchuan Zeng, Zhiying Zou, Xinyuan Cheng, Zhifan Sun, Zheye Deng, Kawai Chung, Yuzhuo Ao, Fan Yixiang, Cheng Jiayang, Ercong Nie, Ginny Wong, Helmut Schmid, Hinrich Schuetze, Simon See, Yangqiu Song
| Challenge: | Existing evaluations of ToM in LLMs are limited to English, neglecting the linguistic diversity that shapes human cognition. |
| Approach: | They propose a multilingual benchmark that evaluates ToM across five languages . they find that models excel in multilingual language understanding, but their ToM performance varies across languages. |
| Outcome: | The proposed benchmark evaluates LLMs across five languages and incorporates diverse task scenarios. |
EventGround: Narrative Reasoning by Grounding to Eventuality-centric Knowledge Graphs (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing frameworks for leveraging background knowledge of narratives are limited. |
| Approach: | They propose a framework to ground free-texts to eventuality-centric KGs for narrative reasoning . their framework is based on a set of probabilistic probabilistic models that are grounded in the real world . |
| Outcome: | The proposed framework outperforms baseline models while providing interpretable evidence. |
InteGround: On the Evaluation of Verification and Retrieval Planning in Integrative Grounding (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing grounding approaches work well for simple queries, but many real-world information needs require synthesizing multiple pieces of evidence. |
| Approach: | They introduce "integrative grounding" to evaluate the ability to ground large language models in external knowledge sources. |
| Outcome: | The proposed approach is robust to redundant evidence, but rationalizes using internal knowledge when information is incomplete. |
CANDLE: Iterative Conceptualization and Instantiation Distillation from Large Language Models for Commonsense Reasoning (2024.acl-long)
Copied to clipboard
Weiqi Wang, Tianqing Fang, Chunyang Li, Haochen Shi, Wenxuan Ding, Baixuan Xu, Zhaowei Wang, Jiaxin Bai, Xin Liu, Cheng Jiayang, Chunkit Chan, Yangqiu Song
| Challenge: | Existing approaches to generalize commonsense reasoning lack instantiated knowledge and require pre-built concept taxonomies and annotations. |
| Approach: | They propose a framework that iteratively performs contextualized conceptualization and instantiation over commonsense knowledge bases by instructing large language models to generate both types of knowledge with critic filtering. |
| Outcome: | Empirical results show that distilling CANDLE on student models provides benefits across three downstream tasks. |
Exploring the Potential of ChatGPT on Sentence Level Relations: A Focus on Temporal, Causal, and Discourse Relations (2024.findings-eacl)
Copied to clipboard
| Challenge: | Recent studies have demonstrated ChatGPT's remarkable few-shot, even zero-shot learning abilities when compared to other models. |
| Approach: | They quantitatively evaluate the performance of ChatGPT on inter-sentential relations such as temporal relations, causal relations, and discourse relations. |
| Outcome: | The proposed model performs well on temporal relations, causal relations, and discourse relations. |
ECON: On the Detection and Resolution of Evidence Conflicts (2024.emnlp-main)
Copied to clipboard
Cheng Jiayang, Chunkit Chan, Qianqian Zhuang, Lin Qiu, Tianhang Zhang, Tengxiao Liu, Yangqiu Song, Yue Zhang, Pengfei Liu, Zheng Zhang
| Challenge: | Recent studies have shown that AI generated content is more likely to dominate search results, making it difficult to detect when compared to human-produced content. |
| Approach: | They propose a method for generating diverse, validated evidence conflicts to simulate real-world misinformation scenarios. |
| Outcome: | The proposed method enables the detection of conflicting information in real-world scenarios and shows that weaker models struggle with similar answer conflicts while stronger models show robust performance. |
DIXITWORLD: Evaluating Multimodal Abductive Reasoning in Vision-Language Models with Multi-Agent Dixit Gameplay (2026.acl-short)
Copied to clipboard
Yunxiang MO, Tianshi Zheng, Qing Zong, Jiayu Liu, Baixuan Xu, Yauwai Yim, Chunkit Chan, Jiaxin Bai, Yangqiu Song
| Challenge: | Existing evaluations of multimodal abductive reasoning are limited to static, single-agent tasks. |
| Approach: | They propose a multiagent evaluation suite that deconstructs the current evaluations of multimodal abductive reasoning in vision–language models. |
| Outcome: | The evaluation suite is based on two core components: DixitArena and DixitsBench. |
DiscoPrompt: Path Prediction Prompt Tuning for Implicit Discourse Relation Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing works on implicit discourse relation recognition focus on syntax features and lack of connectives. |
| Approach: | They propose a prompt-based path prediction method that integrates the interactive information and intrinsic senses among the hierarchy in IDRR. |
| Outcome: | The proposed method shows significant improvement against competitive baselines. |
SessionIntentBench: A Multi-task Inter-session Intention-shift Modeling Benchmark for E-commerce Customer Behavior Understanding (2026.findings-acl)
Copied to clipboard
Yuqi Yang, Weiqi Wang, Baixuan Xu, Wei Fan, Qing Zong, Chunkit Chan, Zheye Deng, Xin Liu, Yifan Gao, Changlong Yu, Chen Luo, Yang Li, Zheng Li, Qingyu Yin, Bing Yin, Yangqiu Song
| Challenge: | Existing models fail to capture and model customer intention effectively because of insufficient information exploitation and only apparent information like descriptions and titles are used. |
| Approach: | They propose to exploit existing session data to capture and model intention in E-commerce product purchase sessions using a multimodal benchmark. |
| Outcome: | The proposed framework can bridge the gap between intention understanding in simplified research cases like co-buy intention and more complex yet practical scenarios like session history. |
PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Models (2024.acl-long)
Copied to clipboard
Haoran Li, Dadi Guo, Donghao Li, Wei Fan, Qi Hu, Xin Liu, Chunkit Chan, Duanyi Yao, Yuan Yao, Yangqiu Song
| Challenge: | generative large language models (LLMs) exhibit surprising capability and integrate previous tasks into a unified text generation formulation. |
| Approach: | They propose a privacy evaluation benchmark to quantify the privacy leakage of language models. |
| Outcome: | The proposed benchmark compares PPLMs with different privacy implementations to find out how privacy leakage is handled. |