Papers by Miaoran Li
Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning (2025.findings-acl)
Copied to clipboard
Xinghao Chen, Zhijing Sun, Guo Wenjin, Miaoran Zhang, Yanjun Chen, Yirong Sun, Hui Su, Yijie Pan, Dietrich Klakow, Wenjie Li, Xiaoyu Shen
| Challenge: | Large Language Models (LLMs) excel in reasoning tasks through Chain-of-Thought prompting. |
| Approach: | They examine the factors influencing CoT distillation including granularity, format and teacher model. |
| Outcome: | The proposed model is based on four teacher models and seven student models across seven mathematical and commonsense reasoning datasets. |
Benchmarking LLM Faithfulness in RAG with Evolving Leaderboards (2025.emnlp-industry)
Copied to clipboard
Manveer Singh Tamber, Forrest Sheng Bao, Chenyu Xu, Ge Luo, Suleman Kazi, Minseok Bae, Miaoran Li, Ofer Mendelevitch, Renyi Qu, Jimmy Lin
| Challenge: | Large language models (LLMs) excel in various tasks, but often produce hallucinations . retrieved contexts, misrepresent information, or generate outright contradictions . |
| Approach: | They propose a framework that measures hallucination faithfulness of large language models . they introduce a leaderboard that leverages diverse human-annotated hallucinian examples . |
| Outcome: | The proposed framework improves hallucination evaluations by leveraging human-annotated examples. |
Hallucination Detection in Structured Query Generation via LLM Self-Debating (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Hallucination remains a key challenge in applying large language models to structured query generation . we propose the Self-Debating framework to enhance detection performance . |
| Approach: | They propose a framework that prompts an LLM to generate contrastive explanations from opposing perspectives . they also propose 'self-debating' framework to enhance detection performance . |
| Outcome: | The proposed framework outperforms LLM-as-a-Judge baselines in hallucination detection . the framework generates contrastive explanations from opposing perspectives . |
SummaCoz: A Dataset for Improving the Interpretability of Factual Consistency Detection for Summarization (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Summarization is an important application of Large Language Models. |
| Approach: | They integrate human-annotated and model-generated natural language explanations to elucidate how a summary deviates and becomes inconsistent with its source article. |
| Outcome: | The proposed model provides rationales for its judgments and improves its accuracy significantly. |
Self-Checker: Plug-and-Play Modules for Fact-Checking with Large Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for fact-checking text generated by large language models are expensive and time-consuming. |
| Approach: | They propose a plug-and-play framework that harnesses large language models for efficient fact-checking in a few-shot manner. |
| Outcome: | The proposed framework is compared with state-of-the-art models and shows that it can be used to speed up fact-checking in a few-shot manner. |
On the Intractability to Synthesize Factual Inconsistencies in Summarization (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for detecting factual inconsistencies in abstractive summarization are lacking in factual consistency detection. |
| Approach: | They propose to use real model-generated summaries with human annotations to detect factual inconsistencies. |
| Outcome: | The proposed model outperforms the SOTA on CoGenSumm, FactCC, Frank, and SummEval datasets. |
FaithBench: A Diverse Hallucination Benchmark for Summarization by Modern LLMs (2025.naacl-short)
Copied to clipboard
Forrest Sheng Bao, Miaoran Li, Renyi Qu, Ge Luo, Erana Wan, Yujia Tang, Weisi Fan, Manveer Singh Tamber, Suleman Kazi, Vivek Sourabh, Mike Qi, Ruixuan Tu, Chenyu Xu, Matthew Gonzales, Ofer Mendelevitch, Amin Ahmad
| Challenge: | Existing evaluations of hallucinations in large language models suffer from a lack of diversity and recency in the LLM and LLM families considered. |
| Approach: | They propose a summarization hallucination benchmark that challenges models to disagree on hallucines . they use models to generate answers or summaries from textual input . |
| Outcome: | The proposed model combines the best of 10 modern LLMs with ground truth annotations. |