Papers by Yukun Huang
Revealing the Attention Floating Mechanism in Masked Diffusion Models (2026.findings-acl)
Copied to clipboard
Xin Dai, Pengcheng Huang, Zhenghao Liu, Shuo Wang, Yukun Yan, Chaojun Xiao, Yu Gu, Ge Yu, Maosong Sun
| Challenge: | Masked diffusion models (MDMs) leverage bidirectional attention and a denoising process. |
| Approach: | They investigate the attention behaviors of Masked diffusion models by revealing the phenomenon of Attention Floating. |
| Outcome: | The proposed model doubles the performance of autoregressive models in knowledge-intensive tasks. |
Atomic Self-Consistency for Better Long Form Generations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has aimed to improve LLM generations by filtering out hallucinations, thereby improving the accuracy of the information in responses. |
| Approach: | They propose a technique that improves the recall of relevant information in an LLM. |
| Outcome: | The proposed technique improves the recall of relevant information in an LLM. |
How to Contextualize Empirical Data for Risk Analysis with LLMs: A Case Study of Power Outages (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly being considered for high-stakes decision-making, yet their application in statistical risk analysis remains largely underexplored. |
| Approach: | They propose a method for extracting key information from raw data and translating it into structured contextual input within the LLM prompt. |
| Outcome: | The proposed approach significantly improves the LLM’s performance in risk assessment tasks. |
Fuzzy Speculative Decoding for a Tunable Accuracy-Runtime Tradeoff (2025.findings-acl)
Copied to clipboard
| Challenge: | Speculative Decoding (SD) enforces strict distributional equivalence to the target model when accepting candidate tokens. |
| Approach: | They propose a decoding algorithm that generalizes SD by accepting candidate tokens based on the divergences between the target and draft model distributions. |
| Outcome: | Using Fuzzy Speculative Decoding (FSD) we show that the proposed method can achieve significant runtime improvements of over 5 tokens per second faster than SD at only an approximate 2% reduction in benchmark accuracy. |
ClueAnchor: Clue-Anchored Knowledge Reasoning Exploration and Optimization for Retrieval-Augmented Generation (2025.findings-emnlp)
Copied to clipboard
Hao Chen, Yukun Yan, Sen Mei, Wanxiang Che, Zhenghao Liu, Qi Shi, Xinze Li, Yuchun Fan, Pengcheng Huang, Qiushi Xiong, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing RAG systems often underutilize the retrieved documents, authors say . they fail to extract and integrate key clues needed to support faithful and interpretable reasoning . |
| Approach: | a new framework extracts key clues from retrieved content and generates multiple reasoning paths . the framework optimizes the model by selecting the most appropriate reasoning path . |
| Outcome: | Experiments show that ClueAnchor outperforms baseline RAG frameworks in completeness and robustness. |
Real-time Factuality Assessment from Adversarial Feedback (2025.acl-long)
Copied to clipboard
| Challenge: | Existing evaluations for assessing the factuality of news from conventional sources, such as claims on fact-checking websites, result in high accuracies over time for LLM-based detectors. |
| Approach: | They propose a pipeline that leverages natural language feedback from a RAG-based detector to iteratively modify real-time news into deceptive variants that challenge LLMs. |
| Outcome: | The proposed pipeline reduces the binary classification ROC-AUC by 17.5 percent for a strong RAG-based GPT-4o detector. |
Chunks as Arms: Multi-Armed Bandit-Guided Sampling for Long-Context LLM Preference Optimization (2026.acl-long)
Copied to clipboard
Shaohua Duan, Pengcheng Huang, Xinze Li, Zhenghao Liu, Xiaoyuan Yi, Yukun Yan, Shuo Wang, Yu Gu, Ge Yu, Maosong Sun
| Challenge: | Recent studies have explored fine-tuning Large Language Models with synthetic data to enhance their long-context capabilities. |
| Approach: | They propose a framework that leverages a Multi-Armed Bandit rollout strategy to identify the most informative chunks from the given long context for sampling high-quality and diverse responses. |
| Outcome: | The proposed framework achieves 4% improvement on long-context reasoning benchmarks on Llama and Qwen. |
Calibrating Long-form Generations From Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Conventional calibration methods treat answer correctness as binary and do not work for long-form generation where an answer can be partially correct. |
| Approach: | They propose a framework where correctness of LLMs' responses and associated confidence levels are treated as distributions across a range of scores. |
| Outcome: | The proposed framework treats the correctness of the LLMs’ responses and their associated confidence levels as distributions across a range of scores. |
Enhancing Large Vision-Language Models with Ultra-Detailed Image Caption Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing pipelines for generating high-quality, ultra-detailed image captions are limited by the scarcity of image caption data. |
| Approach: | They propose a pipeline for generating high-quality, ultra-detailed image captions that integrates both pre-processing and post-processor stages. |
| Outcome: | The proposed pipeline improves LVLMs' perception and cognitive abilities across multiple vision-language benchmarks. |
DeepFact: Co-Evolving Benchmarks and Agents for Deep Research Factuality (2026.acl-long)
Copied to clipboard
Yukun Huang, Leonardo F. R. Ribeiro, Momchil Hardalov, Bhuwan Dhingra, Markus Dreyer, Venkatesh Saligrama
| Challenge: | Existing fact-checkers usually target general-domain atomic claims . citation-grounded fact- checking ignores claims without explicit citations . |
| Approach: | They propose to use a benchmark to test whether claim-level factuality is transferable . they instantiate **Audit-then-Score** as a versioned DRR factualism benchmark . |
| Outcome: | The proposed benchmark outperforms the best prior deep-research and traditional fact-checkers by 14.3 and 24.9 points. |
ExpandR: Teaching Dense Retrievers Beyond Queries with LLM Guidance (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for enhancing dense retrieval with query augmentation ignore the alignment between generation and ranking objectives. |
| Approach: | They propose a unified LLM-augmented dense retrieval framework that jointly optimizes both the LLM and the retriever. |
| Outcome: | Experimental results show that ExpandR outperforms strong baselines, achieving more than 5% improvement in retrieval performance. |
Empirical Analysis of Decoding Biases in Masked Diffusion Models (2026.acl-long)
Copied to clipboard
Pengcheng Huang, Tianming Liu, Zhenghao Liu, Yukun Yan, Shuo Wang, Tong Xiao, Zulong Chen, Maosong Sun
| Challenge: | Existing MDMs employ uncertainty-based decoding strategies that limit their reasoning ability and ultimately degrade generation quality. |
| Approach: | They propose a framework that regularizes uncertainty-based decoding by incorporating two complementary priors to shape global decoding trajectories and promote content informativeness. |
| Outcome: | The proposed framework outperforms existing decoding strategies by more than 7% while achieving comparable performance to autoregressive models of similar parameter scales. |
AgentMark: Utility-Preserving Behavioral Watermarking for Agents (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have improved text generation and reasoning. |
| Approach: | They propose a behavioral watermarking framework that embeds multi-bit identifiers into planning decisions while preserving utility. |
| Outcome: | The proposed framework embeds multi-bit provenance into planning decisions while preserving utility. |