Papers by Yun Hong
Better Process Supervision with Bi-directional Rewarding Signals (2025.findings-acl)
Copied to clipboard
Wenxiang Chen, Wei He, Zhiheng Xi, Honglin Guo, Boyang Hong, Jiazheng Zhang, Nijun Li, Tao Gui, Yun Li, Qi Zhang, Xuanjing Huang
| Challenge: | Existing processes that reward for each step are one-directional and lack a mechanism to model the distance to the final target. |
| Approach: | They propose a process supervision model that evaluates the correctness of previous steps and the probability of future success. |
| Outcome: | The proposed model outperforms existing supervision models like ORM and PRM on reasoning tasks and improves solution re-design. |
Who Wrote this Code? Watermarking for Code Generation (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods to detect machine-generated text by embedding watermarks fail to function appropriately in code generation tasks due to the task’s nature of having low entropy. |
| Approach: | They propose a logit-modifying watermark method which enhances detection ability and mitigates code quality degeneration by removing low-entropy segments at generating and detecting watermarks. |
| Outcome: | The proposed method outperforms baseline methods in detecting machine-generated code text while preserving code quality. |
Efficient Training for Cross-lingual Speech Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Currently, large language models (LLMs) focus on the text modality, making speech modeling difficult. |
| Approach: | They propose a cross-lingual speech language model that trains on discrete speech tokens to achieve cross-modal and cross-linguistic alignment through continual pre-training. |
| Outcome: | The proposed method achieves cross-modal and cross-lingual alignment through continual pre-training. |
M³GQA: A Multi-Entity Multi-Hop Multi-Setting Graph Question Answering Benchmark (2025.acl-long)
Copied to clipboard
| Challenge: | GraphRAG systems have achieved remarkable progress in enhancing performance and reliability of large language models. |
| Approach: | They propose a GraphRAG benchmark focusing on multi-entity queries with six settings for comprehensive evaluation. |
| Outcome: | The proposed method can construct diverse data with semantically correct ground-truth reasoning paths. |
MANTA: A Scalable Pipeline for Transmuting Massive Web Corpora into Instruction Datasets (2025.findings-emnlp)
Copied to clipboard
| Challenge: | MANTA-1M generates high-quality large-scale instruction fine-tuning datasets from web corpora . scalability and diversity of the datasets are preserved, allowing expansion into domains requiring intensive knowledge. |
| Approach: | a team of researchers introduce a pipeline that fine-tunes large-scale instruction datasets from web corpora with minimal human intervention. |
| Outcome: | MANTA generates high-quality large-scale instruction fine-tuning datasets from web corpora . leveraging high-performance LLMs, MANTE outperforms other methods in knowledge-intensive tasks . |
FACT: Examining the Effectiveness of Iterative Context Rewriting for Multi-fact Retrieval (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are proficient at retrieving single facts from extended contexts, but struggle with tasks requiring simultaneous retrieval of multiple facts. |
| Approach: | They propose a method that refines context through successive rounds of rewriting to address this problem by finding all Crucial Texts (FACT) |
| Outcome: | The proposed method improves multi-fact retrieval performance across tasks, though improvements are less notable in general-purpose QA scenarios. |
Revisiting Intermediate Layer Distillation for Compressing Language Models: An Overfitting Perspective (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for knowledge distillation (KD) are prone to overfitting to training datasets . recent advances in NLP have shown that using PLMs such as BERT and RoBERTa on downstream tasks is effective. |
| Approach: | They propose a consistency-regularized knowledge distillation method which mitigates overfitting of existing methods. |
| Outcome: | The proposed method outperforms existing methods on the GLUE benchmark and synthetic datasets. |
FreezeEmpath: Efficient Training for Empathetic Spoken Chatbots with Frozen LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent research has made significant progress in developing empathetic spoken chatbots based on large language models (LLMs). |
| Approach: | They propose an end-to-end empathetic spoken chatbot trained efficiently that generates emotionally expressive speech and outperforms other emmpathetic models in emphatic dialogue, SER, and SpokenQA tasks. |
| Outcome: | The proposed model outperforms other empathetic models on e-dialog, SER, and SpokenQA tasks and achieves strong results on several speech tasks. |
Stable Language Model Pre-training by Reducing Embedding Variability (2024.emnlp-main)
Copied to clipboard
| Challenge: | Stable pre-training is essential for achieving better-performing language models, but tracking pre-train stability is impractical due to high computational costs. |
| Approach: | They propose to use Token Embedding Variability as a proxy to estimate pre-training stability. |
| Outcome: | The proposed method improves stability and lowers perplexities even at deeper layer counts. |