Papers by Renjie Huang
Progra: Progress-Aware Reinforcement Learning for Multi-Turn Function Calling (2026.findings-acl)
Copied to clipboard
Huacan Chai, Zijie Cao, Maolin Ran, Yingxuan Yang, Jianghao Lin, Xin Peng, Hairui Wang, Renjie Ding, Ziyu Wan, Muning Wen, Weiwen Liu, Weinan Zhang, Fei Huang, Ying Wen
| Challenge: | Existing methods for multi-turn function calling are limited by redundancy and lack explicit integration of progress awareness into training. |
| Approach: | They propose a framework that explicitly integrates progress awareness into LLM training for multi-turn function calling. |
| Outcome: | Empirical results show that Progra outperforms existing methods on two public benchmarks. |
PaddleSpeech: An Easy-to-Use All-in-One Speech Toolkit (2022.naacl-demo)
Copied to clipboard
Hui Zhang, Tian Yuan, Junkun Chen, Xintong Li, Renjie Zheng, Yuxin Huang, Xiaojie Chen, Enlei Gong, Zeyu Chen, Xiaoguang Hu, Dianhai Yu, Yanjun Ma, Liang Huang
| Challenge: | PaddleSpeech is an open-source speech toolkit that supports speech-to-text and text-to speech tasks. |
| Approach: | They describe the design philosophy and core architecture of PaddleSpeech to support several essential speech-to-text and text-to speech tasks. |
| Outcome: | The proposed framework achieves competitive or state-of-the-art performance on various speech datasets and implements the most popular methods. |
Simpler and Faster Learning of Adaptive Policies for Simultaneous Translation (D19-1)
Copied to clipboard
| Challenge: | Recent work on simultaneous translation is difficult because of its latency and quality. |
| Approach: | They propose a supervised-learning framework to learn adaptive policies from parallel text sequences . they use a model that predicts when a target word is read or WRITE if context provides enough information . |
| Outcome: | Experiments on German=>English show that the proposed method can learn flexible policies with better BLEU scores and similar latencies compared to previous work. |
Direct Simultaneous Speech-to-Text Translation Assisted by Synchronized Streaming ASR (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to simultaneous speech-to-text translation suffer from error propagation and extra latency. |
| Approach: | They propose a new paradigm for simultaneous speech-to-text translation using two separate decoders . they use multitask learning to jointly learn these two tasks with a shared encoder . |
| Outcome: | The proposed method achieves substantially better translation quality at similar levels of latency. |
Through the Valley: Path to Effective Long CoT Training for Small Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Long chain-of-thought (CoT) supervision is effective for large language models . but small models trained on limited long CoT data experience performance degradation . |
| Approach: | They identify a phenomenon called Long CoT Degradation in small language models . long CoT data can be used to generate long chain-of-thought (CoT) responses . |
| Outcome: | The results show that models trained on 8k long CoT examples lose up to 75% of their original performance before fine-tuning. |
Simultaneous Translation with Flexible Policy via Restricted Imitation Learning (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to simultaneous translation have been limited and use fixed-latency policies or a complicated two-staged model. |
| Approach: | They propose a single model that adds a “delay” token to the target vocabulary and a restricted dynamic oracle to greatly simplify training. |
| Outcome: | The proposed model achieves better BLEU scores and lower latencies compared to fixed and RL-learned policies on Chinese -> English simultaneous translation. |
STACL: Simultaneous Translation with Implicit Anticipation and Controllable Latency using Prefix-to-Prefix Framework (P19-1)
Copied to clipboard
Mingbo Ma, Liang Huang, Hao Xiong, Renjie Zheng, Kaibo Liu, Baigong Zheng, Chuanqiang Zhang, Zhongjun He, Hairong Liu, Xing Li, Hua Wu, Haifeng Wang
| Challenge: | Simultaneous translation is notoriously dif- ficult due to word-order differences. |
| Approach: | They propose a prefix-to-prefix framework that implicitly learns to anticipate in a single translation model. |
| Outcome: | The proposed framework achieves low latency and reasonable qual- ity on 4 directions. |
Simultaneous Translation Policies: From Fixed to Adaptive (2020.acl-main)
Copied to clipboard
| Challenge: | Adaptive policies can balance translation quality and latency based on context information . previous methods on obtaining adaptive policies rely on complicated training process . |
| Approach: | They propose to obtain adaptive policies by a simple heuristic composition of fixed policies . they propose to use a heurism to obtain policies that can outperform fixed ones . |
| Outcome: | Experiments on Chinese -> English and German -> english show that adaptive policies outperform fixed policies by up to 4 BLEU points for the same latency. |
ToolPRM: Fine-Grained Inference Scaling of Structured Outputs for Function Calling (2026.acl-long)
Copied to clipboard
Jianghao Lin, Yuanyuan Shi, Xin Peng, Renjie Ding, Hairui Wang, Yuxuan Peng, Bizhe Bai, Weixi Song, Fengshuo Bai, Huacan Chai, Weinan Zhang, Fei Huang, Ying Wen
| Challenge: | Existing research on inference scaling focuses on unstructured output generation tasks, such as mathematical problems. |
| Approach: | They propose an inference-scaling framework that combines fine-grained beam search with ToolPRM, a process reward model scoring each intra-call decision. |
| Outcome: | The proposed framework outperforms outcome and coarse-grained reward models in predictive accuracy and yields consistent test-time gains on multiple function-calling benchmarks. |
Opportunistic Decoding with Timely Correction for Simultaneous Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to balancing translation quality and latency are either too aggressive or too conservative. |
| Approach: | They propose an opportunistic decoding technique that always (over-)generates a certain mount of extra words at each step to keep the audience on track with the latest information. |
| Outcome: | The proposed technique reduces latency and increases BLEU with no over-generating . it also corrects mistakes in the overgenerated words when observing more context . |
Fluent and Low-latency Simultaneous Speech-to-Speech Translation with Self-adaptive Training (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Current approaches to simultaneous speech-to-speech translation accumulate more and more latencies in later sentences when the speaker talks faster. |
| Approach: | They propose a method which generates more fluent target speech latency than the baseline . they propose to use self-adaptive translation to adjust the length of translations to accommodate different source speech rates. |
| Outcome: | Xiong et al., 2019) show that the proposed method generates more fluent target speech latency than baseline . authors say it provides more natural communication process than speech-to-text translation . xiong and colleagues say the proposed technique is more efficient than current approaches . |
Multi-Reference Training with Pseudo-References for Neural Translation and Text Generation (D18-1)
Copied to clipboard
| Challenge: | Neural text generation has been quite successful recently, but during training time, only one reference is considered for each example, even though there are often multiple references available. |
| Approach: | They propose an algorithm to generate exponentially many pseudo-references by compressing existing references into lattices and traversing them to generate new pseudo-References. |
| Outcome: | The proposed model significantly improves on baselines in machine translation and image captioning, and is comparable to existing models. |
Knowledge Extraction on Semi-Structured Content: Does It Remain Relevant for Question Answering in the Era of LLMs? (2026.eacl-long)
Copied to clipboard
Kai Sun, Yin Huang, Srishti Mehra, Mohammad Kachuee, Xilun Chen, Renjie Tao, Zhaojiang Lin, Andrea Jessee, Nirav Shah, Alex L Betty, Yue Liu, Anuj Kumar, Wen-tau Yih, Xin Luna Dong
| Challenge: | Existing literature on knowledge extraction for question answering questions whether it is still relevant for question answerrs. |
| Approach: | They extend an existing benchmark with knowledge extraction annotations and evaluate commercial and open-source LLMs of varying sizes. |
| Outcome: | The proposed model can achieve high QA accuracy, but can still benefit from knowledge extraction through augmentation with extracted triples and multi-task learning. |
Improving Simultaneous Translation by Incorporating Pseudo-References with Fewer Reorderings (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing systems for simultaneous translation are still trained on full-sentence bitexts due to the abundance of unnecessary long-distance reorderings. |
| Approach: | They propose to rewrite target side of existing full-sentence corpora into simultaneous-style translation by adding generated pseudo-references to the target side. |
| Outcome: | Experiments on ZhEn and JaEn simultaneous translation show that the proposed method improves on existing full-sentence corpora. |
The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Sparse attention is a promising strategy to extend long-context capabilities in LLMs . but its efficiency–accuracy trade-offs remain unclear due to the lack of comprehensive evaluation . |
| Approach: | They evaluate sparse attention methods across multiple model families and sizes . they find larger sparser models outperform smaller dense ones at equivalent cost . |
| Outcome: | The proposed methods outperform smaller sparse models at equivalent cost and improve the Pareto frontier. |
OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems (2024.acl-long)
Copied to clipboard
Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, Maosong Sun
| Challenge: | Large Language Models (LLMs) and Large Multimodal Models have exceeded general human capabilities in various tasks. |
| Approach: | They present an Olympiad-level bilingual multimodal scientific benchmark featuring 8,476 problems from Olympiad level mathematics and physics competitions. |
| Outcome: | The best performing model, GPT-4V, attains an average score of 17.97% on OlympiadBench, with a mere 10.74% in physics, highlighting the benchmark rigor and the intricacy of physical reasoning. |
Speculative Beam Search for Simultaneous Translation (D19-1)
Copied to clipboard
| Challenge: | Beam search is widely used in (full-sentence) machine translation but its application to simultaneous translation remains highly non-trivial. |
| Approach: | They propose a beam search algorithm that hallucinates several steps into the future to reach a more accurate decision by implicitly benefiting from a target language model. |
| Outcome: | The proposed method improves on language models over diverse language pairs and shows significant improvements over greedy search. |
Learning What Matters: Dynamic Dimension Selection and Aggregation for Interpretable Vision-Language Reward Modeling (2026.acl-long)
Copied to clipboard
Qiyuan Chen, Hongsen Huang, Jiahe Chen, Qian Shao, Jintai Chen, Hongxia Xu, Renjie Hua, Ren Chuan, Jian Wu
| Challenge: | Existing multimodal reward models are interpretable but slow, while discriminative ones are opaque "black boxes." |
| Approach: | They propose a framework that dynamically decomposes evaluation into granular, interpretable dimensions. |
| Outcome: | The proposed framework outperforms open-source reward models on benchmarks like VL-RewardBench. |
Icon2: Aligning Large Language Models Using Self-Synthetic Preference Data via Inherent Regulation (2025.emnlp-main)
Copied to clipboard
Qiyuan Chen, Hongsen Huang, Qian Shao, Jiahe Chen, Jintai Chen, Hongxia Xu, Renjie Hua, Ren Chuan, Jian Wu
| Challenge: | Large Language Models (LLMs) require high quality preference datasets to align with human preferences. |
| Approach: | They propose a framework that leverages inherent regulation of LLMs’ representation space for efficient and tailored preference dataset construction, named Icon2. |
| Outcome: | The proposed framework improves performance on benchmarks like AlpacaEval 2.0 and Arena-Hard while reducing computational costs by up to 48.1%. |
MHALO: Evaluating MLLMs as Fine-grained Hallucination Detectors (2025.findings-acl)
Copied to clipboard
| Challenge: | Hallucination remains a critical challenge for multimodal large language models, undermining their reliability in real-world applications. |
| Approach: | They propose a benchmark specifically designed for evaluating MLLMs’ capability in performing token-level hallucination detection (FHD) . they use curated training data to train a specialized model that significantly outperforms existing models. |
| Outcome: | The proposed model outperforms existing models in the evaluation of 9 MLLMs and reaches an average F1IoU of 40.59%. |
Learning to Stop in Structured Prediction for Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | Beam search optimization solves many problems in neural machine translation, but lacks principled stopping criteria and does not learn how to stop during training. |
| Approach: | They propose a ranking method which enables an optimal beam search stop-ping criteria and a structured prediction loss function which penalizes suboptimal finished candidates produced by beam search during training. |
| Outcome: | Experiments on synthetic and real languages show that the proposed methods improve translation quality and length. |
Incremental Text-to-Speech Synthesis with Prefix-to-Prefix Framework (2020.findings-emnlp)
Copied to clipboard
Mingbo Ma, Baigong Zheng, Kaibo Liu, Renjie Zheng, Hairong Liu, Kainan Peng, Kenneth Church, Liang Huang
| Challenge: | Text-to-speech synthesis (TTS) has seen rapid progress in recent years, but still suffers from latencies. |
| Approach: | They propose a neural incremental TTS approach that synthesizes speech in an online fashion, playing a segment of audio while generating the next. |
| Outcome: | Experiments on English and Chinese TTS show that the proposed approach achieves similar speech naturalness compared to full sentence TTS, but with a constant (1-2 words) latency. |
Proofread: Fixes All Errors with One Tap (2024.acl-demos)
Copied to clipboard
Renjie Liu, Yanxiang Zhang, Yun Zhu, Haicheng Sun, Yuanbo Zhang, Michael Huang, Shanqing Cai, Lei Meng, Shumin Zhai
| Challenge: | Extensive experiments on a human-labeled golden set showed our tuned PaLM2-XS model achieved 85.56% good ratio. |
| Approach: | They propose a two-stage tuning approach to acquire the dedicated Large Language Model for the feature, followed by a reinforcement learning approach for targeted refinement. |
| Outcome: | The proposed model achieves 85.56% good quality on Rewrite and proofread tasks on human-labeled golden sets. |