Papers by Renjie Zheng
PaddleSpeech: An Easy-to-Use All-in-One Speech Toolkit (2022.naacl-demo)
Copied to clipboard
Hui Zhang, Tian Yuan, Junkun Chen, Xintong Li, Renjie Zheng, Yuxin Huang, Xiaojie Chen, Enlei Gong, Zeyu Chen, Xiaoguang Hu, Dianhai Yu, Yanjun Ma, Liang Huang
| Challenge: | PaddleSpeech is an open-source speech toolkit that supports speech-to-text and text-to speech tasks. |
| Approach: | They describe the design philosophy and core architecture of PaddleSpeech to support several essential speech-to-text and text-to speech tasks. |
| Outcome: | The proposed framework achieves competitive or state-of-the-art performance on various speech datasets and implements the most popular methods. |
Simpler and Faster Learning of Adaptive Policies for Simultaneous Translation (D19-1)
Copied to clipboard
| Challenge: | Recent work on simultaneous translation is difficult because of its latency and quality. |
| Approach: | They propose a supervised-learning framework to learn adaptive policies from parallel text sequences . they use a model that predicts when a target word is read or WRITE if context provides enough information . |
| Outcome: | Experiments on German=>English show that the proposed method can learn flexible policies with better BLEU scores and similar latencies compared to previous work. |
Flaming-hot Initiation with Regular Execution Sampling for Large Language Models (2025.findings-naacl)
Copied to clipboard
Weizhe Chen, Zhicheng Zhang, Guanlin Liu, Renjie Zheng, Wenlei Shi, Chen Dun, Zheng Wu, Xing Jin, Lin Yan
| Challenge: | Large language models (LLMs) have demonstrated remarkable capabilities across various domains since the release of ChatGPT . a key challenge in developing these general capabilities is efficiently sourcing diverse, high-quality data. |
| Approach: | They introduce Flaming-hot Initiation with Regular Execution (FIRE) sampling to efficiently find good responses by promoting diversity. |
| Outcome: | The proposed method enhances inference-time generation quality and benefits training in the alignment stage. |
Direct Simultaneous Speech-to-Text Translation Assisted by Synchronized Streaming ASR (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to simultaneous speech-to-text translation suffer from error propagation and extra latency. |
| Approach: | They propose a new paradigm for simultaneous speech-to-text translation using two separate decoders . they use multitask learning to jointly learn these two tasks with a shared encoder . |
| Outcome: | The proposed method achieves substantially better translation quality at similar levels of latency. |
Simultaneous Translation with Flexible Policy via Restricted Imitation Learning (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to simultaneous translation have been limited and use fixed-latency policies or a complicated two-staged model. |
| Approach: | They propose a single model that adds a “delay” token to the target vocabulary and a restricted dynamic oracle to greatly simplify training. |
| Outcome: | The proposed model achieves better BLEU scores and lower latencies compared to fixed and RL-learned policies on Chinese -> English simultaneous translation. |
STACL: Simultaneous Translation with Implicit Anticipation and Controllable Latency using Prefix-to-Prefix Framework (P19-1)
Copied to clipboard
Mingbo Ma, Liang Huang, Hao Xiong, Renjie Zheng, Kaibo Liu, Baigong Zheng, Chuanqiang Zhang, Zhongjun He, Hairong Liu, Xing Li, Hua Wu, Haifeng Wang
| Challenge: | Simultaneous translation is notoriously dif- ficult due to word-order differences. |
| Approach: | They propose a prefix-to-prefix framework that implicitly learns to anticipate in a single translation model. |
| Outcome: | The proposed framework achieves low latency and reasonable qual- ity on 4 directions. |
Simultaneous Translation Policies: From Fixed to Adaptive (2020.acl-main)
Copied to clipboard
| Challenge: | Adaptive policies can balance translation quality and latency based on context information . previous methods on obtaining adaptive policies rely on complicated training process . |
| Approach: | They propose to obtain adaptive policies by a simple heuristic composition of fixed policies . they propose to use a heurism to obtain policies that can outperform fixed ones . |
| Outcome: | Experiments on Chinese -> English and German -> english show that adaptive policies outperform fixed policies by up to 4 BLEU points for the same latency. |
Opportunistic Decoding with Timely Correction for Simultaneous Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to balancing translation quality and latency are either too aggressive or too conservative. |
| Approach: | They propose an opportunistic decoding technique that always (over-)generates a certain mount of extra words at each step to keep the audience on track with the latest information. |
| Outcome: | The proposed technique reduces latency and increases BLEU with no over-generating . it also corrects mistakes in the overgenerated words when observing more context . |
Fluent and Low-latency Simultaneous Speech-to-Speech Translation with Self-adaptive Training (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Current approaches to simultaneous speech-to-speech translation accumulate more and more latencies in later sentences when the speaker talks faster. |
| Approach: | They propose a method which generates more fluent target speech latency than the baseline . they propose to use self-adaptive translation to adjust the length of translations to accommodate different source speech rates. |
| Outcome: | Xiong et al., 2019) show that the proposed method generates more fluent target speech latency than baseline . authors say it provides more natural communication process than speech-to-text translation . xiong and colleagues say the proposed technique is more efficient than current approaches . |
Multi-Reference Training with Pseudo-References for Neural Translation and Text Generation (D18-1)
Copied to clipboard
| Challenge: | Neural text generation has been quite successful recently, but during training time, only one reference is considered for each example, even though there are often multiple references available. |
| Approach: | They propose an algorithm to generate exponentially many pseudo-references by compressing existing references into lattices and traversing them to generate new pseudo-References. |
| Outcome: | The proposed model significantly improves on baselines in machine translation and image captioning, and is comparable to existing models. |
Improving Simultaneous Translation by Incorporating Pseudo-References with Fewer Reorderings (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing systems for simultaneous translation are still trained on full-sentence bitexts due to the abundance of unnecessary long-distance reorderings. |
| Approach: | They propose to rewrite target side of existing full-sentence corpora into simultaneous-style translation by adding generated pseudo-references to the target side. |
| Outcome: | Experiments on ZhEn and JaEn simultaneous translation show that the proposed method improves on existing full-sentence corpora. |
Bridge-Coder: Transferring Model Capabilities from High-Resource to Low-Resource Programming Language (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) excel at generating code for high-resource programming languages (HRPLs) however, they struggle significantly with low-resourced programming languages such as D, exacerbating the digital divide. |
| Approach: | They propose a method to generate LRPL data using LLM's general knowledge, HRPL proficiency, and in-context learning capabilities. |
| Outcome: | The proposed method improves on R, D, Racket, and Bash, while maintaining the same quality. |
Speculative Beam Search for Simultaneous Translation (D19-1)
Copied to clipboard
| Challenge: | Beam search is widely used in (full-sentence) machine translation but its application to simultaneous translation remains highly non-trivial. |
| Approach: | They propose a beam search algorithm that hallucinates several steps into the future to reach a more accurate decision by implicitly benefiting from a target language model. |
| Outcome: | The proposed method improves on language models over diverse language pairs and shows significant improvements over greedy search. |
OpenResearcher: Unleashing AI for Accelerated Scientific Research (2024.emnlp-demo)
Copied to clipboard
Yuxiang Zheng, Shichao Sun, Lin Qiu, Dongyu Ru, Cheng Jiayang, Xuefeng Li, Jifan Lin, Binjie Wang, Yun Luo, Renjie Pan, Yang Xu, Qingkai Min, Zizhao Zhang, Yiwen Wang, Wenjie Li, Pengfei Liu
| Challenge: | Global scientific publications are growing annually by about 4%-5% (Pinedo et al., 2024). |
| Approach: | They introduce an AI-assisted platform that answers diverse questions from researchers using Retrieval-Augmented Generation (RAG) they develop various tools to understand queries, search from the scientific literature, filter retrieved information, provide accurate and comprehensive answers, and self-refine answers. |
| Outcome: | OpenResearcher is built on Retrieval-Augmented Generation (RAG) to integrate Large Language Models (LLMs) with up-to-date, domain-specific knowledge. |
Learning to Stop in Structured Prediction for Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | Beam search optimization solves many problems in neural machine translation, but lacks principled stopping criteria and does not learn how to stop during training. |
| Approach: | They propose a ranking method which enables an optimal beam search stop-ping criteria and a structured prediction loss function which penalizes suboptimal finished candidates produced by beam search during training. |
| Outcome: | Experiments on synthetic and real languages show that the proposed methods improve translation quality and length. |
Incremental Text-to-Speech Synthesis with Prefix-to-Prefix Framework (2020.findings-emnlp)
Copied to clipboard
Mingbo Ma, Baigong Zheng, Kaibo Liu, Renjie Zheng, Hairong Liu, Kainan Peng, Kenneth Church, Liang Huang
| Challenge: | Text-to-speech synthesis (TTS) has seen rapid progress in recent years, but still suffers from latencies. |
| Approach: | They propose a neural incremental TTS approach that synthesizes speech in an online fashion, playing a segment of audio while generating the next. |
| Outcome: | Experiments on English and Chinese TTS show that the proposed approach achieves similar speech naturalness compared to full sentence TTS, but with a constant (1-2 words) latency. |