Papers by Rong Ye
NeurST: Neural Speech Translation Toolkit (2021.acl-demo)
Copied to clipboard
| Challenge: | a toolkit for speech translation is available for free and provides step-by-step recipes for feature extraction, data preprocessing, distributed training, and evaluation. |
| Approach: | They propose to use NeurST to facilitate speech translation research for NLP researchers . they show experimental results for different benchmark datasets which can be regarded as reliable baselines . |
| Outcome: | The proposed framework provides reliable benchmarks for speech translation research. |
Argue with Me Tersely: Towards Sentence-Level Counter-Argument Generation (2023.emnlp-main)
Copied to clipboard
Jiayu Lin, Rong Ye, Meng Han, Qi Zhang, Ruofei Lai, Xinyu Zhang, Zhao Cao, Xuanjing Huang, Zhongyu Wei
| Challenge: | Existing work describes paragraph-level counter-argument generation task as paragraph-based . however, sentence-level generation can be quite different due to its unique constraints and brevity-focused challenges. |
| Approach: | They propose a benchmark framework for sentence-level counter-argument generation . they use an annotated debate forum dataset to generate high-quality counter-argments . |
| Outcome: | The proposed framework and evaluator are competitive in counter-argument generation tasks. |
AgentSense: Benchmarking Social Intelligence of Language Agents through Interactive Scenarios (2025.naacl-long)
Copied to clipboard
Xinyi Mou, Jingcong Liang, Jiayu Lin, Xinnong Zhang, Xiawei Liu, Shiyue Yang, Rong Ye, Lei Chen, Haoyu Kuang, Xuanjing Huang, Zhongyu Wei
| Challenge: | Large language models are increasingly employed to empower autonomous agents to simulate human behavior. |
| Approach: | They propose to evaluate LLM-driven agents through multi-turn interactions using a bottom-up approach to create diverse social scenarios constructed from extensive scripts. |
| Outcome: | The proposed model evaluates LLM-driven agents through multi-turn interactions emphasizing goal completion and implicit reasoning. |
Cross-modal Contrastive Learning for Speech Translation (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches for speech translation focus on using additional data from MT and automatic speech recognition (ASR). |
| Approach: | They propose a cross-modal contrastive learning method for end-to-end speech-totext translation. |
| Outcome: | The proposed method outperforms existing methods on a popular benchmark MuST-C. |
Hi-ArG: Exploring the Integration of Hierarchical Argumentation Graphs in Language Pretraining (2023.emnlp-main)
Copied to clipboard
Jingcong Liang, Rong Ye, Meng Han, Qi Zhang, Ruofei Lai, Xinyu Zhang, Zhao Cao, Xuanjing Huang, Zhongyu Wei
| Challenge: | Recent studies have discussed its capability to assist language models for various applications. |
| Approach: | They propose a structure to organize arguments using the **Hi**erarchical **Ar**gumentation **G**raph (Hi-ArG) and propose two approaches to exploit Hi-AarG, including a text-graph multi-modal model GreaseArR and a framework augmented with graph information. |
| Outcome: | The proposed structure supersedes existing language models on two argumentation tasks while incorporating graph information during further training improves vanilla language models. |
DUB: Discrete Unit Back-translation for Speech Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Discrete unit back-translation (DUB) is a back-translated speech-to-text translation (ST) technique that can be applied to ST . a modality gap between speech and text makes it difficult to transfer these techniques to ST due to the modality of the speech-text model. |
| Approach: | They propose a method to represent speech with discrete units instead of continuous features in direct ST. |
| Outcome: | The proposed method achieves comparable performance to existing methods that rely on large-scale external data. |
Debatrix: Multi-dimensional Debate Judge with Iterative Chronological Analysis Based on LLM (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have focused on short dialogues, but mainly on short debates. |
| Approach: | They propose to use Large Language Models to construct an automated debate judge to evaluate multi-turn debates. |
| Outcome: | The proposed system improves on the PanelBench benchmark, which compares its performance to actual debate outcomes. |
Enhancing Dual-Encoders with Question and Answer Cross-Embeddings for Answer Retrieval (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to solve question answering (QA) problems are limited by the need for text generation and answer retrieval. |
| Approach: | They propose to introduce QA interaction features in scoring function but at the cost of low efficiency in inference stage. |
| Outcome: | The proposed framework significantly outperforms the state-of-the-art method on multiple answer retrieval datasets. |
STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods to learn speech representations for end-to-end speech-totext translation (ST) neglect the representation discrepancy across modalities. |
| Approach: | They propose a method to calibrate the representation discrepancy between modalities by mixing up the representation sequences of different modality inputs. |
| Outcome: | The proposed method alleviates the cross-modal representation discrepancy and improves on a strong baseline on eight translation directions. |
WACO: Word-Aligned Contrastive Learning for Speech Translation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing ST methods perform poorly when only a limited amount of parallel data are available for training. |
| Approach: | They propose a Word-Aligned COntrastive learning method for low-resource speech-to-text translation that bridges word-level representations for both speech and text modalities via contrastive learning. |
| Outcome: | The proposed method outperforms the best baseline by 9+ BLEU points with only 1-hour parallel ST data. |
Improving Speech Translation by Fusing Speech and Text (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In speech translation, multimodal data to address limitations of individual modalities has shown significant effectiveness. |
| Approach: | They propose a cross-modal model which supports three input modalities for speech, text and fused speech-text. |
| Outcome: | The proposed model achieves an average of 34.0 BLEU on MuST-C, GigaST and newstest benchmark. |