Papers by Rao Ma
Unsupervised Dual Paraphrasing for Two-stage Semantic Parsing (2020.acl-main)
Copied to clipboard
| Challenge: | Existing semantic parsing frameworks rely on nontrivial human labor to generate canonical utterances. |
| Approach: | They propose a framework that uses an unsupervised paraphrase model to parse canonical utterances. |
| Outcome: | The proposed framework is effective and compatible with supervised training. |
IntrAgent: An LLM Agent for Content-Grounded Information Retrieval through Literature Review (2026.acl-long)
Copied to clipboard
| Challenge: | Scientific research relies on accurate information retrieval from literature to support analytical decisions. |
| Approach: | They propose a task that automates fine-grained information retrieval *faithfully* grounded in the provided content in response to research-driven queries. |
| Outcome: | The proposed agent achieves 13.2% higher cross-domain accuracy than state-of-the-art RAG and research-agent baselines across seven backbone LLMs. |
Investigating the Emergent Audio Classification Ability of ASR Foundation Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Text and vision foundation models can perform many tasks in a zero-shot setting . however, there has been little work on the zero-shoot abilities of ASR foundation models . |
| Approach: | They investigate the ability of ASR foundation models to perform zero-shot audio classification using text prompts and a decoding probability generator. |
| Outcome: | The proposed model outperforms state-of-the-art models on audio classification datasets without training them on extra data or adding any parameters. |
LegoSLM: Connecting LLM with Speech Encoder using CTC Posteriors (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained speech encoders and large language models can perform suboptimal performance on a range of spoken language processing tasks. |
| Approach: | They propose to combine large-scale pre-trained speech encoders and large-language models for better performance on automatic speech recognition tasks. |
| Outcome: | The proposed model can get an average of 49% WER reduction over the baseline model on 8 MLS testsets. |
Cross-Lingual Transfer Learning for Speech Translation (2025.naacl-short)
Copied to clipboard
| Challenge: | Increasing interest in building multilingual foundation models for NLP and speech research has led to limited data collection for training ST systems. |
| Approach: | They propose to use Whisper to explore the behavior of multilingual speech foundation models with restricted data. |
| Outcome: | The proposed model can translate to Chinese with a single language, and it can perform transcriptions in other languages. |
3AM: An Ambiguity-Aware Multi-Modal Machine Translation Dataset (2024.lrec-main)
Copied to clipboard
Xinyu Ma, Xuebo Liu, Derek F. Wong, Jun Rao, Bei Li, Liang Ding, Lidia S. Chao, Dacheng Tao, Min Zhang
| Challenge: | Existing studies have shown that visual information in existing MMT datasets is insufficient, causing models to disregard it and overestimate their capabilities. |
| Approach: | They propose to use 3AM to create an ambiguity-aware multimodal machine translation dataset. |
| Outcome: | The proposed dataset includes more ambiguity and a greater variety of captions and images than other MMT datasets. |
Muting Whisper: A Universal Acoustic Adversarial Attack on Speech Foundation Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | 'special' tokens in large speech foundation models such as Whisper are used to guide their language generation process, but can be exploited by adversarial attacks to manipulate the model's behavior. |
| Approach: | They propose a method to learn a universal acoustic realization of Whisper's |endoftext|> token, which encourages the model to ignore the speech and only transcribe the special token, effectively muting the model. |
| Outcome: | The proposed method can mute Whisper models for over 97% of speech samples and can be used to bypass speech moderation systems and protect private speech data. |
ScratchEval: Are GPT-4o Smarter than My Child? Evaluating Large Multimodal Models with Visual Programming Challenges (2025.naacl-short)
Copied to clipboard
| Challenge: | Recent advances in large multimodal models (LMMs) have demonstrated impressive code generation capabilities, primarily evaluated through image-to-code benchmarks. |
| Approach: | They propose a visual programming reasoning benchmark based on Scratch, a block-based visual programming language widely used in children’s programming education. |
| Outcome: | The proposed framework evaluates the visual programming ability of large multimodal models by integrating visual elements and embedded programming logic. |
Universal Acoustic Adversarial Attacks for Flexible Control of Speech-LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | acoustic adversarial attacks on speech LLMs may make them more vulnerable to adversarials . flexible speech encoders and large language models have enabled speech Llms to handle a wide range of processing tasks. |
| Approach: | They investigate universal adversarial attacks on speech LLMs by pre-trained speech encoders and large language models. |
| Outcome: | The proposed model can handle a wide range of spoken language processing tasks. |
Free your mouse! Command Large Language Models to Generate Code to Format Word Documents (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent LLMs have significantly improved code generation, making it increasingly accessible to users. |
| Approach: | They propose an automatic document formatting method, Text-to-Format, driven by various prompting strategies and a high-quality dataset DocFormEval data. |
| Outcome: | The proposed method improves the efficiency and experience of users in formatting the document and improves document formatting task. |