Papers by Andreas Stolcke
Lightweight Safety Guardrails Using Fine-tuned BERT Embeddings (2025.coling-industry)
Copied to clipboard
| Challenge: | Existing approaches to filter out inappropriate user prompts or system outputs have been successful, but using fine-tuned LLMs as guardrails introduces increased latency and higher maintenance costs. |
| Approach: | They propose to fine-tune a lightweight architecture that reduces the model size from LlamaGuard’s 7 billion parameters to approximately 67 million parameters. |
| Outcome: | Sentence-BERT reduces the model size from 7 billion parameters to approximately 67 million while maintaining comparable performance on the AEGIS safety benchmark. |
Spoken Conversational Agents with Large Language Models (2025.emnlp-tutorials)
Copied to clipboard
| Challenge: | This tutorial focuses on the evolution of voice-native LLMs . it reviews the adaptation of text LLM to audio, cross-modal alignment, and joint speech–text training . |
| Approach: | This tutorial examines the evolution of voice-native LLMs in conversational agents . it compares cascaded and voice-based LLM systems to end-to-end retrieval-and vision-grounded systems . |
| Outcome: | This tutorial examines the evolution of voice-native LLMs . it compares the performance of voice assistants to current open-domain agents . |
Session-level Language Modeling for Conversational Speech (D18-1)
Copied to clipboard
| Challenge: | Xiong et al., 2017) generalizes language models for conversational speech recognition . recurrent neural networks (RNNs) read a list of words sequentially and predict the next word at each position. |
| Approach: | They propose to generalize language models for conversational speech recognition to capture conversation-level phenomena such as adjacency pairs, lexical entrainment, and topical coherence. |
| Outcome: | The proposed model reduces perplexity and improves word error rate over standard models in the conversational telephone speech domain. |
Provenance: A Light-weight Fact-checker for Retrieval Augmented LLM Generation Output (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods for fact checking RAG outputs rely on large language models. |
| Approach: | They propose a method that computes a factuality score that can be thresholded to yield a binary decision to check RAG outputs. |
| Outcome: | The proposed method is low latency and low cost at run-time and no need for LLM fine-tuning. |
CUE Vectors: Modular Training of Language Models Conditioned on Diverse Contextual Signals (2022.findings-acl)
Copied to clipboard
| Challenge: | Using contextual universal embeddings, we train neural language models on one type of contextual data and adapts to novel context types. |
| Approach: | They propose a framework to modularize the training of neural language models that use diverse forms of context by eliminating the need to jointly train context and within-sentence encoders. |
| Outcome: | The proposed framework trains LMs on one type of contextual data and adapts to novel context types. |
Attention-based Contextual Language Model Adaptation for Speech Recognition (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing language models do not incorporate utterance level contextual information . however, for some domains like voice assistants, additional context provides a rich input signal . |
| Approach: | They propose a method for training neural speech recognition models on text and contextual data. |
| Outcome: | The proposed model reduces perplexity by 7.0% relative over a standard LM . it also improves perxicity by 2.8% relative to a state-of-the-art model for contextual LM. |
SpeechLLMs for Large-scale Contextualized Zero-shot Slot Filling (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Slot filling is a key subtask in spoken language understanding (SLU) . recent advent of speech-based large language models has opened new avenues for speech understanding . |
| Approach: | They propose to improve slot-filling task by creating an empirical upper bound for the task . they propose to use a speech-based large language model to integrate speech and text modalities . |
| Outcome: | The proposed model improves slot filling performance while reducing generalization gaps. |
Unifying Streaming and Non-streaming Zipformer-based ASR (2025.acl-industry)
Copied to clipboard
Bidisha Sharma, Karthik Pandia D S, Shankar Venkatesan, Jeena J Prakash, Shashi Kumar, Malolan Chetlur, Andreas Stolcke
| Challenge: | Existing frameworks for streaming and non-streaming ASR models have been used to reduce development, training and deployment costs. |
| Approach: | They propose to use dynamic right-context through chunked attention masking to train zipformer-based ASR models. |
| Outcome: | The proposed framework reduces word error by relative 7.9% with a small degradation in user-perceived latency. |