Papers by Kate Knill
Investigating the Emergent Audio Classification Ability of ASR Foundation Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Text and vision foundation models can perform many tasks in a zero-shot setting . however, there has been little work on the zero-shoot abilities of ASR foundation models . |
| Approach: | They investigate the ability of ASR foundation models to perform zero-shot audio classification using text prompts and a decoding probability generator. |
| Outcome: | The proposed model outperforms state-of-the-art models on audio classification datasets without training them on extra data or adding any parameters. |
Cross-Lingual Transfer Learning for Speech Translation (2025.naacl-short)
Copied to clipboard
| Challenge: | Increasing interest in building multilingual foundation models for NLP and speech research has led to limited data collection for training ST systems. |
| Approach: | They propose to use Whisper to explore the behavior of multilingual speech foundation models with restricted data. |
| Outcome: | The proposed model can translate to Chinese with a single language, and it can perform transcriptions in other languages. |
Muting Whisper: A Universal Acoustic Adversarial Attack on Speech Foundation Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | 'special' tokens in large speech foundation models such as Whisper are used to guide their language generation process, but can be exploited by adversarial attacks to manipulate the model's behavior. |
| Approach: | They propose a method to learn a universal acoustic realization of Whisper's |endoftext|> token, which encourages the model to ignore the speech and only transcribe the special token, effectively muting the model. |
| Outcome: | The proposed method can mute Whisper models for over 97% of speech samples and can be used to bypass speech moderation systems and protect private speech data. |
Universal Acoustic Adversarial Attacks for Flexible Control of Speech-LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | acoustic adversarial attacks on speech LLMs may make them more vulnerable to adversarials . flexible speech encoders and large language models have enabled speech Llms to handle a wide range of processing tasks. |
| Approach: | They investigate universal adversarial attacks on speech LLMs by pre-trained speech encoders and large language models. |
| Outcome: | The proposed model can handle a wide range of spoken language processing tasks. |
Grammatical error detection in transcriptions of spoken English (2020.coling-main)
Copied to clipboard
| Challenge: | CrowdED corpus of spoken English monologues on business topics was crowdsourced from native speakers of English and learners of English with German as their first language. |
| Approach: | They propose to use the corpus recordings to correct existing speech transcriptions and edit them to make them more fluent. |
| Outcome: | The proposed transcription corrections and annotations can be used for automatic transcription post-editing and grammatical error correction for spoken English. |