Papers by Julian Salazar
Robust and Unbounded Length Generalization in Autoregressive Transformer-Based Text-to-Speech (2025.naacl-long)
Copied to clipboard
Eric Battenberg, RJ Skerry-Ryan, Daisy Stanton, Soroosh Mariooryad, Matt Shannon, Julian Salazar, David Teh-Hwa Kao
| Challenge: | Autoregressive (AR) Transformer-based sequence models have difficulty generalizing to sequences longer than those seen during training. |
| Approach: | They propose a system that provides cross-attention operations with relative location information. |
| Outcome: | The proposed system matches the naturalness and expressiveness of a baseline T5-based system while eliminating problems with repeated or dropped words. |
Align-Refine: Non-Autoregressive Speech Recognition via Iterative Realignment (2021.naacl-main)
Copied to clipboard
| Challenge: | Non-autoregressive encoder-decoder models improve decoding speed, but generation quality suffers . editing at the level of output sequences limits model flexibility. |
| Approach: | They propose *iterative realignment* which iteratively realigns connectionist temporal alignments. |
| Outcome: | The proposed model matches an autoregressive baseline with a 14x speedup on the WSJ dataset; on LibriSpeech, it achieves an LM-free test-other WER of 9.0% (19% relative improvement on comparable work). |
Prompting with Phonemes: Enhancing LLMs’ Multilinguality for Non-Latin Script Languages (2025.naacl-long)
Copied to clipboard
Hoang H Nguyen, Khyati Mahajan, Vikas Yadav, Julian Salazar, Philip S. Yu, Masoud Hashemi, Rishabh Maheshwary
| Challenge: | Multilingual LLMs have achieved remarkable benchmark performance, but continue to underperform on non-Latin script languages. |
| Approach: | They propose to integrate phonemic transcriptions as complementary signals to induce script-invariant representations by integrating phonemic and orthographic transcriptions. |
| Outcome: | The proposed approach improves performance for Latin and non-Latin script languages, with 12.6% performance improvement and 15.1% performance improvement compared to randomized ICL retrieval. |
Don’t Use English Dev: On the Zero-Shot Cross-Lingual Evaluation of Contextual Embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual contextual embeddings have demonstrated state-of-the-art performance in zero-shot cross-lingual transfer learning. |
| Approach: | They show that English dev accuracy makes it difficult to obtain reproducible results . they recommend providing oracle scores alongside zero-shot results if possible . |
| Outcome: | mBERT and XLM have shown strong performance on cross-lingual recognition, text classification, dependency parsing, and other tasks. |
Meta-Learning the Difference: Preparing Large Language Models for Efficient Adaptation (2022.tacl-1)
Copied to clipboard
| Challenge: | Large pretrained language models are often domain- or task-adapted via finetuning or prompting. |
| Approach: | They propose to use domain-adaptive pretraining to prepare large pretrained language models for domain- or task-adaptation by learning to learn the difference between general and adapted PLMs. |
| Outcome: | Experiments on few-shot dialogue completion, low-resource abstractive summarization, and multi-domain language modeling show improvements in adaptation time and performance over finetuning or preparation via domain-adaptive pretraining. |
Unsupervised Bitext Mining and Translation via Self-Trained Contextual Embeddings (2020.tacl-1)
Copied to clipboard
| Challenge: | Existing methods to extract parallel sentences from unaligned text yield surprisingly good results. |
| Approach: | They propose an unsupervised method to create pseudo-parallel corpora for machine translation (MT) from unaligned text using multilingual BERT to create source and target sentence embeddings for nearest-neighbor search and adapt the model via self-training. |
| Outcome: | The proposed method outperforms existing methods and outperformed previous state-of-the-art methods by boosting translation performance by up to 3.5 BLEU on the WMT’14 French-English and WMT'16 German-English tasks. |
Zero-Shot End-to-End Spoken Language Understanding via Cross-Modal Selective Self-Training (2024.eacl-long)
Copied to clipboard
| Challenge: | End-to-end (E2E) spoken language understanding models are constrained by the cost of collecting speech-semantics pairs. |
| Approach: | They propose a model that learns E2E SLU without speech-semantics pairs . they propose cross-modal selective self-training (CMSST) to address imbalance and noise issues . |
| Outcome: | The proposed model learns E2E SLU without speech-semantics pairs . the proposed model requires the domains of speech-text and text-sensitization to match . |
Masked Language Model Scoring (2020.acl-main)
Copied to clipboard
| Challenge: | Pretrained masked language models require finetuning for most tasks. |
| Approach: | They evaluate pretrained masked language models out of the box via their pseudo-log-likelihood scores (PLLs) they attribute this success to PLL’s unsupervised expression of linguistic acceptability without a left-to-right bias, greatly improving on scores from GPT-2 . |
| Outcome: | The proposed model outperforms autoregressive language models in a variety of tasks. |