Papers by Patrick Liu
When Does Translation Require Context? A Data-driven, Multilingual Exploration (2023.acl-long)
Copied to clipboard
| Challenge: | Recent studies in context-aware MT attempt to target a small set of discourse phenomena during evaluation, however not in a fully systematic way. |
| Approach: | They develop a multilingual discourse-aware benchmark to evaluate model performance on discourse phenomena in a given dataset. |
| Outcome: | The proposed model improves on previously studied phenomena while uncovering others which were not addressed. |
Not-Just-Scaling Laws: Towards a Better Understanding of the Downstream Impact of Language Model Design Decisions (2025.emnlp-main)
Copied to clipboard
Emmy Liu, Amanda Bertsch, Lintang Sutawika, Lindia Tjuatja, Patrick Fernandes, Lara Marinov, Michael Chen, Shreya Singhal, Carolin Lawrence, Aditi Raghunathan, Kiril Gashteovski, Graham Neubig
| Challenge: | Language model performance is largely dependent on pretraining decisions, but scaling laws based on only these two aspects do not always explain downstream task performance. |
| Approach: | They meta-analyze 92 open-source pretrained models to quantify their impact on performance. |
| Outcome: | The framework lays a foundation for more systematic investigation of how model development choices shape final capabilities. |
Enhancing Self-Consistency and Performance of Pre-Trained Language Models through Natural Language Inference (2022.emnlp-main)
Copied to clipboard
Eric Mitchell, Joseph Noh, Siyan Li, Will Armstrong, Ananth Agarwal, Patrick Liu, Chelsea Finn, Christopher Manning
| Challenge: | Existing frameworks for boosting consistency and accuracy of pre-trained NLP models without fine-tuning or re-training are lacking. |
| Approach: | They propose a framework for boosting the consistency and accuracy of pre-trained NLP models using pre-trainer natural language inference models without fine-tuning or re-training. |
| Outcome: | The proposed framework boosts consistency and accuracy of pre-trained NLP models using pre-train natural language inference models without fine-tuning or re-training. |
PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them (2021.tacl-1)
Copied to clipboard
Patrick Lewis, Yuxiang Wu, Linqing Liu, Pasquale Minervini, Heinrich Küttler, Aleksandra Piktus, Pontus Stenetorp, Sebastian Riedel
| Challenge: | Open-domain Question Answering models that directly leverage question-answer (QA) pairs show promise in terms of speed and memory compared with conventional models which retrieve and read from text corpora. |
| Approach: | They propose a question-answer (QA)-pair retriever to facilitate improved QA-patch models by introducing Probably Asked Questions (PAQ) they propose QA pair retriever, RePAQ, which preempts and caches test questions, enabling it to match the accuracy of recent retrieve-and-read models, whilst being significantly faster. |
| Outcome: | The proposed model outperforms baseline models by 5% but trails RePAQ by 15% . it can be configured for size (under 500MB) or speed (over 1K questions per second) while retaining high accuracy. |
DialGuide: Aligning Dialogue Model Behavior with Developer Guidelines (2023.findings-emnlp)
Copied to clipboard
Prakhar Gupta, Yang Liu, Di Jin, Behnam Hedayatnia, Spandana Gella, Sijia Liu, Patrick Lange, Julia Hirschberg, Dilek Hakkani-Tur
| Challenge: | Dialogue models are able to generate fluent and interesting responses, but they can be difficult to control and may produce non-engaging, unsafe results. |
| Approach: | They propose a framework for controlling dialogue model behavior using natural language rules, or guidelines, which provide information about the context they are applicable to and what should be included in the response. |
| Outcome: | The proposed framework is effective in three open-domain dialogue response generation tasks and is consistent with the developer's expectations and intent. |
Fisher Information-based Efficient Curriculum Federated Learning with Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing frameworks for learning Large Language Models (LLMs) require adaptive data processing and low-rank adjustment to improve accuracy and fine-tuning speed. |
| Approach: | They propose a fisher information-based adaptive federated curriculum learning framework with two novel methods to improve FL fine-tuning process. |
| Outcome: | The proposed framework improves performance and fine-tuning speed compared with baseline approaches. |
Scaling Parameter-Constrained Language Models with Quality Data (2024.emnlp-industry)
Copied to clipboard
Ernie Chang, Matteo Paltenghi, Yang Li, Pin-Jie Lin, Changsheng Zhao, Patrick Huber, Zechun Liu, Rastislav Rabatin, Yangyang Shi, Vikas Chandra
| Challenge: | Scaling laws in language modeling quantify training loss as a function of dataset size and model parameters, but neglect the critical role of data quality in model generalization. |
| Approach: | They propose to use effective training tokens as a combination of text diversity and syntheticity as measured by a teacher model to calculate scaling laws. |
| Outcome: | The proposed term effective training tokens is a combination of two readily-computed indicators of text diversity and syntheticity as measured by a teacher model. |
Multi-Dimensional Evaluation of Text Summarization with In-Context Learning (2023.findings-acl)
Copied to clipboard
Sameer Jain, Vaishakh Keshava, Swarnashree Mysore Sathyendra, Patrick Fernandes, Pengfei Liu, Graham Neubig, Chunting Zhou
| Challenge: | In-context learning-based evaluators are competitive with learned evaluation frameworks for text summarization tasks. |
| Approach: | They propose to use large language models as multi-dimensional evaluators using in-context learning to evaluate text summarization tasks. |
| Outcome: | The proposed frameworks are competitive with existing frameworks on relevance and factual consistency, the authors show . |
MEDs for PETs: Multilingual Euphemism Disambiguation for Potentially Euphemistic Terms (2024.findings-eacl)
Copied to clipboard
Patrick Lee, Alain Chirino Trujillo, Diana Cuevas Plancarte, Olumide Ojo, Xinyi Liu, Iyanuoluwa Shode, Yuan Zhao, Anna Feldman, Jing Peng
| Challenge: | Euphemisms are a linguistic device used to soften or neutralize language that may otherwise be harsh or awkward to state directly. |
| Approach: | They train a multilingual transformer model to disambiguate potentially euphemistic terms in multilingual and cross-lingual settings. |
| Outcome: | The proposed model performs better than monolingual models on the disambiguation task compared to monolingual ones in multilingual and cross-lingual settings. |
Improving Cross-task Generalization of Unified Table-to-text Models with Compositional Task Configurations (2023.findings-acl)
Copied to clipboard
Jifan Chen, Yuhao Zhang, Lan Liu, Rui Dong, Xinchi Chen, Patrick Ng, William Yang Wang, Zhiheng Huang
| Challenge: | Existing methods for multitask learning typically use a dataset name as input prefix, which limits the effectiveness of multitask training. |
| Approach: | They propose compositional task configurations, a set of prompts prepended to the encoder to improve cross-task generalization of unified models. |
| Outcome: | The proposed model outperforms the UnifiedSKG baseline by noticeable margins in both in-domain and zero-shot settings. |
ConEC: Earnings Call Dataset with Real-world Contexts for Benchmarking Contextual Speech Recognition (2024.lrec-main)
Copied to clipboard
Ruizhe Huang, Mahsa Yarmohammadi, Jan Trmal, Jing Liu, Desh Raj, Leibny Paola Garcia, Alexei V. Ivanov, Patrick Ehlen, Mingzhi Yu, Dan Povey, Sanjeev Khudanpur
| Challenge: | Existing work on contextual speech recognition (ASR) systems focuses on recognizing words that are not frequently seen in training data, such as rare words, but word error rate on rare words remains over 20%. |
| Approach: | They propose to use public-domain earnings calls and supplementary materials to evaluate contextual ASR approaches grounded on real-world applications. |
| Outcome: | The proposed frameworks are noisier than artificially synthesized contexts that contain the ground truth, yet still make great room for future improvement of contextual ASR technology. |
MobileLLM-Flash: Latency-Guided On-Device LLM Design for Industry Scale Deployment (2026.acl-industry)
Copied to clipboard
Hanxian Huang, Igor Fedorov, Andrey Gromov, Bernard Beckerman, Naveen Suda, David Eriksson, Maximilian Balandat, Rylan Conway, Patrick Huber, Chinnadhurai Sankar, Ayushi Dalmia, Zechun Liu, Lemeng Wu, Tarek Elgamal, Adithya Sagar, Vikas Chandra, Raghuraman Krishnamoorthi
| Challenge: | MobileLLM-Flash is a family of foundation models for efficient on-device use with strong capabilities. |
| Approach: | They propose a method for designing on-device large language models under mobile latency constraints using hardware-in-the-loop architecture search. |
| Outcome: | The proposed model is amenable to industry-scale deployment and is compatible with mobile runtimes like Executorch. |
Mitigating Hallucinations in LM-Based TTS Models via Distribution Alignment Using GFlowNets (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing mitigation strategies for Text-to-Speech systems require excessive training resources or inference latency. |
| Approach: | They propose a GFlOwNet-guided distribution AlignmenT framework that mitigates hallucinations without relying on massive resources or inference latency. |
| Outcome: | The proposed framework reduces over 50% character error rates and lowers uncertainty by up to 58% on challenging test cases. |
Challenges in Generalization in Open Domain Question Answering (2022.findings-naacl)
Copied to clipboard
| Challenge: | Recent work on Open Domain Question Answering has shown that there is a large discrepancy in model performance between novel test questions and those that largely overlap with training questions. |
| Approach: | They introduce and annotate questions according to three categories that measure training set overlap, compositional generalization, and novel-entity generalization. |
| Outcome: | The proposed models perform better on established datasets and lower on comp-gen/novel-entity questions than on the full test set. |