Papers by Yi-An Lai
Backward Compatibility During Data Updates by Weight Interpolation (2024.eacl-long)
Copied to clipboard
| Challenge: | Retraining a model with a larger amount of training data introduces negative flips . retraining the model with the updated data introduce negative flipping . |
| Approach: | They propose a backward compatible weight interpolation method to improve model predictions without regression bugs. |
| Outcome: | The proposed method reduces negative flips without sacrificing accuracy . it is straight forward to implement and does not increase inference cost. |
Diversity, Density, and Homogeneity: Quantitative Characteristic Metrics for Text Collections (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing descriptive statistics are inadequate to summarize text collections by quantitative measures. |
| Approach: | They propose a set of characteristic metrics that quantitatively measure the dispersion, sparsity, and uniformity of a text collection. |
| Outcome: | The proposed metrics are highly correlated with text classification performance of a renowned model, which could inspire future applications. |
DeAL: Decoding-time Alignment for Large Language Models (2025.acl-long)
Copied to clipboard
James Y. Huang, Sailik Sengupta, Daniele Bonadiman, Yi-An Lai, Arshit Gupta, Nikolaos Pappas, Saab Mansour, Katrin Kirchhoff, Dan Roth
| Challenge: | Large Language Models (LLMs) are expected to generate content aligned with human preferences. |
| Approach: | They propose a framework that allows the user to customize reward functions and enables Decoding-time Alignment of LLMs (DeAL). |
| Outcome: | The proposed framework allows the user to customize reward functions and enables Decoding-time Alignment of LLMs. |
Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System (2022.acl-long)
Copied to clipboard
| Challenge: | Existing pre-trained language models often form a cascaded generation problem . this can lead to error accumulation across different sub-tasks and greater data annotation overhead. |
| Approach: | They propose a plug-and-play model for task-oriented dialogue that learns primary TOD task completion skills from heterogeneous dialog corpora. |
| Outcome: | The proposed model learns primary TOD task completion skills from heterogeneous dialog corpora. |
Regression Bugs Are In Your Model! Measuring, Reducing and Analyzing Regressions In NLP Model Updates (2021.acl-long)
Copied to clipboard
| Challenge: | Using negative flips, we quantify, reduce and analyze regression errors in deep neural networks. |
| Approach: | They propose to quantify, reduce and analyze regression errors in NLP models by negative flips. |
| Outcome: | The proposed model update regression has a prevalent presence across tasks in the GLUE benchmark. |
Improving Prediction Backward-Compatiblility in NLP Model Upgrade with Gated Fusion (2023.findings-eacl)
Copied to clipboard
| Challenge: | a regression error during model upgrade often outweighs the benefits of accuracy gain . a novel method that promotes backward compatibility during model upgrades is proposed . |
| Approach: | They propose a method that promotes backward compatibility via learning to mix predictions between old and new models. |
| Outcome: | The proposed method outperforms existing methods and achieves negative flip rate reductions by 73.2% on two model upgrade scenarios. |
Context Analysis for Pre-trained Masked Language Models (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models that learn contextualized word representations from a large un-annotated corpus have become a standard component for many downstream NLP tasks. |
| Approach: | They propose to use a masking and gradient approach to evaluate the impact of context on the word representation. |
| Outcome: | The proposed model architectures are architecture agnostic and gradient based. |