Papers by Davis Liang
Generating Hashtags for Short-form Videos with Guided Signals (2023.acl-long)
Copied to clipboard
Tiezheng Yu, Hanchao Yu, Davis Liang, Yuning Mao, Shaoliang Nie, Po-Yao Huang, Madian Khabsa, Pascale Fung, Yi-Chia Wang
| Challenge: | Short-form video hashtag recommendation (SVHR) is a classification or ranking problem that selects hashtags from a set of limited candidates. |
| Approach: | They propose a short-form video hashtag recommendation task that better represents how hashtags are created naturally by retrieving relevant hashtags from a large-scale hashtag pool as extra guidance signals. |
| Outcome: | The proposed model outperforms strong classification baselines on two short-form video datasets and the guidance signals boost the performance by 8.11 and 2.17 absolute ROUGE-1 scores on average. |
XLM-V: Overcoming the Vocabulary Bottleneck in Multilingual Masked Language Models (2023.emnlp-main)
Copied to clipboard
Davis Liang, Hila Gonen, Yuning Mao, Rui Hou, Naman Goyal, Marjan Ghazvininejad, Luke Zettlemoyer, Madian Khabsa
| Challenge: | Large multilingual models rely on a single vocabulary shared across 100+ languages . this vocabulary bottleneck limits the representational capabilities of multilingual model XLM-R . |
| Approach: | They propose a new approach for scaling to large multilingual vocabularies by de-emphasizing token sharing between languages with little lexical overlap and assigning vocabulary capacity to achieve sufficient coverage for each individual language. |
| Outcome: | The proposed model outperforms XLM-R on all language tasks and is particularly effective on low-resource tasks. |
The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants (2024.acl-long)
Copied to clipboard
Lucas Bandarkar, Davis Liang, Benjamin Muller, Mikel Artetxe, Satya Narayan Shukla, Donald Husa, Naman Goyal, Abhinandan Krishnan, Luke Zettlemoyer, Madian Khabsa
| Challenge: | Existing benchmarks for text comprehension only cover 30 languages, but lack of labeled data is a major obstacle to building functional systems in most languages. |
| Approach: | They present a multiple-choice machine reading comprehension dataset spanning 122 languages . they use it to evaluate the capabilities of multilingual masked language models and large language models . |
| Outcome: | The proposed dataset enables the evaluation of text models in high-, medium- and low-resource languages. |
A Study on Knowledge Distillation from Weak Teacher for Scaling Up Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | a study shows that DWT can be effective in the vision domain and natural language processing pre-training stages. |
| Approach: | They examine three key factors to optimize Distillation from Weak Teacher (DWT) DWT is a method of transferring knowledge from a weaker teacher model to a larger student model to improve its performance. |
| Outcome: | a new study examines three key factors to optimize DWT in NLP pre-training scenarios . the impact of teacher model quality and guidelines for adjusting the weighting value for DW T loss are examined . |
From Feedback to Checklists: Grounded Evaluation of AI-Generated Clinical Notes (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing automated metrics fail to align with real-world physician preferences. |
| Approach: | They propose a pipeline that distills real user feedback into structured checklists for note evaluation that are interpretable, grounded in human feedback, and enforceable by LLM-based evaluators. |
| Outcome: | The proposed checklist outperforms baseline evaluations in coverage, diversity, and predictive power for human ratings. |
RoAST: Robustifying Language Models via Adversarial Perturbation with Selective Training (2023.findings-emnlp)
Copied to clipboard
Jaehyung Kim, Yuning Mao, Rui Hou, Hanchao Yu, Davis Liang, Pascale Fung, Qifan Wang, Fuli Feng, Lifu Huang, Madian Khabsa
| Challenge: | Several perspectives of robustness for pre-trained language models have been studied independently, but lacking a unified consideration in multiple perspectives. |
| Approach: | They propose a technique to enhance the multi-perspective robustness of LMs by introducing adversarial perturbation while the model parameters are selectively updated upon their relative importance. |
| Outcome: | The proposed technique improves the robustness of LMs by incorporating four perspectives on model robustness. |
Improve Transformer Models with Better Relative Position Embeddings (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for generating position embeddings are not fully utilized in NLP tasks. |
| Approach: | They propose to generalize the absolute position embedding to a generalized relative position embedded method . they also propose to use the relative embeddable method to improve the accuracy of large models . |
| Outcome: | The proposed method improves accuracy on the SQuAD1.1 dataset compared to previous methods . it can be easily adopted as a drop-in replacement for improving accuracy of large models . |
Masked Language Model Scoring (2020.acl-main)
Copied to clipboard
| Challenge: | Pretrained masked language models require finetuning for most tasks. |
| Approach: | They evaluate pretrained masked language models out of the box via their pseudo-log-likelihood scores (PLLs) they attribute this success to PLL’s unsupervised expression of linguistic acceptability without a left-to-right bias, greatly improving on scores from GPT-2 . |
| Outcome: | The proposed model outperforms autoregressive language models in a variety of tasks. |
Co-training and Co-distillation for Quality Improvement and Compression of Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Knowledge Distillation (KD) compresses expensive pre-trained language models . however, most smaller models fail to surpass performance of larger model . |
| Approach: | They propose a framework that co-trains two models while mutually distilling knowledge to improve performance and inference speed together. |
| Outcome: | The proposed framework outperforms the original larger model by 1.66 on the GLUE benchmark. |