Papers by Ruibo Liu

Language Model Augmented Relevance Score (2021.acl-long)

Copied to clipboard

Challenge:	Existing metrics that compare the candidate with the human reference do not consider the context, resulting in poor correlation with human judgements.
Approach:	They propose a language model-aware metric that augments the human reference while considering the context to provide evaluation scores that correlate highly with human judgements.
Outcome:	The proposed metric achieves higher correlation with human reference judgements and differentiates well-formed candidates from adversarial samples to a larger degree.

HATA: Trainable and Hardware-Efficient Hash-Aware Top-k Attention for Scalable Large Model Inference (2025.findings-acl)

Copied to clipboard

Ping Gong, Jiawei Yi, Shengnan Wang, Juncheng Zhang, Zewen Jin, Ouxiang Zhou, Ruibo Liu, Guanbin Xu, Youhui Bai, Bowen Ye, Kun Yuan, Tong Yang, Gong Zhang, Renhai Chen, Feng Wu, Cheng Li

Green & Sustainable NLP

Challenge:	Existing top-k attention methods struggle to strike a balance between efficiency and accuracy.
Approach:	They propose a top-k attention approach that integrates low-overhead techniques into the Top-k Attention process to achieve 7.2 speedup compared to vanilla full attention.
Outcome:	The proposed approach achieves 7.2 speedup compared to current top-k attention methods while maintaining model accuracy.

EnDex: Evaluation of Dialogue Engagingness at Scale (2022.findings-emnlp)

Copied to clipboard

Guangxuan Xu, Ruibo Liu, Fabrice Harel-Canada, Nischal Reddy Chandra, Nanyun Peng

Dialogue Systems & Conversational Agents

Challenge:	Existing models that measure engagement use expensive human annotas and abstract definitions of the term.
Approach:	They propose a human-reaction based model to evaluate dialogue engagingness . they propose combining distant-supervision with a theoretical foundation for engagement .
Outcome:	The proposed model is trained on 80k Reddit-based engagement datasets . it uses distant-supervision from human-reaction feedback to evaluate dialogue engagementness .

Your Vision-Language Model Itself Is a Strong Filter: Towards High-Quality Instruction Tuning with Data Selection (2024.findings-acl)

Copied to clipboard

Ruibo Chen, Yihan Wu, Lichang Chen, Guodong Liu, Qi He, Tianyi Xiong, Chenxi Liu, Junfeng Guo, Heng Huang

Language Models

Challenge:	Existing data selection methods for instruction-following large language models rely on unreliable scores or use downstream tasks for selection.
Approach:	They propose a method that utilizes the VLM itself as a filter to select high-quality instruction-tuning data.
Outcome:	The proposed method can reach better results compared to full data settings with merely about 15% samples and can achieve superior performance against competitive baselines.

ChatMusician: Understanding and Generating Music Intrinsically with LLM (2024.findings-acl)

Copied to clipboard

Ruibin Yuan, Hanfeng Lin, Yi Wang, Zeyue Tian, Shangda Wu, Tianhao Shen, Ge Zhang, Yuhang Wu, Cong Liu, Ziya Zhou, Liumeng Xue, Ziyang Ma, Qin Liu, Tianyu Zheng, Yizhi Li, Yinghao Ma, Yiming Liang, Xiaowei Chi, Ruibo Liu, Zili Wang, Chenghua Lin, Qifeng Liu, Tao Jiang, Wenhao Huang, Wenhu Chen, Jie Fu, Emmanouil Benetos, Gus Xia, Roger Dannenberg, Wei Xue, Shiyin Kang, Yike Guo

Multimodality

Challenge:	Despite LLMs' impressive capabilities in musical knowledge, music reasoning remains an unsolved task.
Approach:	They propose an open-source large language model (LLM) that integrates intrinsic musical abilities into LLaMA2 and GPT-3.5.
Outcome:	The proposed model can understand and generate music with a pure text tokenizer without external multi-modal neural structures or tokenizers.

MoLA: MoE LoRA with Layer-wise Expert Allocation (2025.findings-naacl)

Copied to clipboard

Chongyang Gao, Kezhen Chen, Jinmeng Rao, Ruibo Liu, Baochen Sun, Yawen Zhang, Daiyi Peng, Xiaoyuan Guo, Vs Subrahmanian

Green & Sustainable NLP

Challenge:	Recent efforts to integrate low-rank adaptation (LoRA) with the Mixture-of-Experts (MoE) have achieved performance comparable to full-parameter fine-tuning by tuning much fewer parameters.
Approach:	They propose a parameter-efficient MoE method for low-rank adaptation with the Mixture-of-Experts (MoE) they use layers of LoRA experts to allocate more LoRA expert to middle layers .
Outcome:	The proposed method outperforms baseline models on six well-known NLP and commonsense QA benchmarks on LLAMA-2, Mistral, and Gemma.

Multi-resolution Annotations for Emoji Prediction (2020.emnlp-main)

Copied to clipboard

Weicheng Ma, Ruibo Liu, Lili Wang, Soroush Vosoughi

Visual Data in NLP

Challenge:	Emojis are able to express various linguistic components, such as emotions, sentiments, events, etc. emojis have the merit of preserving information more densely, compared to words, argues a new study.
Approach:	They propose to use passage-level and aspect-level emoji annotations to predict the proper emmojis associated with text.
Outcome:	The proposed method is heuristically generated and validated with a pre-trained BERT model.

Modulating Language Models with Emotions (2021.findings-acl)

Copied to clipboard

Ruibo Liu, Jason Wei, Chenyan Jia, Soroush Vosoughi

Language Models

Challenge:	Existing methods for generating context-aware language that embodies diverse emotions are dull or generic due to limited training data for diverse emotions.
Approach:	They propose a modulated layer normalization technique that generates emotional responses using large pre-trained models.
Outcome:	The proposed method outperforms baseline methods on the MojiTalk dataset while maintaining diversity, fluency, and coherence.

Aligning Generative Language Models with Human Values (2022.findings-naacl)

Copied to clipboard

Ruibo Liu, Ge Zhang, Xinyu Feng, Soroush Vosoughi

Language Models

Challenge:	Existing methods for learning human values do not consider contextual and abstract nature of human values.
Approach:	They propose a reinforcement learning based method that embeds human values judgements into each step of language generation.
Outcome:	The proposed method improves on human values judgements and shows higher alignment performance.

Data Boost: Text Data Augmentation Through Reinforcement Learning Guided Conditional Generation (2020.emnlp-main)

Copied to clipboard

Ruibo Liu, Guangxuan Xu, Chenyan Jia, Weicheng Ma, Lili Wang, Soroush Vosoughi

Information Extraction & Text Mining

Challenge:	Existing methods for data augmentation produce low readability or semantic consistency.
Approach:	They propose a framework which augments data through reinforcement learning guided conditional generation.
Outcome:	The proposed framework improves F1 performance on three different classification tasks by 8.7% on average when given only 10% of the whole data for training.

MMRA: A Benchmark for Evaluating Multi-Granularity and Multi-Image Relational Association Capabilities in Large Visual Language Models (2026.findings-eacl)

Copied to clipboard

Siwei Wu, King Zhu, Yu Bai, Yiming Liang, Yizhi Li, Haoning Wu, Jiaheng Liu, Ruibo Liu, Xingwei Qu, Xuxin Cheng, Ge Zhang, Wenhao Huang, Chenghua Lin

Multimodality

Challenge:	Current multimodal benchmarks focus on facts within individual images, but neglect associative relations among multiple images.
Approach:	They propose a multi-image relational association task and a MMRA benchmark to evaluate LVLMs.
Outcome:	The proposed benchmarks show that entity-level multi-image perception tasks pose greater challenges than image-level tasks.

Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering (2025.naacl-long)

Copied to clipboard

Chenglei Si, Yanzhe Zhang, Ryan Li, Zhengyuan Yang, Ruibo Liu, Diyi Yang

Multimodality

Challenge:	Generative AI has made rapid advances in multimodal understanding and code generation.
Approach:	They construct a first real-world benchmark for multimodal large language models that directly convert visual designs into code implementations by manually curating 484 diverse real-life webpages as test cases.
Outcome:	The proposed model can generate code implementations that directly render into the given reference webpages, given the screenshots as input.

LIME: Less Is More for MLLM Evaluation (2025.findings-acl)

Copied to clipboard

King Zhu, Qianbo Zang, Shian Jia, Siwei Wu, Feiteng Fang, Yizhi Li, Shuyue Guo, Tianyu Zheng, Jiawei Guo, Bo Li, Haoning Wu, Xingwei Qu, Jian Yang, Ruibo Liu, Xiang Yue, Jiaheng Liu, Chenghua Lin, Hamid Alinejad-Rokny, Min Yang, Shiwen Ni, Wenhao Huang, Ge Zhang

Multimodality

Challenge:	Existing MLLM benchmarks and unified evaluation frameworks cannot accurately and efficiently reflect the ability of MLMLs.
Approach:	They propose a semi-automated benchmark curated using a pipeline that filters out uninformative samples and eliminates answer leakage by focusing on tasks that require image-based understanding.
Outcome:	The proposed benchmark reduces the number of samples by 76% and evaluation time by 77% while it can more effectively distinguish different models’ abilities.