Papers by Shuai Lu
WhiteningBERT: An Easy Unsupervised Sentence Embedding Approach (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models perform well on learning sentence semantics when fine-tuned with supervised data. |
| Approach: | They conduct a thorough examination of pretrained model based unsupervised sentence embeddings. |
| Outcome: | The proposed approach improves on whitening-based vector normalization with less than 10 lines of code. |
SmartBench: Is Your LLM Truly a Good Chinese Smartphone Assistant? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation benchmarks for Large Language Models focus on objective tasks like mathematics and coding in English, which do not reflect the practical use cases of on-device LLMs in real-world mobile scenarios. |
| Approach: | They propose a benchmark to evaluate the capabilities of on-device Large Language Models in Chinese mobile contexts. |
| Outcome: | The proposed framework evaluates on-device LLMs and MLLMs in Chinese . it provides a standardized framework for evaluating LLM performance on real smartphones . |
Long-Range Modeling of Source Code Files with eWASH: Extended Window Access by Syntax Hierarchy (2021.emnlp-main)
Copied to clipboard
Colin Clement, Shuai Lu, Xiaoyu Liu, Michele Tufano, Dawn Drain, Nan Duan, Neel Sundaresan, Alexey Svyatkovskiy
| Challenge: | Statistical language modeling and translation with transformers have found many successful applications in program understanding and generation tasks. |
| Approach: | They propose an architecture-independent approach for leveraging syntactic hierarchies of source code . they use syntax trees to extract syntak hierarchical structures and integrate them into context window . |
| Outcome: | The proposed approach achieves state-of-the-art in code completion and summarization for Python in the CodeXGLUE benchmark. |
AdDriftBench: A Benchmark for Detecting Data Drift and Label Drift in Short Video Advertising (2025.findings-emnlp)
Copied to clipboard
Yinghao Song, Xiangji Zeng, Shuai Cui, Lu Sun, Zhaowei Liu, Yuan Yuan, Yulu Wang, Hai Zhou, Zhaohan Gong
| Challenge: | Short video advertising scenarios present unique challenges due to data drift (DD) and label drift (LD). |
| Approach: | They propose to use data drift and label drift to evaluate models under rapidly shifting content distributions and labeling scenarios to assess their generalization capabilities. |
| Outcome: | The proposed model performs moderately in short video advertising contexts, particularly in handling fine-grained semantics and adapting to shifting instructions. |
ReACC: A Retrieval-Augmented Code Completion Framework (2022.acl-long)
Copied to clipboard
| Challenge: | Recent work has shown that statistical language modeling with transformers can greatly improve the performance in code completion tasks. |
| Approach: | They propose a retrieval-augmented code completion framework that combines a source code retriever and an auto-regressive language model for programming language. |
| Outcome: | The proposed framework achieves state-of-the-art on CodeXGLUE benchmark. |
A Systematic Survey of Automatic Prompt Optimization Techniques (2025.emnlp-main)
Copied to clipboard
Kiran Ramnath, Kang Zhou, Sheng Guan, Soumya Smruti Mishra, Xuan Qi, Zhengyuan Shen, Shuai Wang, Sangmin Woo, Sullam Jeoung, Yawei Wang, Haozhu Wang, Han Ding, Yuzhe Lu, Zhichao Xu, Yun Zhou, Balasubramaniam Srinivasan, Qiaojing Yan, Yueyan Chen, Haibo Ding, Panpan Xu, Lin Lee Cheong
| Challenge: | Recent advances in prompt engineering have created impediments for end users to adopt . however, prompt engineering remains an impedance due to rapid advances in models, tasks, and associated best practices. |
| Approach: | They propose to define APO as a 5-part unifying framework and categorize all relevant works based on their salient features. |
| Outcome: | The proposed framework aims to improve the performance of large language models on various tasks. |
Sparsity-Accelerated Training for Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated proficiency across various NLP tasks but often require additional training, such as continual pre-training and supervised fine-tuning. |
| Approach: | They propose to leverage sparsity in pre-trained LLMs to accelerate training by disregarding computations for unimportant neurons. |
| Outcome: | The proposed framework achieves comparable or superior performance to standard training while significantly accelerating the process. |
Alignment for Efficient Tool Calling of Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in tool learning have enabled large language models to integrate external tools, enhancing their task performance by expanding their knowledge boundaries. |
| Approach: | They propose a framework that combines probabilistic knowledge boundary estimation with dynamic decision-making to allow LLMs to better assess when to invoke tools based on their confidence. |
| Outcome: | The proposed framework shows significant improvements in tool efficiency by reducing unnecessary tool usage. |
Constructing Multilingual Code Search Dataset Using Neural Machine Translation (2023.acl-srw)
Copied to clipboard
| Challenge: | Existing datasets for code search are monolingual, but their query data are only in English. |
| Approach: | They construct a multilingual code search dataset in four natural and four programming languages using a neural machine translation model and apply back-translation data filtering to it. |
| Outcome: | The proposed model pre-trained with all natural and programming language data performs best under almost all settings. |
Code Execution with Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
Chenxiao Liu, Shuai Lu, Weizhu Chen, Daxin Jiang, Alexey Svyatkovskiy, Shengyu Fu, Neel Sundaresan, Nan Duan
| Challenge: | Pre-trained code intelligence models ignore the execution trace and only rely on source code and syntactic structures to understand code execution. |
| Approach: | They develop a mutation-based data augmentation technique to create a Python dataset and task for code execution that challenges existing models. |
| Outcome: | The proposed model outperforms existing models on code execution and shows its potential for zero-shot code-to-code search and text-to code generation. |
CARD: Cross-modal Agent Framework for Generative and Editable Residential Design (2025.emnlp-main)
Copied to clipboard
| Challenge: | Architectural design automation has made significant progress, but the complexity of open-world environments makes residential design a challenging task. |
| Approach: | They propose a framework that leverages a system of specialized cross-modal agents to adapt to open-world residential design. |
| Outcome: | The proposed framework enables users to generate and edit residential design without requiring specialized expertise. |
UniXcoder: Unified Cross-Modal Pre-training for Code Representation (2022.acl-long)
Copied to clipboard
| Challenge: | Pre-trained models for programming languages have demonstrated great success on code intelligence . however, such pre-tried models are sub-optimal for auto-regressive tasks . |
| Approach: | They propose a unified cross-modal pre-trained model for programming language that leverages cross-module contents like AST and code comment to enhance code representation. |
| Outcome: | The proposed model achieves state-of-the-art on most code-related tasks and compares with existing models on zero-shot code-to-code search. |
FloorPlan-LLaMa: Aligning Architects’ Feedback and Domain Knowledge in Architectural Floor Plan Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing evaluation methods for floor plan generation rely on statistical metrics like FID, GED, and PSNR, which fail to evaluate using domain knowledge. |
| Approach: | They propose to use a first floor plan dataset to train a floor plan generation model based on a multi-dimensional preference score and a textual analysis to integrate architects’ professional expertise and preferences. |
| Outcome: | The proposed model outperforms baseline models in text-conditional and class-condition tasks and is more rational and aligns better with human preferences. |
Enhancing Large Language Models in Coding Through Multi-Perspective Self-Consistency (2024.acl-long)
Copied to clipboard
| Challenge: | Existing work utilizes verification properties to verify and re-rank solutions in a majority voting manner, but this assumption may not hold. |
| Approach: | They propose a multi-perspective self-consistency framework that incorporates both inter- and intra-consistency across outputs from multiple perspectives. |
| Outcome: | The proposed framework significantly boosts performance of foundation models on various benchmarks, including HumanEval (+15.91%), MBPP (+6.43%) and CodeContests (+9.37%). |
NeuSym-RAG: Hybrid Neural Symbolic Retrieval with Multiview Structuring for PDF Question Answering (2025.acl-long)
Copied to clipboard
Ruisheng Cao, Hanchong Zhang, Tiancheng Huang, Zhangyi Kang, Yuxin Zhang, Liangtai Sun, Hanqi Li, Yuxun Miao, Shuai Fan, Lu Chen, Kai Yu
| Challenge: | Existing approaches to retrieval augmented generation neglect PDF structure and layout . individual PDFs often exceed prompt limits and user queries may span multiple documents. |
| Approach: | They propose a hybrid neural symbolic retrieval framework which combines both paradigms in an interactive process. |
| Outcome: | The proposed framework organizes semi-structured PDF content into relational database and vectorstore . it defeats both RAG and structured baselines on three PDF-based QA datasets . |
LearnAct: Few-Shot Mobile GUI Agent with a Unified Demonstration Benchmark (2026.findings-acl)
Copied to clipboard
Guangyi Liu, Pengxiang Zhao, Liang Liu, Zhiming Chen, Yuxiang Chai, Yaozhen Liang, WenHao Wang, Siheng Chen, Zhengxi Lu, Shuai Ren, Hao Wang, Shibo He, Yong Liu, Wenchao Meng
| Challenge: | Mobile GUI agents show promise in automating tasks but face significant generalization challenges in long-tail scenarios. |
| Approach: | They propose a benchmark framework for mobile GUI agents that measures the performance of GUI agents by analyzing their performance. |
| Outcome: | The LearnGUI benchmark outperforms existing methods in offline and online evaluations and demonstrates consistent gains across model architectures. |
De-Bias for Generative Extraction in Unified NER Task (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for Named entity recognition (NER) are not consistent with the task, which makes the model vulnerable to incorrect biases. |
| Approach: | They propose to use generative model to recognize entities from sentences . they analyze incorrect biases in the generation process from a causal perspective . |
| Outcome: | The proposed method improves the performance of the generative NER model in various datasets. |
Selene: Pioneering Automated Proof in Software Verification (2024.acl-long)
Copied to clipboard
| Challenge: | Currently, software verification is resource-intensive and manpower-consuming. |
| Approach: | They propose a project-level automated proof benchmark based on the seL4 operating system . they propose augmentations to enhance the flexibility of the framework and lightweight verification environment . |
| Outcome: | The proposed framework provides a comprehensive framework for end-to-end proof generation and a lightweight verification environment. |
TVWorld: Foundations for Remote-Control TV Agents (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing work on large vision–language models focuses on point-and-click interaction, while remote-control interaction is underexplored. |
| Approach: | They propose a topology-aware training framework that injects topology awareness into LVLMs. |
| Outcome: | The proposed model achieves 68.3% success rate on TVWorld-N, surpassing closed-source benchmarks and state-of-the-art (SOTA) benchmarks show that existing agents lack topology awareness for focus-based, long-horizon TV navigation. |
Intervention-Based Alignment of Code Search with Execution Feedback (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing code search training datasets approximate text-code co-occurrences as positive execution feedback, but this approximation may misalign models’ retrieval decisions from ground-truth correctness. |
| Approach: | They propose a code intervention-based reinforcement learning approach that perturbs training code to result in misalignment, then tests models’ decisions and corrects them with the execution feedback by reinforcement learning. |
| Outcome: | The proposed method induces the execution feedback from perturbation, without actual execution, and then tests models’ decisions and corrects them with the execution input by reinforcement learning. |
Towards Compositional Generalization in Code Search (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing code search models that focus on code as an unstructured sequence fail to generalize when the lexical perturbation without changing structures and labels is applied in test codes. |
| Approach: | They propose a compositional generalization model that extracts structural elements and a code template that targets compositional genericization. |
| Outcome: | The proposed model is complementary to flow graphs in GraphCodeBERT, by enhancing structural context around variables. |
Gold-Medal-Level Olympiad Geometry Solving with Efficient Heuristic Auxiliary Constructions (2026.acl-long)
Copied to clipboard
Boyan Duan, Xiao Liang, Shuai Lu, Yaoxiang Wang, Yelong Shen, Kai-Wei Chang, Ying Nian Wu, Mao Yang, Weizhu Chen, Yeyun Gong
| Challenge: | Existing methods for geometry theorem proving in Euclidean geometry are challenging and require a neural network to perform. |
| Approach: | They propose a method for adding auxiliary points in geometry that runs on CPUs without relying on neural network-based inference. |
| Outcome: | The proposed method achieves silver-medal-level human performance on IMO-30 benchmark. |
AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models (2024.findings-naacl)
Copied to clipboard
Wanjun Zhong, Ruixiang Cui, Yiduo Guo, Yaobo Liang, Shuai Lu, Yanlin Wang, Amin Saied, Weizhu Chen, Nan Duan
| Challenge: | Traditional benchmarks for evaluating foundation models often fail to accurately represent their general abilities for human-centric tasks. |
| Approach: | They propose a bilingual benchmark to assess foundation models in the context of human-centric standardized exams such as college entrance exams, law school admission tests, and math competitions. |
| Outcome: | The proposed benchmark exceeds the average human performance on SAT, LSAT, and math competitions with 95% accuracy and 92.5% on the Chinese college entrance English exam. |
Understanding the Therapeutic Relationship between Counselors and Clients in Online Text-based Counseling using LLMs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | In traditional face-to-face therapy, the assessment of therapeutic alliance is not directly translated to text-based settings. |
| Approach: | They propose an automatic approach to understand the development of therapeutic alliance in text-based counseling by using large language models. |
| Outcome: | The proposed approach demonstrates that the framework is effective in identifying the therapeutic alliance in text-based counseling. |
Locate and Label: A Two-stage Identifier for Nested Named Entity Recognition (2021.acl-long)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a well-studied task in natural language processing. |
| Approach: | They propose a method that generates span proposals and labels them with categories . they use boundary information of entities and partially matched spans to locate them . |
| Outcome: | The proposed method outperforms state-of-the-art models on nested NER datasets. |
Competition-Level Problems are Effective LLM Evaluators (2024.findings-acl)
Copied to clipboard
Yiming Huang, Zhenghao Lin, Xiao Liu, Yeyun Gong, Shuai Lu, Fangyu Lei, Yaobo Liang, Yelong Shen, Chen Lin, Nan Duan, Weizhu Chen
| Challenge: | Large language models (LLMs) have demonstrated impressive reasoning capabilities, yet there is ongoing debate about their capabilities and the potential data contamination problem. |
| Approach: | They propose to evaluate the reasoning capabilities of large language models in solving recent competition-level programming problems in Codeforces. |
| Outcome: | The proposed model has experienced a cliff-like decline in problems after September 2021, which shows the potential data contamination and the challenges for any existing LLM to solve unseen complex reasoning problems. |
Can Language Model Moderators Improve the Health of Online Discourse? (2024.naacl-long)
Copied to clipboard
Hyundong Cho, Shuai Liu, Taiwei Shi, Darpan Jain, Basem Rizk, Yuyang Huang, Zixun Lu, Nuan Wen, Jonathan Gratch, Emilio Ferrara, Jonathan May
| Challenge: | Existing efforts to automate conversational moderation have focused on banning harmful comments or deleting them, but such efforts can inadvertently push users towards echo chambers that exacerbate polarization. |
| Approach: | They propose a framework to assess models’ moderation capabilities independently of human intervention and propose 'conversational moderation' they propose to use language models as conversational moderators to provide specific feedback on toxic behavior but struggle to influence users to increase their levels of respect and cooperation. |
| Outcome: | The proposed framework assesses models’ moderation capabilities independently of human intervention and shows that appropriately prompted models provide specific and fair feedback on toxic behavior but struggle to influence users to increase their levels of respect and cooperation. |
Generative Prompt Tuning for Relation Classification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing prompt tuning methods for RC are limited by label spaces and rigid prompt restrictions. |
| Approach: | They propose a generative prompt tuning method to reformulate relation classification as an infilling problem by adding cloze-style phrases to masked language modeling problems. |
| Outcome: | The proposed method exploits rich semantics of entity and relation types and can predict label verbalizations with varying lengths at multiple predicted positions. |