Papers by Lei Shu
DomBERT: Domain-oriented Language Model for Aspect-based Sentiment Analysis (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that learning domain-specific language models are equally important for general-purpose and domain-based learning. |
| Approach: | They propose a domain-oriented learning task that combine the benefits of both general and domain-specific worlds. |
| Outcome: | The proposed task solves the problems in an aspect-based sentiment analysis task. |
Controllable Text Generation with Focused Variation (2020.findings-emnlp)
Copied to clipboard
Lei Shu, Alexandros Papangelis, Yi-Chia Wang, Gokhan Tur, Hu Xu, Zhaleh Feizollahi, Bing Liu, Piero Molino
| Challenge: | Focused-Variation Network (FVN) is a new model to control language generation. |
| Approach: | They propose a model that learns discrete latent spaces for each attribute inside codebooks and uses them to generate fluent text. |
| Outcome: | The proposed model can generate fluent and mostly coherent text on two text generation datasets with annotated content and style, and show state-of-the-art performance as assessed by automatic and human evaluations. |
Explore the Reasoning Capability of LLMs in the Chess Testbed (2025.naacl-short)
Copied to clipboard
| Challenge: | a recent study shows that large language models struggle with long-term, complex reasoning tasks. |
| Approach: | They propose to integrate annotated strategy and tactic into large language models to improve reasoning capability. |
| Outcome: | The proposed model performs better than GPT, Claude, and Gemini models . it integrates annotated strategy and tactic into the model . |
Continual Training of Language Models for Few-Shot Learning (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on applying large language models (LMs) achieves impressive performance in many NLP applications. |
| Approach: | They propose to continuously post-train an LM with unlabeled domains to expand its knowledge without forgetting previous skills. |
| Outcome: | The proposed system improves few-shot end-task learning in these domains. |
RPC-Bench: A Fine-grained Benchmark for Research Paper Comprehension (2026.acl-long)
Copied to clipboard
Yelin Chen, Fanjin Zhang, Suping Sun, Yunhe Pang, Yuanchun Wang, Jian Song, XiaoYan Li, Lei Hou, Shu Zhao, Jie Tang, Juanzi Li
| Challenge: | Existing benchmarks for understanding research papers offer limited fine-grained evaluation at scale. |
| Approach: | They propose a large-scale question-answering benchmark built from review–rebuttal exchanges of high-quality computer science papers. |
| Outcome: | The proposed model is based on human-verified QA pairs and contains 15K questions. |
BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis (N19-1)
Copied to clipboard
| Challenge: | Existing work on question-answering has limited training examples for RRC . question-announced questions are a key component of online commerce . |
| Approach: | They propose to turn customer reviews into a large source of knowledge that can be exploited to answer user questions. |
| Outcome: | The proposed approach improves review reading comprehension on popular language model BERT . it also improves aspect extraction and aspect sentiment classification tasks . |
CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have focused on continual learning of aspect sentiment classification (ASC) tasks in domain incremental learning (DIL) |
| Approach: | They propose a continual learning method that learns a sequence of tasks incrementally . they propose CLASSIC, which uses a domain incremental learning setting . |
| Outcome: | The proposed model is highly effective in a domain incremental learning setting. |
Towards an On-device Agent for Text Rewriting (2024.findings-naacl)
Copied to clipboard
Yun Zhu, Yinxiao Liu, Felix Stahlberg, Shankar Kumar, Yu-Hui Chen, Liangchen Luo, Lei Shu, Renjie Liu, Jindong Chen, Lei Meng
| Challenge: | Large Language Models (LLMs) have demonstrated impressive capabilities for text rewriting, however creating a smaller yet potent language model presents two formidable challenges: costly data collection and absence of emergent capabilities. |
| Approach: | They propose a new instruction tuning method to develop a mo-bile text rewriting model that leverages LLM-generated data and heuristic reinforcement learning, eliminating the need for human data collection. |
| Outcome: | The proposed model surpasses the current state-of-the-art LLMs in text rewriting while maintaining a significantly reduced model size using public benchmark EditEval and our new benchmark. |
Double Embeddings and CNN-based Sequence Labeling for Aspect Extraction (P18-2)
Copied to clipboard
| Challenge: | Recent supervised deep learning models have achieved state-of-the-art performance, but there are two other considerations that are important. |
| Approach: | They propose a supervised aspect extraction model using general-purpose embeddings and domain-specific embeddables. |
| Outcome: | The proposed model outperforms state-of-the-art methods without supervision and achieves very good results. |
Fusion-Eval: Integrating Assistant Evaluators with LLMs (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Recent studies have employed large language models (LLMs) as reference-free metrics for NLG evaluation, enhancing adaptability to new tasks tasks. |
| Approach: | They propose a method that leverages large language models to integrate insights from various assistant evaluators. |
| Outcome: | The proposed approach achieves a 0.962 system-level Kendall-Tau correlation with humans on SummEval and a 0.7444 turn-level Spearman correlation on TopicalChat, which is significantly higher than baseline methods. |
Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System (2022.acl-long)
Copied to clipboard
| Challenge: | Existing pre-trained language models often form a cascaded generation problem . this can lead to error accumulation across different sub-tasks and greater data annotation overhead. |
| Approach: | They propose a plug-and-play model for task-oriented dialogue that learns primary TOD task completion skills from heterogeneous dialog corpora. |
| Outcome: | The proposed model learns primary TOD task completion skills from heterogeneous dialog corpora. |
TaCL: Improving BERT Pre-training with Token-aware Contrastive Learning (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing pre-trained MLMs produce an anisotropic distribution of token representations . this is not ideal for tasks that require discriminative semantic meanings of distinct tokens - a problem that exists in pre-training models . |
| Approach: | They propose a continual pre-training approach that encourages BERT to learn an isotropic distribution of token representations. |
| Outcome: | The proposed approach improves on a wide range of English and Chinese benchmarks. |
An Efficient Conversational Smart Compose System (2023.acl-demo)
Copied to clipboard
Yun Zhu, Xiayu Chen, Lei Shu, Bowen Tan, Xinying Song, Lijuan Liu, Maria Wang, Jindong Chen, Ning Ruan
| Challenge: | a cloud-based smart compose system is designed to improve human-to-human conversation efficiency. |
| Approach: | They propose a cloud-based smart compose system to improve conversation efficiency . they propose heuristics to achieve the best trade-off between quality and latency . |
| Outcome: | The proposed system reduces latency without losing composing quality further. |
ODIST: Open World Classification via Distributionally Shifted Instances (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work to achieve open-world classification capability in natural language processing and computer vision focuses on decision boundary finding. |
| Approach: | They propose a method that can create out-of-domain instances from in-domain training instances with the help of a pre-trained generative language model. |
| Outcome: | The proposed method can create out-of-domain instances from the in-domain training instances with the help of a pre-trained generative language model. |
Modeling Multi-Action Policy for Task-Oriented Dialogues (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to learn dialogue management only predict one action per turn, limiting expressive power of the conversational agent and introducing unnecessary turns of interactions. |
| Approach: | They propose a model based on a recurrent cell called gated Continue-Act-Slots that overcomes the limitations of existing models and proposes a novel policy model that predicts multiple acts for each turn. |
| Outcome: | The proposed model outperforms existing models on the task of predicting multiple acts for each turn. |
Understanding Pre-trained BERT for Aspect-based Sentiment Analysis (2020.coling-main)
Copied to clipboard
| Challenge: | Recent studies show impressive results on aspects-based sentiment analysis tasks. |
| Approach: | They analyze the attentions and learned representations of BERT for aspects-based sentiment analysis tasks. |
| Outcome: | The proposed model can be used for aspects-based sentiment analysis (ABSA) but it is not clear how it can provide important features for downstream tasks. |
Adapting a Language Model While Preserving its General Knowledge (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing DA-training methods do not explicitly identify what knowledge should be preserved and what should be changed by the domain corpus. |
| Approach: | They propose to use an unlabeled corpus of aparticular domain to train a pre-trained general-purpose language model to adapt the LM so that end-tasks in the domain can give improved performances. |
| Outcome: | The proposed method improves the performance of pre-trained general-purpose language models by contrasting the representations of the general and the full (both general and domain knowledge) to learn an integrated representation with both general and specific knowledge. |
Enhancing Reinforcement Learning with Dense Rewards from Language Model Critic (2024.emnlp-main)
Copied to clipboard
| Challenge: | Reinforcement learning (RL) can align language models with non-differentiable reward signals, such as human preferences, but the sparsity of these signals can lead to inefficient and unstable learning. |
| Approach: | They propose a framework that utilizes the critique capability of Large Language Models to produce intermediate-step rewards during RL training. |
| Outcome: | The proposed framework improves sample efficiency and the overall performance of the policy model, supported by both automatic and human evaluation. |
Dolphin: Document Image Parsing via Heterogeneous Anchor Prompting (2025.findings-acl)
Copied to clipboard
Hao Feng, Shu Wei, Xiang Fei, Wei Shi, Yingdong Han, Lei Liao, Jinghui Lu, Binghong Wu, Qi Liu, Chunhui Lin, Jingqun Tang, Hao Liu, Can Huang
| Challenge: | Current document image parsing solutions rely on specialized models or generate content autoregressively. |
| Approach: | They propose a multimodal document image parsing model that integrates specialized models with autogeneous content generation. |
| Outcome: | The proposed model achieves state-of-the-art performance across diverse page-level and element-level settings while ensuring superior efficiency. |