Papers by Dayiheng Liu

Bridging Subword Gaps in Pretrain-Finetune Paradigm for Natural Language Generation (2021.acl-long)

Challenge:	Text infilling is an under-explored challenge in the field of text generation.
Approach:	They propose an iterative inference algorithm based on gradient search that can be broadly applied to any sequence generative model for text infilling tasks.
Outcome:	The proposed method performs well on three different text infilling tasks with different mask ratios and mask strategies compared with five state-of-the-art methods.

Copied to clipboard

Xin Liu, Baosong Yang, Dayiheng Liu, Haibo Zhang, Weihua Luo, Min Zhang, Haiying Zhang, Jinsong Su

Noisy Pair Corrector for Dense Retrieval (2023.findings-emnlp)

Challenge:	Existing methods to pretrain language models are limited by one-size-fits-all vocabulary . embeddings of mismatch tokens can be efficiently initialized in downstream tasks .
Approach:	They propose to extend pretrain-finetune pipeline with an embedding transfer step . plug-and-play embeddable generator is introduced to generate any input token .
Outcome:	The proposed approach allows for more efficient and better performed NLG models.

Copied to clipboard

Hang Zhang, Yeyun Gong, Xingwei He, Dayiheng Liu, Daya Guo, Jiancheng Lv, Jian Guo

Information Retrieval

Challenge:	Existing dense retrieval models assume that query-document pairs are exactly matched, resulting in mismatched-pair noise.
Approach:	They propose a novel approach to train an effective model with mismatched-pair noise.
Outcome:	The proposed model performs well on natural question and triviaQA, code-search benchmarks and SO-DS.

NOVA-63: Native Omni-lingual Versatile Assessments of 63 Disciplines (2025.emnlp-main)

Copied to clipboard

Jinyang Zhang, Kexin Yang, Yu Wan, Muyang Ye, Baosong Yang, Fei Huang, Junyang Lin, Dayiheng Liu

Attention Mechanism with Energy-Friendly Operations (2022.findings-acl)

Challenge:	Existing multilingual benchmarks show severe drawbacks, such as overly translated content, the absence of difficulty control, and disciplinary imbalance, making the benchmarking process unreliable and showing low convincingness.
Approach:	They propose a multilingual benchmark that integrates LLM-assisted formatting, expert quality verification, and multi-level difficulty screening to provide a comprehensive, difficult multilingual assessment.
Outcome:	The proposed benchmark features 93,536 questions sourced from native speakers across 14 languages and 63 academic disciplines.

Copied to clipboard

Yu Wan, Baosong Yang, Dayiheng Liu, Rong Xiao, Derek Wong, Haibo Zhang, Boxing Chen, Lidia Chao

Challenge:	Empirical results show that attention mechanism can be improved from the energy consumption aspects.
Approach:	They propose to replace multiplications with either selective operations or additions to reduce energy consumption.
Outcome:	The proposed model achieves competitable accuracy while saving 99% and 66% energy during alignment calculation and the whole attention procedure.

LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedback (2025.findings-acl)

Copied to clipboard

Bofei Gao, Zefan Cai, Runxin Xu, Peiyi Wang, Ce Zheng, Runji Lin, Keming Lu, Dayiheng Liu, Chang Zhou, Wen Xiao, Tianyu Liu, Baobao Chang

ProphetNet: Predicting Future N-gram for Sequence-to-SequencePre-training (2020.findings-emnlp)

Challenge:	Existing mathematical verifiers are trained with binary classification labels, which are not informative enough for the model to accurately assess the solutions.
Approach:	They propose a natural language feedback-enhanced verifier that can validate the correctness of response generated by policy models by constructing automatically generated training data and a two-stage training paradigm.
Outcome:	The proposed verifier significantly improves in verification and reinforcement learning and alleviates data-demanding problems of the reward model.

Copied to clipboard

Weizhen Qi, Yu Yan, Yeyun Gong, Dayiheng Liu, Nan Duan, Jiusheng Chen, Ruofei Zhang, Ming Zhou

ProcessBench: Identifying Process Errors in Mathematical Reasoning (2025.acl-long)

Challenge:	Existing sequence-to-sequence models are optimized for future n-gram prediction and n stream self-attention mechanism.
Approach:	They propose a self-supervised objective called future n-gram prediction and the proposed n stream self-attention mechanism to optimize the model for sequence-to-sequence learning.
Outcome:	The proposed model achieves state-of-the-art on CNN/DailyMail, Gigaword, and SQuAD 1.1 benchmarks compared to the models using the same scale pre-training corpus.

Copied to clipboard

Chujie Zheng, Zhenru Zhang, Beichen Zhang, Runji Lin, Keming Lu, Bowen Yu, Dayiheng Liu, Jingren Zhou, Junyang Lin

Rationales for Answers to Simple Math Word Problems Confuse Large Language Models (2024.findings-acl)

Challenge:	Existing models fail to generalize to more challenging math problems, authors say . existing benchmarks related to assessing language models' reasoning process are limited .
Approach:	They propose a tool to measure language models' ability to identify erroneous steps in reasoning . they use two types of models: process reward models and critic models .
Outcome:	The proposed model outperforms existing models in evaluating language models' reasoning process . the best open-source model has demonstrated the critique capability competitive with the proprietary model .

Copied to clipboard

Yidan Zhang, Mingfeng Xue, Dayiheng Liu, Zhenan He

Unifying Discrete and Continuous Representations for Unsupervised Paraphrase Generation (2023.emnlp-main)

Challenge:	Recent studies show that large language models have advanced mathematical problem-solving abilities in grade school math word problems.
Approach:	They propose to combine fine-tuning and prompt-based methods to improve performance . they propose to use a hybrid algorithm to fine- tune LLMs on specific tasks .
Outcome:	The proposed methods improve performance on the proposed reasoning process evaluation benchmarks.

Copied to clipboard

Mingfeng Xue, Dayiheng Liu, Wenqiang Lei, Jie Fu, Jian Lan, Mei Li, Baosong Yang, Jun Xie, Yidan Zhang, Dezhong Peng, Jiancheng Lv

Dangling-Aware Entity Alignment with Mixed High-Order Proximities (2022.findings-naacl)

Challenge:	Existing unsupervised paraphrase generation methods require large-scale, manually annotated paraphrase datasets, which are labor-intensive to build.
Approach:	They propose a self-supervised pseudo-data construction method that generates diverse pseudo-paraphrases in distinct surface structures for a given sentence.
Outcome:	The proposed method generates diverse pseudo-paraphrases in distinct surface structures for a given sentence.

Copied to clipboard

Juncheng Liu, Zequn Sun, Bryan Hooi, Yiwei Wang, Dayiheng Liu, Baosong Yang, Xiaokui Xiao, Muhao Chen

Structured Data in NLP

Challenge:	Existing methods for dangling-aware entity alignment are underexplored but important problem.
Approach:	They propose a framework that uses high-order proximities to detect dangling entities and align matchable entities.
Outcome:	The proposed framework detects dangling entities and aligns matchable entities better than existing methods.

Should We Rely on Entity Mentions for Relation Extraction? Debiasing Relation Extraction with Counterfactual Analysis (2022.naacl-main)

Copied to clipboard

Yiwei Wang, Muhao Chen, Wenxuan Zhou, Yujun Cai, Yuxuan Liang, Dayiheng Liu, Baosong Yang, Juncheng Liu, Bryan Hooi

Information Extraction & Text Mining

Challenge:	Existing studies rely on entity information for sentence-level relation extraction (RE) but this can leak superficial and spurious clues of relations.
Approach:	They propose to use entity mentions to extract relations from textual context . they use a causal graph to model dependencies between variables in RE models .
Outcome:	The proposed method yields significant gains on both effectiveness and generalization for RE.

PLAWBENCH: A Rubric-Based Benchmark for Evaluating LLMs in Real-World Legal Practice (2026.acl-long)

Copied to clipboard

Yuzhen Shi, Huanghai Liu, Yiran HU, Song Gaojie, Xu Xinran, Yubo Ma, Tianyi Tang, Li Zhang, Qingjing Chen, Feng Di, Wenbo Lv, Weiheng Wu, Kexin Yang, Sen Yang, Wei Wang, Rongyao Shi, Qiu Yuanyang, Yuemeng Qi, Zhang Jingwen, Sui Xiaoyu, Yifan Chen, Zhang Yi, An Yang, Bowen Yu, Dayiheng Liu, Junyang Lin, Weixing Shen, Bing Zhao, Charles L. A. Clarke, HU Wei

Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models (2025.acl-long)

Challenge:	Existing benchmarks for large language models (LLMs) are coarse, single-dimensional metrics and do not explicitly assess fine-grained legal reasoning.
Approach:	They propose a Practical Law Benchmark to evaluate large language models in real-world legal practice scenarios.
Outcome:	The proposed model is based on 850 questions and 13 scenarios with expert-designed evaluation rubrics.

Copied to clipboard

Zihan Qiu, Zeyu Huang, Bo Zheng, Kaiyue Wen, Zekun Wang, Rui Men, Ivan Titov, Dayiheng Liu, Jingren Zhou, Junyang Lin

Programming Languages in NLP

Challenge:	Existing Mixture-of-Experts training frameworks use a micro-batch to calculate LBL . micro-batches are restricted to a single sequence, preventing expert specialization .
Approach:	They propose to use a global-batch to loosen the load balance constraint for MoEs models . they propose to synchronize fi across micro-batches and then use it to calculate the LBL .
Outcome:	The proposed global-batch LBL improves the domain specialization of experts . the micro-battery LBL is almost at the sequence level, and the router is pushed to distribute the token evenly .

Mask Attention Networks: Rethinking and Strengthen Transformer (2021.naacl-main)

Copied to clipboard

Zhihao Fan, Yeyun Gong, Dayiheng Liu, Zhongyu Wei, Siyuan Wang, Jian Jiao, Nan Duan, Ruofei Zhang, Xuanjing Huang

Unsupervised Preference-Aware Language Identification (2022.findings-acl)

Challenge:	Existing research explores to enhance the two sublayers separately to improve the capability of Transformer for text representation.
Approach:	They propose to combine SAN and Feed-Forward Networks to create a dynamic mask attention network with a learnable mask matrix which can model localness adaptively.
Outcome:	The proposed model outperforms the original Transformer on translation and text summarization tasks.

Copied to clipboard

Xingzhang Ren, Baosong Yang, Dayiheng Liu, Haibo Zhang, Xiaoyu Lv, Liang Yao, Jun Xie

Information Extraction & Text Mining

Challenge:	Existing studies do not consider inter-personal variations due to the lack of user annotated training data.
Approach:	They propose to use user preferences to identify ambiguous texts in multilingual applications without user annotated training data to build a preference-aware LID model.
Outcome:	The proposed model significantly outperforms existing LID systems on handling ambiguous texts.

START: Self-taught Reasoner with Tools (2025.emnlp-main)

Copied to clipboard

Chengpeng Li, Mingfeng Xue, Zhenru Zhang, Jiaxi Yang, Beichen Zhang, Bowen Yu, Binyuan Hui, Junyang Lin, Xiang Wang, Dayiheng Liu

GCPG: A General Framework for Controllable Paraphrase Generation (2022.findings-acl)

Challenge:	Large Reasoning Models (LRMs) have demonstrated remarkable capabilities in complex reasoning through long chain-of-thought, yet they struggle with precise computations and algorithmic operations.
Approach:	They propose a training-free approach that activates LRMs’ latent tool-use capabilities through artificial hints and a framework that enables models to learn effective tool utilization through diverse hint patterns and rejection-based data synthesis.
Outcome:	Experiments show that START significantly improves state-of-the-art LRMs across challenging benchmarks, including competition-level mathematics (AMC23: 95.0%, AIME24: 75.6%) and graduate-level science questions (GPQA: 64.6%).

Copied to clipboard

Kexin Yang, Dayiheng Liu, Wenqiang Lei, Baosong Yang, Haibo Zhang, Xue Zhao, Wenqing Yao, Boxing Chen

Controllable LLM Reasoning via Sparse Autoencoder-Based Steering (2026.acl-long)

Challenge:	Existing studies highlight a special condition under two indispensable aspects of controllable paraphrase generation (CPG) individually, lacking a unified circumstance to explore and analyze their effectiveness.
Approach:	They propose a general controllable paraphrase generation framework that integrates lexical and syntactical conditions into a text sequence and uniformly processes them in an encoder-decoder paradigm.
Outcome:	The proposed framework can combine lexical and syntactical conditions and improve paraphrase generation.

Copied to clipboard

Yi Fang, Wenjie Wang, Mingfeng Xue, Boyi Deng, Fengli Xu, Dayiheng Liu, Fuli Feng

P-MMEval: A Parallel Multilingual Multitask Benchmark for Consistent Evaluation of LLMs (2025.emnlp-main)

Challenge:	Existing methods struggle to control fine-grained reasoning strategies due to conceptual entanglement in LRMs’ hidden states.
Approach:	They propose to decompose strategy-entangled hidden states into a disentangled feature space by using Sparse Autoencoders to identify the few strategy-specific features from the vast pool of SAE features.
Outcome:	The proposed method outperforms existing methods by 15% in control effectiveness.

Copied to clipboard

Yidan Zhang, Yu Wan, Boyi Deng, Baosong Yang, Hao-Ran Wei, Fei Huang, Bowen Yu, Dayiheng Liu, Junyang Lin, Fei Huang, Jingren Zhou

The Lessons of Developing Process Reward Models in Mathematical Reasoning (2025.findings-acl)

Challenge:	Recent advances in large language models showcase varied multilingual capabilities across tasks . previous assessments focused on fundamental natural language processing (NLP) or isolated capability-specific tasks.
Approach:	They propose a multilingual multitask benchmark to assess multilingual capabilities . they use a large-scale benchmark covering fundamental and capability-specialized datasets .
Outcome:	The proposed benchmark compares models and tasks across languages and tasks and examines knowledge transfer from English to other languages.

Copied to clipboard

Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, Junyang Lin

GLGE: A New General Language Generation Evaluation Benchmark (2021.findings-acl)

Challenge:	a recent study shows that process reward models can make mistakes, leading to wrong conclusions.
Approach:	They propose a consensus filtering mechanism that integrates MC estimation with LLM-as-a-judge to improve model performance and data efficiency.
Outcome:	The proposed model outperforms existing open-source alternatives and provides practical guidelines for future research.

Copied to clipboard

Dayiheng Liu, Yu Yan, Yeyun Gong, Weizhen Qi, Hang Zhang, Jian Jiao, Weizhu Chen, Jie Fu, Linjun Shou, Ming Gong, Pengcheng Wang, Jiusheng Chen, Daxin Jiang, Jiancheng Lv, Ruofei Zhang, Winnie Wu, Ming Zhou, Nan Duan

How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition (2024.acl-long)

Challenge:	Multi-task benchmarks focus on a range of Natural Language Understanding (NLU) tasks without considering the Natural Language Generation (NLG) models.
Approach:	They propose a multi-task benchmark for evaluating the generalization capabilities of NLG models across eight language generation tasks.
Outcome:	The proposed benchmarks are based on GLUE and Su-perGLUE for English and several other languages.

Copied to clipboard

Guanting Dong, Hongyi Yuan, Keming Lu, Chengpeng Li, Mingfeng Xue, Dayiheng Liu, Wei Wang, Zheng Yuan, Chang Zhou, Jingren Zhou

RikiNet: Reading Wikipedia Pages for Natural Question Answering (2020.acl-main)

Challenge:	supervised fine-tuning (SFT) is a technique used to enhance multiple abilities in large language models.
Approach:	They propose to study the interplay of data composition between mathematical reasoning, code generation, and general human-aligning abilities during supervised fine-tuning.
Outcome:	The proposed model improves math reasoning and code generation with increasing data amount . the proposed model size and SFT strategies can be used to learn multiple skills with different scaling patterns.

Copied to clipboard

Dayiheng Liu, Yeyun Gong, Jie Fu, Yu Yan, Jiusheng Chen, Daxin Jiang, Jiancheng Lv, Nan Duan

Natural Language Interfaces

Challenge:	Using Wikipedia pages to answer open-domain questions remains challenging in natural language understanding.
Approach:	They propose a model which reads Wikipedia pages for natural question answering . it uses a dynamic paragraph dual-attention reader and a cascaded answer predictor .
Outcome:	The proposed model outperforms the human model on the Natural Questions dataset . it achieves 74.3 F1 and 57.9 F1 on long-answer and short-answer tasks .

Tailor: A Soft-Prompt-Based Approach to Attribute-Based Controlled Text Generation (2023.acl-long)

Copied to clipboard

Kexin Yang, Dayiheng Liu, Wenqiang Lei, Baosong Yang, Mingfeng Xue, Boxing Chen, Jun Xie

UniTE: Unified Translation Evaluation (2022.acl-long)

Challenge:	Existing work focuses on generating sentences satisfying pre-specified attributes such as topic and sentiment, yet suffers from increases in storage and inference time.
Approach:	They propose a method that uses a pre-trained continuous vector to generate a fixed pre-trainable language model to satisfy a specified attribute.
Outcome:	The proposed model can achieve improvements on eleven attribute-specific generation tasks with 0.08% extra training parameters.

Copied to clipboard

Yu Wan, Dayiheng Liu, Baosong Yang, Haibo Zhang, Boxing Chen, Derek Wong, Lidia Chao

Bridging the Domain Gaps in Context Representations for k-Nearest Neighbor Neural Machine Translation (2023.acl-long)

Challenge:	Recent methods for evaluation of translation quality are focused on one task, ignoring commonalities .
Approach:	They propose a unified framework engaged with abilities to handle all three evaluation tasks.
Outcome:	The proposed framework can universally surpass state-of-the-art or winner methods across tasks.

Copied to clipboard

Zhiwei Cao, Baosong Yang, Huan Lin, Suhang Wu, Xiangpeng Wei, Dayiheng Liu, Jun Xie, Min Zhang, Jinsong Su

Machine Translation

Challenge:	Existing methods to improve k-Nearest neighbor machine translation (kNN-MT) are based on the ability to non-parametrically adapt to new domains.
Approach:	They propose a method to boost the datastore retrieval of k-Nearest neighbor machine translation by reconstructing the original datastore.
Outcome:	The proposed method boosts the retrieval and translation quality of k-Nearest neighbor machine translation by reconstructing the original datastore.

HellaSwag-Pro: A Large-Scale Bilingual Benchmark for Evaluating the Robustness of LLMs in Commonsense Reasoning (2025.findings-acl)

Copied to clipboard

Xiaoyuan Li, Moxin Li, Rui Men, Yichang Zhang, Keqin Bao, Wenjie Wang, Fuli Feng, Dayiheng Liu, Junyang Lin

Robustness in NLP

Challenge:	Existing studies show that large language models are robust in commonsense reasoning . however, some variations in questions can lead to incorrect responses .
Approach:	They propose a large-scale bilingual benchmark consisting of 11,200 cases . they conduct extensive experiments on 41 representative LLMs .
Outcome:	The proposed benchmark systematically evaluates the robustness of large language models in commonsense reasoning.

MTR-Bench: A Comprehensive Benchmark for Multi-Turn Reasoning Evaluation (2026.acl-long)

Copied to clipboard

Xiaoyuan Li, Keqin Bao, Yubo Ma, Moxin Li, Wenjie Wang, Rui Men, Yichang Zhang, Fuli Feng, Dayiheng Liu

Knowledge Enhanced Pre-training for Cross-lingual Dense Retrieval (2024.lrec-main)

Challenge:	Recent advances in Large Language Models (LLMs) have shown promising results in complex reasoning tasks.
Approach:	They propose to use a multi-turn reasoning evaluation framework to cover multi-turn interactions with the environments of large language models.
Outcome:	The proposed framework covers diverse reasoning capabilities, fine-grained difficulty granularity, and necessitates multi-turn interactions with the environments.

Copied to clipboard

Hang Zhang, Yeyun Gong, Dayiheng Liu, Shunyu Zhang, Xingwei He, Jiancheng Lv, Jian Guo

MAPO: Boosting Large Language Model Performance with Model-Adaptive Prompt Optimization (2023.findings-emnlp)

Challenge:	Existing mPLMs neglect the importance of knowledge in cross-lingual dense retrieval.
Approach:	They propose a novel mPLM that leverages knowledge to learn language-agnostic semantic representations from a multilingual knowledge base and an annotation of Wiki.
Outcome:	The proposed model achieves strong multilingual and cross-lingual retrieval performance with significant improvements over existing mPLMs.

Copied to clipboard

Yuyan Chen, Zhihao Wen, Ge Fan, Zhengyu Chen, Wei Wu, Dayiheng Liu, Zhixu Li, Bang Liu, Yanghua Xiao

Bridging the Gap between Training and Inference: Multi-Candidate Optimization for Diverse Neural Machine Translation (2022.findings-naacl)

Challenge:	Existing research emphasizes the importance of adapting prompts to specific tasks, rather than specific LLMs.
Approach:	They propose a model-adaptive prompt optimizer method that optimizes original prompts for each LLM in downstream tasks.
Outcome:	The proposed method can optimize prompts for an LLM in downstream tasks.

Copied to clipboard

Huan Lin, Baosong Yang, Liang Yao, Dayiheng Liu, Haibo Zhang, Jun Xie, Min Zhang, Jinsong Su

Dynamic Voting for Efficient Reasoning in Large Language Models (2023.findings-emnlp)

Challenge:	Existing diverse NMT models lack translation diversity due to a discrepancy between training and inference . despite the success of diverse NTM, there is still a lack of translation diversity .
Approach:	They propose a multi-candidate optimization framework for diverse NMT to deal with this defect.
Outcome:	The proposed framework is transparent to basic diverse NMT models, and universally makes better trade-off between diversity and quality.

Copied to clipboard

Mingfeng Xue, Dayiheng Liu, Wenqiang Lei, Xingzhang Ren, Baosong Yang, Jun Xie, Yidan Zhang, Dezhong Peng, Jiancheng Lv

Responsible & Trustworthy NLP

Challenge:	Multi-path voting methods generate multiple reasoning paths for each problem, causing factual errors and illusion generation.
Approach:	They propose a multi-path voting technique that effectively reduces the number of reasoning paths during multi-path voting while preserving accuracies.
Outcome:	The proposed method outperforms Self-consistency using 24.7% of the number of paths on the LetterConcat task.

Competency-Aware Neural Machine Translation: Can Machine Translation Know its Own Translation Quality? (2022.emnlp-main)

Copied to clipboard

Pei Zhang, Baosong Yang, Hao-Ran Wei, Dayiheng Liu, Kai Fan, Luo Si, Jun Xie

Machine Translation

Challenge:	Neural machine translation models are often criticized for failures that happen without competency awareness.
Approach:	They propose a method that extends conventional NMT with a self-estimator to translate a source sentence and estimate its competency.
Outcome:	The proposed method performs on translation tasks intact and on quality estimation tasks better than existing methods.

Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation (2020.emnlp-main)

Copied to clipboard

Dayiheng Liu, Yeyun Gong, Yu Yan, Jie Fu, Bo Shao, Daxin Jiang, Jiancheng Lv, Nan Duan

HSS-Synth: Humanities and Social Sciences Data Synthesis for LLMs (2026.findings-acl)

Challenge:	Existing methods for news headline generation focus on producing a single short sentence . et al., 2017; Gehrmann e.t., 2018; Zhong ee., 2019) focus on single-headline generation.
Approach:	They propose a method to generate multiple headlines with keyphrases of user interests . they propose generating multiple keyphrase-relevant headlines using a transformer decoder .
Outcome:	The proposed method achieves state-of-the-art in terms of quality and diversity.

Copied to clipboard

Ru Peng, Tianyu Zhao, Xijun Gu, Zhiting Fan, Haokai Xu, Jinyang Zhang, Yawen Zeng, Yihong Zhuang, Kexin Yang, Junyang Lin, Dayiheng Liu, Junbo Zhao

Challenge:	High-quality, diverse data are vital for large language models (LLMs) but remain scarce and costly.
Approach:	They define the first HSS domain system covering 14 mainstream fields and introduce HSS-Synth.
Outcome:	the proposed pipeline outperforms 14 leading baselines on 16 benchmarks.

POS-Constrained Parallel Decoding for Non-autoregressive Generation (2021.acl-long)

Copied to clipboard

Kexin Yang, Wenqiang Lei, Dayiheng Liu, Weizhen Qi, Jiancheng Lv

Self-supervised Product Title Rewrite for Product Listing Ads (2022.naacl-industry)

Challenge:	Existing non-autoregressive generation systems face multimodality problem due to conditionally independent decoding.
Approach:	They propose to incorporate linguistic structure into NAG inference instead of teacher AG . they propose a method that provides a specific POS sequence to constrain the NAG model .
Outcome:	The proposed method improves NAG models on four text generation tasks to a greater extent compared to knowledge distillation.

Copied to clipboard

Xue Zhao, Dayiheng Liu, Junwei Ding, Liang Yao, Mahone Yan, Huibo Wang, Wenqing Yao

Responsible & Trustworthy NLP

Challenge:	Existing work has investigated the title optimization for Product Listing Ads (PLAs) however, little work has examined the effectiveness of this method.
Approach:	They propose a method to rewrite product listing ads titles without considering the fluency and information priority.
Outcome:	The proposed solution reduces the cost and improves CTR in the offline test and real-world online test by a large amount.

Towards User-Driven Neural Machine Translation (2021.acl-long)

Copied to clipboard

Huan Lin, Liang Yao, Baosong Yang, Dayiheng Liu, Haibo Zhang, Weihua Luo, Degen Huang, Jinsong Su

Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space (2020.emnlp-main)

Challenge:	a good translation should implicitly mirror user traits rather than translate the original content semantically.
Approach:	They propose a framework that captures user traits from historical inputs . they propose 'user-driven' NMT to model user behavior under a zero-shot learning fashion .
Outcome:	The proposed framework can capture user traits from historical inputs under zero-shot learning fashion.

Copied to clipboard

Dayiheng Liu, Yeyun Gong, Jie Fu, Yu Yan, Jiusheng Chen, Jiancheng Lv, Nan Duan, Ming Zhou