SEQ2SEQ

What we need to learn if we want to do and not just talk (N18-3)

Challenge:	Typical datasets used for style transfer in NLP contain aligned pairs of two opposite extremes of a style.
Approach:	They propose a technique to derive a dataset of aligned pairs from an unlabeled corpus by using an auxiliary dataset, allowing for in-domain training.
Outcome:	The proposed method significantly outperforms OpenNMT’s Seq2Seq model trained on the Yahoo Formality Dataset and 6 novel datasets.

Copied to clipboard

Rashmi Gangadharaiah, Balakrishnan Narayanaswamy, Charles Elkan

Forecasting Firm Material Events from 8-K Reports (D19-51)

Challenge:	Existing methods for task-oriented dialogs require fluent natural language responses and correct external actions . but they perform poorly in real world dialog tasks, a new study shows .
Approach:	They propose a hybrid model where nearest neighbor is used to generate fluent responses and Seq2Seq type models ensure dialogue coherency and generate accurate external actions.
Outcome:	The proposed model achieves a 78% relative improvement in fluency and 200% improvement in accuracy of external calls.

Copied to clipboard

Shuang (Sophie) Zhai, Zhu (Drew) Zhang

Challenge:	In this paper, we show deep learning models can be used to forecast firm material event sequences based on the contents of the company’s 8-K Current Reports.
Approach:	They exploit state-of-the-art neural architectures, including sequence-to-sequence architecture and attention mechanisms, to build a deep learning model that can forecast firm material event sequences based on company 8-K Current Reports.
Outcome:	The proposed model can forecast firm material event sequences based on the contents of the firm's 8-K Current Reports.

Mention Flags (MF): Constraining Transformer-based Text Generators (2021.acl-long)

Copied to clipboard

Yufei Wang, Ian Wood, Stephen Wan, Mark Dras, Mark Johnson

Entity-Consistent End-to-end Task-Oriented Dialogue System with KB Retriever (D19-1)

Challenge:	Constrained decoding algorithms produce hypotheses satisfying all constraints, but they are computationally expensive and can lower the generated text quality.
Approach:	They propose a Mention Flag mechanism which traces whether lexical constraints are satisfied in outputs of an S2S decoder.
Outcome:	The proposed models maintain higher constraint satisfaction and text quality than baseline models and other constrained decoding algorithms.

Copied to clipboard

Libo Qin, Yijia Liu, Wanxiang Che, Haoyang Wen, Yangming Li, Ting Liu

Dialogue Meaning Representation for Task-Oriented Dialogue Systems (2022.findings-emnlp)

Challenge:	Existing work on sequence-to-sequence dialogues treats the KB query as an attention over the entire KB without the guarantee that the generated entities are consistent with each other.
Approach:	They propose a framework which queries the knowledge base in two steps to improve consistency . they first return the most relevant KB row given a dialogue history .
Outcome:	The proposed framework outperforms baseline models and produces entity-consistent responses.

Copied to clipboard

Xiangkun Hu, Junqi Dai, Hang Yan, Yi Zhang, Qipeng Guo, Xipeng Qiu, Zheng Zhang

Minimize Exposure Bias of Seq2Seq Models in Joint Entity and Relation Extraction (2020.findings-emnlp)

Challenge:	Existing work on dialogue meaning representations is limited in scalability for complex expressions.
Approach:	They propose a pliable and easily extendable representation for task-oriented dialogue . they propose an inheritance hierarchy mechanism focusing on domain extensibility .
Outcome:	The proposed representation can be easily extended to a task-oriented dialogue dataset.

Copied to clipboard

Ranran Haoran Zhang, Qianying Liu, Aysa Xuemo Fan, Heng Ji, Daojian Zeng, Fei Cheng, Daisuke Kawahara, Sadao Kurohashi

GeoIndia: A Seq2Seq Geocoding Approach for Indian Addresses (2024.emnlp-industry)

Challenge:	Existing methods to extract relation triplets from plain text introduce exposure bias . prior work has focused on pipeline methods that ignore intrinsic interactions between subtasks and propagate classification errors through the tasks.
Approach:	They propose a model that reduces the decoding length to three within a triplet and removes the order among triplets.
Outcome:	The proposed model overfits to both datasets while showing better generalization.

Copied to clipboard

Bhavuk Singhal, Anshu Aditya, Lokesh Todwal, Shubham Jain, Debashis Mukherjee

Incorporating Causal Analysis into Diversified and Logical Response Generation (2022.coling-1)

Challenge:	a new geocoding system for Indian addresses addresses is needed for logistics, urban planning and location-based services.
Approach:	They propose a geocoding system for Indian addresses using hierarchical H3-cell prediction using a Seq2Seq framework.
Outcome:	The proposed system outperforms existing geocoding platforms in accuracy and reliability across multiple Indian states.

Copied to clipboard

Jiayi Liu, Wei Wei, Zhixuan Chu, Xing Gao, Ji Zhang, Tan Yan, Yulin Kang

Keyphrase Generation with Fine-Grained Evaluation-Guided Reinforcement Learning (2021.findings-emnlp)

Challenge:	Existing generation-based models generate generic and safe responses such as "So am I" or "I don't know"
Approach:	They propose to predict the mediators to preserve relevant information and auto-regressively incorporate the mediator into generating process.
Outcome:	The proposed model generates relevant and informative responses and outperforms the state-of-the-art in terms of automatic metrics and human evaluations.

Copied to clipboard

Yichao Luo, Yige Xu, Jiacheng Ye, Xipeng Qiu, Qi Zhang

Structured Data in NLP

Challenge:	Existing KG evaluation metrics are only aware of the exact correctness of predictions on phrase-level and ignore semantic similarities between similar predictions and targets, which inhibits the model from learning deep linguistic patterns.
Approach:	They propose a fine-grained evaluation metric to improve the previous KG framework . the evaluation metrics are only aware of the exact correctness of predictions on phrase-level .
Outcome:	The proposed method outperforms the existing frameworks among all evaluation scores.

Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting (2021.emnlp-main)

Copied to clipboard

Wangchunshu Zhou, Tao Ge, Canwen Xu, Ke Xu, Furu Wei

Learning to Rewrite Negation Queries in Product Search (2025.coling-industry)

Challenge:	Existing text infilling objectives for pretrained language models require self-supervision by masking out tokens or spans in text.
Approach:	They propose to extend text infilling to a self-supervised sequence-to-sequence (Seq2Sequen) task.
Outcome:	The proposed task improves the model's performance on various natural language generation tasks.

Copied to clipboard

Mengtian Guo, Mutasem Al-Darabsah, Choon Hui Teo, Jonathan May, Tarun Agarwal, Rahul Bhagat

From Disjoint Sets to Parallel Data to Train Seq2Seq Models for Sentiment Transfer (2020.findings-emnlp)

Challenge:	Negations in product search are often used to articulate unwanted product features or components.
Approach:	They propose a query rewriting approach to enhance product search performance . they use large language models to extract query reawrites from product text . their results pave the way for further research on enhancing search performance of queries with negations .
Outcome:	The proposed approach improves search performance by 3.17% for queries with negations.

Copied to clipboard

Paulo Cavalin, Marisa Vasconcelos, Marcelo Grave, Claudio Pinhanez, Victor Henrique Alves Ribeiro

Responsible & Trustworthy NLP

Challenge:	Existing methods for sentiment transfer have relied on unsupervised methods due to lack of parallel corpora.
Approach:	They propose a method for creating parallel data to train Seq2Seq neural networks for sentiment transfer.
Outcome:	The proposed method outperforms existing unsupervised methods in sentiment transfer tasks.

Learning Matching Models with Weak Supervision for Response Selection in Retrieval-based Chatbots (P18-2)

Copied to clipboard

Yu Wu, Wei Wu, Zhoujun Li, Ming Zhou

Information Retrieval

Challenge:	Existing methods to learn matching models for retrieval-based chatbots are lacking.
Approach:	They propose a method that uses a sequence-to-sequence architecture model as a weak annotator to judge the matching degree of unlabeled pairs and performs learning with both the weak signals and the unlabed data.
Outcome:	The proposed method improves on two public data sets on matching models on retrieval-based chatbots.

Improving Slot Filling in Spoken Language Understanding with Joint Pointer and Attention (P18-2)

Copied to clipboard

Lin Zhao, Zhe Feng

Extending Neural Generative Conversational Model using External Knowledge Sources (D18-1)

Challenge:	Experimental results show the effectiveness of our slot filling model at addressing the OOV problem.
Approach:	They propose a generative neural network model for slot filling based on a sequence-to-sequence model and a pointer network.
Outcome:	The proposed model is able to predict slot values on spoken language data.

Copied to clipboard

Prasanna Parthasarathi, Joelle Pineau

Constrained Decoding for Neural NLG from Compositional Representations in Task-Oriented Dialogue (P19-1)

Challenge:	Existing generative dialogue models lack coherence and are content poor . however, current models lack the capacity to handle large unstructured knowledge sources.
Approach:	They propose an architecture to incorporate unstructured knowledge sources to enhance the next utterance prediction in chit-chat type of generative dialogue models.
Outcome:	The proposed architecture improves the next utterance prediction in chit-chat type of generative dialogue models by incorporating external knowledge from Wikipedia summaries and the NELL knowledge base.

Copied to clipboard

Anusha Balakrishnan, Jinfeng Rao, Kartikeya Upasani, Michael White, Rajen Subba

Stronger Baselines for Grammatical Error Correction Using a Pretrained Encoder-Decoder Model (2020.aacl-main)

Challenge:	Generating fluent natural language responses from structured semantic representations is a critical step in task-oriented conversational systems.
Approach:	They propose using tree-structured semantic representations for better discourse-level structuring and sentence-level planning and introduce a challenging dataset using this representation for the weather domain.
Outcome:	The proposed model improves discourse-level structuring and sentence-level planning on a weather domain and can be decoded to improve semantic correctness.

Copied to clipboard

Satoru Katsumata, Mamoru Komachi

Generating Reasonable and Diversified Story Ending Using Sequence to Sequence Model with Adversarial Training (C18-1)

Challenge:	Strong pretraining approaches for grammatical error correction require extensive use of a pseudo-parallel corpus.
Approach:	They propose to use bidirectional and auto-regressive transformers as a generic pretrained encoder-decoder model for grammatical error correction (GEC) they find that monolingual and multilingual BART models achieve high performance in GEC, with one of the results being comparable to the current strong results in English GEC.
Outcome:	The proposed model achieves comparable results to the current strong results in English GEC.

Copied to clipboard

Zhongyang Li, Xiao Ding, Ting Liu

Sign Language Video Segmentation Using Temporal Boundary Identification (2025.acl-srw)

Challenge:	Story generation is a challenging problem in artificial intelligence (AI) . previous work focused on learning statistical models of event sequences from large-scale text corpora .
Approach:	They propose to use adversarial training to generate reasonable story endings . their model includes a generator that defines the policy of generating a story ending .
Outcome:	The proposed model achieves better performance on the task of Story Cloze Test with an accuracy of 62.6% compared with state-of-the-art baseline methods.

Copied to clipboard

Kavu Maithri Rao, Yasser Hamidullah, Eleftherios Avramidis

Debiasing Generative Named Entity Recognition by Calibrating Sequence Likelihood (2023.acl-short)

Challenge:	Sign language segmentation focuses on identifying temporal boundaries within video . previous methods have relied on frame-level and phrase-level segmentation.
Approach:	They propose to use synchronized subtitle data to facilitate temporal boundary recognition by a sequence-to-sequence model with and without attention for subtitle boundary identification.
Outcome:	The proposed model outperforms baseline models on optical flow data and aligned subtitles from BOBSL and YouTube-ASL.

Copied to clipboard

Yu Xia, Yongwei Zhao, Wenhao Wu, Sujian Li

SQL-to-Text Generation with Graph-to-Sequence Model (D18-1)

Challenge:	Existing approaches to recognize flat, overlapped and discontinuous entities uniformly have been used for Named Entity Recognition.
Approach:	They propose a reranking-based approach that redistributes the likelihood among candidate sequences depending on their performance via a contrastive loss.
Outcome:	The proposed method boosts baseline and yields competitive or better results compared with the state-of-the-art methods on 8 widely-used datasets for Named Entity Recognition.

Copied to clipboard

Kun Xu, Lingfei Wu, Zhiguo Wang, Yansong Feng, Vadim Sheinin

Autoencoder as Assistant Supervisor: Improving Text Representation for Chinese Social Media Text Summarization (P18-2)

Challenge:	Existing approaches to generate SQL-to-text using seq2seq models do not capture graph-structured information in SQL query.
Approach:	They propose a graph-to-sequence model to encode global structure information into node embeddings.
Outcome:	The proposed model outperforms the Seq2Seq and Tree2Sq baselines on the WikiSQL and Stackoverflow datasets.

Copied to clipboard

Shuming Ma, Xu Sun, Junyang Lin, Houfeng Wang

Syntax-Enhanced Neural Machine Translation with Syntax-Aware Word Representations (N19-1)

Challenge:	Existing abstractive text summarization models learn a semantic representation of the source text and the summaries from it.
Approach:	They evaluate the model on a popular Chinese social media dataset and compare it to other models.
Outcome:	The proposed model achieves state-of-the-art performance on a popular Chinese social media dataset.

Copied to clipboard

Meishan Zhang, Zhenghua Li, Guohong Fu, Min Zhang

Pushing Paraphrase Away from Original Sentence: A Multi-Round Paraphrase Generation Approach (2021.findings-acl)

Challenge:	Syntax integration has been demonstrated highly effective in neural machine translation (NMT).
Approach:	They propose a method to integrate source-side syntax implicitly for neural machine translation . they use hidden representations of a well-trained end-to-end dependency parser to concatenate them with ordinary word embeddings to enhance basic NMT models.
Outcome:	The proposed method outperforms existing methods on two translation tasks . it can be easily integrated into the widely-used sequence-to-sequence (Seq2Sequen) framework .

Copied to clipboard

Zhe Lin, Xiaojun Wan

Tailored Sequence to Sequence Models to Different Conversation Scenarios (P18-1)

Challenge:	Recent years, neural paraphrase generation models have demonstrated superior performance, but the output paraphrase still lacks diversity.
Approach:	They propose a back-translation guided multi-round paraphrase generation framework which leverages multi- round paraphrases to improve diversity while preserving semantic information.
Outcome:	The proposed model improves diversity while preserving semantic information.

Copied to clipboard

Hainan Zhang, Yanyan Lan, Jiafeng Guo, Jun Xu, Xueqi Cheng

Variational Attention for Sequence-to-Sequence Models (C18-1)

Challenge:	Sequence to sequence (Seq2Sequeq) models fail to meet the diverse requirements for different conversation scenarios, such as customer service and chatbot.
Approach:	They propose two optimized criteria for Sequence to sequence (Seq2Sequeq) to meet different conversation scenarios, i.e., maximum generated likelihood for specific-requirement scenario, and conditional value-at-risk for diverse-requrement scenarios.
Outcome:	The proposed models satisfies diverse requirements for different conversation scenarios and yields better performances than existing models.

Copied to clipboard

Hareesh Bahuleyan, Lili Mou, Olga Vechtomova, Pascal Poupart

Alirector: Alignment-Enhanced Chinese Grammatical Error Corrector (2024.findings-acl)

Challenge:	Existing variational autoencoders encode data to latent variables and then decode them into target data.
Approach:	They propose a variational attention mechanism where the attention vector is also modeled as Gaussian distributed random variables.
Outcome:	The proposed method reduces the variational latent space bypassing phenomenon as it increases diversity of generated sentences.

Copied to clipboard

Haihui Yang, Xiaojun Quan

Interpretable Charge Predictions for Criminal Cases: Learning to Generate Court Views from Fact Descriptions (N18-1)

Challenge:	Existing methods to address overcorrection in Chinese grammatical error correction (CGEC) are difficult to adapt to decoder-only large language models (LLMs).
Approach:	They propose an alignment-enhanced corrector for the overcorrection problem that applies to both Seq2Seq models and decoder-only large language models (LLMs).
Outcome:	The proposed corrector alleviates the overcorrection problem in Chinese grammatical error correction (CGEC) using generative models and decoder-only large language models.

Copied to clipboard

Hai Ye, Xin Jiang, Zhunchen Luo, Wenhan Chao

Explainability & Interpretability in NLP

Challenge:	Existing work on court view generation from fact descriptions has improved the working efficiency of legal assistant systems.
Approach:	They propose to decode court views conditioned on encoded charge labels from the fact description in a criminal case to improve interpretability of charge prediction systems.
Outcome:	The proposed model can generate court views conditioned on encoded charge labels.

Seq2Path: Generating Sentiment Tuples as Paths of a Tree (2022.findings-acl)

Copied to clipboard

Yue Mao, Yi Shen, Jingchao Yang, Xiaoying Zhu, Longjun Cai

Sentiment Analysis

Challenge:	Existing generative methods for extracting sentiment tuples do not have orders between the t-uples . a novel parallel generative framework for ABSA is proposed .
Approach:	They propose a parallel generative framework to generate sentiment tuples as paths of a tree . they train the model with an independent target and introduce a discriminative token .
Outcome:	The proposed method achieves state-of-the-art on AOPE, ASTE, TASD, UABSA, ACOS . it trains with the loss of ordinary Seq2Seq averaged over paths, and inferences automatically select valid paths.

GenerativeRE: Incorporating a Novel Copy Mechanism and Pretrained Model for Joint Entity and Relation Extraction (2021.findings-emnlp)

Copied to clipboard

Jiarun Cao, Sophia Ananiadou

Focus-Constrained Attention Mechanism for CVAE-based Response Generation (2020.findings-emnlp)

Challenge:	Existing models for extracting relation triplets suffer from incompletion and disorder problems when they extract multi-token entities from input sentences.
Approach:	They propose a special entity labelling method that fine-tunes the pre-trained model and learns the special entity labels simultaneously.
Outcome:	The proposed model achieves 4.6% and 0.9% improvement over current methods in the NYT24 and NYT29 benchmark datasets.

Copied to clipboard

Zhi Cui, Yanran Li, Jiayi Zhang, Jianwei Cui, Chen Wei, Bin Wang

Learning to Progressively Recognize New Named Entities with Sequence to Sequence Models (C18-1)

Challenge:	Existing models generate high-frequency but trivial responses such as "I don't know" or "I'm ok" due to the discrepancy in discourse-level information, standard models generate one-to-many relationships.
Approach:	They propose to transform coarse-grained discourse-level information into fine-grounded word-level knowledge by introducing a fine-grain focus signal and a focus-constrained attention mechanism to take full advantage of focus.
Outcome:	The proposed model can generate more diverse and informative responses compared with state-of-the-art models.

Copied to clipboard

Lingzhen Chen, Alessandro Moschitti

Understanding and Improving Sequence-to-Sequence Pretraining for Neural Machine Translation (2022.acl-long)

Challenge:	Existing models for Named Entity Recognition (NER) are trained on data with the same NE label set, but they are not able to recognize previously unseen NE categories.
Approach:	They propose to use a sequence to sequence model for Named Entity Recognition (NER) and propose to reshape and re-parametrize the output layer of the first learned model to enable the recognition of new NEs.
Outcome:	The proposed model can recognize previously unseen NE categories while keeping the knowledge of previously seen categories.

Copied to clipboard

Wenxuan Wang, Wenxiang Jiao, Yongchang Hao, Xing Wang, Shuming Shi, Zhaopeng Tu, Michael Lyu

DyKgChat: Benchmarking Dialogue Generation Grounding on Dynamic Knowledge Graphs (D19-1)

Challenge:	Existing studies on self-supervised pretraining for machine translation have focused on the jointly pretrained decoder .
Approach:	They propose a method to improve neural machine translation by jointly pretrained decoder . they propose two strategies to remedy the domain and objective discrepancies .
Outcome:	The proposed approach improves translation performance and model robustness on three language pairs.

Copied to clipboard

Yi-Lin Tuan, Yun-Nung Chen, Hung-yi Lee

Question Generation Using Sequence-to-Sequence Model with Semantic Role Labels (2023.eacl-main)

Challenge:	Existing work has not shown that knowledge-grounded models can zero-shot adapt to updated, unseen knowledge graphs.
Approach:	They propose a task to apply dynamic knowledge graphs to neural conversation models . they propose 'dyKgChat' that selects an output from two networks at each time step .
Outcome:	The proposed model outperforms existing knowledge-grounded conversation models in evaluation metrics.

Copied to clipboard

Alireza Naeiji, Aijun An, Heidar Davoudi, Marjan Delpisheh, Muath Alzghool

A Learning-Exploring Method to Generate Diverse Paraphrases with Multi-Objective Deep Reinforcement Learning (2020.coling-main)

Challenge:	Existing question generation methods that generate multiple questions from text are labor-intensive and do not capture the complexity of ways a human asks questions.
Approach:	They propose a question generation method that combines the benefits of rule-based and neural sequence-to-sequence (Seq2Sequen) models.
Outcome:	The proposed method significantly improves the state-of-the-art neural question generation approaches on three real-world data sets.

Copied to clipboard

Mingtong Liu, Erguang Yang, Deyi Xiong, Yujie Zhang, Yao Meng, Changjian Hu, Jinan Xu, Yufeng Chen

Exploring Question-Specific Rewards for Generating Deep Questions (2020.coling-main)

Challenge:	Paraphrase generation is of great importance for many downstream tasks in natural language processing.
Approach:	They propose a method to generate sentences as learning objectives from the learned data distribution and employ reinforcement learning to combine these new learning objectives for model training.
Outcome:	The proposed method gains significant diversity and improves generation quality over state-of-the-art datasets.

Copied to clipboard

Yuxi Xie, Liangming Pan, Dongzhe Wang, Min-Yen Kan, Yansong Feng

Evaluating Rewards for Question Generation Models (N19-1)

Challenge:	Recent question generation approaches use the sequence-to-sequence framework to optimize the log likelihood of ground-truth questions using teacher forcing.
Approach:	They propose to optimize for QG-specific objectives via reinforcement learning to improve question quality.
Outcome:	The proposed model improves the fluency, relevance, and answerability of generated questions.

Copied to clipboard

Tom Hosking, Sebastian Riedel

Graph-to-Tree Neural Networks for Learning Structured Input-Output Translation with Applications to Semantic Parsing and Math Word Problem (2020.findings-emnlp)

Challenge:	Recent approaches to question generation have used modifications to a Seq2Seq architecture inspired by advances in machine translation.
Approach:	They propose to use a Seq2Seq architecture to train models to generate one-step-ahead predictions, but at test time, the model is asked to generate a whole sequence, causing errors to propagate through the generation process.
Outcome:	The proposed model is trained to generate a plausible question, conditioned on an input document and answer span within that document.

Copied to clipboard

Shucheng Li, Lingfei Wu, Shiwei Feng, Fangli Xu, Fengyuan Xu, Sheng Zhong

Multilinguality

Challenge:	Graph2Tree model encodes graph-structured input and decodes tree-structures output.
Approach:	They propose a novel Graph-to-Tree Neural Network consisting of a graph encoder and a hierarchical tree decoder that encodes an augmented graph-structured input and decodes a tree-structure-output.
Outcome:	The proposed model outperforms or matches the performance of other state-of-the-art models on two problems, neural semantic parsing and math word problem.

NL2pSQL: Generating Pseudo-SQL Queries from Under-Specified Natural Language Questions (D19-1)

Copied to clipboard

Fuxiang Chen, Seung-won Hwang, Jaegul Choo, Jung-Woo Ha, Sunghun Kim

FRSUM: Towards Faithful Abstractive Summarization via Enhancing Factual Robustness (2022.findings-emnlp)

Challenge:	Existing studies focus on generating SQL codes from natural language questions . however, questions cover more diverse tasks including table manipulation or performance issues .
Approach:	They propose a task to generate pSQL codes from natural language questions . they define two new metrics suitable for the task, Canonical-BLEU and SQL-BLUE .
Outcome:	The proposed task generates well-formed queries on under-specified database issues.

Copied to clipboard

Wenhao Wu, Wei Li, Jiachen Liu, Xinyan Xiao, Ziqiang Cao, Sujian Li, Hua Wu

Modeling Event Background for If-Then Commonsense Reasoning Using Context-aware Variational Autoencoder (D19-1)

Challenge:	Existing models of abstractive summarization are able to generate fluent and coherent summaries, but they still suffer from the unfaithful generation problem.
Approach:	They propose to improve the faithfulness of existing models by enhancing their factual robustness by using a novel training strategy, namely FRSUM, which teaches the model to defend against both explicit adversarial samples and implicit factual adversarials.
Outcome:	The proposed training strategy improves faithfulness of various models, such as T5, BART, and T5 .

Copied to clipboard

Li Du, Xiao Ding, Ting Liu, Zhongyang Li

Reasoning

Challenge:	Understanding event and event-centered commonsense reasoning is crucial for natural language processing (NLP).
Approach:	They propose a If-Then commonsense reasoning dataset Atomic and an RNN-based Seq2Seq model to facilitate this.
Outcome:	The proposed model improves the accuracy and diversity of inferences compared with baseline methods.

GraphMR: Graph Neural Network for Mathematical Reasoning (2021.emnlp-main)

Copied to clipboard

Weijie Feng, Binbin Liu, Dongpeng Xu, Qilong Zheng, Yun Xu

Reasoning

Challenge:	Existing studies have shown the effectiveness of sequence-to-sequence (Seq2Seque) on mathematics solving.
Approach:	They propose a graph-to-sequence neural network which can learn hierarchical information of graphs inputs to solve mathematical problems and speculate answers.
Outcome:	The proposed neural network outperforms other neural networks in hidden information learning and mathematics resolving.

Deconvolution-Based Global Decoding for Neural Machine Translation (C18-1)

Copied to clipboard

Junyang Lin, Xu Sun, Xuancheng Ren, Shuming Ma, Jinsong Su, Qi Su

Machine Translation

Challenge:	Existing models for Neural Machine Translation (NMT) use Recurrent Neural Network (RNN) to generate translation word by word following a sequential order.
Approach:	They propose a Neural Machine Translation (NMT) model that decodes the sequence with the guidance of its structural prediction of the target-side context.
Outcome:	The proposed model is more competitive compared with the state-of-the-art methods and reduces repetition with the instruction from the target-side context for decoding.

Table-to-Text Generation with Effective Hierarchical Encoder on Three Dimensions (Row, Column and Time) (D19-1)

Copied to clipboard

Heng Gong, Xiaocheng Feng, Bing Qin, Ting Liu

Form2Seq : A Framework for Higher-Order Form Structure Extraction (2020.emnlp-main)

Challenge:	Seq2Seq models for table-to-text generation have achieved remarkable progress, but modeling table representation in one dimension is inadequate.
Approach:	They propose to model each table cell considering other records in the same row and to enrich table’s representation by modeling each cell in context of other cells in the similar column or with historical data respectively.
Outcome:	The proposed model outperforms baseline and state-of-the-art models on ROTOWIRE, a benchmark dataset of NBA basketball games.

Copied to clipboard

Milan Aggarwal, Hiresh Gupta, Mausoom Sarkar, Balaji Krishnamurthy

Complexity-Weighted Loss and Diverse Reranking for Sentence Simplification (N19-1)

Challenge:	Document structure extraction is a widely researched area for decades due to image resolution and poor semantics.
Approach:	They propose a sequence-to-sequence framework for document structure extraction using text . they use a text-based framework to classify low-level constituent elements into ten types .
Outcome:	The proposed framework outperforms existing methods for document structure extraction on ICDAR 2013 dataset.

Copied to clipboard

Reno Kriz, João Sedoc, Marianna Apidianaki, Carolina Zheng, Gaurav Kumar, Eleni Miltsakaki, Chris Callison-Burch

Conditional set generation using Seq2seq models (2022.emnlp-main)

Challenge:	Recent research has applied sequence-to-sequence (Seq2Sequen) models to text simplification . generic models tend to copy directly from the original sentence, resulting in outputs that are long and complex.
Approach:	They propose to incorporate word complexities into the loss function during training and generate a large set of diverse candidate simplifications at test time.
Outcome:	The proposed model can perform competitively with state-of-the-art systems while generating simpler sentences.

Copied to clipboard

Aman Madaan, Dheeraj Rajagopal, Niket Tandon, Yiming Yang, Antoine Bosselut

Learning When to Concentrate or Divert Attention: Self-Adaptive Attention Temperature for Neural Machine Translation (D18-1)

Challenge:	Several NLP tasks are instances of set generation.
Approach:	They propose a model-independent data augmentation approach that enlarges the model with the signals of order-invariance and cardinality.
Outcome:	The proposed method improves performance on four benchmark datasets with no additional annotations.

Copied to clipboard

Junyang Lin, Xu Sun, Xuancheng Ren, Muyu Li, Qi Su

Machine Translation

Challenge:	Neural Machine Translation models treat decoding at each time step equally with the same matrix . conventional methods treat decoder outputs at all time steps with the identical weight matrix causing inaccuracy .
Approach:	They propose a model with a mechanism to control the softness of attention by means of an attention temperature.
Outcome:	The proposed model outperforms baseline models on Chinese-English and English-Vietnamese translations.

Contrastive Learning enhanced Author-Style Headline Generation (2022.emnlp-main)

Copied to clipboard

Hui Liu, Weidong Guo, Yige Chen, Xiangyang Li

Intent Classification and Slot Filling for Privacy Policies (2021.acl-long)

Challenge:	Current work only uses the article itself in the headline generation, but have not taken the writing style of headlines into account.
Approach:	They propose a model which takes historical headlines into account to integrate the stylistic features of the author into the model and integrate them into the decoder.
Outcome:	The proposed model can integrate the stylistic features of the author into the model and generate a headline that is appropriate for the article and consistent with the author’s style.

Copied to clipboard

Wasi Ahmad, Jianfeng Chi, Tu Le, Thomas Norton, Yuan Tian, Kai-Wei Chang

Responsible & Trustworthy NLP

Challenge:	Sentences written in privacy policies explain privacy practices and the constituent text spans convey further specific information.
Approach:	They propose an English corpus of 5,250 intent and 11,788 slot annotations . they propose two alternative neural approaches to model the corpus as a sequence-to-sequence learning task.
Outcome:	The proposed corpus predicts intent classification and slot filling, while the sequence tagging method outperforms slot filler by a large margin.

Exploring Sequence-to-Sequence Learning in Aspect Term Extraction (P19-1)

Copied to clipboard

Dehong Ma, Sujian Li, Fangzhao Wu, Xing Xie, Houfeng Wang

ELI5: Long Form Question Answering (P19-1)

Challenge:	Aspect term extraction (ATE) aims at identifying all aspect terms in a sentence . sequence labeling based methods cannot make full use of overall meaning of sentence if they have dependencies between labels.
Approach:	They propose to formalize ATE as a sequence-to-sequence (Seq2Seque) learning task . they propose gated unit networks and position-aware attention mechanism to make it suit to ATE .
Outcome:	The proposed learning task is effective when labels correspond to words one by one . the proposed learning system is gated unit networks and position-aware attention mechanism .

Copied to clipboard

Angela Fan, Yacine Jernite, Ethan Perez, David Grangier, Jason Weston, Michael Auli

Question Answering

Challenge:	Existing question answering datasets provide extractive or short answers, but less attention has been paid to open-ended questions that require explanations.
Approach:	They present a large-scale corpus for long form question answering . they use a Reddit forum to provide elaborate answers to open-ended questions .
Outcome:	The proposed model outperforms Seq2Seq, language modeling, and other models in human evaluations.

Translation vs. Dialogue: A Comparative Analysis of Sequence-to-Sequence Modeling (2020.coling-main)

Copied to clipboard

Wenpeng Hu, Ran Le, Bing Liu, Jinwen Ma, Dongyan Zhao, Rui Yan

RACE: Retrieval-augmented Commit Message Generation (2022.emnlp-main)

Challenge:	Existing models for machine translation and dialogue response generation require a large number of handcrafted features.
Approach:	They propose to interpret a general neural model comparatively by using the seq2seq model in two mainstream NLP tasks.
Outcome:	The proposed model is used in two mainstream NLP tasks and is compared with a standard model.

Copied to clipboard

Ensheng Shi, Yanlin Wang, Wei Tao, Lun Du, Hongyu Zhang, Shi Han, Dongmei Zhang, Hongbin Sun

OTSeq2Set: An Optimal Transport Enhanced Sequence-to-Set Model for Extreme Multi-label Text Classification (2022.emnlp-main)

Challenge:	Existing approaches to automatically generate commit messages are repetitive or redundant.
Approach:	They propose a retrieval-augmented neural commit message generation method which treats the retrieved similar commit as an exemplar and leverages it to generate an accurate commit message.
Outcome:	The proposed method outperforms baselines on a large dataset with five programming languages and can boost existing Seq2Seq models in commit message generation.

Copied to clipboard

Jie Cao, Yin Zhang

Text Classification

Challenge:	Extreme multi-label text classification (XMTC) is a task of finding the most relevant subset labels from an extremely large label set.
Approach:	They propose an autoregressive sequence-to-set model for XMTC tasks named OTSeq2Set.
Outcome:	The proposed model outperforms other competitive baselines on 4 benchmark datasets.

TemplateGEC: Improving Grammatical Error Correction with Detection Template (2023.acl-long)

Copied to clipboard

Yinghao Li, Xuebo Liu, Shuo Wang, Peiyuan Gong, Derek F. Wong, Yang Gao, Heyan Huang, Min Zhang

Grammatical Error Correction via Mixed-Grained Weighted Training (2023.findings-emnlp)

Challenge:	Existing methods for grammatical error correction (GEC) have been developed.
Approach:	They propose a method which integrates the detection labels from a Seq2Edit model to construct a template as the input.
Outcome:	The proposed method can perform human-in-the-loop error correction tasks.

Copied to clipboard

Jiahao Li, Quan Wang, Chiwei Zhu, Zhendong Mao, Yongdong Zhang

Conversational Semantic Parsing (2020.emnlp-main)

Challenge:	Empirical evaluation shows that MainGEC achieves consistent and significant performance improvements on two benchmark datasets.
Approach:	They propose to use mixed-grained weighted training to improve the training effect for GEC by analyzing the inherent discrepancies in annotated training data.
Outcome:	Empirical results show that the proposed method achieves significant performance improvements on two benchmark datasets.

Copied to clipboard

Armen Aghajanyan, Jean Maillard, Akshat Shrivastava, Keith Diedrick, Michael Haeger, Haoran Li, Yashar Mehdad, Veselin Stoyanov, Anuj Kumar, Mike Lewis, Sonal Gupta

Non-autoregressive Text Editing with Copy-aware Latent Alignments (2023.emnlp-main)

Challenge:	Structured representations for task-oriented assistant systems are limited due to the limitations of the representation.
Approach:	They propose a semantic representation for task-oriented conversational systems that can represent co-reference and context carryover.
Outcome:	The proposed model improves the best results on ATIS, SNIPS, TOP and DSTC2 by up to 5 points for slot-carryover.

Copied to clipboard

Yu Zhang, Yue Zhang, Leyang Cui, Guohong Fu

Hierarchical Transformer for Task Oriented Dialog Systems (2021.naacl-main)

Challenge:	Seq2Edit approaches still face several challenges such as inflexibility in generation and difficulty in generalizing to other languages.
Approach:	They propose a non-autoregressive text editing method that models the edit process with latent CTC alignments and introduces the copy operation into the edit space.
Outcome:	The proposed method outperforms existing Seq2Edit models and achieves similar or even better results than Seq1Edit with over 4 speedup.

Copied to clipboard

Bishal Santra, Potnuru Anusha, Pawan Goyal

A Unified Generative Framework for Various NER Subtasks (2021.acl-long)

Challenge:	Existing models for dialog generation are challenging to train using the standard Seq2Seq models.
Approach:	They propose a framework for Hierarchical Transformer Encoders that can be morphed into any hierarchical transformer by using specially designed attention masks and positional encodings.
Outcome:	The proposed framework can be morphed into any hierarchical encoder, including HRED and HIBERT like models, by using specially designed attention masks and positional encodings.

Copied to clipboard

Hang Yan, Tao Gui, Junqi Dai, Qipeng Guo, Zheng Zhang, Xipeng Qiu

A Benchmark and Dataset for Post-OCR text correction in Sanskrit (2022.findings-emnlp)

Challenge:	Named Entity Recognition (NER) is the task of identifying spans that represent entities in sentences.
Approach:	They propose to formulate NER subtasks as entity span sequence generation task . framework can be used to solve all three kinds of NER tasks without tagging schema .
Outcome:	The proposed framework achieves state-of-the-art (SoTA) or near SoTA performance on eight English NER datasets.

Copied to clipboard

Ayush Maheshwari, Nikhil Singh, Amrith Krishna, Ganesh Ramakrishnan

NüshuRescue: Reviving the Endangered Nüshu Language with AI (2025.coling-main)

Challenge:	Sanskrit is a classical language with 30 million manuscripts available for digitisation . however, it is considered to be low-resource when it comes to available digital resources.
Approach:	They propose to use a post-OCR text correction dataset to correct errors from OCR predictions from 30 different books in the Indian subcontinent.
Outcome:	The proposed model outperforms OCR models on graphemic and lexical levels and shows that it is more accurate than previous models.

Copied to clipboard

Ivory Yang, Weicheng Ma, Soroush Vosoughi

Machine Translation

Challenge:	Nüshu is a rare syllabic script used by Yao women in china for self-expression . a lack of data makes the reconstruction labor-intensive and costly .
Approach:	They propose an AI-driven framework to train large corpora on endangered languages . Nüshu is a rare syllabic script used by Yao women in china for self-expression .
Outcome:	NüshuRescue automates evaluation and expands target corpora to accelerate linguistic revitalization.

Language-Independent Approach for Morphological Disambiguation (2022.coling-1)

Copied to clipboard

Alymzhan Toleu, Gulmira Tolegen, Rustam Mussabayev

BASS: Boosting Abstractive Summarization with Unified Semantic Graph (2021.acl-long)

Challenge:	Existing approaches for predicting complex morphological tags treat each analysis as a tag and apply sequence labeling models to perform tagging.
Approach:	They propose a language-independent approach which integrates all words, roots, POS and morpheme tags into vectors and computes the inner products between analyses and the contexts.
Outcome:	The proposed approach outperforms existing models on seven different languages while running about 6 and 33 times faster than MarMot and Seq2Seq, respectively.

Copied to clipboard

Wenhao Wu, Wei Li, Xinyan Xiao, Jiachen Liu, Ziqiang Cao, Sujian Li, Hua Wu, Haifeng Wang

Precisely the Point: Adversarial Augmentations for Faithful and Informative Text Generation (2022.emnlp-main)

Challenge:	Abstractive summarization for long-document or multi-document remains challenging for Seq2Seq as it does not analyze long-distance relations in text.
Approach:	They propose a framework for Boosting Abstractive Summarization based on a unified Semantic graph which aggregates co-referent phrases distributing across a long range of context and conveys rich relations between phrases.
Outcome:	The proposed framework improves document representation and summary generation process by leveraging the graph structure.

Copied to clipboard

Wenhao Wu, Wei Li, Jiachen Liu, Xinyan Xiao, Sujian Li, Yajuan Lyu

Improving Seq2Seq Grammatical Error Correction via Decoding Interventions (2023.findings-emnlp)

Challenge:	Existing models for text generation are weak enough to handle perturbations in inputs, leading to degeneration in faithfulness and informativeness.
Approach:	They propose a framework for improving faithfulness and informativeness of Seq2Seq models by perturbing word representations and word swapping.
Outcome:	The proposed framework improves faithfulness and informativeness of Seq2Seq models under automatic and human evaluation settings.

Copied to clipboard

Houquan Zhou, Yumeng Liu, Zhenghua Li, Min Zhang, Bo Zhang, Chen Li, Ji Zhang, Fei Huang

Cognate Detection for Historical Language Reconstruction of Proto-Sabean Languages: the Case of Ge’ez, Tigrinya, and Amharic (2025.coling-main)

Challenge:	Existing approaches to grammatical error correction (GEC) are sequence-to-sequence and sequence-edit.
Approach:	They propose a unified decoding intervention framework that employs an external critic to assess the appropriateness of the token to be generated incrementally.
Outcome:	The proposed framework outperforms baselines and state-of-the-art methods on English and Chinese datasets.

Copied to clipboard

Elleni Sisay Temesgen, Hellina Hailu Nigatu, Fitsum Assamnew Andargie

Challenge:	As languages evolve, we risk losing ancestral languages.
Approach:	They propose to use cognates to reconstruct proto-languages from cognates in child languages that have likely evolved from the same word in the proto-linguistics.
Outcome:	The proposed method is based on automatic cognate detection and in-context learning with GPT-4o to generate the proto-language from the cognates and use Sequence-to-Sequence models.

A Sequence-to-Structure Approach to Document-level Targeted Sentiment Analysis (2023.findings-emnlp)

Copied to clipboard

Nan Song, Hongjie Cai, Rui Xia, Jianfei Yu, Zhen Wu, Xinyu Dai

Sentiment Analysis

Challenge:	Aspect-based sentiment analysis (ABSA) has received wide attention in NLP for nearly two decades . previous studies focused on sentence-level ABSA, but document-level research has not received enough attention.
Approach:	They propose a Sequence-to-Structure approach to address the document-level targeted sentiment analysis task, which aims to extract the opinion targets consisting of multi-level entities from a review document and predict their sentiments.
Outcome:	The proposed approach outperforms baselines on six domains on the document-level targeted sentiment analysis task.

Incorporating Linguistic Constraints into Keyphrase Generation (P19-1)

Copied to clipboard

Jing Zhao, Yuxiang Zhang

A Deep Reinforced Sequence-to-Set Model for Multi-Label Classification (P19-1)

Challenge:	Existing keyphrase generation methods generate overlapping phrases (including sub-phrases or super-phrase) Existing methods are far from satisfactory for a wide range of natural language processing tasks.
Approach:	They propose a parallel Seq2Seq network with coverage attention to alleviate the overlapping phrase problem by integrating linguistic constraints of keyphrase into the basic Seq2-Sequeq network on the source side and employ the multi-task learning framework on the target side.
Outcome:	The proposed method outperforms the state-of-the-art CopyRNN on scientific datasets and is also more effective in news domain.

Copied to clipboard

Pengcheng Yang, Fuli Luo, Shuming Ma, Junyang Lin, Xu Sun

Text Classification

Challenge:	Multi-label classification (MLC) aims to assign multiple labels to each sample.
Approach:	They propose a sequence-to-set model that is trained via reinforcement learning and rewards feedback independent of the label order.
Outcome:	The proposed model outperforms baseline models and reduces sensitivity to label order.

How Do Seq2Seq Models Perform on End-to-End Data-to-Text Generation? (2022.acl-long)

Copied to clipboard

Xunjian Yin, Xiaojun Wan

Personal Information Leakage Detection in Conversations (2020.emnlp-main)

Challenge:	Existing models for data-to-text generation are based on pipelines and end-to end architectures.
Approach:	They use multidimensional quality metrics to evaluate models on end-to-end data-totext generation and compare their performance against pipeline models.
Outcome:	The proposed model improves in Omission and Inaccuracy Extrinsic errors but increases errors such as Addition.

Copied to clipboard

Qiongkai Xu, Lizhen Qu, Zeyu Gao, Gholamreza Haffari

Focus-Driven Contrastive Learning for Medical Question Summarization (2022.coling-1)

Challenge:	4.5 billion dollars will be invested in conversational assistants (chatbots) by 2021, according to Opus Research 2 . Among diverse types of chatbots, Google Duplex represents the kind of AI personal assistants that act on behalf of people to perform simple tasks.
Approach:	They propose to protect personal information by warning users of detected suspicious sentences . they propose to use a constrained alignment problem to perform an alignment optimization problem .
Outcome:	The proposed models outperform baseline models on the behavior of personalized chit-chat dialogue systems.

Copied to clipboard

Ming Zhang, Shuai Dou, Ziyang Wang, Yunfang Wu

Confusionset-guided Pointer Networks for Chinese Spelling Check (P19-1)

Challenge:	Existing methods to summarize health questions are not able to capture well question focus and lack the ability to understand sentence-level semantics.
Approach:	They propose a question focus-driven contrastive learning framework to capture question focus and exploit contrastive training at both encoder and decoder to obtain better sentence representations.
Outcome:	The proposed model achieves 5.33, 12.85 and 3.81 points over the baseline model on three medical benchmark datasets.

Copied to clipboard

Dingmin Wang, Yi Tay, Li Zhong

Text Error Correction

Challenge:	Existing methods to detect and fix errors in Chinese are limited due to context.
Approach:	They propose a Confusionset-guided pointer network for Chinese Spell Check task . they propose to use off-the-shelf confusionset to guide character generation .
Outcome:	The proposed model outperforms all competitor models on three human-annotated datasets.

Can You Unpack That? Learning to Rewrite Questions-in-Context (D19-1)

Copied to clipboard

Ahmed Elgohary, Denis Peskov, Jordan Boyd-Graber

GreekBART: The First Pretrained Greek Sequence-to-Sequence Model (2024.lrec-main)

Challenge:	Existing QA datasets lack key NLP problems like coreference and ellipsis resolution.
Approach:	They propose a task of question-in-context rewriting to rewrite a context-dependent question into a self-contained question with the same answer.
Outcome:	The proposed task is based on a dataset of 40,527 questions based in QuAC . it requires models to link questions together to resolve conversational dependencies .

Copied to clipboard

Iakovos Evdaimon, Hadi Abdine, Christos Xypolopoulos, Stamatis Outsios, Michalis Vazirgiannis, Giorgos Stamou

Distilling Knowledge Learned in BERT for Text Generation (2020.acl-main)

Challenge:	Transfer learning has revolutionized the fields of Computer Vision and Natural Language Processing.
Approach:	They introduce a new language model, GreekBART, that is based on a BART-base architecture.
Outcome:	The proposed model outperforms BERT, GPT and other transformer-based models on discriminative tasks.

Copied to clipboard

Yen-Chun Chen, Zhe Gan, Yu Cheng, Jingzhou Liu, Jingjing Liu

A Probabilistic End-To-End Task-Oriented Dialog Model with Latent Belief States towards Semi-Supervised Learning (2020.emnlp-main)

Challenge:	Large-scale pre-trained language models such as BERT have revolutionized the state of the art in many language understanding tasks.
Approach:	They propose a conditional masked language modeling approach to fine tune BERT on target generation tasks by imposing global sequence-level supervision on conventional Seq2Seq models.
Outcome:	The proposed model outperforms strong Transformer baselines on multiple language generation tasks such as machine translation and text summarization.

Copied to clipboard

Yichi Zhang, Zhijian Ou, Min Hu, Junlan Feng

Responsible & Trustworthy NLP

Challenge:	Structured belief states are crucial for goal tracking and database query in task-oriented dialog systems.
Approach:	They propose a probabilistic dialog model where belief states are represented as discrete latent variables and jointly modeled with system responses given user inputs.
Outcome:	The proposed model outperforms supervised-only and semi-supervised baselines on three benchmark datasets.

Set Learning for Generative Information Extraction (2023.emnlp-main)

Copied to clipboard

Jiangnan Li, Yice Zhang, Bin Liang, Kam-Fai Wong, Ruifeng Xu

Transforming Sequence Tagging Into A Seq2Seq Task (2022.emnlp-main)

Challenge:	Recent efforts to employ sequence-to-sequence models to solve IE tasks have been focused on a single problem: structured objects are an unordered set, resulting in a potential order bias.
Approach:	They propose a sequence-to-sequence (Seq2Sequen) model that considers multiple permutations of structured objects to optimize set probability approximately.
Outcome:	The proposed model improves existing frameworks on vast tasks and datasets.

Copied to clipboard

Karthik Raman, Iftekhar Naim, Jiecao Chen, Kazuma Hashimoto, Kiran Yalasangi, Krishna Srinivasan

NL2TL: Transforming Natural Languages to Temporal Logics using Large Language Models (2023.emnlp-main)

Challenge:	Pretrained, large, generative language models have had great success in a wide range of sequence tagging and structured prediction tasks.
Approach:	They propose to use a new format for casting input text sentences and their output labels into the input and target of a Seq2Seq model and introduce it to test their hypothesis.
Outcome:	The proposed format shows to be both simpler and more effective and devoid of hallucination.

Copied to clipboard

Yongchao Chen, Rujul Gandhi, Yang Zhang, Chuchu Fan

gMBA: Expression Semantic Guided Mixed Boolean-Arithmetic Deobfuscation Using Transformer Architectures (2025.findings-acl)

Challenge:	Temporal Logic (TL) can be used to specify complex high-level specifications for systems in many engineering domains.
Approach:	They propose a framework for translation between NL and TL using Large Language Models . they use a dataset to create a model with 23K NL-TL pairs and human annotation .
Outcome:	The proposed framework achieves higher accuracy (> 95%) using only 10% training data compared with baseline model.

Copied to clipboard

Youjeong Roh, Joon-Young Paik, Jingun Kwon, Eun-Sun Cho