HL-EncDec: A Hybrid-Level Encoder-Decoder for Neural Response Generation (C18-1)
Copied to clipboard
| Challenge: | Existing models for conversation systems operate sentences at word-level . word-based models suffer from Unknown Words Issue and Preference Issue . |
| Approach: | They propose a hybrid-level Encoder-Decoder model which utilizes word-level features and character-level ones. |
| Outcome: | The proposed model outperforms non-word-level models in automatic metrics and human annotations on a Chinese corpus. |
Similar Papers
Hierarchy Response Learning for Neural Conversation Generation (D19-1)
Copied to clipboard
| Challenge: | Neural conversation generation models can't perceive and express the intention effectively, causing dull and generic responses. |
| Approach: | They propose a hierarchical response generation model to capture conversation intention . they propose an expression reconstruction model and an expression attention model . |
| Outcome: | The proposed model can generate the responses with more appropriate content and expression. |
Exemplar Encoder-Decoder for Neural Conversation Generation (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to generate conversational systems suffer from lack of diversity in responses and generation of short, repetitive and uninteresting responses. |
| Approach: | They propose a novel conversation model that uses similar examples from training data to generate responses. |
| Outcome: | The proposed model outperforms state-of-the-art sequence to sequence learning on several evaluation metrics on two large data sets. |
MEMD: A Diversity-Promoting Learning Framework for Short-Text Conversation (C18-1)
Copied to clipboard
| Challenge: | Neural encoder-decoder models tend to generate meaningless and generic responses regardless of what the input text is. |
| Approach: | They propose an easy-to-extend learning framework based on latent vectors to provide training guidance without resorting to extra data or complicating network’s inner structure. |
| Outcome: | The proposed framework improves the quality of generated responses according to automatic metrics and human evaluations, yielding more diverse and smooth replies. |
Encoder-Decoder Models Can Benefit from Pre-trained Masked Language Models in Grammatical Error Correction (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for incorporating a masked language model into an EncDec model have potential drawbacks when applied to GEC. |
| Approach: | They propose to incorporate a pre-trained masked language model (MLM) into an encoder-decoder model for grammatical error correction. |
| Outcome: | The proposed method achieves state-of-the-art on BEA-2019 and CoNLL-2014 benchmarks. |
RECAP: Retrieval-Enhanced Context-Aware Prefix Encoder for Personalized Dialogue Response Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to personalized dialogue generation rely on dialogue data paired with user traits, profiles or persona description sentences. |
| Approach: | They propose a hierarchical transformer retriever trained on dialogue domain data to perform personalized retrieval and a context-aware prefix encoder that fuses the retrieved information to the decoder more effectively. |
| Outcome: | The proposed model generates more fluent and personalized responses under a suite of human and automatic metrics and is superior to state-of-the-art baselines on English Reddit conversations. |
What is the best recipe for character-level encoder-only modelling? (2023.acl-long)
Copied to clipboard
| Challenge: | aims to benchmark recent progress in language understanding models that output contextualised representations at the character level. |
| Approach: | They aim to find the best way to build and train character-level BERT-like models by comparing architectural innovations with pretraining objectives. |
| Outcome: | The proposed model outperforms a token-based model on a set of evaluation tasks with a fixed training procedure. |
On the Multilingual Ability of Decoder-based Pre-trained Language Models: Finding and Controlling Language-Specific Neurons (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing decoder-based pre-trained language models demonstrate excellent multilingual capabilities, but it is unclear how they handle multilingualism. |
| Approach: | They propose to examine the neuron-level internal behavior of decoder-based PLMs by finding neurons that fire “uniquely for each language” within decoded PLM models. |
| Outcome: | The proposed models fire “uniquely for each language” and show that language-specific neurons are unique, with a slight overlap (5%) between languages. |
A Simple and Effective Unified Encoder for Document-Level Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models for document-level machine translation use two separate encoders to model the source sentences and document- level contexts. |
| Approach: | They propose a unified encoder that can outperform existing models of dual-encoder models . they propose to use document-level contexts to model the interaction between the contexts and the source sentences . |
| Outcome: | The proposed model outperforms baseline models of dual-encoder models in terms of BLEU and METEOR scores. |
Character-Level Translation with Self-attention (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models for character-level neural machine translation operate on word-level, which makes them memory inefficient because of large vocabulary sizes. |
| Approach: | They propose a transformer-based model and a novel variant that uses convolutions to combine information from nearby characters to facilitate character interactions. |
| Outcome: | The proposed model outperforms the standard transformer model and learns more robust character alignments on bilingual and multilingual translation datasets. |
An Investigation of Suitability of Pre-Trained Language Models for Dialogue Generation – Avoiding Discrepancies (2021.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models have been widely used in open-domain dialogue generation. |
| Approach: | They propose to use decoder-only architecture to achieve excellent performance for dialogue generation. |
| Outcome: | The proposed frameworks are based on transformer-ED, transformer-Dec, transformer MLM and transformer-AR. |