Papers by Xuancheng Ren
A Skeleton-Based Model for Promoting Coherence Among Sentences in Narrative Story Generation (D18-1)
Copied to clipboard
| Challenge: | Existing models for narrative story generation lack semantic dependency among sentences. |
| Approach: | They propose a skeleton-based model that generates the most critical phrases and expands them to a complete sentence. |
| Outcome: | The proposed model can generate significantly more coherent stories according to human evaluation and automatic evaluation. |
Deconvolution-Based Global Decoding for Neural Machine Translation (C18-1)
Copied to clipboard
| Challenge: | Existing models for Neural Machine Translation (NMT) use Recurrent Neural Network (RNN) to generate translation word by word following a sequential order. |
| Approach: | They propose a Neural Machine Translation (NMT) model that decodes the sequence with the guidance of its structural prediction of the target-side context. |
| Outcome: | The proposed model is more competitive compared with the state-of-the-art methods and reduces repetition with the instruction from the target-side context for decoding. |
simNet: Stepwise Image-Topic Merging Network for Generating Detailed and Comprehensive Image Captions (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to image captioning combine visual and semantic attention to generate a detailed and comprehensive caption. |
| Approach: | They propose a stepwise image-topic merging network that integrates visual and semantic attentions to generate a detailed caption. |
| Outcome: | The proposed approach is evaluated on two benchmark datasets and reaches the state-of-the-art performance. |
Structure Regularized Neural Network for Entity Relation Classification for Chinese Literature Text (N18-2)
Copied to clipboard
| Challenge: | Existing methods for relation classification have been used in natural language processing. |
| Approach: | They propose a relation classification task for Chinese literature text using a new dataset. |
| Outcome: | The proposed model outperforms the state-of-the-art methods on Chinese literature text. |
Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies reveal a security threat to natural language processing models, called the Backdoor Attack. |
| Approach: | They propose to hack a model by modifying one single word embedding vector without sacrificing accuracy on clean samples. |
| Outcome: | The proposed method is more efficient and stealthier on sentiment analysis and sentence-pair classification tasks. |
A Global Past-Future Early Exit Method for Accelerating Inference of Pre-trained Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to accelerate inference speed of pre-trained language models are limited to local representations of exit layer . current models are associated with large memory requirement and high computational cost, which slow down inference and further encumber the application of PLMs. |
| Approach: | They propose a method to exit early without passing through all inference layers . they take into consideration all the linguistic information embedded in the past layers a global perspective . |
| Outcome: | The proposed method outperforms existing methods by a large margin . it uses linguistic information embedded in the past layers and future features . the proposed method is scalable and cost-effective . |
A Hierarchical Reinforced Sequence Operation Method for Unsupervised Text Style Transfer (P19-1)
Copied to clipboard
| Challenge: | Existing text style transfer methods face three challenges: 1) the transfer is weakly interpretable; 2) generated outputs struggle in content preservation; 3) the trade-off between content and style is intractable. |
| Approach: | They propose a hierarchical reinforced sequence operation method that proposes operation positions and alters the sentence. |
| Outcome: | The proposed method significantly outperforms existing methods on two text style transfer datasets. |
Incorporating Fine-grained Events in Stock Movement Prediction (D19-51)
Copied to clipboard
| Challenge: | Existing studies mainly adopt coarse-grained events, which loses the specific semantic information of diverse event types. |
| Approach: | They propose to use a finance event dictionary to extract fine-grained events from finance news to train a neural model that uses the extracted events as the distant supervised label to train stock prediction. |
| Outcome: | The proposed method outperforms baselines and has good generalizability. |
Review-Driven Multi-Label Music Style Classification by Exploiting Style Correlations (N19-1)
Copied to clipboard
| Challenge: | Several methods have been proposed for automatic music style classification, but they are limited in two aspects. |
| Approach: | They propose a deep learning approach to automatically learn and exploit style correlations by reviewing music reviews on websites. |
| Outcome: | The proposed approach performs well in capturing style correlations. |
Hierarchical Inductive Transfer for Continual Dialogue Learning (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing frameworks for learning and deployment of neural dialogue models have been used for online chit-chat scenarios. |
| Approach: | They propose a hierarchical inductive transfer framework to learn and deploy dialogue skills continually and efficiently. |
| Outcome: | The proposed framework achieves comparable performance under deployment-friendly model capacity. |
Neural Network Surgery: Injecting Data Patterns into Pre-trained Models with Minimal Instance-wise Side Effects (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing neural network tuning methods cause instance-wise side effects . et al., 2018: a new approach to perform neural network surgery . |
| Approach: | They propose to perform neural network surgery by only changing 10-5 parameters . they propose to use a dynamic selecting method to achieve the best overall performance . |
| Outcome: | The proposed method achieves the best overall performance and induces fewer instance-wise side effects by changing only 10-5 of the parameters. |
Query and Output: Generating Words by Querying Distributed Word Representations for Paraphrase Generation (N18-1)
Copied to clipboard
| Challenge: | Existing models tend to memorize words instead of learning meaning of words . existing models tend not to model semantic information, resulting in incorrect sentences . |
| Approach: | They propose a novel model that generates words by querying distributed word representations . they evaluate model on two paraphrase-oriented tasks, namely text simplification and short abstractive summarization . |
| Outcome: | The proposed model outperforms the baseline model on two paraphrase-oriented tasks . it achieves state-of-the-art performance on these benchmark datasets . |
Rethinking Denoised Auto-Encoding in Language Pre-Training (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained models such as BERT have achieved success in learning sequence representations, but they tend to learn representations that are covariant with the noise of pre-training. |
| Approach: | They propose to train self-trained models to learn noise invariant sequence representations . they encourage consistency between original sequence and corrupted version via unsupervised instance-wise training signals. |
| Outcome: | The proposed model improves on 11 natural language understanding and cross-modal tasks and achieves 0.6% gain on GLUE benchmarks and 0.8% increment on NLVR2 . |
Diversity-Promoting GAN: A Cross-Entropy Based Generative Adversarial Network for Diversified Text Generation (D18-1)
Copied to clipboard
| Challenge: | Existing text generation methods tend to produce repeated and ”boring” expressions. |
| Approach: | They propose a model that assigns low reward for repeatedly generated text and high reward for ”novel” and fluent text, and a novel language-model based discriminator which can distinguish novel text from repeated text without the saturation problem. |
| Outcome: | The proposed model generates more diverse and informative text than existing baselines on review generation and dialogue generation tasks. |
Regularizing Dialogue Generation by Imitating Implicit Scenarios (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for dialogue generation lack the flexibility to handle such freedoms. |
| Approach: | They propose to take into account dialogue history and future conversation to implicitly reconstruct the scenario knowledge. |
| Outcome: | The proposed approach outperforms state-of-the-art models on diversity and relevance and expresses scenario-specific knowledge. |
From Mimicking to Integrating: Knowledge Integration for Pre-Trained Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for natural language processing (NLP) are fine-tuned and released for research and deployments. |
| Approach: | They propose a PLM reuse paradigm that merges teacher-PLM knowledge into a student model. |
| Outcome: | The proposed paradigm can reduce the computational cost and environmental side-effects of retraining the PLM from scratch. |
Delving into the Openness of CLIP (2023.findings-acl)
Copied to clipboard
| Challenge: | Contrastive Language-Image Pre-training (CLIP) allows for open-vocabulary visual recognition, where the model can recognize images from an open class set in a zero-shot manner. |
| Approach: | They propose to use image classification as an image-to-text matching task instead of discrete category IDs to achieve open-vocabulary visual recognition. |
| Outcome: | The proposed model can recognize images from an open vocabulary in a zero-shot manner, but its performance deteriorates as the vocabulary expands. |
Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach (P18-1)
Copied to clipboard
| Challenge: | Existing studies for sentiment-to-sentiment "translation" only change the underlying sentiment and fail to keep the semantic content. |
| Approach: | They propose a cycled reinforcement learning method that combines neutralization module and emotionalization module. |
| Outcome: | The proposed method outperforms state-of-the-art systems on Yelp and Amazon review datasets. |
LatentRefusal: Latent-Signal Refusal for Unanswerable Text-to-SQL Queries (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing refusal strategies for unanswerable and underspecified user queries are brittle due to model hallucinations or add complexity and overhead. |
| Approach: | They propose a latent-signal refusal mechanism that predicts query answerability from hidden activations of an LLM. |
| Outcome: | The proposed scheme reduces schema noise and sparse, localized question–schema mismatch cues that indicate unanswerability. |
Does Higher Order LSTM Have Better Accuracy for Segmenting and Labeling Sequence Data? (C18-1)
Copied to clipboard
| Challenge: | Existing neural models take long distance dependencies into account when predicting the tag of the current token. |
| Approach: | They propose a method to capture long distance tag dependencies and use them for dependency analysis. |
| Outcome: | The proposed model can predict multiple tags for the current token without taking dependencies between tags into account. |
O2NA: An Object-Oriented Non-Autoregressive Approach for Controllable Video Captioning (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for video captioning consider a sequence of frames and biases towards focused objects. |
| Approach: | They propose an Object-Oriented Non-Autoregressive approach to video captioning . it performs three steps: 1) identify the focused objects and predict their locations . 2) generate related attribute words and relation words of these focused objects to form a draft caption . |
| Outcome: | The proposed method achieves competitive results with the state-of-the-art methods but with higher diversity and faster inference speed. |
Building an Ellipsis-aware Chinese Dependency Treebank for Web Text (L18-1)
Copied to clipboard
| Challenge: | ellipsis is a common linguistic phenomenon that some words are left out as they are understood from the context, especially in oral utterance. |
| Approach: | They propose to use a Chinese dependency treebank to facilitate the parsing of web text . they propose to restore omissions and reserve contexts in the web text to improve dependency parsers . |
| Outcome: | The proposed framework enables the parsing of web text from online microblogs. |
Learning When to Concentrate or Divert Attention: Self-Adaptive Attention Temperature for Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Neural Machine Translation models treat decoding at each time step equally with the same matrix . conventional methods treat decoder outputs at all time steps with the identical weight matrix causing inaccuracy . |
| Approach: | They propose a model with a mechanism to control the softness of attention by means of an attention temperature. |
| Outcome: | The proposed model outperforms baseline models on Chinese-English and English-Vietnamese translations. |
Transferring General Multimodal Pretrained Models to Text Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for text recognition rely on large-scale pretraining on human-annotated or synthetic data. |
| Approach: | They propose a method to transfer multimodal pretrained models to text recognition using image captioning. |
| Outcome: | The proposed method outperforms the baselines and achieves state-of-the-art performance in the Chinese text recognition benchmark. |
Rethinking Skip Connection with Layer Normalization (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods to solve the optimization problem of deep neural networks are not linear, but can be used as a modulating mechanism between the input and output. |
| Approach: | They propose to use skip connection to adjust the scale of the input and output to improve the performance. |
| Outcome: | The proposed approach improves performance and convergence of deep neural networks and can be applied to machine translation and image classification datasets. |