Papers by Xisen Jin
Sequicity: Simplifying Task-oriented Dialogue Systems with Single Sequence-to-Sequence Architectures (P18-1)
Copied to clipboard
| Challenge: | Existing solutions to task-oriented dialogue systems follow pipeline designs which introduces complexity and fragility. |
| Approach: | They propose a novel sequence-to-sequence (seq2sequ) model which tracks dialogue believes and a two stage copynet instantiation which emonstrates good scalability. |
| Outcome: | The proposed framework outperforms state-of-the-art pipeline-based methods on large datasets and retains satisfactory entity match rate on out-of vocabulary (OOV) cases where pipeline-designed competitors totally fail. |
On Transferability of Bias Mitigation Effects in Language Model Fine-Tuning (2021.naacl-main)
Copied to clipboard
| Challenge: | PTLMs can exhibit biases against protected groups in a host of modeling tasks . but, fine-tuned LMs may propagate bias to downstream classifiers . |
| Approach: | They propose to use upstream bias mitigation techniques to reduce bias on downstream tasks by fine-tuning an upstream model and applying it to a downstream model. |
| Outcome: | The proposed model reduces bias on hate speech detection, toxicity detection and coreference resolution tasks over bias factors. |
Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot Learning (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models that pursue rapid generalization to new tasks are mostly trained in a single shot on fixed datasets, unable to dynamically expand their knowledge. |
| Approach: | They propose a new learning setup that assumes a model learns from a sequence of diverse NLP tasks arriving sequentially, accumulating knowledge for improved generalization to new tasks. |
| Outcome: | The proposed learning setup improves generalization ability while retaining performance on the tasks learned earlier. |
Recurrent Event Network: Autoregressive Structure Inferenceover Temporal Knowledge Graphs (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for reasoning over temporal knowledge graphs focus on past timestamps and are not able to predict future interactions. |
| Approach: | They propose a novel autoregressive architecture for predicting future interactions using a recurrent event encoder and a neighborhood aggregator. |
| Outcome: | The proposed method achieves state-of-the-art on five public datasets. |
Contextualizing Hate Speech Classifiers with Post-hoc Explanation (2020.acl-main)
Copied to clipboard
| Challenge: | Modern text classifiers struggle to learn a model of hate speech that generalizes to real-world applications. |
| Approach: | They propose a method to regularize BERT classifiers to detect bias towards identity terms by providing explanations for group identifiers and allowing models to learn from the context of group identifiers. |
| Outcome: | The proposed method limiting false positives on out-of-domain data while maintaining and improving in-domain performance. |
Overcoming Catastrophic Forgetting in Massively Multilingual Continual Learning (2023.findings-acl)
Copied to clipboard
Genta Winata, Lingjue Xie, Karthik Radhakrishnan, Shijie Wu, Xisen Jin, Pengxiang Cheng, Mayank Kulkarni, Daniel Preotiuc-Pietro
| Challenge: | Existing methods to handle catastrophic forgetting fail to retain knowledge learnt in the past when sudden shifts occur in training data distributions. |
| Approach: | They propose a learning rate scheduling method that preserves new information without strongly overwriting past knowledge. |
| Outcome: | The proposed method preserves new information without overwriting past knowledge in a multilingual continuous learning framework. |
Visually Grounded Continual Learning of Compositional Phrases (2020.emnlp-main)
Copied to clipboard
| Challenge: | Modern NLP systems rely on offline training and are inefficient for new tasks. |
| Approach: | They propose a visually grounded ContinuaL learning task which simulates the continual acquisition of compositional phrases from streaming visual scenes. |
| Outcome: | The proposed system improves on existing systems, but it's infeasible to store all possible compositions. |
Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora (2022.naacl-main)
Copied to clipboard
| Challenge: | Pretrained language models are typically learned over a large, static corpus and fine-tuned for various downstream tasks. |
| Approach: | They propose to continuously update a pretrained language model to adapt to emerging data and to keep track of the model's performance. |
| Outcome: | The proposed model can adapt to new corpora while retaining knowledge in earlier domains. |