Papers by Haozhe Ji
Denoising Distantly Supervised Open-Domain Question Answering (P18-1)
Copied to clipboard
| Challenge: | Existing DS-QA models ignore rich information contained in other paragraphs and are noisy . Existing systems rely on pre-identified relevant texts, which do not always exist in real-world QA scenarios. |
| Approach: | They propose a model which uses a paragraph selector to filter out noisy paragraphs and a reader to extract the correct answer from denoised paragraphs. |
| Outcome: | The proposed model can capture useful information from noisy data and achieve significant improvements on open domain question answering. |
LaMemo: Language Modeling with Look-Ahead Memory (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to model long-term dependencies are limited to long texts with thousands of words. |
| Approach: | They propose a look-ahead memory that augments the recurrence memory by attending to the right-side tokens and interpolating with the old memory states to maintain long-term information in the history. |
| Outcome: | Experiments on widely used language modeling benchmarks show that LaMemo outperforms baseline models with recurrence memory. |
DiscoDVT: Generating Long Text with Discourse-Aware Discrete Variational Transformer (2021.emnlp-main)
Copied to clipboard
| Challenge: | Generating long passages that maintain long-range coherence is a long-standing problem in natural language generation (NLG). |
| Approach: | They propose a discourse-aware discrete variational Transformer that learns a latent variable sequence that summarizes the global structure of the text and then applies it to guide the generation process at each decoding step. |
| Outcome: | The proposed model can generate long texts with better long-range coherence by learning a latent variable sequence with each latent code. |
Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches that integrate commonsense knowledge into pre-trained language models simply transfer relational knowledge while ignoring rich connections within the knowledge graph. |
| Approach: | They propose a method that leverages structural and semantic information of the knowledge graph to generate commonsense-aware text. |
| Outcome: | The proposed method outperforms baseline models on three text generation tasks that require reasoning over commonsense knowledge. |
JointGT: Graph-Text Joint Representation Learning for Text Generation from Knowledge Graphs (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing pre-trained models for knowledgegraph-to-text generation ignore graph structure during encoding and lack elaborate pre-training tasks to explicitly model graph-text alignments. |
| Approach: | They propose a graph-text joint representation learning model called JointGT which incorporates a structure-aware semantic aggregation module into each Transformer layer to preserve the graph structure. |
| Outcome: | The proposed model achieves state-of-the-art performance on various KG-to-text datasets. |
Generating Commonsense Explanation by Extracting Bridge Concepts from Reasoning Paths (2020.aacl-main)
Copied to clipboard
| Challenge: | Existing tasks that use commonsense reasoning as multi-choice reading comprehension lack direct assessment to machine commonsence and impede its practicability to realistic scenarios. |
| Approach: | They propose a method that first extracts the underlying concepts which are served as bridges in the reasoning chain and then integrates these concepts to generate the final explanation. |
| Outcome: | The proposed model outperforms the state-of-the-art models in automatic and human evaluation. |
SentiLARE: Sentiment-Aware Language Representation Learning with Linguistic Knowledge (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-trained models neglect to consider linguistic knowledge of texts . existing models neglect linguistic information, which is important for sentiment analysis . |
| Approach: | They propose a model that introduces word-level linguistic knowledge into pre-trained models to enhance sentiment analysis by querying SentiWordNet to acquire sentiment polarity. |
| Outcome: | The proposed model obtains state-of-the-art performance on a variety of sentiment analysis tasks. |