| Challenge: | Existing methods for text classification are not scalable to large corpus and ignore heterogeneity of text graph. |
| Approach: | They propose a Transformer-based heterogeneous graph neural network that captures structure and heterogenity from the text graph. |
| Outcome: | The proposed model outperforms state-of-the-art methods on large-sized corpus datasets and significantly reduces computing and memory costs. |
Similar Papers
Connecting the Dots: What Graph-Based Text Representations Work Best for Text Classification using Graph Neural Networks? (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Graph Neural Networks have been used for text classification, but only in domains with limited data characteristics. |
| Approach: | They compare graph representation methods for text classification using different architectures and setups. |
| Outcome: | The proposed graph representation methods outperform other models in document comprehension tasks. |
Text Level Graph Neural Network for Text Classification (D19-1)
Copied to clipboard
| Challenge: | Recent researches have explored graph neural network (GNN) techniques on text classification, but they are faced with the problems of fixed corpus level graph structure which don’t support online testing and high memory consumption. |
| Approach: | They propose a graph neural network model that builds graphs for each input text with global parameters sharing instead of a single graph for the whole corpus. |
| Outcome: | The proposed model outperforms existing models on several text classification datasets even with consuming less memory. |
Heterogeneous Graph Transformer for Graph-to-Sequence Learning (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies ignore the indirect relations between distance nodes, or treat indirect relations and direct relations in the same way. |
| Approach: | They propose a graph-to-sequence (Graph2Seq) encoder which models graph structure to model different relations in individual subgraphs of the original graph. |
| Outcome: | The proposed model outperforms the state-of-the-art on all four benchmarks of AMR-to-text generation and syntax-based neural machine translation. |
Every Document Owns Its Structure: Inductive Text Classification via Graph Neural Networks (2020.acl-main)
Copied to clipboard
| Challenge: | Existing graph-based methods for text classification cannot capture contextual word relationships within each document nor can they produce inductive learning of new words. |
| Approach: | They propose to use Graph Neural Networks to learn the local word representations and then aggregate the word nodes as the document embeddings. |
| Outcome: | The proposed method outperforms state-of-the-art methods on four benchmark datasets. |
Improving Graph-Based Text Representations with Character and Word Level N-grams (2022.aacl-short)
Copied to clipboard
| Challenge: | Graph-based text representation is important in downstream natural language processing tasks. |
| Approach: | They propose a heterogeneous word-character text graph that combines word and character n-gram nodes together with document nodes. |
| Outcome: | The proposed graph outperforms baselines and state-of-the-art models in text classification and automatic summarization. |
HETFORMER: Heterogeneous Transformer with Sparse Attention for Long-Text Extractive Summarization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for summarizing semantic graph structure from raw text are cumbersome and inefficient for long-text documents. |
| Approach: | They propose a Transformer-based pre-trained model with multi-granularity sparse attentions for long-text extractive summarization. |
| Outcome: | The proposed model performs state-of-the-art on single- and multi-document summarization tasks while using less memory and fewer parameters. |
HeterGraphLongSum: Heterogeneous Graph Neural Network with Passage Aggregation for Extractive Long Document Summarization (2022.coling-1)
Copied to clipboard
| Challenge: | Existing models for extractive document summarization are based on sequence-to-sequence (Seq2Sequency) but long-form document summaries using graph-based methods are still an open research issue. |
| Approach: | They propose a heterogeneous graph neural network model to improve the performance of extractive document summarization using graph-based methods. |
| Outcome: | The proposed model can achieve state-of-the-art performance without pre-trained language models. |
Multi Graph Neural Network for Extractive Long Document Summarization (2022.coling-1)
Copied to clipboard
| Challenge: | Heterogeneous Graph Neural Networks (GNN) have been proposed as an emergent approach for extracting document summarization (EDS) but there are still limitations in applying it for long documents due to the lack of inter-sentence connections. |
| Approach: | They propose to build a graph on sentence-level nodes and combine it with HeterGNN to capture the semantic information in terms of both inter and intra-sentence connections. |
| Outcome: | Experiments on two datasets show that the proposed method achieves state-of-the-art in this research field. |
Heterogeneous Graph Neural Networks for Extractive Document Summarization (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models capture cross-sentence relations with recurrent neural networks, but they are hard to capture sentence-level long-distance dependency. |
| Approach: | They propose a graph-based neural network for extractive summarization which contains semantic nodes apart from sentences. |
| Outcome: | The proposed graph-based neural network is the first to incorporate different types of nodes into it and perform a qualitative analysis. |
Beyond Text: Incorporating Metadata and Label Structure for Multi-Label Document Classification using Heterogeneous Graphs (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for multi-label document classification ignore the heterogeneous graphical structures of metadata and labels. |
| Approach: | They propose a neural network based approach to multi-label document classification that uses two heterogeneous graphs to model metadata and labels. |
| Outcome: | The proposed approach outperforms state-of-the-art models on two benchmark datasets. |