Papers by Renshen Wang
FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extraction (2022.acl-long)
Copied to clipboard
Chen-Yu Lee, Chun-Liang Li, Timothy Dozat, Vincent Perot, Guolong Su, Nan Hua, Joshua Ainslie, Renshen Wang, Yasuhisa Fujii, Tomas Pfister
| Challenge: | Form-like document understanding is a surging research topic due to its practical applications . form documents have unique challenges stemming from their structural characteristics . |
| Approach: | They propose a structure-aware sequence model that leverages spatial relationships between tokens in a form for more precise attention score calculation. |
| Outcome: | The proposed model outperforms existing methods with a more compact model size and less pre-training data. |
FormNetV2: Multimodal Graph Contrastive Learning for Form Document Information Extraction (2023.acl-long)
Copied to clipboard
Chen-Yu Lee, Chun-Liang Li, Hao Zhang, Timothy Dozat, Vincent Perot, Guolong Su, Xiang Zhang, Kihyuk Sohn, Nikolay Glushnev, Renshen Wang, Joshua Ainslie, Shangbang Long, Siyang Qin, Yasuhisa Fujii, Nan Hua, Tomas Pfister
| Challenge: | Existing approaches that extend the mask language modeling to other modalities require careful multi-task tuning, complex reconstruction target designs, or additional pre-training data. |
| Approach: | They propose a centralized multimodal graph contrastive learning strategy to unify self-supervised pre-training for all modalities in one loss. |
| Outcome: | The proposed model achieves state-of-the-art performance on FUNSD, CORD, SROIE and Payment benchmarks with a more compact model size. |
ROPE: Reading Order Equivariant Positional Encoding for Graph-based Document Information Extraction (2021.acl-short)
Copied to clipboard
Chen-Yu Lee, Chun-Liang Li, Chu Wang, Renshen Wang, Yasuhisa Fujii, Siyang Qin, Ashok Popat, Tomas Pfister
| Challenge: | Graph Convolutional Networks (GCNs) have limited ability to capture reading orders of given word-level node representations in a graph. |
| Approach: | They propose a new positional encoding technique to capture word-level nodes in a graph. |
| Outcome: | The proposed method improves existing GCNs with an 8.4% F1 score on two datasets and a large-scale payment dataset. |