Beyond Layout Embedding: Layout Attention with Gaussian Biases for Structured Document Understanding (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for encoding layout information rely on millions of learnable parameters . polar coordinates provide superior choice for layout modeling, study finds . |
| Approach: | They propose to model layout attention with Gaussian biases by feeding polar coordinates into 2-D Gausssian kernels. |
| Outcome: | The proposed model improves on three widely used benchmarks. |
Similar Papers
DocPolarBERT: A Pre-trained Model for Document Understanding with Relative Polar Coordinate Encoding of Layout Structures (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing models that take text block positions into account are not efficient for document understanding. |
| Approach: | They propose a layout-aware BERT model that takes into account text block positions in relative polar coordinate system rather than the Cartesian one. |
| Outcome: | The proposed model eliminates the need for absolute positional embeddings on a dataset more than six times smaller than the widely used IIT-CDIP corpus. |
Skim-Attention: Learning to Focus via Document Layout (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to document understanding have high computational and memory costs. |
| Approach: | They propose a new attention mechanism that takes advantage of the structure of a document and its layout. |
| Outcome: | The proposed attention mechanism obtains lower perplexity than previous studies while being more computationally efficient. |
ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understanding (2022.findings-emnlp)
Copied to clipboard
Qiming Peng, Yinxu Pan, Wenjin Wang, Bin Luo, Zhenyu Zhang, Zhengjie Huang, Yuhui Cao, Weichong Yin, Yongfeng Chen, Yin Zhang, Shikun Feng, Yu Sun, Hao Tian, Hua Wu, Haifeng Wang
| Challenge: | Existing methods for visually rich document understanding lack layout-centered knowledge . experimental results show that ERNIE-Layout improves layout awareness . |
| Approach: | They propose a document pre-training solution with layout knowledge enhancement in the whole workflow to learn better representations that combine the features from text, layout, and image. |
| Outcome: | The proposed model outperforms existing models on key downstream tasks. |
LayoutLLM: Large Language Model Instruction Tuning for Visually Rich Document Understanding (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods to enhance document comprehension require fine-tuning for each task and dataset, and are expensive to train and operate. |
| Approach: | They propose a more flexible document analysis method that integrates visual-rich document understanding with large-scale language models (LLMs) by leveraging existing research in document image understanding and LLMs’ superior language understanding capabilities, the proposed model performs an understanding of document images in a single model. |
| Outcome: | The proposed model improves on the baseline model in document image understanding tasks. |
Infinity-Parser: Layout-Aware Reinforcement Learning with High-quality Document Parsing Dataset (2026.findings-acl)
Copied to clipboard
Baode Wang, Biao Wu, Weizhen Li, Meng Fang, Zuming Huang, Jun Huang, Yanjie Liang, Haozhe Wang, Ling Chen, Wei Chu, Yuan Qi
| Challenge: | Existing supervised fine-tuning methods struggle to generalize across document types, leading to poor performance. |
| Approach: | They propose layoutRL, a reinforcement learning framework that optimizes layout understanding through composite rewards integrating normalized edit distance, paragraph count accuracy, and reading order preservation. |
| Outcome: | The proposed model outperforms specialized document parsing systems and general-purpose vision-language models on a broad range of document types, languages, and structural complexities. |
Information Representation Fairness in Long-Document Embeddings: The Peculiar Interaction of Positional and Language Bias (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that embedding models exhibit systematic positional and language biases when documents are longer and consist of multiple segments. |
| Approach: | They propose a permutation-based evaluation framework to quantify embedding biases . they propose an inference-time attention calibration method that redistributes attention more evenly across document positions . |
| Outcome: | The proposed framework reduces the positional and language biases in embedding models . the proposed framework improves the discoverability of later segments . |
Problem Solved? Information Extraction Design Space for Layout-Rich Documents using LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new open-source layout-aware IE test suite is available for download at https://github.com/gayecolakoglu/layIE-LLM. |
| Approach: | They propose an open-source layout-aware IE test suite that provides a layout-based IE pipeline. |
| Outcome: | The proposed method achieves 13.3–37.5 F1 points more than a baseline configuration using the same LLM. |
Modeling Context With Linear Attention for Scalable Document-Level Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Document-level machine translation models lack quadratic complexity in the sequence length due to their attention layers. |
| Approach: | They evaluate a recent linear attention model with a sentential gate to promote a recency inductive bias and compare it to open-source document translation. |
| Outcome: | The proposed model significantly improves translation quality on IWSLT 2015 and OpenSubtitles 2018 with similar or better BLEU scores. |
DocLLM: A Layout-Aware Generative Language Model for Multimodal Document Understanding (2024.acl-long)
Copied to clipboard
Dongsheng Wang, Natraj Raman, Mathieu Sibue, Zhiqiang Ma, Petr Babkin, Simerjot Kaur, Yulong Pei, Armineh Nourbakhsh, Xiaomo Liu
| Challenge: | Documents with rich layouts are a significant portion of enterprise corpora and document AI is still a challenge. |
| Approach: | They propose a lightweight extension to traditional large language models for reasoning over visual documents that takes into account both textual semantics and spatial layout. |
| Outcome: | The proposed model outperforms existing large language models on 14 out of 16 datasets and generalizes well to 4 out of 5 previously unseen datasets. |
LayoutLMv2: Multi-modal Pre-training for Visually-rich Document Understanding (2021.acl-long)
Copied to clipboard
Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, Lidong Zhou
| Challenge: | Existing pre-training tasks for text and layout are effective in visually-rich document understanding tasks. |
| Approach: | They propose to combine pre-training tasks with a multi-modal model to model interaction between text, layout and image in a single multi-module framework. |
| Outcome: | The proposed model outperforms LayoutLM by a large margin on visual-rich document understanding tasks. |