LayoutDIT: Layout-Aware End-to-End Document Image Translation with Multi-Step Conductive Decoder (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods struggle to capture the visual layout in complex document images. |
| Approach: | They propose to integrate layout knowledge into document image translation by using a layout-aware encoder and a multi-step conductive decoder to achieve the translation step by step. |
| Outcome: | The proposed model outperforms state-of-the-art methods with better parameter efficiency. |
Similar Papers
A Query-Response Framework for Whole-Page Complex-Layout Document Image Translation with Relevant Regional Concentration (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for document image translation rely on the vanilla encoder-decoder paradigm . a novel dynamic aggregation mechanism is designed to enhance the text semantics in query features toward translation. |
| Approach: | They propose a Query-Response DIT framework that reformulates the DIT task into a parallel response/translation process of multiple queries. |
| Outcome: | The proposed framework improves translation quality on four translation directions on three benchmarks. |
LayoutLMv2: Multi-modal Pre-training for Visually-rich Document Understanding (2021.acl-long)
Copied to clipboard
Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, Lidong Zhou
| Challenge: | Existing pre-training tasks for text and layout are effective in visually-rich document understanding tasks. |
| Approach: | They propose to combine pre-training tasks with a multi-modal model to model interaction between text, layout and image in a single multi-module framework. |
| Outcome: | The proposed model outperforms LayoutLM by a large margin on visual-rich document understanding tasks. |
From Chaotic OCR Words to Coherent Document: A Fine-to-Coarse Zoom-Out Network for Complex-Layout Document Image Translation (2025.coling-main)
Copied to clipboard
| Challenge: | Document Image Translation (DIT) aims to translate documents in images from one language to another. |
| Approach: | They propose a novel end-to-end network called Zoom-out DIT to improve document translation by combining word positioning, sentence recognition and document organization. |
| Outcome: | The proposed network improves word positioning, sentence recognition and document organization, and improves translation quality. |
LayoutMask: Enhance Text-Layout Interaction in Multi-modal Pre-training for Document Understanding (2023.acl-long)
Copied to clipboard
| Challenge: | Pre-trained models on document images with transformer-based backbones have led to significant performance gains in this field. |
| Approach: | They propose a multi-modal pre-training model that combines text, layout and image . they propose to use local 1D position instead of global 1D positions as layout input . |
| Outcome: | The proposed model can achieve state-of-the-art results on a wide variety of VrDU problems. |
PEIT: Bridging the Modality Gap with Pre-trained Models for End-to-End Image Translation (2023.acl-long)
Copied to clipboard
| Challenge: | Image translation is a task that translates an image containing text in the source language to the target language. |
| Approach: | They propose an end-to-end image translation framework that bridges the modality gap between visual inputs and textual inputs/outputs of machine translation (MT). |
| Outcome: | The proposed framework outperforms existing models on a large-scale image translation corpus . it significantly outperformed both cascaded and strong models on the e-commerce domain . |
Intelligent Document Parsing: Towards End-to-end Document Parsing via Decoupled Content Parsing and Layout Grounding (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods fragment document parsing into pipeline of separated subtasks, resulting in incomplete semantics and error propagation. |
| Approach: | They propose an end-to-end document parsing framework that leverages vision-language priors of MLLMs. |
| Outcome: | The proposed method surpasses existing methods significantly in document parsing . it leverages the vision-language priors of MLLMs to decouple parse and layout grounding based on visual information. |
BabelDOC: Better Layout-Preserving PDF Translation via Intermediate Representation (2026.acl-demo)
Copied to clipboard
| Challenge: | Existing document translation pipelines face a tension between linguistic processing and layout preservation. |
| Approach: | They propose a framework for layout-preserving PDF translation that decouples visual layout metadata from semantic content. |
| Outcome: | The proposed framework improves layout fidelity, visual aesthetics, and terminology consistency over representative baselines while maintaining competitive translation precision. |
Document Image Machine Translation with Dynamic Multi-pre-trained Models Assembling (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing TIMT tasks focus on text-line-level images. |
| Approach: | They propose to extend the existing TIMT task and introduce a new framework to translate a source document image to markdown-formatted target translation. |
| Outcome: | The proposed task aims to translate a source document image with long context and complex layout structure to markdown-formatted target translation. |
DocPolarBERT: A Pre-trained Model for Document Understanding with Relative Polar Coordinate Encoding of Layout Structures (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing models that take text block positions into account are not efficient for document understanding. |
| Approach: | They propose a layout-aware BERT model that takes into account text block positions in relative polar coordinate system rather than the Cartesian one. |
| Outcome: | The proposed model eliminates the need for absolute positional embeddings on a dataset more than six times smaller than the widely used IIT-CDIP corpus. |
LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understanding (2022.acl-long)
Copied to clipboard
| Challenge: | Existing structured document understanding models only deal with document data of specific language(s) this is extremely limited for other languages, especially in the case of lacking pre-training structured document data. |
| Approach: | They propose a language-independent Layout Transformer (LiLT) for structured document understanding . they propose to pre-train structured documents in a single language and fine-tune them on other languages . |
| Outcome: | The proposed model achieves competitive or even superior performance on diverse downstream benchmarks on eight languages. |