Papers by Yaping Zhang
LayoutDIT: Layout-Aware End-to-End Document Image Translation with Multi-Step Conductive Decoder (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods struggle to capture the visual layout in complex document images. |
| Approach: | They propose to integrate layout knowledge into document image translation by using a layout-aware encoder and a multi-step conductive decoder to achieve the translation step by step. |
| Outcome: | The proposed model outperforms state-of-the-art methods with better parameter efficiency. |
EmoHarbor: Evaluating Personalized Emotional Support by Simulating the User’s Internal World (2026.acl-long)
Copied to clipboard
| Challenge: | EmoHarbor is an evaluation framework that rewards generic empathetic responses but fails to assess whether the support is genuinely personalized to users’ unique psychological profiles and contextual needs. |
| Approach: | They propose an automated evaluation framework that adopts a User-as-a-Judge paradigm by simulating the user's inner world. |
| Outcome: | The proposed framework decomposes users' internal processes into three specialized roles and defines 10 evaluation dimensions of personalized support quality. |
From Generic Empathy to Personalized Emotional Support: A Self-Evolution Framework for User Preference Alignment (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) show great potential for expressing empathy, but often deliver generic responses that fail to address users’ specific needs. |
| Approach: | They propose a self-evolution framework to help LLMs improve their responses to better align with users’ implicit preferences concerning personality, emotional state, and specific context. |
| Outcome: | The proposed model significantly improves the model's performance in emotional support, reducing unhelpful responses and minimizing discrepancies between user preferences and model outputs. |
SweetieChat: A Strategy-Enhanced Role-playing Framework for Diverse Scenarios Handling Emotional Support Agent (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated promising potential in providing empathetic support during interactions, but their responses are often verbose or overly formulaic, failing to adequately address the diverse emotional support needs of real-world scenarios. |
| Approach: | They propose a strategy-enhanced role-playing framework that emulates real-world interactions and a dataset that is used to develop an emotional support agent. |
| Outcome: | The proposed framework emulates real-world interactions and promotes a broader range of dialogues and Emotional Support Agent training. |
A Query-Response Framework for Whole-Page Complex-Layout Document Image Translation with Relevant Regional Concentration (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for document image translation rely on the vanilla encoder-decoder paradigm . a novel dynamic aggregation mechanism is designed to enhance the text semantics in query features toward translation. |
| Approach: | They propose a Query-Response DIT framework that reformulates the DIT task into a parallel response/translation process of multiple queries. |
| Outcome: | The proposed framework improves translation quality on four translation directions on three benchmarks. |
Single-to-mix Modality Alignment with Multimodal Large Language Model for Document Image Machine Translation (2025.acl-long)
Copied to clipboard
| Challenge: | Document Image Machine Translation (DIMT) faces generalization challenges due to limited training data and the complex interplay between visual and textual information. |
| Approach: | They propose a single-to-mix Modality alignment framework leveraging Multimodal Large Language Models (MLLMs) this framework aligns an imageonly encoder with multimodal representations of an MLLM pre-trained on large-scale document image datasets. |
| Outcome: | The proposed framework improves translation quality in cross-domain generalization and challenging document image scenarios. |
Document Image Machine Translation with Dynamic Multi-pre-trained Models Assembling (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing TIMT tasks focus on text-line-level images. |
| Approach: | They propose to extend the existing TIMT task and introduce a new framework to translate a source document image to markdown-formatted target translation. |
| Outcome: | The proposed task aims to translate a source document image with long context and complex layout structure to markdown-formatted target translation. |
CCIM: Cross-modal Cross-lingual Interactive Image Translation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing research on text image machine translation (TIMT) lacks recognized source language information resulting in a decrease in translation performance. |
| Approach: | They propose a cross-modal cross-lingual interactive model which incorporates source language information by synchronizing source and target language results. |
| Outcome: | The proposed model outperforms end-to-end models and has faster decoding speed with smaller model size than cascade models. |
Born a BabyNet with Hierarchical Parental Supervision for End-to-End Text Image Machine Translation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing research on text image machine translation (TIMT) is divided into two types: Cascade methods combine text image recognition and MT models to recognize source language text images. |
| Approach: | They propose a method which is optimized with hierarchical parental supervision to improve translation performance. |
| Outcome: | The proposed method significantly outperforms existing methods on synthetic and real-world tests on both synthetic and realistic images. |
Improving MLLM’s Document Image Machine Translation via Synchronously Self-reviewing Its OCR Proficiency (2025.findings-acl)
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) have shown strong performance in document image tasks, especially Optical Character Recognition (OCR). However, they struggle with Document Image Machine Translation (DIMT), which requires handling both cross-modal and cross-lingual challenges. |
| Approach: | They propose a novel fine-tuning paradigm that allows the model to generate OCR text before producing translation text, which allows it to leverage its strong monolingual OCR ability while learning to translate text across languages. |
| Outcome: | The proposed model can leverage its strong monolingual OCR ability while learning to translate text across languages. |
From Chaotic OCR Words to Coherent Document: A Fine-to-Coarse Zoom-Out Network for Complex-Layout Document Image Translation (2025.coling-main)
Copied to clipboard
| Challenge: | Document Image Translation (DIT) aims to translate documents in images from one language to another. |
| Approach: | They propose a novel end-to-end network called Zoom-out DIT to improve document translation by combining word positioning, sentence recognition and document organization. |
| Outcome: | The proposed network improves word positioning, sentence recognition and document organization, and improves translation quality. |