Papers by Hao Tian
ERNIE-Doc: A Retrospective Long-Document Modeling Transformer (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models for document-level language pretraining are not suitable for long documents due to their quadratically increasing memory and time consumption. |
| Approach: | They propose a document-level language pretraining model based on Recurrence Transformers. |
| Outcome: | The proposed model outperforms existing models on language understanding tasks. |
SKEP: Sentiment Knowledge Enhanced Pre-training for Sentiment Analysis (2020.acl-main)
Copied to clipboard
| Challenge: | sentiment knowledge is ignored in sentiment analysis, despite its use in pretraining. |
| Approach: | They propose to use sentiment knowledge to learn a unified sentiment representation for multiple sentiment analysis tasks. |
| Outcome: | The proposed method outperforms strong pre-training baseline on three kinds of sentiment tasks. |
PairCoder: Pair Programming-Inspired Two-Agent Collaboration for Code Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing multi agent frameworks for large language models are brittle on code generation tasks. |
| Approach: | They propose a framework that brings pair programming to autonomous LLM collaboration. |
| Outcome: | Using PairCoder, large language models achieve better results on code generation tasks and reduce token usage by 40% to 70% on eight representative backbones. |
Train Once, and Decode As You Like (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to machine translation support autoregressive, semi-autoregressive and refinement-based non-auto-regressives. |
| Approach: | They propose a unified approach for supporting different generation manners of machine translation including autoregressive, semi-autoregressive and refinement-based non-auto-regressives. |
| Outcome: | The proposed approach achieves better or competitive translation performance compared with strong baseline models in all the settings. |
Path-enhanced Pre-trained Language Model for Knowledge Graph Completion (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models have achieved remarkable knowledge graph completion (KGC) success. |
| Approach: | They propose a path-enhanced pre-trained language model-based knowledge graph completion method which uses multi-view generation to infer missing facts in triple-level and path-level simultaneously. |
| Outcome: | The proposed method significantly improves the performance of the knowledge graph completion task. |
A Framework of Knowledge Graph-Enhanced Large Language Model Based on Question Decomposition and Atomic Retrieval (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to enhance LLMs with knowledge graphs have limited results . knowledge graph question answering (KGQA) provides interpretable reasoning for large language models . |
| Approach: | They propose a framework for KG-enhanced LLM based on question decomposition and atomic retrieval . they propose question decomposing tree as framework for LLM reasoning . |
| Outcome: | The proposed framework outperforms existing reasoning-based baselines on KGQA datasets. |
Friendly Topic Assistant for Transformer Based Abstractive Summarization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive document summarization is a comprehensive task in natural language processing. |
| Approach: | They propose a topic assistant that rearranges and learns document semantics . they propose TA that is compatible with Transformer-based models and user-friendly . |
| Outcome: | The proposed model is compatible with Transformer-based models and user-friendly. |
Clip-Tuning: Towards Derivative-free Prompt Learning with a Mixture of Rewards (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work does not take full advantage of over-parameterized characteristics of large pre-trained language models. |
| Approach: | They propose a method that uses frozen "thinned" networks to obtain a mixture of rewards and advance the derivative-free prompt learning. |
| Outcome: | The proposed method outperforms previous gradient-free prompt learning methods and achieves parity with gradient-based counterparts on seven language understanding benchmarks under few-shot settings. |
Exploring Logically Dependent Multi-task Learning with Causal Inference (2020.emnlp-main)
Copied to clipboard
| Challenge: | Hierarchical multi-task learning models can utilize task dependencies by stacking encoders and outperform democratic ones. |
| Approach: | They propose a model that utilizes the labels of all lower-level tasks and a Gumbel sampling model to deal with cascading errors. |
| Outcome: | The proposed model outperforms democratic models on six out of seven subtasks and achieves state-of-the-art on the two English and one Chinese datasets. |
ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for pretraining cross-lingual models are limited in their size due to the limited amount of parallel corpora. |
| Approach: | They propose a method that encourages the model to align multiple languages with monolingual corpora to overcome the constraint of the parallel corpus size. |
| Outcome: | The proposed method outperforms existing cross-lingual models and delivers new state-of-the-art results in various cross-linguistic downstream tasks. |
FAER: Benchmarking VLMs for Failure-Aware Embodied Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Visual-language models (VLMs) are the core component of embodied agents in perceiving the environment and making decisions. |
| Approach: | They propose a failure-aware benchmark to evaluate the performance of visual language models (VLMs) in long-horizon tasks. |
| Outcome: | The proposed benchmark evaluates the performance of 16 widely utilized VLMs and 4 LLMs for FAER tasks. |
To What Extent Do Natural Language Understanding Datasets Correlate to Logical Reasoning? A Method for Diagnosing Logical Reasoning. (2022.coling-1)
Copied to clipboard
| Challenge: | Reasoning and knowledge-related skills are considered as fundamental skills for natural language understanding (NLU) tasks. |
| Approach: | They propose a method to diagnose correlations between an NLU dataset and a specific skill. |
| Outcome: | The proposed method is able to diagnose correlations between dataset and logical reasoning skill on 8 MRC and 3 NLI datasets. |
Chain-of-Thought Tuning: Masked Language Models can also Think Step By Step in Natural Language Understanding (2023.emnlp-main)
Copied to clipboard
| Challenge: | Chain-of-Thought (CoT) is a technique that guides large language models to decompose complex tasks into multi-step reasoning processes. |
| Approach: | They propose a two-step reasoning framework based on prompt tuning to implement step-by-step thinking for MLMs on NLU tasks. |
| Outcome: | The proposed framework outperforms baselines and achieves state-of-the-art performance on two NLU tasks. |
De-Confounded Variational Encoder-Decoder for Logical Table-to-Text Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Logical table-to-text generation is challenging where deep learning models capture surface-level spurious correlations rather than the causal relationships between the table x and the sentence y. |
| Approach: | They propose to use variational inference to estimate the confounders in the latent space and cooperate with the causal intervention based on Pearl’s do-calculus to alleviate the spurious correlations. |
| Outcome: | The proposed model outperforms baselines and achieves new state-of-the-art performance on two logical table-to-text datasets in terms of logical fidelity. |
A Semantically Consistent and Syntactically Variational Encoder-Decoder Framework for Paraphrase Generation (2020.coling-main)
Copied to clipboard
| Challenge: | Paraphrase generation is a longstanding problem in natural language processing (NLP) Neural network-based methods have shown great progress on paraphrase generation. |
| Approach: | They propose a framework that integrates variational inference on a target-related latent variable to introduce the diversity. |
| Outcome: | The proposed framework outperforms baseline models on the metrics based on n-gram matching and semantic similarity, and it can generate multiple different paraphrases by assembling different syntactic variables. |
Augmenting Reasoning Capabilities of LLMs with Graph Structures in Knowledge Base Question Answering (2024.findings-emnlp)
Copied to clipboard
Yuhang Tian, Dandan Song, Zhijing Wu, Changzhi Zhou, Hao Wang, Jun Yang, Jing Xu, Ruanmin Cao, HaoYu Wang
| Challenge: | Recent work uses Large Language Models (LLMs) for semantic parsing to address Knowledge Base Question Answering tasks. |
| Approach: | They propose a framework that augments reasoning capabilities of LLMs with Graph Structures in Knowledge Base Question Answering to retrieve question-related graph structures. |
| Outcome: | The proposed framework outperforms existing methods on GrailQA and WebQSP under the few-shot setting. |
Context Length Alone Hurts LLM Performance Despite Perfect Retrieval (2025.findings-emnlp)
Copied to clipboard
Yufeng Du, Minyang Tian, Srikanth Ronanki, Subendhu Rongali, Sravan Babu Bodapati, Aram Galstyan, Azton Wells, Roy Schwartz, Eliu A Huerta, Hao Peng
| Challenge: | Large language models (LLMs) often fail to scale their performance on long-context tasks performance in line with the context lengths they support. |
| Approach: | They propose a model-agnostic mitigation strategy that transforms a long-context task into a short-concept one by prompting the model to recite the retrieved evidence before attempting to solve the problem. |
| Outcome: | The proposed model improves on a long-context task up to 4% on RULER. |
Diagnosing the First-Order Logical Reasoning Ability Through LogicNLI (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have focused on diagnosing LMs' reasoning abilities in natural language understanding tasks. |
| Approach: | They propose a diagnostic method for first-order logic reasoning with a proposed benchmark, LogicNLI. |
| Outcome: | The proposed method disentangles the target FOL reasoning from commonsense inference and can be used to diagnose LMs from four perspectives: accuracy, robustness, generalization, and interpretability. |
Unraveling the Mechanics of Learning-Based Demonstration Selection for In-Context Learning (2025.acl-long)
Copied to clipboard
| Challenge: | Recent learning-based demonstration selection methods have proven beneficial to in-context learning (ICL) by choosing more useful exemplars. |
| Approach: | They propose two methods to capture task-agnostic similarities between input and output of LLMs. |
| Outcome: | The proposed methods integrate task-agnostic similarities of different levels between input and output of exemplars and test cases to eliminate costly data collection. |
MTR: A Dataset Fusing Inductive, Deductive, and Defeasible Reasoning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets for logical reasoning focus on monotonic logic and a single form of reasoning. |
| Approach: | They propose to use a dataset to study the human-like reasoning in machine reading comprehension. |
| Outcome: | The proposed dataset shows that state-of-the-art neural models perform noticeably worse than expected. |
GRV-KBQA: A Three-Stage Framework for Knowledge Base Question Answering with Decoupled Logical Structure, Semantic Grounding and Structure-Aware Validation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for Knowledge Base Question Answering generate non-executable queries and inefficiencies in query execution. |
| Approach: | a framework that decouples logical structure generation from semantic grounding is proposed . the framework explicitly enforces KB constraints to improve alignment between generated logical forms and KB structures. |
| Outcome: | GRV-KBQA decouples logical structure generation from semantic grounding and incorporates structure-aware validation to enhance accuracy. |
ERNIE-Code: Beyond English-Centric Cross-lingual Pretraining for Programming Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | ERNIE-Code is a unified pre-trained language model for 116 NLs and 6 PLs. |
| Approach: | They propose a unified pre-trained language model for 116 NLs and 6 PLs . they employ span-corruption language modeling that learns patterns from monolingual NL or PL . |
| Outcome: | The proposed model outperforms previous multilingual models for NL or NL across end tasks. |
Task-Level Thinking Steps Help Large Language Models for Challenging Classification Task (2023.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results prove the superiority of our proposed method on challenging classification tasks. |
| Approach: | They propose a task-level thinking step that eliminates bias introduced by demonstrations . they propose 'progressive revision framework' which can improve the thinking steps by correcting hard demonstrations. |
| Outcome: | The proposed method achieves best performance on three kinds of classification tasks in zero-shot and few-shot settings. |
PRISM-MCTS: Learning from Reasoning Trajectories with Metacognitive Reflection (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing reasoning models are limited by inefficiency and computational redundancy . PRISM-MCTS integrates a process reward model with a dynamic shared memory . |
| Approach: | They propose a reasoning framework that integrates a process reward model with a dynamic shared memory. |
| Outcome: | PRISM-MCTS integrates a process reward model with a dynamic shared memory . it halves trajectory requirements on GPQA while surpassing MCTS-RAG and Search-o1 . |
ERNIE-Gram: Pre-Training with Explicitly N-Gram Masked Language Modeling for Natural Language Understanding (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to model coarse-grained linguistic information do not integrate coarse-gram information into pre-training. |
| Approach: | They propose an explicitly n-gram masking method to enhance integration of coarse-grained linguistic information into pre-training. |
| Outcome: | The proposed method outperforms existing models on English and Chinese text corpora and fine-tunes on 19 downstream tasks. |
LongTableBench: Benchmarking Long-Context Table Reasoning across Real-World Formats and Domains (2025.findings-emnlp)
Copied to clipboard
Liyao Li, Jiaming Tian, Hao Chen, Wentao Ye, Chao Ye, Haobo Wang, Ningtao Wang, Xing Fu, Gang Chen, Junbo Zhao
| Challenge: | Evaluating 52 LLMs reveals that only the strongest models maintain robust performance under increasing context lengths and format diversity. |
| Approach: | They propose a benchmark for evaluating long-context reasoning over semi-structured tables across diverse formats, tasks, and domains. |
| Outcome: | The proposed model outperforms compression-based approaches on tasks requiring semantic integration. |
360∘REA: Towards A Reusable Experience Accumulation with 360∘ Assessment for Multi-Agent System (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies focus on optimizing the agent team or employing self-reflection to iteratively solve complex tasks. |
| Approach: | They propose a hierarchical multi-agent framework that uses 360 assessment to accumulate experience through fine-grained assessment. |
| Outcome: | The proposed framework is based on corporate organizational practices and employs a dual-level experience pool for agents to accumulate experience through fine-grained assessment. |
ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understanding (2022.findings-emnlp)
Copied to clipboard
Qiming Peng, Yinxu Pan, Wenjin Wang, Bin Luo, Zhenyu Zhang, Zhengjie Huang, Yuhui Cao, Weichong Yin, Yongfeng Chen, Yin Zhang, Shikun Feng, Yu Sun, Hao Tian, Hua Wu, Haifeng Wang
| Challenge: | Existing methods for visually rich document understanding lack layout-centered knowledge . experimental results show that ERNIE-Layout improves layout awareness . |
| Approach: | They propose a document pre-training solution with layout knowledge enhancement in the whole workflow to learn better representations that combine the features from text, layout, and image. |
| Outcome: | The proposed model outperforms existing models on key downstream tasks. |
LAiW: A Chinese Legal Large Language Models Benchmark (2025.coling-main)
Copied to clipboard
Yongfu Dai, Duanyu Feng, Jimin Huang, Haochen Jia, Qianqian Xie, Yifang Zhang, Weiguang Han, Wei Tian, Hao Wang
| Challenge: | Xie et al., 2023) show that large language models (LLMs) can generate legal text, but lack the legal syllogism . legal experts are cautious about their practical application due to the opaque nature of the LLMs. |
| Approach: | They propose a Chinese legal LLM benchmark structured around the legal syllogism . they evaluate LLMs across three levels of capability, each reflecting a more complex stage of legal . |
| Outcome: | The proposed benchmark identifies that LLMs lack the legal syllogism, which hinders trust and understanding from legal experts. |
CompKBQA: Component-wise Task Decomposition for Knowledge Base Question Answering (2025.emnlp-main)
Copied to clipboard
Yuhang Tian, Dandan Song, Zhijing Wu, Pan Yang, Changzhi Zhou, Jun Yang, Hao Wang, Huipeng Ma, Chenhao Li, Luan Zhang
| Challenge: | Existing knowledge base question answering methods struggle with complex queries. |
| Approach: | They propose a framework that optimizes the process of fine-tuning a LLM for generating logical forms by enabling it to learn relevant sub-tasks like skeleton generation, topic entity generation, and relevant relations generation. |
| Outcome: | The proposed framework achieves state-of-the-art on two benchmark KBQA datasets, WebQSP and CWQ. |
JumpCoder: Go Beyond Autoregressive Coder via Online Modification (2024.acl-long)
Copied to clipboard
| Challenge: | Existing code large language models lack reversibility and autoregressive sequential generation is incapable of correcting previous missing statements as humans do. |
| Approach: | They propose a model-agnostic framework that enables human-like online modification and non-sequential generation to augment code large language models. |
| Outcome: | The proposed framework enables human-like modification and non-sequential generation to augment code large language models. |