Papers by Tiejun Zhao

45 papers
Disentangling Reasoning Capabilities from Language Models with Compositional Reasoning Transformers (2023.findings-acl)

Copied to clipboard

Challenge: ReasonFormer is a unified reasoning framework for complex decision-making . it is based on the dual-process theory of cognitive science, where two cognitive systems interact to form a whole reasoning process.
Approach: They propose a unified reasoning framework that mirrors the modular reasoning process of humans . they decouple the representation module and the reasoning modules to capture different levels of cognition .
Outcome: The proposed framework shows that humans can perform better in complex decision-making tasks.
LLM-based Translation Inference with Iterative Bilingual Understanding (2025.findings-acl)

Copied to clipboard

Challenge: Existing studies show that the ability of large language models to generate contextual understanding of the sentence can degrade translation quality.
Approach: They propose a method that generates contextual understanding for both source and target languages separately.
Outcome: The proposed method outperforms strong comparison methods in multiple domains.
Cross Copy Network for Dialogue Generation (2020.emnlp-main)

Copied to clipboard

Challenge: Despite the success of sequence-to-sequence models, dialogue logics are often ignored.
Approach: They propose a network architecture to explore the current dialog context and similar dialogue instances’ logical structure simultaneously.
Outcome: The proposed network architecture is superior to existing state-of-the-art models.
Robust Machine Reading Comprehension by Learning Soft labels (2020.coling-main)

Copied to clipboard

Challenge: Neural models have achieved great success on the task of machine reading comprehension, which are typically trained on hard labels.
Approach: They propose a robust training method for machine reading comprehension models to address label sparseness problem by using three strategies to train models on soft labels.
Outcome: The proposed method improves the baseline model performance and achieves state-of-the-art performance on NewsQA and QUOREF.
Make Imagination Clearer! Stable Diffusion-based Visual Imagination for Multimodal Machine Translation (2025.acl-long)

Copied to clipboard

Challenge: Experimental results show that our model significantly outperforms existing multimodal MT and text-only MT.
Approach: They propose a stable diffusion-based imagination network into a multimodal large language model to generate an image for each source sentence.
Outcome: The proposed model outperforms existing multimodal and text-only MT and achieves an average improvement of 14 BLEU points on Multi30K and MSCOCO multimodal MT benchmarks.
A Chain-of-Task Framework for Instruction Tuning of LLMs Based on Chinese Grammatical Error Correction (2025.coling-main)

Copied to clipboard

Challenge: Existing approaches to address Grammatical Error Correction (GEC) tasks are based on large scale labeled data, which leads to extremely high data annotation costs.
Approach: They propose a Chain-of-Task framework to reduce over-correction in large language models . they propose supervised fine-tuning strategy and an algorithm for automatic dataset annotation .
Outcome: The proposed framework achieves state-of-the-art on both FCGEC (in-domain) and NaCGEC (out-of domain) test sets.
DKME: Rethinking Coupled Knowledge Memory for Lifelong Model Editing of Large Language Models (2026.findings-acl)

Copied to clipboard

Challenge: Existing memory-based editors suffer from catastrophic forgetting as edits accumulate.
Approach: They propose a method which injects factual updates into large language models without retraining or finetuning into existing memory-based editors.
Outcome: Experiments on HalluEditBench, CKnowEdit, and WikiDatacounterfact show that the proposed model achieves a more favorable trade-off between editing success and locality compared to baselines while maintaining more stable performance as the edit scale increases.
An Empirical Study of LLM-as-a-Judge for LLM Evaluation: Fine-tuned Judge Model is not a General Substitute for GPT-4 (2025.findings-acl)

Copied to clipboard

Challenge: Recent studies have fine-tuned judge models based on open-source LLMs to evaluate the quality of other LLM.
Approach: They propose to use open-source LLMs to evaluate Large Language Models (LLMs) their empirical results show that the models underperform GPT-4 in several dimensions .
Outcome: The proposed models outperform GPT-4 on several dimensions including generalizability, fairness and adaptability.
Improving Neural Machine Translation with Neural Syntactic Distance (N19-1)

Copied to clipboard

Challenge: Neural syntactic distance (NSD) is used to represent constituent trees using a sequence whose length is identical to the number of words in the sentence.
Approach: They propose five strategies to improve NMT with explicit use of syntactic information . et al., 2014) propose a set of five strategies that incorporate syntastic information into the encoder and/or decoder of the baseline model.
Outcome: The proposed strategies improve translation performance of the baseline model (+2.1 (En–Ja), +1.3 (Ja–En), +1.2 (En-Ch), and +1.0 (Ch–En) BLEU.
Sentence-Level Agreement for Neural Machine Translation (P19-1)

Copied to clipboard

Challenge: Empirical results show that a sentence-level agreement module can significantly improve the performance of neural machine translation (NMT)
Approach: They propose a sentence-level agreement module to minimize the difference between the representation of source and target sentences.
Outcome: Empirical results show the proposed agreement module significantly improves translation performance.
Document-Level Relation Extraction with Sentences Importance Estimation and Focusing (2022.naacl-main)

Copied to clipboard

Challenge: Document-level relation extraction models are not robust and exhibit bizarre behaviors when non-evidence sentences are removed.
Approach: They propose a document-level relation extraction framework that uses a sentence importance score and a focusing loss to encourage DocRE models to focus on evidence sentences.
Outcome: The proposed framework improves overall performance and makes DocRE models more robust.
UniRPG: Unified Discrete Reasoning over Table and Text as Program Generation (2022.emnlp-main)

Copied to clipboard

Challenge: Existing methods for question answering using knowledge resources are mixed-of-experts and semantic parsing-based.
Approach: They propose a semantic-parsing-based approach to perform Unified discrete Reasoning over heterogeneous knowledge resources as Program Generation.
Outcome: The proposed approach improves interpretability and scalability over table and text . it achieves promising performance on the TAT-QA dataset without annotation .
LoRA-drop: Efficient LoRA Parameter Pruning based on Output Evaluation (2025.coling-main)

Copied to clipboard

Challenge: Low-Rank Adaptation (LoRA) is currently the most commonly used PEFT method for fine-tuning models with billions of parameters.
Approach: They propose to use low-rank Adaptation to evaluate LoRA parameter features and then retain LoRA for important layers and the other layers share the same LoRA.
Outcome: The proposed method achieves comparable performance to full fine-tuning and LoRA while retaining 50% of the LoRA parameters on average.
Learning to Decouple Relations: Few-Shot Relation Classification with Entity-Guided Attention and Confusion-Aware Training (2020.coling-main)

Copied to clipboard

Challenge: Existing few-shot relation classifiers struggle to distinguish them with few annotated instances due to high co-occurrence of some relations .
Approach: They propose a few-shot relation classification model with two mechanisms to decouple easily-confused relations.
Outcome: The proposed model achieves comparable and even better results to strong baselines in terms of accuracy.
Selection Bias Explorations and Debias Methods for Natural Language Sentence Matching Datasets (P19-1)

Copied to clipboard

Challenge: Natural Language Sentence Matching (NLSM) is a popular NLP task.
Approach: They propose to use QuoraQP to train and evaluate NLSM models using a selection bias framework.
Outcome: The proposed framework can improve generalization ability of trained models and give more trustworthy evaluation results for real-world adoptions.
Improving Low-resource Question Answering by Augmenting Question Information (2023.findings-emnlp)

Copied to clipboard

Challenge: Low-resource questions pose a significant challenge within the field of Question-Answering (QA) tasks.
Approach: They propose a method that leverages large models' internal knowledge to enhance the quality of augmented data by Prompt Answer, Question Generation, and Question Filter.
Outcome: The proposed method outperforms existing augmentation strategies on high-resource QA tasks like SQUAD1.1 and TriviaQA.
CLMLF:A Contrastive Learning and Multi-Layer Fusion Method for Multimodal Sentiment Detection (2022.findings-naacl)

Copied to clipboard

Challenge: Existing methods for multimodal sentiment detection do not consider token-level feature fusion.
Approach: They propose a method for multimodal sentiment detection using a combination of text and image to encode and fuse token-level features.
Outcome: The proposed method can fuse multimodal features with token-level features on three publicly available multimodal datasets.
MuSC: Improving Complex Instruction Following with Multi-granularity Self-Contrastive Training (2025.acl-long)

Copied to clipboard

Challenge: Existing methods for complex instruction-following with elaborate constraints rely on a weaker model, especially GPT-4, limiting their application.
Approach: They propose a Multi-granularity Self-Contrastive Training framework to improve instruction alignment without relying on a stronger model.
Outcome: The proposed framework improves instruction-following with elaborate constraints without external supervision on coarse and fine granularity.
MuGER2: Multi-Granularity Evidence Retrieval and Reasoning for Hybrid Question Answering (2022.findings-emnlp)

Copied to clipboard

Challenge: Conventional HQA models retrieve coarse- or fine-grained evidence to reason the answer . however, they neglect a more general scenario requiring reasoning over heterogeneous data to answer a question.
Approach: They propose a multi-granularity evidence retrieval and reasoning approach to answer questions over heterogeneous data using tables and passages linked to table cells.
Outcome: The proposed approach significantly boosts the performance on the HybridQA dataset.
Iterative Nearest Neighbour Machine Translation for Unsupervised Domain Adaptation (2023.findings-acl)

Copied to clipboard

Challenge: Existing methods for supervised domain adaptation of machine translation focus on fine-tuning, which is non-extensible.
Approach: They propose to perform unsupervised domain adaptation in a non-parametric manner by using in-domain monolingual data and performing nearest neighbour inference on both forward and backward directions.
Outcome: The proposed method significantly improves the in-domain translation performance and achieves state-of-the-art results among non-parametric methods.
Diagnosing and Remedying Representation Deficiencies for Deterministic Reasoning in KGQA (2026.acl-long)

Copied to clipboard

Challenge: Existing approaches to solve non-deterministic reasoning problems in large language models are limited by their complexity and lack of a clear understanding of the problem.
Approach: They propose a method to diagnose and correct non-deterministic reasoning behaviors in large language models.
Outcome: The proposed method outperforms baselines and WebQSP benchmarks on the widely used WebQ SP and CWQ benchmarks.
On the Effectiveness of Sentence Encoding for Intent Detection Meta-Learning (2022.naacl-main)

Copied to clipboard

Challenge: Recent studies on few-shot intent detection have attempted to formulate the task as a meta-learning problem.
Approach: They propose to modify a few-shot intent detection task to produce a non-trivially strong performance without further domain-specific adaptation.
Outcome: The proposed model improves on the prototypical network variants with task-specific fine-tuning.
Word-level Cross-lingual Structure in Large Language Models (2025.coling-main)

Copied to clipboard

Challenge: Large Language Models (LLMs) have demonstrated exceptional performance across a broad spectrum of cross-lingual Natural Language Processing (NLP) tasks.
Approach: They propose to use Word-level Cross-lingual Structure to prove that the word-level embedding on the hidden layers isomorphic between languages.
Outcome: The proposed method significantly improves on two representative LLM foundations, LLaMA2 and BLOOM.
Jointly Learning Guidance Induction and Faithful Summary Generation via Conditional Variational Autoencoders (2022.findings-naacl)

Copied to clipboard

Challenge: Existing methods for abstractive summarization generate factual consistency summaries with a high level of accuracy and coherence.
Approach: They propose a framework that induces the guidance information and generates summary equipment with the guidance synchronously.
Outcome: The proposed framework generates fluent summaries with no constraint on the words and phrases, and is more faithful than the existing state-of-the-art approaches.
Dynamic Planning for LLM-based Graphical User Interface Automation (2024.findings-emnlp)

Copied to clipboard

Challenge: Existing approaches to planning for GUI tasks are limited due to long historical dialogues.
Approach: They propose a novel approach to dynamic planning based on environmental feedback and execution history to guide action prediction in GUI tasks.
Outcome: The proposed approach surpasses the strong GPT-4V baseline by +12.7% in accuracy.
DUAL-REFLECT: Enhancing Large Language Models for Reflective Translation through Dual Learning Feedback Mechanisms (2024.acl-short)

Copied to clipboard

Challenge: Existing self-reflection methods lack effective feedback information, limiting the translation performance of large language models (LLMs).
Approach: They propose a framework that leverages the dual learning of translation tasks to provide effective feedback, thereby enhancing the models’ self-reflective abilities and improving translation performance.
Outcome: The proposed framework improves the models’ self-reflective abilities and improves translation accuracy and eliminating ambiguities across translation tasks.
Robust Unsupervised Neural Machine Translation with Adversarial Denoising Training (2020.coling-main)

Copied to clipboard

Challenge: Unsupervised neural machine translation (UNMT) has attracted great interest in the machine translation community.
Approach: They propose to explicitly take noisy data into consideration to improve the robustness of UNMT based systems.
Outcome: The proposed methods significantly improved the robustness of the conventional UNMT systems in noisy scenarios.
CoLaDa: A Collaborative Label Denoising Framework for Cross-lingual Named Entity Recognition (2023.acl-long)

Copied to clipboard

Challenge: Existing approaches to named entity recognition (NER) are limited by the cost of labeling and labeling, especially for low-resource languages.
Approach: They propose a model-collaboration-based denoising scheme that enables models trained on different data sources to collaboratively denoise pseudo labels used by each other.
Outcome: The proposed framework achieves superior results on benchmark datasets and can generalize to distant languages.
Improving Translation Quality Estimation with Bias Mitigation (2023.acl-long)

Copied to clipboard

Challenge: State-of-the-art translation Quality Estimation models are biased, relying on monolingual features while ignoring the bilingual semantic alignment.
Approach: They propose a method to mitigate the bias of translation quality estimation models by contrastive learning between clean and noisy sentence pairs.
Outcome: The proposed method improves the estimation performance while mitigating the bias.
Cross-lingual Feature Extraction from Monolingual Corpora for Low-resource Unsupervised Bilingual Lexicon Induction (2022.coling-1)

Copied to clipboard

Challenge: Unsupervised bilingual lexicon induction models fail on low-resource language pairs due to insufficient initialization.
Approach: They propose a method to learn cross-lingual features from monolingual corpora for low-resource UBLI by integrating cross-linguistic representations with pre-trained word embeddings in a fully unsupervised initialization.
Outcome: The proposed method outperforms state-of-the-art methods on low-resource language pairs and improves representational ability and robustness of existing embedding models.
Decomposed Meta-Learning for Few-Shot Named Entity Recognition (2022.findings-acl)

Copied to clipboard

Challenge: Named entity recognition systems aim at recognizing unseen entity types based on a few labeled examples.
Approach: They propose a decomposed meta-learning approach to solve few-shot span detection and few- shot entity typing problems by introducing a model-agnostic meta-loop algorithm.
Outcome: The proposed approach achieves superior performance over prior methods on benchmarks.
Understanding and Improving Hidden Representations for Neural Machine Translation (N19-1)

Copied to clipboard

Challenge: Existing studies have explored some methods for understanding hidden representations, but they have not sought to improve the translation quality rationally according to their understanding.
Approach: They propose to construct a sequence of nested relative tasks and measure the feature generalization ability of the learned hidden representation over these tasks.
Outcome: The proposed methods achieve consistent improvements (up to +1.3 BLEU) on two widely-used datasets.
Discriminative Reasoning for Document-level Relation Extraction (2021.findings-acl)

Copied to clipboard

Challenge: Existing models use graph networks to implicitly model reasoning skills . but it is yet to be seen whether modeling these reasoning skills implicitly is competitive with intuitive reasoning skills between one entity pair in this document.
Approach: They propose a discriminative reasoning framework to explicitly model the paths of reasoning skills between entity pairs in a document.
Outcome: The proposed method outperforms the previous state-of-the-art on the large-scale DocRE dataset.
OPERA: Operation-Pivoted Discrete Reasoning over Text (2022.naacl-main)

Copied to clipboard

Challenge: Existing methods to predict logical forms ignore the utilization of symbolic operations and lack reasoning ability and interpretability.
Approach: They propose an operation-pivoted discrete reasoning framework that uses symbolic operations as neural modules to facilitate reasoning ability and interpretability.
Outcome: Extensive experiments on DROP and RACENum datasets show the reasoning ability of OPERA.
Forest-Based Neural Machine Translation (P18-1)

Copied to clipboard

Challenge: Compared with string-to-string systems, tree-based NMT methods use more syntactic information and can incorporate prior knowledge.
Approach: They propose a tree-based neural machine translation method that translates a linearized packed forest under a simple sequence-to-sequence framework.
Outcome: The proposed method outperforms tree-based approaches in the BLEU score of the proposed model.
Knowledge Distillation for Multilingual Unsupervised Neural Machine Translation (2020.acl-main)

Copied to clipboard

Challenge: Unsupervised neural machine translation (UNMT) can only translate between a single language pair and cannot produce translation results for multiple language pairs at the same time.
Approach: They propose a method to translate between 13 languages using a single encoder and a decoder . they propose two knowledge distillation methods to further enhance multilingual UNMT performance .
Outcome: The proposed method improves translation performance for all languages using multilingual data.
Demographics Should Not Be the Reason of Toxicity: Mitigating Discrimination in Text Classifications with Instance Weighting (2020.acl-main)

Copied to clipboard

Challenge: Recent research has found that text classification datasets contain certain unintended biases, such as text containing demographic identity-terms that are more likely to be abusive.
Approach: They propose a model-agnostic debiasing framework that recovers the non-discrimination distribution using instance weighting, which does not require extra resources or annotations apart from a pre-defined set of demographic identity-terms.
Outcome: The proposed framework alleviates the unintended biases without hurting models’ generalization ability.
Self-Evaluation of Large Language Model based on Glass-box Features (2024.findings-emnlp)

Copied to clipboard

Challenge: Existing evaluation methods rely on external evaluators, focusing on training and prompting strategies, but model-aware glass-box features are overlooked.
Approach: They propose to use model-aware glass-box features to evaluate an LLM's output.
Outcome: The proposed model-aware features are reliable quality indicators for self-evaluation on public benchmarks.
Self-Training for Unsupervised Neural Machine Translation in Unbalanced Training Data Scenarios (2021.naacl-main)

Copied to clipboard

Challenge: Existing methods that use monolingual corpora for translation are not suitable for low-resource languages such as Estonian.
Approach: They propose unsupervised neural machine translation (UNMT) that relies on monolingual corpora to train a robust UNMT system and improve its performance.
Outcome: The proposed methods outperform conventional UNMT systems on several language pairs.
Neural Document Summarization by Jointly Learning to Score and Select Sentences (P18-1)

Copied to clipboard

Challenge: Sentence scoring and sentence selection are two main steps in extractive document summarization systems.
Approach: They propose an end-to-end neural network framework for extractive document summarization by jointly learning to score and select sentences.
Outcome: The proposed framework outperforms the state-of-the-art summarization models on the CNN/Daily Mail dataset.
Unsupervised Bilingual Word Embedding Agreement for Unsupervised Neural Machine Translation (P19-1)

Copied to clipboard

Challenge: Unsupervised bilingual word embedding (UBWE) has helped unsupervised neural machine translation (UNMT) achieve remarkable results in several language pairs.
Approach: They propose two methods that train UNMT with UBWE agreement . they propose to use UBwe to initialize word embedding in UNMT .
Outcome: The proposed methods outperform conventional methods on several language pairs.
Benchmarking LLMs for Translating Classical Chinese Poetry: Evaluating Adequacy, Fluency, and Elegance (2025.emnlp-main)

Copied to clipboard

Challenge: Existing large language models fall short of translating culturally significant content . existing models fall behind in achieving such translations, authors say .
Approach: They propose a suitable benchmark for translating classical Chinese poetry into English . they propose RAT, a retrieval-augmented machine translation method that enhances the translation process .
Outcome: The proposed method improves translation quality in terms of adequate, fluent, and elegant translations.
Issues with Entailment-based Zero-shot Text Classification (2021.acl-short)

Copied to clipboard

Challenge: Pre-trained BERT models with no fine-tuning can yield competitive performance against BERT fine- tuned for NLI.
Approach: They propose to use any target label into a sentence of hypothesis and verify whether it could be entailed by the input.
Outcome: The proposed models perform better than models fine-tuned for BERT, but the results are in general negative.
Understanding Data Augmentation in Neural Machine Translation: Two Perspectives towards Generalization (D19-1)

Copied to clipboard

Challenge: Existing studies measure the superiority of DA methods in terms of their performance on a specific test set, but some do not exhibit consistent improvements across translation tasks.
Approach: They propose to evaluate DA methods from two perspectives to determine their generalization ability . they find that DA method's test performance does not exhibit consistent improvements across translation tasks .
Outcome: The proposed methods do not exhibit consistent improvements across translation tasks.
Memory-augmented Query Reconstruction for LLM-based Knowledge Graph Reasoning (2025.findings-acl)

Copied to clipboard

Challenge: Existing methods that confuse tool utilization with knowledge reasoning harm readability and give rise to tool invocation hallucinations.
Approach: They propose to decouple LLM from tool invocation tasks by establishing a memory module with explicit descriptions of query statements and a query memory module to facilitate the KGQA process.
Outcome: The proposed method achieves state-of-the-art on WebQSP and CWQ benchmarks.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations