Papers by Fei Tan

43 papers
HACHIMI: Scalable and Controllable Student Persona Generation via Orchestrated Agents (2026.findings-acl)

Copied to clipboard

Challenge: ad-hoc prompting and hand-crafted profiles with limited control over educational theory and population distributions are often used for student personas.
Approach: They propose a framework that generates theory-aligned, quota-controlled personas . they factorize each persona into a theory-anchored educational schema .
Outcome: HACHIMI generates theory-aligned, quota-controlled personas for grades 1-12 . results show near-perfect schema validity, accurate quots, and substantial diversity .
ReFSQL: A Retrieval-Augmentation Framework for Text-to-SQL Generation (2023.findings-emnlp)

Copied to clipboard

Challenge: Existing methods that align natural language with SQL Language underestimate inherent structural characteristics of SQL and lead to structure errors.
Approach: They propose a retrieval-argument framework that aligns natural language with SQL Language and trains one encoder-decoder-based model to fit all questions.
Outcome: The proposed framework improves accuracy and robustness of text-to-SQL generation on five datasets.
Text Diffusion Model with Encoder-Decoder Transformers for Sequence-to-Sequence Generation (2024.naacl-long)

Copied to clipboard

Challenge: Existing diffusion models are applied to continuous feature space while texts are sequences of discrete categorical tokens.
Approach: They propose to use an encoder-decoder Transformer architecture to approach sequence-to-sequence text generation.
Outcome: The proposed model improves on five sequence-to-sequence generation tasks compared to other diffusion-based models regarding text quality and inference time.
End-to-End Modeling via Information Tree for One-Shot Natural Language Spatial Video Grounding (2022.acl-long)

Copied to clipboard

Challenge: Existing methods for grounding video frames with dense annotations require enormous amount of human effort.
Approach: They propose to ground natural language in video frames with only one frame labeled . they propose an end-to-end model that eliminates interference of irrelevant frames .
Outcome: The proposed model can ground natural language in all video frames with only one frame labeled . the proposed model eliminates interference of irrelevant frames based on branch search and cropping techniques .
LightNER: A Lightweight Tuning Paradigm for Low-resource NER via Pluggable Prompting (2022.coling-1)

Copied to clipboard

Challenge: Existing approaches for Named Entity Recognition (NER) use extensive labeled data for model training, which struggles in low-resource scenarios.
Approach: They propose a lightweight tuning paradigm for low-resource NER via pluggable prompting . they construct a learnable verbalizer of entity categories without any label-specific classifiers .
Outcome: The proposed model outperforms baselines and class transfer models in low-resource scenarios.
CWSeg: An Efficient and General Approach to Chinese Word Segmentation (2023.acl-industry)

Copied to clipboard

Challenge: Existing methods for Chinese word segmentation have achieved state-of-the-art performance, but they pose challenges in the deployment.
Approach: They propose to augment PLM-based Chinese word segmentation schemes by developing cohort training and versatile decoding strategies.
Outcome: The proposed model can be used to augment existing PLM-based models and improve their performance on Chinese LLaMA and Alpaca datasets.
CMR Scaling Law: Predicting Critical Mixture Ratios for Continual Pre-training of Language Models (2024.emnlp-main)

Copied to clipboard

Challenge: Large Language Models (LLMs) excel in diverse tasks but often underperform in specialized fields due to limited domain-specific or proprietary corpus.
Approach: They propose a power-law relationship between loss, mixture ratio, and training tokens scale and formalize the trade-off between general and domain-specific capabilities.
Outcome: The proposed model achieves the desired domain transfer while maintaining general ability and highest utilization of available resources.
Good Visual Guidance Make A Better Extractor: Hierarchical Visual Prefix for Multimodal Entity and Relation Extraction (2022.findings-naacl)

Copied to clipboard

Challenge: Existing approaches for named entity recognition and relation extraction suffer from error sensitivity when irrelevant object images are incorporated in texts.
Approach: They propose a hierarchical visual prefix fusion NeTwork for visual-enhanced entity and relation extraction using pluggable visual prefixed visual features.
Outcome: The proposed method achieves state-of-the-art on three benchmark datasets.
SDA: Simple Discrete Augmentation for Contrastive Sentence Representation Learning (2024.lrec-main)

Copied to clipboard

Challenge: Existing methods for data augmentation have not been well explored.
Approach: They propose to use punctuation insertion, modal verbs, and double negation to produce diverse forms of sentences.
Outcome: The proposed methods perform better on diverse datasets with semantic similarity and standard negation.
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning (2021.emnlp-main)

Copied to clipboard

Challenge: Recent pretrained language models extend from millions to billions of parameters.
Approach: They propose a technique which forwards on a whole network while backwarding on resetting the gradients of the non-child network during the backward process.
Outcome: The proposed technique outperforms the vanilla fine-tuning technique on various downstream tasks and can achieve better generalization performance by large margins.
Towards Unified Prompt Tuning for Few-shot Text Classification (2022.findings-emnlp)

Copied to clipboard

Challenge: Prompt-based fine-tuning has boosted performance of Pre-trained Language Models (PLMs) on few-shot text classification, but PLMs are unfamiliar with prompt-style expressions during pre-training, which limits the few- shot learning performance on downstream tasks.
Approach: They propose a framework for prompt-based fine-tuning that captures prompting semantics from non-target NLP datasets and propose 'Prompt-Options-Verbalizer' for joint prompt learning across different NLP tasks.
Outcome: Experiments show that the proposed framework outperforms state-of-the-art prompt-based fine-tuning frameworks on few-shot text classification tasks.
The Mirage of Model Editing: Revisiting Evaluation in the Wild (2025.acl-long)

Copied to clipboard

Challenge: despite near-perfect results, effectiveness of model editing in real-world applications remains unclear.
Approach: They propose QAEdit and WILD to better reflect real-world use of model editing . they propose a benchmark aligned with widely used question answering datasets and a task-agnostic evaluation framework .
Outcome: The proposed QAEdit benchmark and WILD evaluation framework show that current models perform worse than previously reported.
Rethinking Depression Prediction from a Fine-Grained Subscore Modeling Perspective via Multi-Task Learning (2026.acl-long)

Copied to clipboard

Challenge: Existing methods for depression assessment rely on standardized ratings, but they are time-consuming and subject to inter-rater variability.
Approach: They propose a fine-grained model for subscore prediction via multi-task learning that can be used to predict depression severity using multiple tasks.
Outcome: The proposed model outperforms baselines and Qwen3-14B direct scoring on the public E-DAIC dataset and to a large-scale private clinical dataset.
Deeply Coupled Cross-Modal Prompt Learning (2023.findings-acl)

Copied to clipboard

Challenge: Existing prompt-tuning methods focus on language branch or learn vision-language interaction in a shallow mechanism.
Approach: They propose a Deeply coupled Cross-modal Prompt learning method based on CLIP to facilitate the interplay between vision and language with a Cross-Modal Prompting Attention mechanism.
Outcome: The proposed method enables the interplay between vision and language with a Cross-Modal Prompt Attention mechanism.
TNT: Text Normalization based Pre-training of Transformers for Content Moderation (2020.emnlp-main)

Copied to clipboard

Challenge: Language model pre-training (self-supervised or unsupervised learning) has been widely used in a multitude of language processing tasks such as named entity recognition, sentiment analysis, question answering and content moderation.
Approach: They propose a new language pre-training model TNT for content moderation that uses a combination of masking strategy and text normalization to learn from text.
Outcome: The proposed model outperforms baselines on hate speech classification task and is a potential approach to misspelling correction.
Knowledge Rumination for Pre-trained Language Models (2023.emnlp-main)

Copied to clipboard

Challenge: Existing studies have shown that pre-trained language models lack the capacity to handle knowledge-intensive tasks alone.
Approach: They propose a new paradigm to help pre-trained language models utilize latent knowledge without retrieving it from external corpus.
Outcome: The proposed paradigm can be applied to pre-trained language models without retrieving external knowledge from the corpus.
Reward Difference Optimization For Sample Reweighting In Offline RLHF (2024.findings-emnlp)

Copied to clipboard

Challenge: Large Language Models (LLMs) are becoming more capable, but their maximum likelihood objective for the next token prediction falls short in capturing such crucial human values.
Approach: They propose a reward difference prediction method that uses reward difference coefficients to reweigh sample pairs in offline RLHF and a difference model that considers rich interactions between a pair of responses.
Outcome: The proposed method is effective in both automatic metrics and human evaluation.
PRDetect: Perturbation-Robust LLM-generated Text Detection Based on Syntax Tree (2025.findings-naacl)

Copied to clipboard

Challenge: Recent methods for detecting LLM-generated text have shown impressive performance, but in real-world scenarios, users often introduce perturbations to the text.
Approach: They propose a method that detects syntactic trees that are minimally affected by perturbations and exhibit distinct differences between human-written and LLM-generated text.
Outcome: The proposed method shows that it is significantly better against perturbations on the HC3 and GPT-3.5-mixed datasets and also has the shortest time expenditure.
Repulsive Attention: Rethinking Multi-head Attention as Bayesian Inference (2020.emnlp-main)

Copied to clipboard

Challenge: Existing studies show that multi-head attention is an effective module in deep neural networks, but there are no explicit mechanisms guaranteeing this property.
Approach: They propose a non-parametric approach that explicitly improves the repulsiveness in multi-head attention and consequently strengthens model’s expressiveness.
Outcome: The proposed approach improves the repulsiveness in multi-head attention and strengthens model’s expressiveness.
HiGMem: A Hierarchical and LLM-Guided Memory System for Long-Term Conversational Agents (2026.findings-acl)

Copied to clipboard

Challenge: Existing memory systems rely on vector similarity for retrieval, resulting in bloated evidence sets . existing systems produce little additional recall, but this approach lowers retrieval precision .
Approach: They propose a two-level event-turn memory system that uses event summaries as semantic anchors to predict which related turns are worth reading.
Outcome: The proposed system achieves the best F1 on four of five question categories and improves adversarial F1 from 0.54 to 0.78 over A-Mem while retrieving an order of magnitude fewer turns.
Reasoning with Language Model Prompting: A Survey (2023.acl-long)

Copied to clipboard

Challenge: Reasoning is an essential ability for complex problem-solving and can provide back-end support for various real-world applications.
Approach: They present cutting-edge research on reasoning with language model prompting and provide systematic resources to help beginners.
Outcome: The proposed approaches have not been systematically reviewed and analyzed.
Consultant Decoding: Yet Another Synergistic Mechanism (2025.findings-acl)

Copied to clipboard

Challenge: Large language models (LLMs) have attracted widespread attention and adoption across diverse domains due to their exceptional performance and robust generalization abilities.
Approach: They propose a synergetic mechanism for Consultant Decoding (CD) that achieves a 2.5-fold increase in inference speed compared to the target model while maintaining comparable generation quality.
Outcome: The proposed mechanism achieves 2.5-fold increase in inference speed while maintaining comparable generation quality (100% of the target model’s performance).
Balancing Speciality and Versatility: a Coarse to Fine Framework for Supervised Fine-tuning Large Language Model (2024.findings-acl)

Copied to clipboard

Challenge: Aligned Large Language Models exhibit remarkable versatility, capable of handling diverse real-world tasks.
Approach: They propose a coarse to fine framework to fine-tune aligned Large Language Models to achieve a balance between speciality and versatility.
Outcome: The proposed framework outperforms baseline methods across diverse tasks and model scales.
HyPe: Better Pre-trained Language Model Fine-tuning with Hidden Representation Perturbation (2023.acl-long)

Copied to clipboard

Challenge: Existing techniques to fine-tune pre-trained language models on downstream tasks are inadequate.
Approach: They propose a technique to perturb hidden Transformers representations by enhancing generalization of hidden representations from different layers.
Outcome: The proposed technique outperforms vanilla fine-tuning and enhances generalization of hidden representations from different layers.
UI-Copilot: Advancing Long-Horizon GUI Automation via Tool-Integrated Policy Optimization (2026.acl-long)

Copied to clipboard

Challenge: Experimental results show that UI-Copilot-7B achieves state-of-the-art performance on challenging MemGUI-Bench, outperforming strong 7B-scale GUI agents such as GUI-Owl-7B and UITARS-1.5-7B.
Approach: They propose a collaborative framework where the GUI agent focuses on task execution while a lightweight copilot provides on-demand assistance for memory retrieval and numerical computation.
Outcome: The proposed framework outperforms GUI-Owl-7B and UI-TARS-1.5-7B on MemGUI-Bench and delivers 17.1% improvement on AndroidWorld over the base Qwen model.
Too Consistent to Detect: A Study of Self-Consistent Errors in LLMs (2025.emnlp-main)

Copied to clipboard

Challenge: Existing detection methods fail to account for **self-consistent error** . study identifies self-consistency errors and evaluates them .
Approach: They propose a method that fuses hidden state evidence from an external verifier LLM to detect self-consistent errors.
Outcome: The proposed method significantly enhances performance on self-consistent errors across three LLM families.
Blinded by Generated Contexts: How Language Models Merge Generated and Retrieved Contexts When Knowledge Conflicts? (2024.acl-long)

Copied to clipboard

Challenge: Recent advances in augmenting Large Language Models (LLMs) with auxiliary information have significantly revolutionized their efficacy in knowledge-intensive tasks.
Approach: They propose a systematic framework to identify whether LLMs’ responses are attributed to either generated or retrieved contexts.
Outcome: The proposed framework identifies whether LLMs’ responses are attributed to either generated or retrieved contexts.
BERT-Beta: A Proactive Probabilistic Approach to Text Moderation (2021.emnlp-main)

Copied to clipboard

Challenge: Existing approaches to text moderation are reactive and do not account for user generated content.
Approach: They propose a text toxicity propensity model to characterize extent to which a user generated text attracts toxic comments and introduce a beta regression model to do the probabilistic modeling.
Outcome: The proposed model performs well in comprehensive experiments and is scalable.
Beyond Error Propagation in Neural Machine Translation: Characteristics of Language Also Matter (D18-1)

Copied to clipboard

Challenge: Neural machine translation suffers from exposure bias and error propagation problem.
Approach: They conduct a series of analyses to deeply understand the accuracy drop problem . they find that the left part of the translated sentence is often better than its right part .
Outcome: The results show that the left part of the translated sentence is often better than its right part in left-to-right decoding models.
Fusing Heterogeneous Factors with Triaffine Mechanism for Nested Named Entity Recognition (2022.findings-acl)

Copied to clipboard

Challenge: Named entity recognition (NER) is a fundamental natural language processing task that extracts entities from texts.
Approach: They propose a triaffine mechanism which integrates heterogeneous factors into a single model to fuse these factors into one model to achieve better span representation.
Outcome: The proposed method outperforms previous span-based methods and achieves state-of-the-art F1 scores on nested NER datasets GENIA and KBP2017.
daDPO: Distribution-Aware DPO for Distilling Conversational Abilities (2025.findings-acl)

Copied to clipboard

Challenge: Knowledge distillation (KD) with Direct Preference Optimization (DPO) has emerged as a promising approach to enhance the conversational abilities of smaller models using a larger teacher model.
Approach: They propose a framework that integrates the teacher's distributional information into DPO distillation while preserving theoretical guarantees.
Outcome: The proposed framework outperforms existing methods in restoring performance for pruned models and enhancing smaller models within the same LLM family.
Lost in Diffusion: Uncovering Hallucination Patterns and Failure Modes in Diffusion Large Language Models (2026.findings-acl)

Copied to clipboard

Challenge: dLLMs have emerged as a promising non-autoregressive paradigm for text generation, but their hallucination mechanisms remain underexplored.
Approach: They present the first controlled comparative study to evaluate hallucination patterns in Diffusion Large Language Models.
Outcome: The proposed model exhibits higher propensity for hallucination than AR counterparts controlled for architecture, scale, and pre-training weights.
Cross-lingual Cross-modal Pretraining for Multimodal Retrieval (2021.naacl-main)

Copied to clipboard

Challenge: Recent pretrained vision-language models have achieved impressive performance on cross-modal retrieval tasks in English.
Approach: They propose a new approach to learn cross-lingual cross-modal representations for matching images and captions in multiple languages using an annotated corpus.
Outcome: The proposed model achieves impressive performance on two multimodal multilingual image caption benchmarks: Multi30k with German captions and MSCOCO with Japanese captions.
Reverse Constitutional AI: A Framework for Controllable Toxic Data Generation via Probability-Clamped RLAIF (2026.findings-acl)

Copied to clipboard

Challenge: Existing approaches to red teaming focus on searching for individual adversarial inputs.
Approach: They propose a framework for automated adversarial data generation that inverts harmless constitution into constitution of toxicity and iteratively refining model outputs through critique–revision pipeline.
Outcome: The proposed framework generates diverse, high-quality toxic data without human annotation and significantly improves semantic coherence without sacrificing adversarial strength.
HABERTOR: An Efficient and Effective Deep Hatespeech Detector (2020.emnlp-main)

Copied to clipboard

Challenge: HABERTOR model is a highly efficient and effective alternative to BERT for the hatespeech classification task.
Approach: They propose to modify BERT's HABERTOR model to generate its own vocabularies and pre-trained it using the largest scale hatespeech dataset.
Outcome: The proposed model is faster, more efficient and more robust than existing methods for hatespeech classification.
Parallel Instance Query Network for Named Entity Recognition (2022.acl-long)

Copied to clipboard

Challenge: Named entity recognition is a fundamental task in natural language processing.
Approach: They propose a method that sets up global and learnable instance queries to extract entities from a sentence in a parallel manner.
Outcome: The proposed method outperforms existing state-of-the-art models on nested and flat datasets.
CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark (2022.acl-long)

Copied to clipboard

Challenge: a new benchmark for biomedical language understanding is being developed in Chinese . most benchmarks are limited to English, which makes it difficult to replicate success in other languages.
Approach: They propose to use Chinese biomedical language understanding evaluation benchmarks to evaluate Chinese models.
Outcome: The proposed benchmarks show that the current models perform worse than the human ceiling.
What Makes Pre-trained Language Models Better Zero-shot Learners? (2023.acl-long)

Copied to clipboard

Challenge: Current methods for prompt learning in zero-shot scenarios rely on a development set with sufficient human-annotated data to select the best-performing prompt template.
Approach: They propose a method for screening reasonable prompt templates in zero-shot text classification using language discrepancy.
Outcome: The proposed method improves prediction performance in a realistic zero-shot setting, eliminating the need for labelled examples.
The Fall of ROME: Understanding the Collapse of LLMs in Model Editing (2024.findings-emnlp)

Copied to clipboard

Challenge: Recent studies have found that model editing methods can cause large language models to collapse with just a single edit.
Approach: They propose a method that uses prefixed keys and adds prefixes during testing to prevent model collapse.
Outcome: The proposed method prevents model collapse while maintaining effectiveness, the authors show . Rank-One Model Editing (ROME) has been found to cause model collapse with just a single edit .
BaseCal: Unsupervised Confidence Calibration via Base Model Signals (2026.acl-long)

Copied to clipboard

Challenge: Post-trained LLMs typically compromise reliability with severe overconfidence, resulting in inaccurate responses.
Approach: They propose a solution that feeds PoLLMs into the base LLM to get confidence.
Outcome: The proposed solution reduces expected calibration error (ECE) by 42.90% compared to the best unsupervised baselines.
More Data or Better Data? A Critical Analysis of Data Selection and Synthesis for Mathematical Reasoning (2025.emnlp-industry)

Copied to clipboard

Challenge: Despite various proposed data construction methods, their practical utility in real-world pipelines remains underexplored.
Approach: They conduct a comprehensive analysis of open-source datasets and data synthesis techniques for mathematical reasoning under a unified pipeline designed to mirror training and deployment scenarios.
Outcome: The proposed pipelines mirror training and deployment scenarios and are suitable for industrial applications.
TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing (2021.acl-demo)

Copied to clipboard

Challenge: Existing approaches to textual robustness evaluation focus on slightly modifying the input data, which maintains the original meaning and results in a different prediction.
Approach: They propose a multilingual robustness evaluation toolkit for NLP that integrates universal text transformations, task-specific transformations and adversarial attack.
Outcome: The toolkit includes universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analyses.
ReSURE: Regularizing Supervision Unreliability for Multi-turn Dialogue Fine-tuning (2025.emnlp-main)

Copied to clipboard

Challenge: Existing methods address data quality via static prefiltering, which decouples quality control from training and fails to mitigate turn-level error propagation.
Approach: They propose an adaptive learning method that dynamically down-weights unreliable supervision without explicit filtering.
Outcome: Experiments on single-source and mixed-quality datasets show improved stability and response quality.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations