Papers by Ji Xin
The Art of Abstention: Selective Prediction and Error Regularization for Natural Language Processing (2021.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models have improved the state-of-the-art results on many NLP applications. |
| Approach: | They propose a simple error regularization trick that improves confidence estimation without substantially increasing the computation budget. |
| Outcome: | The proposed regularization improves confidence estimation without increasing computation budget. |
Cross Copy Network for Dialogue Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Despite the success of sequence-to-sequence models, dialogue logics are often ignored. |
| Approach: | They propose a network architecture to explore the current dialog context and similar dialogue instances’ logical structure simultaneously. |
| Outcome: | The proposed network architecture is superior to existing state-of-the-art models. |
AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments (2026.acl-long)
Copied to clipboard
Zhiheng Xi, Dingwen Yang, Jiaqi Liu, Jixuan Huang, Honglin Guo, Baodai Huang, Tinggang Chen, Qi Zhang, Zhonghang Lu, Chenyu Liu, Jiajun Sun, Jiazheng Zhang, Dingwei Zhu, Xin Guo, Junzhe Wang, Zhihao Zhang, Yuming Yang, Junjie Ye, Minghe Gao, Dongrui Liu, Jiaming Ji, Guohao Li, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing benchmarks evaluate agents in simplified, idealized settings, relying on pre-packaged tool interfaces, overlooking critical steps, and assume inputs are clean and fully specified. |
| Approach: | They propose a framework that evaluates language agents in simplified, idealized settings . they show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 . |
| Outcome: | Experiments on 15 proprietary and open-source models show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 . |
How Pre-trained Language Models Capture Factual Knowledge? A Causal-Inspired Analysis (2022.findings-acl)
Copied to clipboard
Shaobo Li, Xiaoguang Li, Lifeng Shang, Zhenhua Dong, Chengjie Sun, Bingquan Liu, Zhenzhou Ji, Xin Jiang, Qun Liu
| Challenge: | Recent studies show that pre-trained language models can fill in the missing factual words in cloze-style prompts such as ”Dante was born in [MASK]” . |
| Approach: | They propose to quantitatively measure and evaluate the word-level patterns that PLMs depend on to generate the missing factual words. |
| Outcome: | The proposed model fills in the missing factual words in cloze-style prompts by relying on effective clues or shortcut patterns. |
Are Emotion and Rhetoric Neurons in LLM? Neuron Recognition and Adaptive Masking for Emotion-Rhetoric Prediction Steering (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies on neurons focus on emotion and rhetoric, neglecting their intrinsic connections. |
| Approach: | They propose a framework for fine-grained steering of emotion and rhetoric in large language models . they propose 'neuro-based' masking method that integrates multi-dimensional screening . |
| Outcome: | The proposed method achieves directed induction of non-target sentences and enhancement of emotion tasks via rhetoric neurons. |
Zero-Shot Dense Retrieval with Momentum Adversarial Domain Invariant Representations (2022.findings-acl)
Copied to clipboard
| Challenge: | Dense retrieval (DR) methods first encode texts into a dense embedding space and then conduct text retrieval using efficient nearest neighbor search. |
| Approach: | They propose Momentum adversarial Domain Invariant Representation learning to train a domain classifier that distinguishes source versus target domains and adversarially updates the DR encoder to learn domain invariant representations. |
| Outcome: | The proposed method outperforms baselines on 10+ ranking datasets collected in the BEIR benchmark in the zero-shot setting, with more than 10% relative gains on datasets with enough sensitivity for DR models’ evaluation. |
Critic-CoT: Boosting the Reasoning Abilities of Large Language Model via Chain-of-Thought Critic (2025.findings-acl)
Copied to clipboard
Xin Zheng, Jie Lou, Boxi Cao, Xueru Wen, Yuqiu Ji, Hongyu Lin, Yaojie Lu, Xianpei Han, Debing Zhang, Le Sun
| Challenge: | Existing approaches to improve the reasoning performance of large language models rely on intuitive instance-level feedback, which limits the reasoning capabilities. |
| Approach: | They propose a framework that pushes LLMs toward System-2-like critic capability by using a step-wise CoT reasoning paradigm and automatic construction of weak-supervision data without human annotation. |
| Outcome: | The proposed model significantly improves task-solving performance by filtering out invalid solutions or iterative refinement. |
Can Language Models Follow Multiple Turns of Entangled Instructions? (2025.findings-emnlp)
Copied to clipboard
Chi Han, Xin Liu, Haodong Wang, Shiyang Li, Jingfeng Yang, Haoming Jiang, Zhengyang Wang, Qingyu Yin, Liang Qiu, Changlong Yu, Yifan Gao, Zheng Li, Bing Yin, Jingbo Shang, Heng Ji
| Challenge: | Despite of significant achievements in improving instruction-following capabilities of large language models, the ability to process multiple potentially entangled or conflicting instructions remains a considerable challenge. |
| Approach: | They construct multi-turn instruction with 1.1K high-quality multi-turned conversations using the human-in-the-loop approach and examine their capabilities. |
| Outcome: | The proposed model shows that it is difficult to integrate multiple turns and balance competing objectives when instructions intersect or conflict. |
Operator Selection and Ordering in a Pipeline Approach to Efficiency Optimizations for Transformers (2023.findings-acl)
Copied to clipboard
| Challenge: | Natural language processing tasks rely on complex neural models . transformer-based models are typically slow to execute, making it a non-trivial challenge to apply them in real-world applications. |
| Approach: | They propose to consider an efficiency method as an operator applied on a model . they find that the commutativity and cumulativeness of efficiency operators are plausible . |
| Outcome: | The proposed method is commutative and cumulative, and the results are estimated by combining methods. |
Are Missing Links Predictable? An Inferential Benchmark for Knowledge Graph Completion (2021.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for Knowledge Graph Completion (KGC) are unsatisfactory . |
| Approach: | They propose to use rule-guided train/test generation instead of conventional random split to ensure that each testing sample is predictable with supportive data in the training set. |
| Outcome: | The proposed model improves on existing benchmarks in inferential ability, assumptions, and patterns. |
LongHeads: Multi-Head Attention is Secretly a Long Context Processor (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models struggle to process lengthy inputs due to limited length generalization and attention’s quadratic computational demands. |
| Approach: | They propose a training-free framework that allows each head to attend to important context chunks instead of allowing each head a full sentence . |
| Outcome: | The proposed framework unlocks multi-head attention's untapped potential by allowing each head to attend to important context chunks instead of the full sentence. |
UReader: Universal OCR-free Visually-situated Language Understanding with Multimodal Large Language Model (2023.findings-emnlp)
Copied to clipboard
Jiabo Ye, Anwen Hu, Haiyang Xu, Qinghao Ye, Ming Yan, Guohai Xu, Chenliang Li, Junfeng Tian, Qi Qian, Ji Zhang, Qin Jin, Liang He, Xin Lin, Fei Huang
| Challenge: | Existing studies for visually-situated language understanding have shown shallow zero-shot visual text recognition ability when fed a low-resolution image with salient text information. |
| Approach: | They propose a model for universal OCR-free visually-situated language understanding based on the Multimodal Large Language Model (MLLM) their model is jointly finetuned on a wide range of visually situated language understanding tasks via a unified instruction format. |
| Outcome: | The proposed model achieves state-of-the-art ocr-free performance in 8 out of 10 visually-situated language understanding tasks across 5 domains: documents, tables, charts, natural images, and webpage screenshots. |
Inserting Information Bottlenecks for Attribution in Transformers (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained transformers are a popular approach for understanding features important for prediction. |
| Approach: | They apply information bottlenecks to analyze attribution of features for prediction on a black-box model. |
| Outcome: | The proposed method outperforms two competing methods in degradation tests on four datasets. |
What Part of the Neural Network Does This? Understanding LSTMs by Measuring and Dissecting Neurons (D19-1)
Copied to clipboard
| Challenge: | Biological neural systems consist of a huge number of neurons, and can react to the environment in complicated ways. |
| Approach: | They propose a metric to quantify the sensitivity of neurons to each label and conduct experiments to prove it. |
| Outcome: | The proposed metric is based on a set of experiments that show that dropping an arbitrary neuron significantly degrades the accuracy of the model. |
TRIDENT: Enhancing Large Language Model Safety with Tri-Dimensional Diversified Red-Teaming Data Synthesis (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) excel in natural language processing tasks but are vulnerable to harmful content and being exploited for malicious purposes. |
| Approach: | They propose a framework to measure the risk coverage of alignment datasets across three dimensions: Lexical Diversity, Malicious Intent, and Jailbreak Tactics. |
| Outcome: | The proposed framework measures risk coverage across Lexical Diversity, Malicious Intent, and Jailbreak Tactics. |
BERxiT: Early Exiting for BERT with Better Fine-Tuning and Extension to Regression (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods to make exiting decisions are limited to classification tasks . large-scale pre-trained language models such as BERT have brought performance gain but at the cost of heavy computational burden. |
| Approach: | They propose a fine-tuning strategy and a learning-to-exit module to accelerate BERT inference . they propose to make trade-offs between model quality and efficiency by early exiting . |
| Outcome: | The proposed approach improves early exiting for BERT, with better trade-offs . it can be combined with other acceleration methods, and the proposed strategy can be applied to regression tasks. |
Showing Your Work Doesn’t Always Work (2020.acl-main)
Copied to clipboard
| Challenge: | countless experimental papers lack empirical rigor, disregarding necessities such as statistical significance tests and computational environments. |
| Approach: | They propose to report the expected validation effectiveness of the best-tuned model with respect to the computational budget. |
| Outcome: | The proposed model favors negative errors and yields poor bootstrapped confidence intervals, the authors argue . they find that the proposed model is biased and uses error-prone assumptions . |
JointGT: Graph-Text Joint Representation Learning for Text Generation from Knowledge Graphs (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing pre-trained models for knowledgegraph-to-text generation ignore graph structure during encoding and lack elaborate pre-training tasks to explicitly model graph-text alignments. |
| Approach: | They propose a graph-text joint representation learning model called JointGT which incorporates a structure-aware semantic aggregation module into each Transformer layer to preserve the graph structure. |
| Outcome: | The proposed model achieves state-of-the-art performance on various KG-to-text datasets. |
TARN-VIST: Topic Aware Reinforcement Network for Visual Storytelling (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for visual storytelling ignore latent topic information. |
| Approach: | They propose a topic-aware reinforcement network for VIsual StoryTelling that takes topic information into account to generate a coherent story. |
| Outcome: | The proposed method outperforms most of the competing models across multiple evaluation metrics. |
Pre-training Language Models with Deterministic Factual Knowledge (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that Pre-trained Language Models fail to capture factual knowledge robustly. |
| Approach: | They propose to let PLMs learn the deterministic relationship between context and masked content to improve their ability to capture factual knowledge. |
| Outcome: | The proposed methods improve accuracy and consistency of factual knowledge capturing and boost performance of other knowledge-intensive tasks. |
DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference (2020.acl-main)
Copied to clipboard
| Challenge: | Large-scale pre-trained language models such as BERT are notorious for being slow in both training and inference. |
| Approach: | They propose a method to accelerate BERT inference by inserting extra classification layers between each transformer layer of BERT. |
| Outcome: | The proposed method saves up to 40% inference time with minimal degradation in model quality. |
JurisBench: A Deep Benchmark for Assessing Large Language Models in Professional Legal Practice (2026.acl-long)
Copied to clipboard
Ziang Chen, Guannan Li, Fanlin Ji, Yipeng Kang, Jiaqi Li, Muhan Zhang, Yangtao Zhang, Li Tianjiao, Jiannan Wang, Xin Guo, Song-Chun Zhu, Bin Ling
| Challenge: | Existing legal benchmarks evaluate isolated tasks or exam-style questions, failing to capture the procedural interdependencies and adjudicative rigor inherent in professional practice. |
| Approach: | They propose a vertical, depth-oriented, domain-specific benchmark to evaluate Large Language Models (LLMs) in Chinese civil litigation. |
| Outcome: | The proposed benchmarks show that large language models exhibit an "illusion of competence" the results highlight a critical gap between fluent linguistic output and judicial reliability . |
EcomScriptBench: A Multi-task Benchmark for E-commerce Script Planning via Step-wise Intention-Driven Product Association (2025.acl-long)
Copied to clipboard
Weiqi Wang, Limeng Cui, Xin Liu, Sreyashi Nag, Wenju Xu, Chen Luo, Sheikh Muhammad Sarwar, Yang Li, Hansu Gu, Hui Liu, Changlong Yu, Jiaxin Bai, Yifan Gao, Haiyang Zhang, Qi He, Shuiwang Ji, Yangqiu Song
| Challenge: | Goal-oriented script planning is used by humans to plan for typical activities . however, this capability remains underexplored due to several challenges . |
| Approach: | They propose a framework that enables product-enriched scripts by associating products with each step based on the semantic similarity between the actions and their purchase intentions. |
| Outcome: | The proposed framework can generate product-enriched scripts from 2.4 million scripts . human annotations are conducted to provide gold labels for a sampled subset . |
LongAlign: A Recipe for Long Context Alignment of Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies to build long context language models focus on context extension and continual training on long text. |
| Approach: | They propose a recipe for instruction fine-tuning on input sequences of similar length . they adopt packing and sorted batching strategies to speed up supervised fine-uning . |
| Outcome: | The proposed model outperforms existing recipes for LLMs in long context tasks by 30% while maintaining proficiency in handling short, generic tasks. |
Simple and Effective Unsupervised Redundancy Elimination to Compress Dense Vectors for Passage Retrieval (2021.emnlp-main)
Copied to clipboard
| Challenge: | Dense passage retrieval improves ranking accuracy in open-domain question answering but at the cost of large space and memory requirements. |
| Approach: | They propose a simple unsupervised pipeline that includes principal component analysis (PCA), product quantization, and hybrid search to improve space efficiency. |
| Outcome: | The proposed pipeline achieves good accuracy–space trade-offs, for example, 48 compression with less than 3% drop in top-100 retrieval accuracy on average or 96 compression without drop in space requirements. |
Put It Back: Entity Typing with Language Model Enhancement (D18-1)
Copied to clipboard
| Challenge: | Existing Entity typing models suffer from noisy labels due to distant supervision . |
| Approach: | They propose to enhance existing entity typing models with language model enhancement to measure compatibility between context sentences and labels. |
| Outcome: | The proposed model significantly outperforms the state-of-the-art model on benchmark datasets and is available on github. |
Pre-training CLIP against Data Poisoning with Optimal Transport-based Matching and Alignment (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that Contrastive Language-Image Pre-training (CLIP) models are vulnerable to data poisoning and backdoor attacks due to massive training image-caption pairs crawled from the Internet. |
| Approach: | They propose an Optimal Transport-based framework to reconstruct image-caption pairs and propose an optimal transport-based distance measure to re-assign new captions based on the proposed optimal transport distance. |
| Outcome: | The proposed framework reduces the attack success rates of poisoning attacks to 0% in most cases. |
LexGenius: An Expert-Level Benchmark for Large Language Models in Legal General Intelligence (2026.findings-acl)
Copied to clipboard
Wenjin Liu, Haoran Luo, Xin Feng, Xiang Ji, Lijuan Zhou, Rui Mao, Jiapu Wang, Shirui Pan, Erik Cambria
| Challenge: | Existing benchmarks for legal general intelligence (GI) are result-oriented and do not evaluate the legal intelligence of large language models (LLMs). |
| Approach: | They propose a Chinese legal benchmark for evaluating legal GI in large language models . they use recent legal cases and exam questions to create multiple-choice questions . |
| Outcome: | The proposed benchmarks lack a systematic evaluation of the legal intelligence of large language models (LLMs) the results show that even the best LLMs lagging behind human legal professionals. |
Voice Query Auto Completion (2021.emnlp-main)
Copied to clipboard
Raphael Tang, Karun Kumar, Kendra Chalkley, Ji Xin, Liming Zhang, Wenyan Li, Gefei Yang, Yajie Mao, Junho Shin, Geoffrey Craig Murray, Jimmy Lin
| Challenge: | Existing methods fail to complete voice queries from incomplete prefixes because they use orthographic prefix and substrings instead of the true phonetic prefix. |
| Approach: | They propose to condition QAC approaches on intermediate transcriptions to complete voice queries. |
| Outcome: | The proposed method obtains an 18% relative improvement over previous methods on a speech-enabled smart television with real-life voice search traffic. |
Certified Error Control of Candidate Set Pruning for Two-Stage Relevance Ranking (2022.emnlp-main)
Copied to clipboard
| Challenge: | In information retrieval, candidate set pruning is used to speed up two-stage relevance ranking but lacks accurate error control and empirical guarantees. |
| Approach: | They propose a method that guarantees the test error after pruning is controlled under a user-specified threshold with high probability. |
| Outcome: | The proposed method reduces the average set size from 1000 to 27, increasing reranking speed by about 37 times while keeping MRR@10 greater than a pre-specified value of 0.38 with about 90% empirical coverage. |