Papers by Jing Yan
Language Model Detoxification in Dialogue with Contextualized Stance Control (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on Language Model detoxification has focused on reducing the toxicity of the generation itself without consideration of the context. |
| Approach: | They propose a method to do context-dependent detoxification without taking into account the stance of the generated response. |
| Outcome: | The proposed method can learn the context-dependent stance control strategies while keeping a low self-toxicity of the underlying LM. |
Beyond Literal Descriptions: Understanding and Locating Open-World Objects Aligned with Human Intentions (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for visual grounding rely on the assumption that the given expression must be literal . this impedes the practical deployment of agents in real-world scenarios. |
| Approach: | They propose a visual grounding task that uses intention expressions to locate foreground entities . they build a large-scale IVG dataset with free-form intention expression to promote VG . |
| Outcome: | The proposed method is based on a large-scale intention-driven visual-language (V-L) dataset with free-form intention expressions. |
CodeJudge-Eval: Can Large Language Models be Good Judges in Code Understanding? (2025.coling-main)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have showcased impressive code generation capabilities, primarily evaluated through language-to-code benchmarks. |
| Approach: | They propose a benchmark to assess LLMs’ code understanding abilities from the perspective of code judging rather than code generation. |
| Outcome: | The proposed benchmark evaluates 12 well-known large language models to determine the correctness of provided code solutions. |
Muffin or Chihuahua? Challenging Multimodal Large Language Models with Multipanel VQA (2024.acl-long)
Copied to clipboard
Yue Fan, Jing Gu, Kaiwen Zhou, Qianqi Yan, Shan Jiang, Ching-Chen Kuo, Yang Zhao, Xinze Guan, Xin Wang
| Challenge: | Multipanel images are a common form of visual representations, and humans can achieve approximately 99% accuracy on these questions. |
| Approach: | They propose a benchmark that tests multipanel visual reasoning models with 6,600 triplets of questions, answers, and multipanel images. |
| Outcome: | The proposed benchmark features 6,600 triplets of questions, answers, and multipanel images that challenge state-of-the-art Multimodal Large Language Models (MLLMs) human users can attain approximately 99% accuracy on these questions, compared with previous benchmarks. |
Federated LoRA Fine-Tuning with Pipelined Error-Mitigated Aggregation and Matrix-Wise Freezing (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning large language models often suffer from biased model aggregation and are hindered by significant communication and computation burden. |
| Approach: | They propose a Federated low-rank adaptation system for large language models that leverages pipelined error-mitigated model aggregation and adaptive matrix-wise parameter freezing to mitigate aggregations. |
| Outcome: | The proposed system improves time-to-target by 2.17-8.48 on real-world datasets. |
Limitations of Language Models in Arithmetic and Symbolic Induction (2023.acl-long)
Copied to clipboard
| Challenge: | Recent work has shown that large pretrained Language Models (LMs) can perform remarkably well on a range of NLP tasks but they have limitations on basic symbolic manipulation tasks such as copy, reverse, and addition. |
| Approach: | They propose to use explicit positional markers, fine-grained computation steps, and LMs with callable programs to teach large pretrained Language Models. |
| Outcome: | The proposed model can perform 100% accuracy in OOD and repeating symbols. |
SPD-Faith Bench: Diagnosing and Improving Faithfulness in Chain-of-Thought for Multimodal Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on multimodal faithfulness have focused on perceptual hallucinations, raising concerns about the validity of reasoning traces. |
| Approach: | They propose a diagnostic benchmark that enforces explicit visual comparison to assess faithfulness of reasoning traces. |
| Outcome: | The proposed framework improves visual routing and aligns reasoning with perception. |
TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities (2023.acl-demo)
Copied to clipboard
Zhe Zhao, Yudong Li, Cheng Hou, Jing Zhao, Rong Tian, Weijie Liu, Yiren Chen, Ningyuan Sun, Haoyan Liu, Weiquan Mao, Han Guo, Weigang Gou, Taiqiang Wu, Tao Zhu, Wenhang Shi, Chen Chen, Shan Huang, Sihong Chen, Liqun Liu, Feifei Li, Xiaoshuai Chen, Xingwu Sun, Zhanhui Kang, Xiaoyong Du, Linlin Shen, Kimmo Yan
| Challenge: | Several pre-training models of different modalities are showing a rising trend of homogeneity in their model structures. |
| Approach: | They propose a toolkit that supports pre-training models of different modalities. |
| Outcome: | The proposed toolkit can match the performance of the original implementations on text, vision, and audio benchmarks. |
Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Models (2025.findings-acl)
Copied to clipboard
Yingqian Cui, Pengfei He, Jingying Zeng, Hui Liu, Xianfeng Tang, Zhenwei Dai, Yan Han, Chen Luo, Jing Huang, Zhen Li, Suhang Wang, Yue Xing, Jiliang Tang, Qi He
| Challenge: | Chain-of-Thought (CoT) reasoning has improved the performance of large language models (LLMs) however, the detailed reasoning process in CoT often incurs long generation times and high computational costs due to the inclusion of unnecessary steps. |
| Approach: | They propose a method to identify critical reasoning steps using perplexity as a measure of their importance. |
| Outcome: | The proposed method achieves a better balance between reasoning accuracy and efficiency of CoT. |
LLM Agents for Education: Advances and Applications (2025.findings-emnlp)
Copied to clipboard
Zhendong Chu, Shen Wang, Jian Xie, Tinghui Zhu, Yibo Yan, Jingheng Ye, Aoxiao Zhong, Xuming Hu, Jing Liang, Philip S. Yu, Qingsong Wen
| Challenge: | Large Language Model (LLM) agents are transforming education by automating complex tasks and enhancing both teaching and learning processes. |
| Approach: | This survey analyzes recent advances in applying Large Language Model agents to educational settings . it highlights ethical issues, hallucination and overreliance, and integration with existing ecosystems . |
| Outcome: | The authors analyze the technologies enabling LLM agents and highlight key challenges in deploying them in educational settings. |
ERGO: Event Relational Graph Transformer for Document-level Event Causality Identification (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods to identify event-event causal relations in a document are noisy and require heuristic rules or external tools. |
| Approach: | They propose a document-level event-event causality identification framework that uses heuristic rules to design edges between events. |
| Outcome: | The proposed framework outperforms existing state-of-the-art methods on two benchmark datasets. |
Can LLM Agents Simulate Multi-Turn Human Behavior? Evidence from Real Online Customer Behavior Data (2026.acl-long)
Copied to clipboard
Yuxuan Lu, Jing Huang, Yan Han, Bingsheng Yao, Sisong Bei, Yaochen Xie, Yisi Sang, Qi He, Dakuo Wang
| Challenge: | Recent research shows that LLM Agents can generate “believable” human behaviors via prompt-only methods, leaving open questions of whether they can accurately generate step-by-step actions in multi-turn interaction tasks. |
| Approach: | They propose to use shopping data to evaluate LLMs' ability to accurately generate step-by-step actions in a multi-turn interaction task. |
| Outcome: | The proposed model achieves 17.26% action generation accuracy and 33.86% F1 score on final purchase prediction, representing improvements of 5.4% and 13.85% over baselines. |
Self-adaptive Dataset Construction for Real-World Multimodal Safety Scenarios (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing dataset construction methods fail to cover the complexity of multimodal safety scenarios . lack of a unified evaluation metric makes them unproven . |
| Approach: | They propose a risk-oriented image-oriented self-adaptive dataset construction method for RMS . they automatically generate an RMS dataset comprising 35,610 image–text pairs with guidance responses . |
| Outcome: | The proposed method automatically generates an RMS dataset comprising 35,610 image–text pairs with guidance responses. |
Directional Skip-Gram: Explicitly Distinguishing Left and Right Context for Word Embeddings (N18-2)
Copied to clipboard
| Challenge: | Existing word embedding models are limited by semantic resources, which are hard to obtain or annotate. |
| Approach: | They propose a directional skip-gram model that explicitly distinguishes between left and right contexts in word prediction. |
| Outcome: | The proposed model outperforms other models on different datasets in semantic and syntactic evaluations. |
LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shifts (2025.acl-long)
Copied to clipboard
Qibing Ren, Hao Li, Dongrui Liu, Zhanxu Xie, Xiaoya Lu, Yu Qiao, Lei Sha, Junchi Yan, Lizhuang Ma, Jing Shao
| Challenge: | Current safety training focuses on teaching models to reject harmful queries, but recent research shows that adversarial attacks or jailbreak methods bypass these safety mechanisms. |
| Approach: | They propose to use a new attack method to craft multi-turn toxic prompts that gradually lead LLMs to reveal unsafe content. |
| Outcome: | The proposed method outperforms existing methods in diversity, effectiveness, and efficiency across aligned LLMs. |
Controllable Dialogue Simulation with In-context Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to generate annotated dialogues require crowdsourcing, which is expensive and time-consuming. |
| Approach: | They propose a dialogue simulation method based on large language model in-context learning that generates new dialogues and annotations in a controllable way. |
| Outcome: | The proposed method can expand a small set of dialogue data with minimum or zero human involvement and parameter update. |
FrontCoder: Scaling Visual Fidelity in Front-End Code Generation (2026.findings-acl)
Copied to clipboard
Jun Feng, Jian Yang, Wei Zhang, Jing Wang, Keyi Chen, Xiaokun Yang, Weicheng Gu, Yihang Lou, Yan Bai, Xianglong Liu
| Challenge: | Existing work on front-end code generation fails to provide visual fidelity and rendering quality for front- end developers. |
| Approach: | They propose a three-stage pipeline to enhance front-end code generation capabilities in LLMs . they use synthetic data, quality-controlled supervised fine-tuning, and reinforcement learning . |
| Outcome: | The proposed model achieves competitive performance with frontier models while maintaining generation efficiency. |
Projecting Out the Malice: A Global Subspace Approach to LLM Detoxification (2026.acl-long)
Copied to clipboard
Zenghao Duan, Zhiyi Yin, Zhichao Shi, Liang Pang, Shaoling Jing, Zihe Huang, Jiayi Wu, Yu Yan, Jingcheng Deng, Huawei Shen, Xueqi Cheng
| Challenge: | Large language models (LLMs) exhibit exceptional performance but pose inherent risks of generating toxic content. |
| Approach: | They propose a method that removes toxic subspaces from FFN parameters . they propose to use a lightweight method to eliminate toxic subespaces . |
| Outcome: | The proposed method achieves SOTA detoxification while preserving general capabilities without large-scale retraining. |
Topic Memory Networks for Short Text Classification (D18-1)
Copied to clipboard
| Challenge: | Existing classification models for short texts are weak due to data sparsity . |
| Approach: | They propose topic memory networks for short text classification with a novel topic memory mechanism to encode latent topic representations indicative of class labels. |
| Outcome: | The proposed model outperforms state-of-the-art models on short text classification, while generating coherent topics. |
CodeAttack: Revealing Safety Generalization Challenges of Large Language Models via Code Completion (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models exhibit remarkable generative capabilities but can be misused for harmful purposes. |
| Approach: | They propose a framework that transforms natural language inputs into code inputs. |
| Outcome: | The proposed framework bypasses the safety guardrails of all models more than 80% of the time. |
BiPaR: A Bilingual Parallel Dataset for Multilingual and Cross-lingual Reading Comprehension on Novels (D19-1)
Copied to clipboard
| Challenge: | Using BiPaR, we build monolingual, multilingual and cross-lingual MRC on novels. |
| Approach: | They propose a bilingual parallel novel-style machine reading comprehension dataset BiPaR . they collect 3,667 bilingual parallel paragraphs from Chinese and English novels . |
| Outcome: | The proposed dataset supports multilingual and cross-lingual reading comprehension. |
Pretraining Without Attention (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that state-space models (SSMs) outperform standard and deep learning for long-range sequence modeling. |
| Approach: | They propose a model that combines SSM layers with a multiplicative gating architecture that has been effective in simplified sequence modeling architectures. |
| Outcome: | The proposed model outperforms standard and standard sequence modeling architectures on speech generation and the long range arena benchmarks. |
Lifelong Learning of Hate Speech Classification on Social Media (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing work on automated hate speech classification assumes that the dataset is fixed and the classes are pre-defined. |
| Approach: | They propose to use Variational Representation Learning and a load-balancing self-organizing inductive neural network to learn hate speech classification on social media. |
| Outcome: | The proposed model improves on the lifelong learning techniques on social media. |
LMR-BENCH: Evaluating LLM Agent’s Ability on Reproducing Language Modeling Research (2025.emnlp-main)
Copied to clipboard
Shuo Yan, Ruochen Li, Ziming Luo, Zimu Wang, Daoyang Li, Liqiang Jing, Kaiyu He, Peilin Wu, Juntong Ni, George Michalopoulos, Yue Zhang, Ziyang Zhang, Mian Zhang, Zhiyu Chen, Xinya Du
| Challenge: | Large language model (LLM) agents have demonstrated remarkable potential in advancing scientific discovery, but their capability in reproducing code from research papers remains underexplored. |
| Approach: | They propose to evaluate LLM agents' ability to reproduce scientific research papers by analyzing code reproduction tasks from 23 research papers published in top-tier NLP venues. |
| Outcome: | The proposed benchmark systematically evaluates the capability of large language model (LLM) agents on code reproduction from Language Modeling Research. |
DuQM: A Chinese Dataset of Linguistically Perturbed Natural Questions for Evaluating the Robustness of Question Matching Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | a comprehensive evaluation of QM models should be conducted on natural texts, not on artificial adversarial examples . ral models are often not robust to adversarials, which means they predict unexpected outputs . |
| Approach: | They use a Chinese dataset to evaluate the robustness of QM models . they show that the effect of artificial adversarial examples does not work on natural texts . |
| Outcome: | The proposed model is more robust than other models on natural questions with 32 linguistic perturbations. |
A Hybrid Approach to Automatic Corpus Generation for Chinese Spelling Check (D18-1)
Copied to clipboard
| Challenge: | Chinese spelling check (CSC) is a challenging but meaningful task that serves as a preprocessing in many natural language processing(NLP) applications. |
| Approach: | They propose to construct Chinese spelling check corpus with automatically generated spelling errors, which are either visually or phonologically resembled characters, corresponding to OCR- and ASR-based methods. Experimental results demonstrate the effectiveness of the approach. |
| Outcome: | The proposed method is based on visual or phonologically similar spelling errors, and is validated with respect to three standard test sets. |
FIHA: Automated Fine-grained Hallucinations Evaluations in Large Vision Language Models with Davidson Scene Graphs (2025.findings-acl)
Copied to clipboard
| Challenge: | Current approaches to large vision-language models rely on costly annotations and are not comprehensive in terms of evaluating all aspects. |
| Approach: | They propose an automated method which can access LVLMs hallucination in an LLM-free and annotation-free way and model the dependency between different types of halluciNations. |
| Outcome: | The proposed model can model the dependency between different types of hallucinations and generate Q&A pairs on any image dataset at minimal cost. |
Encoding Conversation Context for Neural Keyphrase Extraction from Microblog Posts (N18-1)
Copied to clipboard
| Challenge: | Existing keyphrase extraction methods suffer from data sparsity problem when conducted on short and informal texts. |
| Approach: | They propose a neural keyphrase extraction framework for microblog posts that takes conversation context into account and uses four types of neural encoders to represent conversation context. |
| Outcome: | The proposed framework outperforms state-of-the-art keyphrase extraction methods on Twitter and Weibo datasets. |