Papers by Ling Zheng
Spiral of Silence in Large Language Model Agents (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing theories of Spiral of Silence do not apply to large language models . |
| Approach: | They propose an evaluation framework for examining SoS in large language models . they consider four controlled conditions that vary the availability of "History" and "Persona" signals . |
| Outcome: | The proposed framework examines the SoS-like dynamics in large language models . it shows that history and persona together produce strong majority dominance . |
SpatialWebAgent: Leveraging Large Language Models for Automated Spatial Information Extraction and Map Grounding (2025.acl-demo)
Copied to clipboard
| Challenge: | Understanding and extracting spatial information from text is vital for a wide range of applications, says nielsen . inherent complexity of geographic expressions in natural language presents significant hurdles for traditional extraction methods. |
| Approach: | They propose a system that leverages large language models to extract spatial information from natural language. |
| Outcome: | SpatialWebAgent is designed to extract, standardize, and ground spatial information from natural language text directly onto maps. |
Exploring the Impact of Negative Samples of Contrastive Learning: A Case Study of Sentence Embedding (2022.findings-acl)
Copied to clipboard
| Challenge: | Unsupervised contrastive learning is emerging as a powerful technique for extracting knowledge from unlabeled data. |
| Approach: | They propose a momentum contrastive learning model with negative sample queue for sentence embedding with a simulated model with EMA update mechanism. |
| Outcome: | The proposed model achieves a Spearman’s correlation of 77.27% on the semantic text similarity task and a maximum traceable distance metric. |
Attention-Enhancing Backdoor Attacks Against BERT-based Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing textual backdoor attacks focus on generating stealthy triggers or modifying model weights. |
| Approach: | They propose a Trojan Attention Loss (TAL) which enhances the Trojan behavior by directly manipulating attention patterns. |
| Outcome: | The proposed method improves the effectiveness of the backdoor attacks on different backbone models and tasks. |
Aligning Large Language Models with Human Preferences through Representation Engineering (2024.acl-long)
Copied to clipboard
Wenhao Liu, Xiaohua Wang, Muling Wu, Tianlong Li, Changze Lv, Zixuan Ling, Zhu JianHao, Cenyuan Zhang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Existing methods for achieving this alignment involve employing reinforcement learning from human feedback (RLHF) Existing approaches involve using RLHF to fine-tune LLMs based on human labels . however, RLRF is susceptible to instability during fine- tuning and presents challenges in implementation. |
| Approach: | They propose to use reinforcement learning from human feedback to fine-tune large language models with human preferences to achieve precise control of model behavior. |
| Outcome: | Experiments show that RAHF can be used to capture and manipulate representations to align with a broad spectrum of human preferences or values rather than being confined to a single concept or function. |
Parameter Efficient Multi-task Fine-tuning by Learning to Transfer Token-wise Prompts (2023.findings-emnlp)
Copied to clipboard
Muling Wu, Wenhao Liu, Jianhan Xu, Changze Lv, Zixuan Ling, Tianlong Li, Longtao Huang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Prompt tuning has been proven to be successful on various tasks by incorporating a small number of trainable parameters while freezing large pre-trained language models. |
| Approach: | They propose a token-wise prompt tuning method that uses a bank of finer-grained soft prompt tokens to generate an instance-dependent prompt. |
| Outcome: | The proposed method performs far better than full parameter fine-tuned models and achieves state-of-the-art by tuning only 0.035% parameters on 14 datasets. |
Benchmarking Foundation Models with Retrieval-Augmented Generation in Olympic-Level Physics Problem Solving (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new study examines the potential of retrieval-augmented generation (RAG) with foundation models to enhance expert-level reasoning. |
| Approach: | They introduce PhoPile, a high-quality multimodal dataset specifically designed for Olympiad-level physics. |
| Outcome: | The proposed model can be used to solve Olympiad-level physics problems. |
LEANCODE: Understanding Models Better for Code Simplification of Pre-trained Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models often require significant computational resources, often constraining input word or code token lengths. |
| Approach: | They propose to use the encoder-decoder attention scores to represent the importance of a code token across multiple contexts to reduce training and prediction time. |
| Outcome: | The proposed approach outperforms the SOTAs DietCode and SlimCode in code search and summarization tasks. |
Heterogeneous Graph Neural Networks to Predict What Happen Next (2020.coling-main)
Copied to clipboard
| Challenge: | Existing work on event representation cannot capture discontinuous event segments . Existing models cannot represent heterogeneous relations and discontinuous events . |
| Approach: | They propose a heterogeneous-event graph network to model missing events . they employ each unique word and individual event as nodes in the graph . |
| Outcome: | The proposed model outperforms baseline models on one-step and multi-step inference tasks. |
DIAG-NRE: A Neural Pattern Diagnosis Framework for Distantly Supervised Neural Relation Extraction (P19-1)
Copied to clipboard
| Challenge: | Existing methods for labeling relational facts require significant expert labor to write relation-specific patterns, which makes them too sophisticated to generalize quickly. |
| Approach: | They propose a neural pattern diagnosis framework that can summarize and refine relation-specific patterns with human experts in the loop. |
| Outcome: | The proposed framework can summarize and refine high-quality relational patterns from noise data with human experts in the loop. |
Beyond Examples: Towards Automated Thought-level In-Context Reasoning for Large Language Models (2026.acl-long)
Copied to clipboard
Jinyang Wu, Mingkuan Feng, Shuai Zhang, Feihu Che, Zhengqi Wen, Chonghua Liao, Ling Yang, Haoran Luo, Zheng Lian, Jianhua Tao
| Challenge: | In-context learning (ICL) struggles with complex reasoning due to superficial, example-level implicit imitation. |
| Approach: | They propose an automated method that shifts from surface-level examples to more guidance-oriented thought patterns. |
| Outcome: | The proposed method achieves 80.6% accuracy on MATH and 62.5% on AMC, surpassing GPT-4o’s 77.2% and 57.5% accuracy. |
Task-Agnostic Detector for Insertion-Based Backdoor Attacks (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for textual backdoor detection are task-specific and less effective beyond sentence classification. |
| Approach: | They propose a task-agnostic method for backdoor detection that leverages final layer logits and an efficient pooling technique. |
| Outcome: | TABDet can jointly learn from diverse task-specific models, demonstrating superior detection efficacy over traditional methods. |
GRAG: Graph Retrieval-Augmented Generation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Naive Retrieval-Augmented Generation (RAG) focuses on individual documents during retrieval and is not suitable for networked documents. |
| Approach: | They propose a novel divide-and-conquer strategy that retrieves optimal subgraph structure in linear time. |
| Outcome: | The proposed approach outperforms current state-of-the-art methods on graph reasoning benchmarks. |
Advancing Parameter Efficiency in Fine-tuning via Representation Editing (2024.acl-long)
Copied to clipboard
Muling Wu, Wenhao Liu, Xiaohua Wang, Tianlong Li, Changze Lv, Zixuan Ling, Zhu JianHao, Cenyuan Zhang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Parameter Efficient Fine-Tuning (PEFT) has gained significant attention for its ability to achieve competitive results while updating only a small subset of trainable parameters. |
| Approach: | They propose a new approach to fine-tuning neural models that scales and biases the representation produced at each layer. |
| Outcome: | The proposed approach reduces the number of trainable parameters by a factor of 25,700 compared to full parameter fine-tuning and by . 32 compared with LoRA. |
Relation-aware Ensemble Learning for Knowledge Graph Embedding (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to explore semantics of knowledge graphs have been proposed to explore these semantics in distinct ways. |
| Approach: | They propose to leverage existing methods in relation-aware manner to learn an ensemble by leveraging existing methods. |
| Outcome: | The proposed method has the same computation cost as general ensemble methods but with much better performance on benchmark datasets. |
RJE: A Retrieval-Judgment-Exploration Framework for Efficient Knowledge Graph Question Answering with LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Knowledge graph question answering (KGQA) aims to answer natural language questions using knowledge graphs. |
| Approach: | They propose a framework that retrieves refined reasoning paths and evaluates their sufficiency. |
| Outcome: | The proposed framework outperforms existing baselines while enabling small open-source LLMs to achieve competitive results without fine-tuning LLM. |
OFFSIDE: Benchmarking Unlearning Misinformation in Multimodal Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for MU are limited by a lack of image diversity and coarse-grained unlearning targets. |
| Approach: | They propose a benchmark to evaluate misinformation unlearning in MLLMs . OFFSIDE supports advanced unlearning targets such as fine-grained unlearning and visual rumor removal. |
| Outcome: | OFFSIDE supports advanced unlearning targets, such as fine-grained unlearning and visual rumor removal. |
Enhancing Unsupervised Semantic Parsing with Distributed Contextual Representations (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to learn models on corpus of pairs of sentences require labor-intensive annotation. |
| Approach: | They propose to leverage distributed contextual word and phrase representations pre-trained on unlabelled texts to deal with homonymy and polysemy. |
| Outcome: | The proposed model achieves better accuracy on question-answering and relation extraction tasks. |
Promoting Data and Model Privacy in Federated Learning through Quantized LoRA (2024.findings-emnlp)
Copied to clipboard
Zhu JianHao, Changze Lv, Xiaohua Wang, Muling Wu, Wenhao Liu, Tianlong Li, Zixuan Ling, Cenyuan Zhang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Existing federated learning frameworks require substantial data and computational resources to develop large language models. |
| Approach: | They propose a method that distributes a quantized version of the model’s parameters during training and combine it with a popular fine-tuning method to significantly reduce communication costs. |
| Outcome: | The proposed method enables accurate estimations for parameter updates while preventing clients from accessing a model whose performance is comparable to the centrally hosted one. |