Papers by Jianfei Zhang
Selecting Demonstrations for Many-Shot In-Context Learning via Gradient Matching (2025.findings-acl)
Copied to clipboard
| Challenge: | In-Context Learning (ICL) empowers Large Language Models for rapid task adaptation without fine-tuning. |
| Approach: | They propose a method that aligns fine-tuning gradients between entire training set and selected examples to enable in-context learning and fine-uning. |
| Outcome: | The proposed method outperforms random selection on large LLMs from 4-shot to 128-shot scenarios across 9 datasets. |
The Role of Mixed-Language Documents for Multilingual Large Language Model Pretraining (2026.acl-long)
Copied to clipboard
Jiandong Shao, Raphael Tang, Crystina Zhang, Karin Sevegnani, Pontus Stenetorp, Jianfei Yang, Yao Lu
| Challenge: | Existing research suggests that multilingual large language models can achieve impressive cross-lingual understanding despite largely monolingual pretraining. |
| Approach: | They compare a monolingual-only corpus with a standard web corpus that removes all multilingual documents and then retrain the models from scratch under controlled conditions. |
| Outcome: | The results show that removing bilingual data causes translation performance to drop 56% in BLEU, whereas code-switching contributes minimally. |
Lost in Pronunciation: Detecting Chinese Offensive Language Disguised by Phonetic Cloaking Replacement (2025.emnlp-industry)
Copied to clipboard
Haotan Guo, Jianfei He, Jiayuan Ma, Hongbin Na, Zimu Wang, Haiyang Zhang, Qi Chen, Wei Wang, Zijing Shi, Tao Shen, Ling Chen
| Challenge: | Phonetic Cloaking Replacement (PCR) is a problem in content moderation in China. |
| Approach: | They organize PCR into a four-way surface-form taxonomy and compile PCR-ToxiCN, a dataset of 500 phonetically cloaked offensive posts gathered from the RedNote platform. |
| Outcome: | The proposed model achieves only an F1-score and zero-shot chain-of-thought prompting pushes performance even lower. |
Disentangling Preference Representation and Text Generation for Efficient Individual Preference Alignment (2025.coling-main)
Copied to clipboard
| Challenge: | Human values are inherently diverse, making it insufficient to align LLMs solely with general preferences. |
| Approach: | They propose a flexible paradigm for individual preference alignment that disentangles preference representation from text generation in LLMs. |
| Outcome: | The proposed method produces aligned quality and better than PEFT-based methods while reducing training time for each new individual preference by 80% to 90%. |
Mitigating Hallucination in Large Vision-Language Models through Aligning Attention Distribution to Information Flow (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Decode-Only models propagate information from left to right, but the model's attention still focuses on the visual representations, resulting in hallucinations. |
| Approach: | They propose to leverage the core information embedded in semantic representations to enhance the model's visual understanding by leveraging the attention distributions. |
| Outcome: | The proposed method reduces hallucinations by 80% by aligning the attention distribution with the actual information flow. |
PhonoThink: Improving Large Language Models’ Reasoning on Chinese Phonological Ambiguities (2025.emnlp-main)
Copied to clipboard
| Challenge: | Effectively resolving phonological ambiguities is crucial for robust natural language processing, as these ambiguity are pervasive in tasks ranging from speech-to-text, spelling correction, to offensive language detection. |
| Approach: | They propose a framework to enhance LLMs’ phonological capability through a multiple-stage training approach. |
| Outcome: | The proposed framework enables the base model to achieve comparable performance to a much larger model. |
Leveraging Estimated Transferability Over Human Intuition for Model Selection in Text Ranking (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for text ranking are based on intuition, but their estimated transferability may not align well with the objectives of text ranking. |
| Approach: | They propose to compute expected rank as transferability, explicitly reflecting the model’s ranking capability. |
| Outcome: | The proposed method shows significant improvements over previous classification-oriented TE methods, human intuition, and ChatGPT with minor time consumption. |
ProCQA: A Large-scale Community-based Programming Question Answering Dataset for Code Search (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches to code question answering use bi-modal and unimodal pretraining to align text and code representations. |
| Approach: | They propose a modality-agnostic contrastive pre-training approach to improve alignment of text and code representations of current code language models. |
| Outcome: | The proposed model exhibits significant performance improvements across a wide range of code retrieval benchmarks. |
Enhancing Dual-Encoders with Question and Answer Cross-Embeddings for Answer Retrieval (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to solve question answering (QA) problems are limited by the need for text generation and answer retrieval. |
| Approach: | They propose to introduce QA interaction features in scoring function but at the cost of low efficiency in inference stage. |
| Outcome: | The proposed framework significantly outperforms the state-of-the-art method on multiple answer retrieval datasets. |