Papers by Junzhuo Li
FineCops-Ref: A new Dataset and Task for Fine-Grained Compositional Referring Expression Comprehension (2024.emnlp-main)
Copied to clipboard
| Challenge: | Referring Expression Comprehension (REC) is a cross-modal task that objectively evaluates the capabilities of language understanding, image comprehension, and language-to-image grounding. |
| Approach: | They propose to use a new reference expression comprehension (REC) dataset to evaluate the capabilities of language understanding, image comprehension, and language-to-image grounding. |
| Outcome: | The proposed model is able to reject scenarios where the target object is not visible in the image, a key aspect often overlooked in existing models and approaches. |
Tab-CQA: A Tabular Conversational Question Answering Dataset on Financial Reports (2023.acl-industry)
Copied to clipboard
| Challenge: | Existing conversational question answering datasets are usually constructed from unstructured texts in English. |
| Approach: | They propose a Chinese tabular conversational question answering dataset based on financial reports . they select 2,463 tables and manually generate 2,463, conversations with 35,494 QA pairs . |
| Outcome: | The proposed dataset is based on Chinese financial reports extracted from listed companies in the past 30 years. |
Dynamic Expert Specialization: Towards Catastrophic Forgetting-Free Multi-Domain MoE Adaptation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to adapt Mixture-of-Experts models to multiple domains are prohibitive computation, cross-domain interference or require separate runs per domain. |
| Approach: | They propose a dynamic expert specialization framework for multi-domain adaptation of Mixture-of-Experts models. |
| Outcome: | The proposed framework reduces forgetting by 89% compared to full fine-tuning as domains scale from 2 to 6 and achieves faster convergence than conventional methods. |
Capturing Nuanced Preferences: Preference-Aligned Distillation for Small Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for aligning small language models with human values model preference knowledge from large language models (LLMs) however, this limitation hinders student SLMs from capturing nuanced preferences for multiple responses. |
| Approach: | They propose a framework which models teacher's preference knowledge as a probability distribution over all potential preferences, thereby providing more nuanced supervisory signals. |
| Outcome: | The proposed framework outperforms existing methods on four benchmark tasks and achieves 20% improvement on AlpacaEval 2 and Arena-Hard. |
DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have demonstrated that pretrained language models memorize and regurgitate a significant portion of training data, including atypical data points that appear only once in the training data. |
| Approach: | They propose a method to locate and erase risky neurons in order to eliminate the impact of privacy data in the model in batches. |
| Outcome: | The proposed method eliminates the impact of privacy data in the model in batches without affecting the model's performance. |
Language Representation Projection: Can We Transfer Factual Knowledge across Languages in Multilingual Language Models? (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that multilingual pretrained models can recall factual knowledge without additional fine-tuning. |
| Approach: | They propose two parameter-free language representation projection modules to transfer factual knowledge between English and non-English languages. |
| Outcome: | The proposed modules improve factual knowledge retrieval accuracy and transferability across diverse non-English languages. |
Optimal Expert-Attention Allocation in Mixture-of-Experts: A Scalable Law for Dynamic Model Design (2026.acl-industry)
Copied to clipboard
| Challenge: | a novel extension of neural scaling laws to Mixture-of-Experts models is proposed . a ratio of expert-attention compute is crucial for efficient MoE models . |
| Approach: | They propose an extension of neural scaling laws to Mixture-of-Experts (MoE) models . they define the ratio r as the fraction of total FLOPs per token dedicated to expert and attention layers . |
| Outcome: | The proposed model can be tuned beyond size and data with the proposed model. |
Decoding Knowledge Attribution in Mixture-of-Experts: A Framework of Basic-Refinement Collaboration and Efficiency Analysis (2025.acl-long)
Copied to clipboard
| Challenge: | Existing attribution methods for dense models fail to capture dynamic routing-expert interactions in sparse MoE architectures. |
| Approach: | They propose to analyze sparse MoE architectures against dense models to capture dynamic routing-expert interactions. |
| Outcome: | The proposed algorithm shows that sparse models achieve higher efficiency per layer . it also shows that deep Qwen-MoE mitigates expert failures while minimizing complexity . |
KaFSP: Knowledge-Aware Fuzzy Semantic Parsing for Conversational Question Answering over a Large-Scale Knowledge Base (2022.acl-long)
Copied to clipboard
| Challenge: | Existing semantic parsing frameworks for conversational question answering do not handle uncertain reasoning . qa over large knowledge bases has attracted broad interest due to the popularity of intelligent virtual assistants . |
| Approach: | They propose a fuzzy semantic parsing framework that defines fuzzy comparison operations in grammar for uncertain reasoning based on fuzzy set theory. |
| Outcome: | The proposed framework achieves significant improvements over state-of-the-art models on a large-scale conversational question answering benchmark. |
Internal Chain-of-Thought: Empirical Evidence for Layer‐wise Subtask Scheduling in LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | et al., 2024) show large language models exhibit internal chain-of-thought, meaning they decompose and execute composite tasks layer-by-layer. |
| Approach: | They propose a method to decode hidden states using LogitLens . they also propose 'chain of thought' to decompose and execute composite tasks . |
| Outcome: | The proposed method decodes hidden states and shows consistent execution pattern . it opens avenues for fine-grained, instruction-level activation steering. |