Papers by Tingyu Li
CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generation (2024.acl-long)
Copied to clipboard
Weixiang Yan, Haitian Liu, Yunkun Wang, Yunzhe Li, Qian Chen, Wen Wang, Tingyu Lin, Weishan Zhao, Li Zhu, Hari Sundaram, Shuiguang Deng
| Challenge: | Existing benchmarks for evaluating the code understanding and generation capacities of Large Language Models are insufficient . existing benchmarks focus on a narrow range of popular programming languages and specific tasks . |
| Approach: | They propose an execution-based, multilingual, multitask evaluation benchmark for LLMs . they evaluate coding performance from three dimensions: length, difficulty, efficiency . |
| Outcome: | The proposed benchmark covers 43 programming languages and eight coding tasks. |
Mitigating Negative Interference in Multilingual Knowledge Editing through Null-Space Constraints (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing monolingual knowledge editing methods are expensive and require multiple models to maintain factual consistency. |
| Approach: | They propose a null-space constrained framework to precisely isolate language-specific knowledge updates that can be mapped onto other languages’ subspaces. |
| Outcome: | The proposed framework can project parameter updates for each language onto the orthogonal complement of other languages’ subspaces while preserving multilingual generalization capabilities. |
JiraiBench: A Bilingual Benchmark for Evaluating Large Language Models’ Detection of Human risky health behavior Content in Jirai Community (2026.eacl-long)
Copied to clipboard
Yunze Xiao, Tingyu He, Lionel Z. Wang, Yiming Ma, Xingyu Song, Xiaohang Xu, Mona T. Diab, Irene Li, Ka Chung Ng
| Challenge: | a cross-lingual dataset captures a transnational cultural phenomenon . risky health behaviors (RHB) are often linked to complex mental health conditions . |
| Approach: | They present the first cross-lingual dataset that captures a transnational cultural phenomenon . their dataset of more than 15,000 annotated social media posts forms the core of JiraiBench . |
| Outcome: | The study shows that cultural context can be more influential than linguistic similarity . the study also shows that the Japanese prompts better handle Chinese content . |
Leveraging Dual Process Theory in Language Agent Framework for Real-time Simultaneous Human-AI Collaboration (2025.acl-long)
Copied to clipboard
Shao Zhang, Xihuai Wang, Wenhao Zhang, Chaoran Li, Junru Song, Tingyu Li, Lin Qiu, Xuezhi Cao, Xunliang Cai, Wen Yao, Weinan Zhang, Xinbing Wang, Ying Wen
| Challenge: | Large language models (LLMs) excel in turn-by-turn human-AI collaboration but struggle with simultaneous tasks requiring real-time interaction. |
| Approach: | They propose a language agent framework that integrates *System 1* and *System 2* for efficient real-time simultaneous human-AI collaboration. |
| Outcome: | The proposed framework improves on existing LLM-based agents and human collaborators by integrating Theory of Mind and asynchronous reflection to infer human intentions and perform reasoning-based autonomous decisions. |
Rethinking Composed Image Retrieval Evaluation: A Fine-Grained Benchmark from Image Editing (2026.acl-long)
Copied to clipboard
Tingyu Song, Yanzhao Zhang, Mingxin Li, Zhuoning Guo, Dingkun Long, Pengjun Xie, Siyue Zhang, Yilun Zhao, Shu Wu
| Challenge: | Composed Image Retrieval (CIR) is a complex task in multimodal understanding . current CIR benchmarks lack a robust evaluation pipeline and limited query categories . |
| Approach: | They construct a fine-grained CIR benchmark that allows for precise control over modification types and content. |
| Outcome: | The proposed benchmark covers 5,000 high-quality queries structured across five main categories and fifteen subcategories. |
When Inverse Data Outperforms: Exploring the Pitfalls of Mixed Data in Multi-Stage Fine-Tuning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for o1-level performance focus on unidirectional supervised fine-tuning (SFT), overlooking the intricate interplay between diverse reasoning patterns. |
| Approach: | They construct a reverse reasoning dataset and examine how it is supervised . they find that naively mixing forward and reverse data during SFT weakens the directional distinction . |
| Outcome: | The proposed model improves accuracy by 1.6%–6.8% over a standard model. |
Retrieval Augmented Instruction Tuning for Open NER with Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies have focused on integrating large language models (LLMs) with information extraction (IE) however, the best approach to incorporate information with LLMs for IE remains an open question. |
| Approach: | They propose to use a Chinese IT dataset to perform RA-IT for IE . they use semantically similar examples from the training dataset as the context . |
| Outcome: | The proposed approach is evaluated in English and Chinese scenarios. |
DGPO: Beyond Pairwise Preferences with Directional Consistent Groupwise Optimization (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for directional consistency alignment of large language models are limited . a recent study suggests reverse supervision as a complement to forward reasoning . |
| Approach: | They propose a framework that aggregates supervision signals at the group level and explicitly models direction-aware alignment through multi-candidate comparisons. |
| Outcome: | The proposed framework achieves 3.2% accuracy improvement across five benchmarks and multiple datasets. |
Large Language Model Evaluation via Matrix Nuclear-Norm (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) are computationally intensive due to their O(n3) time complexity with Singular Value Decomposition (SVD). |
| Approach: | They propose a metric to quantify the data compression proficiency of large language models and a convex approximation of matrix rank to capture both predictive discriminability and diversity. |
| Outcome: | The proposed model achieves speeds 8 to 24 times faster than Matrix Entropy for the CEREBRAS-GPT model as models increase from 111M to 6.7B . |
Specialization through Collaboration: Understanding Expert Interaction in Mixture-of-Expert Large Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | Mixture-of-Experts (MoE) based large language models are popular for multitasking . however, whether each expert can specialize to a task remains unclear . |
| Approach: | They propose to use a dictionary learning approach to analyze expert collaboration mechanisms in MoE LLMs. |
| Outcome: | The proposed model outperforms existing methods by 2.5% while enabling 50% expert reduction. |
Empirical Study of Zero-Shot NER with ChatGPT (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been a key component of natural language processing (NLP) . |
| Approach: | They propose to decompose the NER task into simpler subproblems by labels and propose a syntactic augmentation strategy to stimulate model's intermediate thinking. |
| Outcome: | The proposed methods achieve remarkable improvements for zero-shot NER across seven benchmarks, including Chinese and English datasets. |
Self-Improving for Zero-Shot Named Entity Recognition with Large Language Models (2024.naacl-short)
Copied to clipboard
| Challenge: | Existing studies exploring the performance of large language models on named entity recognition tasks have focused on training task-specific LLMs for NER. |
| Approach: | They propose a training-free self-improving framework that utilizes an unlabeled corpus to stimulate the self-learning ability of LLMs. |
| Outcome: | The proposed framework improves performance on the named entity recognition task by using an unlabeled corpus. |
A Class-Rebalancing Self-Training Framework for Distantly-Supervised Named Entity Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Distant supervision reduces the reliance on human annotation in named entity recognition tasks. |
| Approach: | They propose a class-rebalancing self-training framework for improving distantly-supervised named entity recognition by using a flexible threshold and a hybrid pseudo label. |
| Outcome: | The proposed model achieves state-of-the-art on five flat and two nested datasets and compares with other methods on the same dataset. |