Papers by Yifei Li

47 papers
POSITION BIAS MITIGATES POSITION BIAS: Mitigate Position Bias Through Inter-Position Knowledge Distillation (2025.emnlp-main)

Copied to clipboard

Challenge: Positional bias (PB) manifests as non-uniform sensitivity across contextual locations . previous studies have addressed PB by modifying the underlying architectures or employing extensive contextual awareness training.
Approach: They propose a position-to-position knowledge distillation framework that leverages position-induced disparities to counteract PB.
Outcome: The proposed framework reduces positional bias and improves performance on retrieval and reasoning tasks.
Muse: Towards Reproducible Long-Form Song Generation with Fine-Grained Style Control (2026.findings-acl)

Copied to clipboard

Challenge: Recent commercial systems such as Suno demonstrate strong capabilities in long-form song generation, but academic research remains non-reproducible due to the lack of publicly available training data.
Approach: They propose a system for long-form song generation with fine-grained style conditioning that includes a licensed synthetic dataset and a song generation model, Muse.
Outcome: The proposed system achieves competitive performance on phoneme error rate, text–music style similarity, and audio aesthetic quality while enabling controllable segment-level generation across different musical structures.
VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models (2024.emnlp-main)

Copied to clipboard

Challenge: Recent research has focused on pushing weight-only quantization to extremely low-bit due to numerical representation limitations.
Approach: They propose a vector-based quantization approach that pushes LLMs to extremely low-bit . they propose scalar-based weight quantization that reduces memory requirements and optimizes storage costs .
Outcome: The proposed method reduces model quantization perplexity by 0.01-0.34 on LLaMA-2, 0.38-0.68 on mistral-7B, 4.41-7.34, on llaMA-3 on QA tasks on average.
From Charts to Code: A Hierarchical Benchmark for Multimodal Models (2026.acl-long)

Copied to clipboard

Challenge: Chart2Code is a new benchmark for evaluating the natural language to chart code generation capabilities of large multimodal models.
Approach: They introduce Chart2Code, a new benchmark for evaluating the natural language to chart code generation capabilities of large multimodal models.
Outcome: The proposed benchmark is the first to scale task complexity while capturing diverse scenarios.
Reasoning as Gradient: Scaling MLE Agents Beyond Tree Search (2026.findings-acl)

Copied to clipboard

Challenge: LLM-based agents for machine learning engineering rely on tree search to rank candidates.
Approach: They propose an LLM-based agent that operationalizes gradient-based optimization.
Outcome: The proposed agent achieves a state-of-the-art 35.1% any-medal rate on MLE-Bench with a limited budget on a single GPU.
Alleviating Sparsity of Open Knowledge Graphs with Ternary Contrastive Learning (2022.findings-emnlp)

Copied to clipboard

Challenge: Existing approaches to learning KG triplets ignore ternary propagation patterns and ignore zero-shot, few-shot and synonymity problems.
Approach: They propose a framework for contrastive learning based on ternary propagation patterns among head, relation and tail.
Outcome: Experiments on benchmarks show that TernaryCL is superior to state-of-the-art models.
Chain of Ideas: Revolutionizing Research Via Novel Idea Development with LLM Agents (2025.findings-emnlp)

Copied to clipboard

Challenge: Existing methods for idea generation either trivially prompt LLMs or expose LLM to extensive literature without indicating useful information.
Approach: They propose a chain-of-ideas agent that organizes literature in a chains structure . they propose evaluating idea-generation methods from different perspectives .
Outcome: The proposed agent outperforms existing methods and matches human quality in idea generation.
Extracting Person Names from User Generated Text: Named-Entity Recognition for Combating Human Trafficking (2022.findings-acl)

Copied to clipboard

Challenge: Existing methods for Named-Entity Recognition (NER) on escort ads are not sufficient to extract person names from the text of the ad.
Approach: They propose to use a model to extract person names from escort ads to capture ambiguous names and adapt to adversarial changes in the text.
Outcome: The proposed model shows 19% improvement on average in the F1 classification score compared to previous state-of-the-art in two domain-specific datasets.
Locomo-Plus: Beyond-Factual Cognitive Memory Evaluation Framework for LLM Agents (2026.acl-long)

Copied to clipboard

Challenge: Existing benchmarks and evaluation protocols focus on surface-level factual recall.
Approach: They propose a benchmark for assessing cognitive memory under cue–trigger semantic disconnect.
Outcome: The proposed framework reveals failures not captured by existing benchmarks.
Enhancing Court View Generation with Knowledge Injection and Guidance (2024.lrec-main)

Copied to clipboard

Challenge: Existing methods of natural language generation (NLG) rely on the extensive parameters of pretrained language models (PLMs) but their effectiveness may be compromised by insufficient domain-specific knowledge.
Approach: They propose a knowledge-injected prompt encoder to incorporate domain knowledge during the training stage, thereby reducing computational overhead.
Outcome: The proposed approach outperforms established baselines on real-world data in responsivity of claims and in the ability to transfer domain knowledge.
Contrastive Learning with Generated Representations for Inductive Knowledge Graph Embedding (2023.findings-acl)

Copied to clipboard

Challenge: Existing methods for inductive knowledge Graphs are limited by sparsity and implicit transfer.
Approach: They propose a Contrastive Learning framework with graph guided Variational autoencoder on Meta-KGs to capture and transfer entities.
Outcome: The proposed framework outperforms state-of-the-art methods with extensive experiments.
Making Language Models Better Reasoners with Step-Aware Verifier (2023.acl-long)

Copied to clipboard

Challenge: Large language models have made impressive progress in few-shot learning but still face difficulties in reasoning tasks such as GSM8K.
Approach: They propose a new approach that uses a verifier to filter out incorrect answers based on a weighted voting scheme to improve reasoning ability of language models.
Outcome: The proposed approach improves GSM8K reasoning rate by 17.9% to 58.1%.
Nested Named Entity Recognition as Corpus Aware Holistic Structure Parsing (2022.coling-1)

Copied to clipboard

Challenge: Named entity recognition is a natural language processing task . nested NER is based on a linear structure, but there is no research on applying corpus-level information to NER.
Approach: They propose a holistic structure parsing algorithm to reveal the entire NEs in a sentence . they introduce points-wise mutual information and other frequency features from corpus-aware statistics .
Outcome: The proposed model outperforms existing models on widely-used benchmarks and achieves state-of-the-art.
AttributionBench: How Hard is Automatic Attribution Evaluation? (2024.findings-acl)

Copied to clipboard

Challenge: generative search engines enhance the reliability of large language model responses by providing cited evidence.
Approach: They propose to use a benchmark to evaluate whether a large language model supports the generated responses or not .
Outcome: The proposed benchmark shows that even a fine-tuned GPT-3.5 only achieves around 80% macro-F1 under a binary classification formulation.
AgentThink: A Unified Framework for Tool-Augmented Chain-of-Thought Reasoning in Vision-Language Models for Autonomous Driving (2025.findings-emnlp)

Copied to clipboard

Challenge: Vision-Language Models struggle with hallucinations, inefficient reasoning, and limited real-world validation hinders accurate perception and robust step-by-step reasoning.
Approach: AgentThink integrates Chain-of-Thought reasoning with dynamic, agent-style tool invocation for autonomous driving tasks.
Outcome: Experiments on the DriveLMM-o1 benchmark show AgentThink significantly boosts overall reasoning scores by 53.91% and enhances answer accuracy by 33.54% .
UniLR: Unleashing the Power of LLMs on Multiple Legal Tasks with a Unified Legal Retriever (2025.acl-long)

Copied to clipboard

Challenge: Existing retrieval methods are designed for general domains, struggling with legal knowledge, or tailored for specific legal tasks, unable to handle diverse legal knowledge types.
Approach: They propose a novel retrieval method that integrates specialized knowledge into LLMs.
Outcome: The proposed method can perform multiple legal retrieval tasks for LLMs.
HiFT: A Hierarchical Full Parameter Fine-Tuning Strategy (2024.emnlp-main)

Copied to clipboard

Challenge: Existing approaches to fine-tuning language models use zeroth-order optimizers to conserve GPU memory.
Approach: They propose a full-parameter fine-tuning strategy which updates a subset of parameters at each training step.
Outcome: The proposed approach reduces the amount of gradients and optimizer state parameters residing in GPU memory at the same time, thereby reducing GPU memory usage.
MUR: Momentum Uncertainty guided Reasoning for Large Language Models (2026.acl-long)

Copied to clipboard

Challenge: Existing methods for optimizing reasoning quality are limited by overthinking.
Approach: They propose a method that allocates thinking budgets to critical reasoning steps by tracking and aggregating step-wise uncertainty over time.
Outcome: The proposed method reduces computation by over 45% on average while improving accuracy by 0.33–3.46%.
Unleashing the Power of LLMs in Court View Generation by Stimulating Internal Knowledge and Incorporating External Knowledge (2024.findings-naacl)

Copied to clipboard

Challenge: Large Language Models (LLMs) have made remarkable strides in language generation, but they encounter difficulties in the knowledge-intensive legal domain.
Approach: They propose to decompose court views into different parts, stimulate internal knowledge, and incorporate external information to unleash the power of LLMs in the task.
Outcome: The proposed method generates more accurate and reliable court views on two real-world datasets LAIC2021 and CJO2022.
What if Othello-Playing Language Models Could See? (2025.findings-emnlp)

Copied to clipboard

Challenge: a multi-modal model trained on move sequences and board images is a popular testbed for language models .
Approach: They propose a multi-modal model trained jointly on move sequences and board images.
Outcome: The proposed multi-modal model trains on move sequences and board images.
A Co-Attention Neural Network Model for Emotion Cause Analysis with Emotional Context Awareness (D18-1)

Copied to clipboard

Challenge: Existing methods ignore the contexts around the emotion word which can provide an emotion cause clue.
Approach: They propose a co-attention neural network model for emotion cause analysis with emotional context awareness.
Outcome: The proposed model outperforms the state-of-the-art methods.
Neural Parameter Search for Slimmer Fine-Tuned Models and Better Transfer (2025.acl-long)

Copied to clipboard

Challenge: Foundational models and their checkpoints have advanced deep learning, boosting performance across applications.
Approach: They propose a method for pruning fine-tuned models by calculating differences between them and original model.
Outcome: The proposed method can improve performance across vision, NLP, and multi-modal benchmarks.
CMMLU: Measuring massive multitask language understanding in Chinese (2024.findings-acl)

Copied to clipboard

Challenge: Existing large language models struggle to achieve an accuracy of even 60%, which is the pass mark for Chinese exams.
Approach: They propose to use CMMLU to evaluate Chinese multilingual and Chinese LLMs in a comprehensive benchmark that covers various subjects and settings.
Outcome: The proposed benchmark covers natural sciences, social sciences, engineering, and the humanities and aims to improve on existing models.
Boundary Detection with BERT for Span-level Emotion Cause Analysis (2021.findings-acl)

Copied to clipboard

Challenge: Emotion cause analysis (ECA) is an emerging topic in natural language processing, which aims to identify the reasons behind a given emotion.
Approach: They propose to detect the precise boundaries of text spans conveying accurate emotion causes from the given context by a sequence labeling and position identification problem.
Outcome: The proposed methods outperform existing models on two benchmark datasets on the emotion cause analysis task.
MeepleLM: A Virtual Playtester Simulating Diverse Subjective Experiences (2026.acl-long)

Copied to clipboard

Challenge: Recent advances in large language models have expanded the role of board games as creative co-designers . however, current systems lack the capacity to offer constructive critique grounded in the emergent user experience .
Approach: They propose a large language model that internalizes persona-specific reasoning patterns to accurately simulate the subjective feedback of diverse player archetypes.
Outcome: The proposed model outperforms commercial models in community alignment and critique quality.
Conceptor-Aided Debiasing of Large Language Models (2023.emnlp-main)

Copied to clipboard

Challenge: Pre-trained large language models reflect inherent social biases of their training corpus.
Approach: They propose two methods to identify and remove the bias subspace in pre-trained large language models such as BERT and GPT by applying conceptors to a conceptor NOT operation.
Outcome: The proposed method achieves state-of-the-art (SoTA) debiasing while maintaining LLMs’ performance on the GLUE benchmark.
Unveiling Factual Recall Behaviors of Large Language Models through Knowledge Neurons (2024.emnlp-main)

Copied to clipboard

Challenge: Recent advances in Large Language Models have underscored their exceptional reasoning prowess with natural language understanding across a broad spectrum of tasks.
Approach: They examine whether Large Language Models actively recall or retrieve their internal repositories of factual knowledge when faced with reasoning tasks.
Outcome: The proposed model improves reasoning performance while suppressing it leads to notable degradation.
Can MLLMs Understand the Deep Implication Behind Chinese Images? (2025.acl-long)

Copied to clipboard

Challenge: MLLMs perform poorly on traditional culture images, indicating limitations in understanding high-level semantics and lacking a deep knowledge base of Chinese traditional culture.
Approach: They propose to use Chinese images to assess MLLMs' higher-order perception and understanding of Chinese visual content.
Outcome: The proposed model incorporates images that represent Chinese traditional culture, such as famous Chinese traditional paintings, to ensure the authenticity of the Chinese context.
Sequential-NIAH: A Needle-In-A-Haystack Benchmark for Extracting Sequential Needles from Long Contexts (2025.emnlp-main)

Copied to clipboard

Challenge: Recent models have extended Corresponding Author. context lengths to millions of tokens while maintaining reasoning and comprehension capabilities.
Approach: They propose a benchmark to evaluate the ability of large language models to extract sequential information items from long contexts.
Outcome: The proposed model achieves maximum accuracy of 63.50% on six well-known LLMs.
To See a World in a Spark of Neuron: Disentangling Multi-Task Interference for Training-Free Model Merging (2025.emnlp-main)

Copied to clipboard

Challenge: Existing approaches to model merging ignore the fundamental roles of neurons, connectivity and activation.
Approach: They propose a framework that relies on neuronal mechanisms to mitigate task interference . they decomposed task-specific representations into two complementary subspaces . their results offer new insights into mitigating task interference and improving knowledge fusion .
Outcome: The proposed framework reduces task interference within neurons and improves knowledge fusion.
FoodieQA: A Multimodal Dataset for Fine-Grained Understanding of Chinese Food Culture (2024.emnlp-main)

Copied to clipboard

Challenge: FoodieQA is a manually curated, fine-grained image-text dataset capturing the intricate features of food cultures across various regions in China.
Approach: They evaluate vision–language Models and large language models on unseen food images and corresponding questions.
Outcome: The proposed dataset evaluates vision–language Models and large language models on unseen food images and corresponding questions.
CogDual: Enhancing Dual Cognition of LLMs via Reinforcement Learning with Implicit Rule-Based Rewards (2025.emnlp-main)

Copied to clipboard

Challenge: Existing approaches to role-playing language models rely on prompt engineering or supervised fine-tuning to emulate character behaviors but neglect the underlying cognitive mechanisms driving these behaviors.
Approach: They propose a novel RPLA adopting a cognize-then-respond reasoning paradigm that leverages dual cognition for more contextually grounded and psychologically coherent responses.
Outcome: The proposed RPLA outperforms baselines and generalizes effectively across diverse role-playing tasks.
LoRASC: Expressive and Generalizable Low-rank Adaptation for Large Models via Slow Cascaded Learning (2024.findings-emnlp)

Copied to clipboard

Challenge: Existing low-rank adaptations have limited expressiveness, a tendency to overfit, and sensitivity to hyperparameter settings.
Approach: They propose a technique to enhance LoRA’s expressiveness and generalization capabilities while preserving its training efficiency.
Outcome: The proposed method outperforms baselines, mitigates overfitting, enhances model stability, and improves OOD robustness.
Summarize-Exemplify-Reflect: Data-driven Insight Distillation Empowers LLMs for Few-shot Tabular Classification (2025.findings-emnlp)

Copied to clipboard

Challenge: Recent studies show the promise of large language models for few-shot tabular classification but highlight challenges due to the variability in structured data.
Approach: They propose a framework that distills data into actionable insights to enable robust and effective classification by large language models.
Outcome: The proposed framework integrates rule summarization, strategic exemplification, and insight reflection through deep collaboration between LLMs and data modeling techniques.
Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations (2024.acl-long)

Copied to clipboard

Challenge: Existing methods for process-oriented math reward models rely on manual annotation.
Approach: They propose a process-oriented math process reward model called Math-shepherd which assigns a reward score to each step of math problem solutions.
Outcome: The proposed model breaks the bottleneck of manual supervision in two scenarios.
AutoSDT: Scaling Data-Driven Discovery Tasks Toward Open Co-Scientists (2025.emnlp-main)

Copied to clipboard

Challenge: AutoSDT-5K is the only automatically collected and the largest open dataset for data-driven scientific discovery.
Approach: They propose an automatic pipeline that collects high-quality coding tasks in real-world data-driven discovery workflows.
Outcome: The proposed pipeline synthesizes accurate tasks and tasks from a dataset of 5,404 tasks covering four scientific disciplines and 756 Python packages.
Semantic-Eval : A Semantic Comprehension Evaluation Framework for Large Language Models Generation without Training (2025.acl-long)

Copied to clipboard

Challenge: Large language models (LLMs) have emerged as key drivers of progress in the field of natural language processing.
Approach: They propose a framework that assesses LLM-generated text based on semantic understanding.
Outcome: The proposed framework surpasses traditional evaluation metrics and lags behind GPT-4.
A Critical Analysis of Document Out-of-Distribution Detection (2023.findings-emnlp)

Copied to clipboard

Challenge: Existing document understanding models focus on single-modal inputs such as images or texts.
Approach: They propose to use a spatial-aware adapter to adapt transformer-based language models to document domain to exploit multi-modal information.
Outcome: The proposed model significantly improves the OOD detection performance compared to using a standard language model and to competitive baselines.
Uncertainty Unveiled: Can Exposure to More In-context Examples Mitigate Uncertainty for Large Language Models? (2025.findings-acl)

Copied to clipboard

Challenge: Recent advances in handling long sequences have unlocked new possibilities for long-context in-contact learning (ICL).
Approach: They investigate how increased examples influence predictive uncertainty . they quantify uncertainty across different “shot” configurations and focus on EU .
Outcome: The proposed model reduces uncertainty in simple and complex tasks by injecting task-specific knowledge.
TableLlama: Towards Open Large Generalist Models for Tables (2024.naacl-long)

Copied to clipboard

Challenge: Existing methods for interpreting, augmenting, and querying semi-structured tables require pretraining on tables or special model architecture design.
Approach: They construct a dataset with a variety of tables and tasks for instruction tuning and evaluating LLMs.
Outcome: The proposed model achieves comparable or better performance on 7 out of 8 in-domain tasks compared with the base model on 6 out-of-domain datasets.
Soul-Mix: Enhancing Multimodal Machine Translation with Manifold Mixup (2024.acl-long)

Copied to clipboard

Challenge: Multimodal machine translation (MMT) aims to improve the performance of machine translation with the help of visual information.
Approach: They propose a multimodal machine translation mixup method that integrates visual information into conventional text-only neural machine translation systems.
Outcome: The proposed method outperforms existing models on a multi-directional dataset with fewer parameters and achieves new state-of-the-art performance.
Knowledge Fusion By Evolving Weights of Language Models (2024.findings-acl)

Copied to clipboard

Challenge: Experimental results on mainstream language models show that Evolver outperforms previous state-of-the-art models by large margins due to the high training costs of large language models.
Approach: They propose a method to integrate multiple models from diverse training scenarios into a unified model.
Outcome: The proposed method outperforms state-of-the-art models on mainstream language models by large margins.
From Scores to Preferences: Redefining Evaluation Paradigm for Speech Quality Reward Modeling (2026.findings-acl)

Copied to clipboard

Challenge: Experimental results show that the MOS-aware GRM significantly improves fine-grained speech quality discrimination.
Approach: They propose a MOS-aware reward model that incorporates MOS gap into reward function during reinforcement learning.
Outcome: The proposed model significantly improves fine-grained speech quality discrimination.
OneRec-Think: In-Text Reasoning for Generative Recommendation (2026.acl-long)

Copied to clipboard

Challenge: Existing generative models lack the capacity for explicit and controllable reasoning, a key advantage of LLMs.
Approach: They propose a framework that integrates dialogue, reasoning, and personalized recommendation.
Outcome: Experiments across public benchmarks show state-of-the-art performance.
Unsolvable Problem Detection: Robust Understanding Evaluation for Large Multimodal Models (2025.acl-long)

Copied to clipboard

Challenge: Multiple-choice question answering (MCQA) is widely used to assess the understanding capability of Large Multimodal Models (LMMs).
Approach: They propose a task to evaluate the robust understanding capability of Large Multimodal Models (LMMs) they introduce a benchmark to assess performance across various ability dimensions .
Outcome: The proposed model can withhold answers when encountering unsolvable problems of MCQA, proving it understands the answer.
Answer-Supervised Question Reformulation for Enhancing Conversational Machine Comprehension (D19-58)

Copied to clipboard

Challenge: Existing question reformulation models are based on supervised question labels without considering feedback information from answers.
Approach: They propose a question reformulation model that integrates conversational history information with reinforcement learning.
Outcome: The proposed model is more effective in conversational machine comprehension with reinforcement learning.
Aspect Sentiment Quad Prediction as Paraphrase Generation (2021.emnlp-main)

Copied to clipboard

Challenge: Existing studies focus on predicting the four elements in one shot, instead of predicting them all.
Approach: They propose a task to jointly detect all sentiment elements in quads for a given opinionated sentence.
Outcome: The proposed method can generate the semantics of the sentiment elements in the natural language form.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations