Papers by Jiacheng Li

37 papers
RoadMapper: A Multi-Agent System for Roadmap Generation of Solving Complex Research Problems (2026.findings-acl)

Copied to clipboard

Challenge: Existing tools to generate structured content for research tasks are limited in their ability to generate high-quality roadmaps.
Approach: They propose a benchmark to evaluate the ability of large language models (LLMs) to generate high-quality roadmaps for solving complex research problems.
Outcome: The proposed system can improve LLMs’ ability for roadmap generation while saving 84% of the time required by human experts.
An Empirical Study of Position Bias in Modern Information Retrieval (2025.findings-emnlp)

Copied to clipboard

Challenge: a new evaluation framework is used to assess the extent and impact of position bias in information retrieval.
Approach: They introduce a position-aware retrieval benchmark and a diagnostic metric to quantify position bias . they compare models with BM25, dense embedding models, ColBERT-style late-interaction models .
Outcome: The proposed framework evaluates retrieval models for position bias from a worst-case perspective.
Training Dynamics for Text Summarization Models (2022.findings-acl)

Copied to clipboard

Challenge: Pre-trained language models have shown impressive results when fine-tuned on large summarization datasets.
Approach: They analyze the training dynamics for generation models, focusing on summarization . they find that a propensity to copy the input is learned early in the training process .
Outcome: The proposed model learns at different stages of fine-tuning, the authors show . they show that factual errors are learnt in later stages, but not at high-loss tokens .
TritonBench: Benchmarking Large Language Model Capabilities for Generating Triton Operators (2025.findings-acl)

Copied to clipboard

Challenge: Triton is a high-level Python-like programming language for building efficient GPU kernels.
Approach: They propose a TritonBench benchmark that provides a comprehensive evaluation of Tritonic operators on widely deployed GPUs.
Outcome: The proposed benchmarks show that current LLMs struggle to generate efficient Triton operators on widely deployed GPUs aligned with industry applications.
Open-world Semi-supervised Generalized Relation Discovery Aligned in a Real-world Setting (2023.emnlp-main)

Copied to clipboard

Challenge: Existing approaches to open-world relation extraction assume that all instances of unlabeled data belong to novel classes.
Approach: They propose a method that classifies relations from known and novel classes within unlabeled data.
Outcome: The proposed method outperforms existing methods on Open-world RE benchmarks.
BTC-LLM: Efficient Sub-1-Bit LLM Quantization via Learnable Transformation and Binary Codebook (2026.acl-long)

Copied to clipboard

Challenge: Recent sparsity-aware binarization approaches can achieve sub-1-bit compression, but they face performance degradation, mask-management overhead, and limited hardware compatibility.
Approach: They propose a binary quantization framework that leverages binary pattern clustering and weight transformation to overcome performance degradation and mask-management overhead.
Outcome: The proposed framework achieves state-of-the-art compression (1.11–0.7 bits) it maintains high performance with only a 3.1% accuracy drop in zero-shot benchmarks while delivering a 1.6 speedup over FP16.
SeNsER: Learning Cross-Building Sensor Metadata Tagger (2020.findings-emnlp)

Copied to clipboard

Challenge: Sensor metadata tagging is a key component of smart building applications.
Approach: They propose a framework that learns a sensor metadata tagger for a new building based on its raw metadata and some existing fully annotated building.
Outcome: The proposed framework learns a sensor metadata tagger for a new building based on its raw metadata and some existing fully annotated building.
Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoders (2026.acl-long)

Copied to clipboard

Challenge: Recent advances in large language models have enabled their use as semantic encoders for recommendation, but their roles and behaviors in this setting are still not well understood.
Approach: They propose a benchmark to evaluate large language models as semantic encoders in recommendation scenarios.
Outcome: The proposed benchmark shows that ranking of 11 leading LLMs is low compared to MTEB, highlighting the unique challenges of semantic encoding in recommendation.
FinanceReasoning: Benchmarking Financial Numerical Reasoning More Credible, Comprehensive and Challenging (2025.acl-long)

Copied to clipboard

Challenge: Compared to existing benchmarks, FinanceReasoning provides three key advancements: (1) credibility; (2) comprehensiveness; (3) numerical precision; (4) complexity; (5) complexity; and (6) complexity.
Approach: They propose a benchmark to evaluate the reasoning capabilities of large reasoning models (LRMs) in financial numerical reasoning problems.
Outcome: The proposed benchmark exceeds existing benchmarks in 67.8% of financial concepts and formulas and is credible, comprehensive, and challenging.
STORM-BORN: A Challenging Mathematical Derivations Dataset Curated via a Human-in-the-Loop Multi-Agent Framework (2025.findings-acl)

Copied to clipboard

Challenge: Existing datasets suffer from outdated and insufficient challenging content, neglecting human-like reasoning, and limited reliability due to single-LLM generation.
Approach: They propose a human-in-the-loop, multi-agent data generation framework that integrates reasoning-dense filters, multiagent collaboration, and human mathematicians’ evaluations to ensure the reliability and quality of the dataset.
Outcome: The proposed framework improves accuracy and quality of the 2,000-synthesized datasets by integrating reasoning-dense filters, multi-agent collaboration, and human mathematicians’ evaluations.
Bit-by-Bit: Progressive QAT Strategy with Outlier Channel Splitting for Stable Low-Bit LLMs (2026.acl-long)

Copied to clipboard

Challenge: Existing approaches to training LLMs at ultra-low precisions suffer from convergence instability and substantial training costs.
Approach: They propose a progressive QAT framework with outlier channel splitting to address these issues . they use nested structure of integer quantization grids to enable a "train once, deploy any precision" paradigm .
Outcome: The proposed framework outperforms baselines on both Llama2/3 and W2A16, with an 11 speedup over BF16.
LLMSurgeon: Diagnosing Data Mixture of Large Language Models (2026.acl-long)

Copied to clipboard

Challenge: a lack of transparency in large language models makes auditing their "digital DNA" difficult.
Approach: They propose a framework that casts DMS as an inverse problem under label-shift assumption . they propose LLMScan, a recipe-verifiable evaluation suite built from open-source LLMs .
Outcome: The proposed framework casts DMS as an inverse problem under label-shift assumption . compared with existing frameworks, it recovers domain mixtures with high fidelity .
SCCS: Semantics-Consistent Cross-domain Summarization via Optimal Transport Alignment (2023.findings-acl)

Copied to clipboard

Challenge: Existing methods for multimodal summarization ignore the structure and semantics of the whole video and article.
Approach: They propose a semantic-consistent cross-domain summarization model that extracts features from video and article and uses fusion methods to select representative one.
Outcome: The proposed model produces high-quality multimodal summaries on three MSMO datasets.
Understanding the Language Model to Solve the Symbolic Multi-Step Reasoning Problem from the Perspective of Buffer Mechanism (2025.findings-emnlp)

Copied to clipboard

Challenge: Large language models struggle with complex reasoning tasks, such as mathematical problem-solving.
Approach: They constructed a symbolic multi-step reasoning task to investigate the information propagation mechanisms in Transformer models when solving the task through direct answering and Chain-of-Thought (CoT) reasoning.
Outcome: The proposed algorithm improves on 7 multi-step reasoning datasets, while introducing only 132 trainable parameters.
Data Augmentation for Text Generation Without Any Augmented Data (2021.acl-long)

Copied to clipboard

Challenge: Existing methods for data augmentation need to define or choose proper data mapping functions to create augmented samples.
Approach: They propose to use data mapping functions to augment text samples without using specific mapping functions.
Outcome: The proposed approach can approximate or even surpass popular data augmentation methods on two text generation tasks with a convergence rate guarantee.
How Do LLMs Acquire New Knowledge? A Knowledge Circuits Perspective on Continual Pre-Training (2025.findings-acl)

Copied to clipboard

Challenge: Large Language Models (LLMs) have exceptional capabilities in knowledge-intensive tasks . however, they struggle with knowledge updates due to dynamic nature of world knowledge .
Approach: They propose to identify computational subgraphs that facilitate knowledge storage and processing . they also identify a phase shift from formation to optimization in LLMs .
Outcome: The proposed model can capture factual knowledge from pre-training corpus and encapsulate it as extensive parametric knowledge.
MuCGEC: a Multi-Reference Multi-Source Evaluation Dataset for Chinese Grammatical Error Correction (2022.naacl-main)

Copied to clipboard

Challenge: Using a multi-reference multi-source evaluation dataset, Chinese grammatical error correction (CGEC) is relatively scarce.
Approach: They propose a multi-reference multi-source evaluation dataset for Chinese grammar error correction . the dataset contains 7,063 sentences written by Chinese-as-a-Second-Language learners .
Outcome: The proposed dataset can be used to evaluate Chinese grammar errors in Chinese.
Text-to-Table: A New Way of Information Extraction (2022.acl-long)

Copied to clipboard

Challenge: Existing methods for information extraction are not well understood . text-to-table is a problem that aims to extract information from text data .
Approach: They propose a new problem setting of information extraction, called text-to-table . they formalize text- to-table as a sequence-tosequence problem .
Outcome: The proposed method outperforms existing methods on text-to-table tasks.
Beyond Benchmarks: A Capability-Based Maturity Model for Systematic AI Integration in Hospitals (2026.findings-acl)

Copied to clipboard

Challenge: Current Large Language Models (LLMs) excel in standardized tests focused on medical knowledge recall, but not in real-world healthcare scenarios.
Approach: They propose a "capability-based hospital AI Maturity Model" framework that categorizes capabilities into distinct maturity levels . medical artificial intelligence is currently at a critical transition stage from technical verification to deep clinical integration .
Outcome: The proposed model provides a clear, stepwise evolutionary path for hospitals from foundational infrastructure construction to ubiquitous intelligence.
QaRL: Rollout-Aligned Quantization-Aware RL for Fast and Stable Training under Training–Inference Mismatch (2026.findings-acl)

Copied to clipboard

Challenge: Recent work has shown that reinforcement learning with simple rule-based reward functions (RLVR) can induce emergent reasoning behaviors and yield gains in challenging domains such as math problem solving.
Approach: They propose a rollout-alignment-quantization-aware RL which aligns training-side forward with the quantized rollout to minimize mismatch.
Outcome: The proposed approach outperforms quantized-rollout training by +5.5 on Qwen3-30B-A3B MoE for math problems while maintaining low-bit throughput.
ZeroGen: Efficient Zero-shot Learning via Dataset Generation (2022.emnlp-main)

Copied to clipboard

Challenge: Existing approaches to generate training data with pre-trained language models have been found effective in various scenarios.
Approach: They propose an unsupervised zero-shot learning method that generates a dataset from scratch and trains a tiny task model under supervision of the synthesized dataset.
Outcome: The proposed method is annotated-free and efficient, but can provide useful insights from the perspective of data-free model-agnostic knowledge distillation and unreferenced text generation evaluation.
WISCA: A Lightweight Model Transition Method to Improve LLM Training via Weight Scaling (2026.findings-acl)

Copied to clipboard

Challenge: Recent advances in training optimization for Transformer-based large language models lack systematic optimization of weight patterns during training.
Approach: They propose a Weight Scaling method that rescales weights while preserving model outputs to improve model training efficiency and model quality.
Outcome: The proposed method significantly improves convergence quality and loss reduction in LLMs with Grouped Query Attention architectures and LoRA fine-tuning tasks.
Generating Data for Symbolic Language with Large Language Models (2023.emnlp-main)

Copied to clipboard

Challenge: Large language models (LLMs) bring performance and complexity, but they incur a large computational cost in practice.
Approach: They propose a task-based model which uses large language models to generate symbolic language data by an informative prompt and agreement-based verifier.
Outcome: The proposed model can generate symbolic language data with a few human demonstrations and saves a considerable amount of inference effort.
AEGIS: A Holistic Benchmark for Evaluating Forensic Analysis of AI-Generated Academic Images (2026.acl-long)

Copied to clipboard

Challenge: AEGIS examines whether current models can effectively audit AI-generated images in academic papers.
Approach: They propose a holistic benchmark for forensic analysis of AI-Generated academic ImageS that reveals limitations in academic image forensics.
Outcome: AEGIS compared with existing benchmarks on seven academic categories and features key advances in forensic analysis.
UCTopic: Unsupervised Contrastive Learning for Phrase Representations and Topic Mining (2022.acl-long)

Copied to clipboard

Challenge: Existing phrase representation learning methods combine unigram representations in a context-free manner or rely on extensive annotations to learn context-aware knowledge.
Approach: They propose a novel unsupervised contrastive learning framework for context-aware phrase representations and topic mining.
Outcome: The proposed framework outperforms the state-of-the-art phrase representation model by 38.2% NMI on four entity clustering tasks.
Decoding Scientific Experimental Images: The SPUR Benchmark for Perception, Understanding, and Reasoning (2026.acl-long)

Copied to clipboard

Challenge: Xu and Peng, 2025) . . SPUR is a comprehensive benchmark for scientific experimental image perception, understanding, and reasoning, comprising 4,264 question-answering (QA) pairs derived from 1,084 expert-curated images.
Approach: They propose to use 4,264 question-answering (QA) pairs derived from 1,084 expert-curated images to evaluate the visual perception of multimodal large language models (MLLMs) . they also propose to utilize cross-panel relation understanding to evaluate MLLM’s ability to decipher intricate cross-panel relations.
Outcome: The proposed model is based on 4,264 question-answering pairs derived from 1,084 expert-curated images.
LoRAExit: Empowering Dynamic Modulation of LLMs in Resource-limited Settings using Low-rank Adapters (2024.findings-emnlp)

Copied to clipboard

Challenge: Large Language Models (LLMs) have exhibited remarkable performance across various natural language processing tasks, but deployment on resource-limited settings remains a challenge.
Approach: They propose a dynamic inference architecture that leverages low-rank adaptors for efficient deployment of LLMs.
Outcome: The proposed architecture significantly improves performance when deployed on resource-limited settings.
Weakly Supervised Named Entity Tagging with Learnable Logical Rules (2021.acl-long)

Copied to clipboard

Challenge: Existing methods for building entity tagging systems use weak supervision . previous methods focus on disambiguating entity types based on contexts and expert-provided rules .
Approach: They propose a method that bootstraps high-quality logical rules to train a neural tagger in a fully automated manner.
Outcome: The proposed method outperforms weakly supervised methods on three datasets . it rivals state-of-the-art supervised method with lexicon of over 2,000 terms .
Fine-grained Contrastive Learning for Relation Extraction (2022.emnlp-main)

Copied to clipboard

Challenge: Existing methods assume all silver labels are accurate and treat them equally, but distant supervision is noisy–some silver labels more reliable than others.
Approach: They propose a noise-aware contrastive learning approach that leverages fine-grained information about which silver labels are and are not noisy to improve the quality of learned relationship representations.
Outcome: The proposed approach improves relation extraction performance over state-of-the-art methods on several RE benchmarks.
Justifying Recommendations using Distantly-Labeled Reviews and Fine-Grained Aspects (D19-1)

Copied to clipboard

Challenge: Existing approaches to generating reviews struggle to generate justifications that are relevant to users’ decision-making process.
Approach: They propose an ‘extractive’ approach to identify review segments which justify users’ intentions and use it to distantly label massive review corpora and construct large-scale personalized recommendation justification datasets.
Outcome: The proposed model can generate convincing and diverse justifications from massive review corpora and distantly label massive review data.
An Empirical Study of LLM Reasoning Ability Under Strict Output Length Constraint (2025.emnlp-main)

Copied to clipboard

Challenge: Large Language Models (LLMs) are a powerful tool for test-time scaling, but they are often used under time constraints.
Approach: They propose to use LLMs to make models think before answering questions . they also use self-correction and best-of-N decoding to encourage deeper thinking .
Outcome: The proposed models are able to achieve higher inference accuracy with extra inference computation under time constraints.
Uncertainty-Aware Label Refinement for Sequence Labeling (2020.emnlp-main)

Copied to clipboard

Challenge: Conditional random fields (CRF) for label decoding have been a problem for many tasks.
Approach: They propose a two-stage label decoding framework that model long-term label dependencies while being much more computationally efficient.
Outcome: The proposed method outperforms the CRF-based methods and greatly accelerates the inference process.
TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing (2021.acl-demo)

Copied to clipboard

Challenge: Existing approaches to textual robustness evaluation focus on slightly modifying the input data, which maintains the original meaning and results in a different prediction.
Approach: They propose a multilingual robustness evaluation toolkit for NLP that integrates universal text transformations, task-specific transformations and adversarial attack.
Outcome: The toolkit includes universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analyses.
Can Brain Signals Reveal Inner Alignment with Human Languages? (2023.findings-emnlp)

Copied to clipboard

Challenge: Brain Signals, such as Electroencephalography, and human languages have been explored independently for many downstream tasks, however, the connection between them has not been well explored.
Approach: They introduce a multimodal transformer alignment model to observe coordinated representations between EEG and language.
Outcome: The proposed method achieved an F1-score improvement of 1.7% on ZuCo and 9.3% on Zuco datasets for sentiment analysis, and 7.4% on ZuCO for relation detection.
Can LLMs See Without Pixels? Benchmarking Spatial Intelligence from Textual Descriptions (2026.findings-acl)

Copied to clipboard

Challenge: Existing advances in Spatial Intelligence rely on vision-Language Models . however, a critical question remains: does spatial understanding originate from visual encoders?
Approach: They propose to evaluate the SI performance of Large Language Models without pixel-level input.
Outcome: The proposed benchmark challenges large language models to perform symbolic reasoning rather than visual pattern matching.
How Numerical Precision Affects Arithmetical Reasoning Capabilities of LLMs (2025.findings-acl)

Copied to clipboard

Challenge: Despite the success of transformer-based large language models, understanding and enhancing their mathematical capabilities remains a significant challenge.
Approach: They propose to use numerical precision as a key factor that influences LLMs' effectiveness in arithmetical tasks to determine their effectiveness.
Outcome: The proposed models perform better in arithmetic tasks than transformer-based models with standard numerical precision.
Enhancing Agentic Textual Graph Retrieval with Synthetic Stepwise Supervision (2026.acl-long)

Copied to clipboard

Challenge: Existing methods for integrating textual graphs with LLMs are limited by symbolic inference and high annotation costs.
Approach: They propose a textual graph reasoning framework that integrates textual diagrams with large language models.
Outcome: The proposed approach achieves 15.6% accuracy and 17.2% in F1 score on three common datasets.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations