Papers by Zheng Ge

36 papers
E2-LLM: Efficient and Extreme Length Extension of Large Language Models (2024.findings-acl)

Copied to clipboard

Challenge: Existing techniques for extending context capabilities in LLMs require additional training procedures and access to datasets with long context (e.g., sequences of 32K tokens).
Approach: They propose a solution to extend context capabilities in Large Language Models by training a single process over a sequence of 4K tokens.
Outcome: The proposed solution significantly reduces the cost of continual-pretraining or fine-tuning over short sequences and improves robustness to diverse relative positions.
Word Segmentation by Separation Inference for East Asian Languages (2022.findings-acl)

Copied to clipboard

Challenge: Chinese Word Segmentation (CWS) is a sequence labeling task that divides sentences into words . despite diverse tagging schemas, they all carry implicit position information.
Approach: They propose to model the separation state of every two consecutive characters by tagging them as two tags.
Outcome: The proposed framework outperforms state-of-the-art on Japanese and Korean Word Segmentation datasets.
Unified Thinker: A General Reasoning Core for Image Generation (2026.acl-long)

Copied to clipboard

Challenge: generative models struggle with logic-intensive instruction following, exposing a persistent reasoning–execution gap.
Approach: They propose a task-agnostic reasoning architecture for general image generation . they propose pixel-level feedback to ground the Thinker's policy in pixel feedback .
Outcome: The proposed system significantly improves image reasoning and generation quality.
IntentCoding: Amplifying User Intent in Code Generation (2026.findings-acl)

Copied to clipboard

Challenge: IntentCoding captures the influence of user intent by masking out the intent, and integrates seamlessly with existing decoding procedures.
Approach: They propose a decoding strategy that captures the influence of user intent by masking out the intent and applies a multi-strength ensemble mechanism to amplify the effect of user intention during generation.
Outcome: The proposed model significantly improves both constraint satisfaction and functional correctness compared to greedy decoding approaches.
KARPA: A Training-free Method of Adapting Knowledge Graph as References for Large Language Model’s Reasoning Path Aggregation (2025.findings-acl)

Copied to clipboard

Challenge: Existing methods for large language models (LLMs) are limited by step-by-step decision-making on KGs, or require fine-tuning or pre-training on specific KG.
Approach: They propose a framework that harnesses the global planning abilities of large language models (LLMs) for efficient and accurate KG reasoning.
Outcome: Extensive experiments show that the proposed framework achieves state-of-the-art performance in KGQA tasks, delivering both high efficiency and accuracy.
DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories (2024.findings-acl)

Copied to clipboard

Challenge: Existing benchmarks are poorly aligned with real-world code repositories and are insufficient to evaluate the coding abilities of Large Language Models (LLMs).
Approach: They propose a repository-level benchmark named DevEval to evaluate LLMs' coding abilities in real-world code repositories.
Outcome: The proposed benchmarks show that the LLMs perform better in real-world code repositories than existing benchmarks.
PaCoRe: Learning to Scale Test-Time Compute with Parallel Coordinated Reasoning (2026.acl-long)

Copied to clipboard

Challenge: Parallel Coordinated Reasoning (PaCoRe) overcomes a central limitation of contemporary language models: their inability to scale test-time compute (TTC) far beyond sequential reasoning under a fixed context window.
Approach: They propose a training-and-inference framework to overcome a central limitation of language models: their inability to scale test-time compute (TTC) under a fixed context window.
Outcome: The proposed model scales to multi-million-token effective TTC without exceeding context limits.
Improving Empathetic Dialogue Generation by Dynamically Infusing Commonsense Knowledge (2023.findings-acl)

Copied to clipboard

Challenge: Existing work on generating empathetic responses by utilizing the speaker's emotion has not been successful.
Approach: They propose an approach which incorporates an adaptive module for commonsense knowledge selection to ensure consistency between the generated empathetic responses and the speaker’s situation.
Outcome: The proposed approach outperforms baseline models in both automatic and human evaluations, exhibiting the generation of more coherent and empathetic responses.
Orthogonal Subspace Learning for Language Model Continual Learning (2023.findings-emnlp)

Copied to clipboard

Challenge: Existing methods for continual learning in language models suffer catastrophic forgetting when learning sequential tasks.
Approach: They propose an orthogonal low-rank adaptation approach for continual learning in language models that uses orthogons to learn sequentially.
Outcome: The proposed approach outperforms state-of-the-art methods on continual learning benchmarks and preserves generalization ability of LLMs on unseen tasks.
When Generative Adversarial Networks Meet Sequence Labeling Challenges (2024.emnlp-main)

Copied to clipboard

Challenge: Existing approaches for sequence labeling use a feature extractor and sequence tagger . a recent study shows that SLGAN is versatile and highly effective .
Approach: They propose a framework that harnesses the capabilities of Generative Adversarial Networks to address sequence labeling challenges.
Outcome: The proposed framework exhibits strong adaptability to various sequence labeling tasks.
EA-Agent: A Structured Multi-Step Reasoning Agent for Entity Alignment (2026.acl-long)

Copied to clipboard

Challenge: Entity alignment (EA) aims to identify entities across different knowledge graphs (KGs) that refer to the same real-world object.
Approach: They propose to use large language models to integrate semantic knowledge into EA to identify entities across different knowledge graphs that refer to the same object.
Outcome: The proposed agent outperforms existing methods and achieves state-of-the-art performance on three benchmark datasets.
DeltaNet: Conditional Medical Report Generation for COVID-19 Diagnosis (2022.coling-1)

Copied to clipboard

Challenge: X-ray and CT are the gold standard for COVID-19 diagnosis and treatment . however, due to the excessive number of patients, writing reports becomes a heavy burden for radiologists.
Approach: They propose to use X-ray and CT to generate medical reports automatically . they evaluate DeltaNet on a COVID-19 dataset, where it outperforms state-of-the-art approaches .
Outcome: The proposed system outperforms state-of-the-art methods on a COVID-19 dataset.
Synchronized Video Storytelling: Generating Video Narrations with Structured Storyline (2024.acl-long)

Copied to clipboard

Challenge: Existing studies on dense video captioning and video story generation have made some progress, but in practical applications, we typically require synchronized narrations for ongoing visual scenes.
Approach: They propose a task of Synchronized Video Storytelling to generate synchronized narrations for videos using a benchmark dataset with rich annotations.
Outcome: The proposed framework can generate narrations with the guidance of the generated or predefined storyline and human evaluations validate the effectiveness.
PRIME: A Process-Outcome Alignment Benchmark for Verifiable Reasoning in Mathematics and Engineering (2026.acl-long)

Copied to clipboard

Challenge: Current outcome-centric verification paradigms neglect potential errors in the derivation process.
Approach: They propose a process-aware RLVR training paradigm utilizing verifiers selected via **PRIME**.
Outcome: The proposed approach outperforms the baseline verification paradigm on AIME24, AIME25, and Beyond-AIME models.
ChatMusician: Understanding and Generating Music Intrinsically with LLM (2024.findings-acl)

Copied to clipboard

Challenge: Despite LLMs' impressive capabilities in musical knowledge, music reasoning remains an unsolved task.
Approach: They propose an open-source large language model (LLM) that integrates intrinsic musical abilities into LLaMA2 and GPT-3.5.
Outcome: The proposed model can understand and generate music with a pure text tokenizer without external multi-modal neural structures or tokenizers.
MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark (2025.acl-long)

Copied to clipboard

Challenge: Recent advances in multimodal large language models have led to progress in tackling complex reasoning tasks that combine textual and visual information.
Approach: They introduce a robust version of the Massive Multi-discipline Multimodal Understanding and Reasoning (MMMU) benchmark.
Outcome: The proposed model performs lower on MMMU-Pro than on the previous benchmark, ranging from 16.8% to 26.9%.
Characterizing the Impacts of Instances on Robustness (2023.findings-acl)

Copied to clipboard

Challenge: Existing defense approaches focus on developing new model structures or training algorithms, but they do little to tap the potential of training instances.
Approach: They propose a method that can distinguish between robust and non-robust instances according to the model’s sensitivity to perturbations on individual instances during training.
Outcome: The proposed method can distinguish between robust and non-robust instances according to the model’s sensitivity to perturbations on individual instances during training.
Dialogue Medical Information Extraction with Medical-Item Graph and Dialogue-Status Enriched Representation (2023.findings-emnlp)

Copied to clipboard

Challenge: Existing approaches ignore relationships between medical items and statuses in the multi-turn doctor-patient dialogue.
Approach: They propose a task to extract structured medical information from free text dialogues . they propose 'Dialogue Medical Information Extraction' to model relationships between items .
Outcome: The proposed model outperforms previous models and achieves state-of-the-art performance on the public benchmark data set.
Learning from the Irrecoverable: Error-Localized Policy Optimization for Tool-Integrated LLM Reasoning (2026.acl-long)

Copied to clipboard

Challenge: Tool-integrated reasoning (TIR) enables LLM agents to solve tasks through planning, tool use, and iterative revision, but outcome-only reinforcement learning suffers from sparse, delayed rewards and weak step-level credit assignment.
Approach: They propose a tool-integrated reasoning approach that localizes the first irrecoverable step and leverages it for fine-grained credit assignment.
Outcome: The proposed algorithm outperforms strong Agentic RL benchmarks in math, science QA, and code execution with additional gains in Pass@K and Major@K scaling, rollout ranking quality, and tool-call efficiency.
Are All Prompt Components Value-Neutral? Understanding the Heterogeneous Adversarial Robustness of Dissected Prompt in LLMs (2026.eacl-long)

Copied to clipboard

Challenge: Existing studies treat prompts as flat text, overlooking their internal structure, and different components within a prompt contribute unequally to robustness.
Approach: They propose a framework that decomposes prompts into functional components and a method that selectively modifies components to expose component-wise vulnerabilities.
Outcome: The proposed framework exposes component-wise vulnerabilities while ensuring linguistic plausibility through perplexity-based filtering.
Sample-Efficient Human Evaluation of Large Language Models via Maximum Discrepancy Competition (2025.acl-long)

Copied to clipboard

Challenge: Existing methods for evaluation of large language models are inefficient and inefficient due to inaccuracy of standard metrics in human perception of text quality and inefficiency in sampling informative test examples.
Approach: They propose a sample-efficient human evaluation method for large language models based on the principle of MAximum Discrepancy (MAD) competition.
Outcome: The proposed method achieves the “golden” ranking of LLMs with a minimum set of input instructions, which in turn reveal their relative strengths and weaknesses.
COIG-P: A High-Quality and Large-Scale Chinese Preference Dataset for Alignment with Human Values (2026.findings-eacl)

Copied to clipboard

Challenge: Existing Chinese preference datasets suffer from limited scale, restricted domain coverage, and insufficiently rigorous data validation.
Approach: They propose an LLM-based data annotation pipeline with no human intervention to annotate Chinese preference datasets.
Outcome: The proposed pipeline outperforms existing Chinese preference datasets on AlignBench and Chinese Reward Benchmark.
Do not Abstain! Identify and Solve the Uncertainty (2025.acl-long)

Copied to clipboard

Challenge: Existing solutions rely on evasive responses when confronting uncertain scenarios.
Approach: They propose a benchmark to assess LLMs' ability to recognize and address uncertainty . they generate context-aware inquiries that highlight the confusing aspect of the original query .
Outcome: Experiments with ConfuseBench show that LLMs struggle to identify root cause of uncertainty and solve it.
ConceptMath: A Bilingual Concept-wise Benchmark for Measuring Mathematical Reasoning of Large Language Models (2024.findings-acl)

Copied to clipboard

Challenge: ConceptMath evaluates concept-wise mathematical reasoning of Large Language Models (LLMs) Existing benchmarks that evaluate general mathematical reasoning with an average accuracy fail to probe the fine-grained failure modes of mathematical reasoning on specific datasets.
Approach: They introduce a bilingual, fine-grained benchmark that evaluates concept-wise mathematical reasoning of Large Language Models.
Outcome: The proposed benchmarks evaluate concept-wise mathematical reasoning of Large Language Models with concept-based accuracies.
Using Context-to-Vector with Graph Retrofitting to Improve Word Embeddings (2022.acl-long)

Copied to clipboard

Challenge: Contextualized embeddings are expensive and resource-demanding, hence environmentally unfriendly.
Approach: They propose a method to convert contextualized embeddings from pre-trained models into static embeddables using synonym knowledge and weighted vector distribution.
Outcome: The proposed method outperforms baseline embeddings by a large margin through extrinsic and intrinsic tasks.
ProtoCycle: Reflective Tool-Augmented Planning for Text-Guided Protein Design (2026.findings-acl)

Copied to clipboard

Challenge: Recent deep generative models have already shown encouraging * Equal contribution.
Approach: They propose to use generic instruction-tuned LLMs as direct text-to-sequence generators to achieve this goal.
Outcome: Recent studies show that reflection improves sequence quality and alignment while maintaining competitive foldability.
COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning (2025.findings-naacl)

Copied to clipboard

Challenge: Existing datasets for Chinese instruction tuning are not well-aligned with Chinese users’ interaction patterns.
Approach: They propose to use Chinese instruction tuning datasets to improve instruction fine-tuning for Chinese users.
Outcome: The proposed dataset shows that Chinese models achieve competitive performance in diverse benchmarks.
M2RC-EVAL: Massively Multilingual Repository-level Code Completion Evaluation (2025.acl-long)

Copied to clipboard

Challenge: Existing repository-level code completion benchmarks focus on a limited number of languages . existing benchmarks report overall average scores of different languages ignoring fine-grained abilities .
Approach: They propose to use repository-level code completion benchmarks to evaluate general code intelligence abilities across languages for existing code Large Language Models.
Outcome: The proposed benchmarks improve the code completion abilities of existing LLMs by using two types of annotations on the parsed syntax tree.
MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialogues (2024.acl-long)

Copied to clipboard

Challenge: Large Language Models (LLMs) have greatly enhanced dialogue systems, but evaluation of their capabilities remains a challenge.
Approach: They propose a model to evaluate the fine-grained abilities of Large Language Models in multi-turn dialogues.
Outcome: The proposed model evaluates 21 popular chatbots based on MT-Bench-101 . it includes 3 overarching abilities and 13 distinct tasks within multi-turn dialogue scenarios.
GraphReader: Building Graph-based Agent to Enhance Long-Context Abilities of Large Language Models (2024.findings-emnlp)

Copied to clipboard

Challenge: Existing models for long contexts struggle to handle long inputs due to limited context window and memory usage.
Approach: They propose a graph-based agent system that analyzes long texts into a graphical graph . GraphReader consistently outperforms GPT-4-128k across context lengths from 16k to 256k .
Outcome: The proposed model outperforms existing models on four challenging benchmarks.
VC4VG: Optimizing Video Captions for Text-to-Video Generation (2025.emnlp-main)

Copied to clipboard

Challenge: Recent advances in text-to-video generation highlight the critical role of high-quality video-text pairs in training models capable of producing coherent and instruction-aligned videos.
Approach: They propose a caption optimization framework tailored to the needs of T2V models.
Outcome: The proposed framework improves video caption quality and video generation performance.
LIME: Less Is More for MLLM Evaluation (2025.findings-acl)

Copied to clipboard

Challenge: Existing MLLM benchmarks and unified evaluation frameworks cannot accurately and efficiently reflect the ability of MLMLs.
Approach: They propose a semi-automated benchmark curated using a pipeline that filters out uninformative samples and eliminates answer leakage by focusing on tasks that require image-based understanding.
Outcome: The proposed benchmark reduces the number of samples by 76% and evaluation time by 77% while it can more effectively distinguish different models’ abilities.
MORE-3S:Multimodal-based Offline Reinforcement Learning with Shared Semantic Spaces (2024.lrec-main)

Copied to clipboard

Challenge: Existing approaches to offline reinforcement learning (RL) focus on learning value functions or policy gradients, but they view it as a sequence modeling task.
Approach: They propose a method that integrates multimodal and pre-trained language models to transform offline reinforcement learning into a supervised learning task by integrating state information derived from images and action-related data obtained from text.
Outcome: The proposed approach outperforms baselines on Atari and OpenAI Gym environments while promoting long-term strategic thinking.
Can Large Language Models Detect Errors in Long Chain-of-Thought Reasoning? (2025.acl-long)

Copied to clipboard

Challenge: Recent advances in o1-like models have generated long Chain-of-Thought reasoning steps to improve the reasoning abilities of existing Large Language Models (LLMs).
Approach: They propose a DeltaBench to analyze the quality and effectiveness of o1-like models and measure their ability to detect errors in long COT reasoning.
Outcome: The proposed model can detect errors in long COT reasoning.
OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement (2024.findings-acl)

Copied to clipboard

Challenge: OpenCodeInterpreter-33B provides a high level of performance for code generation, executing, and iterative refinement.
Approach: They propose a family of open-source code systems for generating, executing, and iteratively refining code.
Outcome: The OpenCodeInterpreter-33B performs well on humanEval, MBPP, and EvalPlus benchmarks.
Enhancing Agentic Textual Graph Retrieval with Synthetic Stepwise Supervision (2026.acl-long)

Copied to clipboard

Challenge: Existing methods for integrating textual graphs with LLMs are limited by symbolic inference and high annotation costs.
Approach: They propose a textual graph reasoning framework that integrates textual diagrams with large language models.
Outcome: The proposed approach achieves 15.6% accuracy and 17.2% in F1 score on three common datasets.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations