Papers by Ge Zhang

121 papers
Towards Reward Fairness in RLHF: From a Resource Allocation Perspective (2025.acl-long)

Copied to clipboard

Challenge: if rewards are imperfect, they can adversely affect the alignment of large language models (LLMs).
Approach: They propose a bias-agnostic method to address the issue of reward unfairness from a resource allocation perspective without specifically designing for each type of bias . they apply methods Fairness Regularization and Fairness Coefficient to achieve fairness in rewards.
Outcome: The proposed method achieves fairness in rewards while minimizing biases . it can be applied to verification and reinforcement learning scenarios .
DisCo_Speech: Controllable Zero-Shot Speech Generation with A Disentangled Speech Codec (2026.acl-long)

Copied to clipboard

Challenge: DisCo-Speech is a zero-shot controllable text-to-speech framework . standard codecs entangle timbre and prosody, which hinders independent control in continuation-based LMs.
Approach: They propose a disentangled speech codec and an LM-based generator to solve this problem . they propose fusion and reconstruction that merges content and prosody into unified tokens .
Outcome: DisCo-Speech achieves competitive voice cloning and superior zero-shot prosody control.
Capability Salience Vector: Fine-grained Alignment of Loss and Capabilities for Downstream Task Scaling Law (2025.acl-long)

Copied to clipboard

Challenge: Large language models have demonstrated impressive performance across a wide range of tasks, but this achievement comes with the trade-off of significant computational demands.
Approach: They propose a scaling law that decomposes the overall validation loss and assigns different importance weights to tokens to assess a specific meta-capability.
Outcome: The proposed model can predict the loss trending of models across different levels of computation without a gap between validation loss and model's downstream capabilities.
E2-LLM: Efficient and Extreme Length Extension of Large Language Models (2024.findings-acl)

Copied to clipboard

Challenge: Existing techniques for extending context capabilities in LLMs require additional training procedures and access to datasets with long context (e.g., sequences of 32K tokens).
Approach: They propose a solution to extend context capabilities in Large Language Models by training a single process over a sequence of 4K tokens.
Outcome: The proposed solution significantly reduces the cost of continual-pretraining or fine-tuning over short sequences and improves robustness to diverse relative positions.
Contrastive Attention for Automatic Chest X-ray Report Generation (2021.findings-acl)

Copied to clipboard

Challenge: Recent studies show that learning-based models fail to accurately capture and describe abnormal regions due to data bias.
Approach: They propose a model that compares the current input image with normal images to capture abnormal regions by contrasting the input image and normal images.
Outcome: The proposed model can be easily incorporated into existing models to boost their performance under most metrics.
Parallel Data Augmentation for Formality Style Transfer (2020.acl-main)

Copied to clipboard

Challenge: Formality style transfer is a task of automatically transforming text in one particular formality style into another.
Approach: They propose to augment parallel data with three specific data augmentation methods to improve the model's generalization ability and reduce the overfitting risk.
Outcome: The proposed methods significantly improve performance when used to pre-train the model and lead to the state-of-the-art results in the GYAFC benchmark dataset.
Resource-Limited Joint Multimodal Sentiment Reasoning and Classification via Chain-of-Thought Enhancement and Distillation (2026.findings-acl)

Copied to clipboard

Challenge: Current approaches for Multimodal Sentiment Analysis (MSA) rely on parameter-heavy LLMs for classification, overlooking multimodal sentiment reasoning generation in resource-limited environments.
Approach: They propose a multimodal sentiment reasoning distillation model that employs a teacher-assistant-student paradigm to address deployment constraints in resource-limited environments.
Outcome: The proposed model performs well on a resource-limited JMSRC task with only 3B parameters and shows generalization and interpretability.
HERB: Measuring Hierarchical Regional Bias in Pre-trained Language Models (2022.findings-aacl)

Copied to clipboard

Challenge: Existing methods do not examine social groups categorised by geographical information, leaving the region-related biases in pre-trained LMs unexplored.
Approach: They propose a hierarchical regional bias evaluation method to quantify regional bias in pre-trained language models.
Outcome: The proposed method evaluates regional bias with regard to comprehensive topics and measures potential regional bias that can be propagated to downstream tasks.
KARPA: A Training-free Method of Adapting Knowledge Graph as References for Large Language Model’s Reasoning Path Aggregation (2025.findings-acl)

Copied to clipboard

Challenge: Existing methods for large language models (LLMs) are limited by step-by-step decision-making on KGs, or require fine-tuning or pre-training on specific KG.
Approach: They propose a framework that harnesses the global planning abilities of large language models (LLMs) for efficient and accurate KG reasoning.
Outcome: Extensive experiments show that the proposed framework achieves state-of-the-art performance in KGQA tasks, delivering both high efficiency and accuracy.
DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories (2024.findings-acl)

Copied to clipboard

Challenge: Existing benchmarks are poorly aligned with real-world code repositories and are insufficient to evaluate the coding abilities of Large Language Models (LLMs).
Approach: They propose a repository-level benchmark named DevEval to evaluate LLMs' coding abilities in real-world code repositories.
Outcome: The proposed benchmarks show that the LLMs perform better in real-world code repositories than existing benchmarks.
Automatic Data Visualization Generation from Chinese Natural Language Questions (2024.lrec-main)

Copied to clipboard

Challenge: Existing studies on data visualization generation from natural languages have not been conducted on Chinese Text-to-Vis.
Approach: They propose to generate a Chinese text-to-vis dataset using a multilingual encoder and a cross-lingual ability.
Outcome: The proposed dataset is challenging and deserves further research.
PRISM: Probabilistic Reward Model with Inherent Structural Modeling (2026.acl-long)

Copied to clipboard

Challenge: Existing evaluators compress diverse human judgments into a single scalar, leading to brittle alignment and reward hacking.
Approach: They propose a Gaussian-based reinterpretation of reward evaluation as a conditional distribution and a mixture of Gaussians to capture conflicting preference dimensions.
Outcome: The proposed model outperforms scalar baselines in accuracy and generalization.
Learning from Mistakes: Iterative Prompt Relabeling for Text-to-Image Diffusion Model Training (2024.findings-emnlp)

Copied to clipboard

Challenge: Recent advances in diffusion models have shown impressive performance in many domains, but their ability to follow instructions is still unsatisfactory.
Approach: They propose an algorithm that aligns images to text through iterative image sampling and prompt relabeling with feedback.
Outcome: The proposed algorithm improves on the spatial relation VISOR benchmark by 15.22% compared to previous methods.
GuideTree: Guideline-Induced Review Trees for Long Medical Records (2026.acl-industry)

Copied to clipboard

Challenge: Medical record reviewers must produce consistent, traceable, guideline-compliant outcomes . longcontext inference is expensive and often degrades as inputs grow .
Approach: a new method compiles textual guidelines into a fixed review tree . a cost-aware split-and-prune search is used to update the tree offline . the algorithm produces consistent, traceable, guideline-compliant outcomes .
Outcome: The proposed system outperforms the strongest non-expert baselines by 84.5–92.8 Macro-F1 . it reduces average I/O volume to 74K input+output characters and average latency to 22s .
E-VarM: Enhanced Variational Word Masks to Improve the Interpretability of Text Classification Models (2022.coling-1)

Copied to clipboard

Challenge: Empirical studies show that our approach outperforms the SOTA methods in improving the interpretability of text classification models.
Approach: They propose an enhanced variational word masks approach that exploits the Variational Information Bottleneck to obtain task-specific words.
Outcome: Empirical results show that the proposed method outperforms the SOTA methods in improving the interpretability of the model.
MMRC: A Large-Scale Benchmark for Understanding Multimodal Large Language Model in Real-World Conversation (2025.acl-long)

Copied to clipboard

Challenge: Existing multimodal large language models lack the ability to memorize, recall, and reason in sustained interactions.
Approach: They propose a multimodal real-world conversation benchmark for evaluating open-ended abilities of multimodal large language models.
Outcome: The proposed benchmarks show that the models perform better in open-ended conversations.
AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling (2024.acl-long)

Copied to clipboard

Challenge: Existing language models that use discrete representations for unified processing of various modalities are limited to text generation and do not include multimodal output.
Approach: They propose a multimodal language model that utilizes discrete representations for unified processing of various modalities.
Outcome: The proposed model can be trained stably without any alterations to existing models or training paradigms.
LJPCheck: Functional Tests for Legal Judgment Prediction (2024.findings-acl)

Copied to clipboard

Challenge: Existing LJP models fail to evaluate specific aspects of their performance, such as legal fairness and judicial fairness.
Approach: They propose a suite of functional tests for LJP models to comprehend LJp models’ behaviors and offer diagnostic insights.
Outcome: Extensive tests reveal weaknesses in LJP models and provide diagnostic insights.
NER-guided Comprehensive Hierarchy-aware Prompt Tuning for Hierarchical Text Classification (2024.lrec-main)

Copied to clipboard

Challenge: Hierarchical text classification (HTC) is a challenging task in natural language processing due to its complex taxonomic label hierarchy.
Approach: They propose to use prompts to model hierarchical text classification (HTC) they propose to introduce conditional random fields and Global Pointer to establish hierarchic dependencies .
Outcome: The proposed approach achieves state-of-the-art (SoTA) performance on three public datasets.
Clear Up Confusion: Advancing Cross-Domain Few-Shot Relation Extraction through Relation-Aware Prompt Learning (2024.naacl-short)

Copied to clipboard

Challenge: Existing approaches to few-shot Relation Extraction (RE) are prone to confusion when applying knowledge to a target domain with entirely new types of relations.
Approach: They propose a relation-aware prompt learning method with pre-training to clear confusion by decomposing relation types through an innovative label prompt.
Outcome: The proposed method outperforms previous sota methods and yields better results on cross-domain few-shot RE tasks.
PaCoRe: Learning to Scale Test-Time Compute with Parallel Coordinated Reasoning (2026.acl-long)

Copied to clipboard

Challenge: Parallel Coordinated Reasoning (PaCoRe) overcomes a central limitation of contemporary language models: their inability to scale test-time compute (TTC) far beyond sequential reasoning under a fixed context window.
Approach: They propose a training-and-inference framework to overcome a central limitation of language models: their inability to scale test-time compute (TTC) under a fixed context window.
Outcome: The proposed model scales to multi-million-token effective TTC without exceeding context limits.
Self-Edit: Fault-Aware Code Editor for Code Generation (2023.acl-long)

Copied to clipboard

Challenge: Existing Large language models (LLMs) have low pass rates and accuracy on competitive programming tasks.
Approach: They propose a generate-and-edit approach that uses execution results of generated code from LLMs to improve code quality on competitive programming tasks.
Outcome: The proposed method improves pass@1 by 89% on APPS-dev, 31% on apps-test, and 48% on HumanEval over nine popular code generation LLMs with parameter sizes ranging from 110M to 175B.
Orthogonal Subspace Learning for Language Model Continual Learning (2023.findings-emnlp)

Copied to clipboard

Challenge: Existing methods for continual learning in language models suffer catastrophic forgetting when learning sequential tasks.
Approach: They propose an orthogonal low-rank adaptation approach for continual learning in language models that uses orthogons to learn sequentially.
Outcome: The proposed approach outperforms state-of-the-art methods on continual learning benchmarks and preserves generalization ability of LLMs on unseen tasks.
CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges (2024.acl-long)

Copied to clipboard

Challenge: Large Language Models excel in simple tasks such as generating standalone code units, but real-world software development often involves complex code repositories with complex dependencies and extensive documentation.
Approach: They propose a novel LLM-based agent framework that employs external tools for effective repo-level code generation.
Outcome: The proposed framework outperforms commercial products like Github Copilot in the humanEval benchmark and shows that it is adaptable and efficient across multiple code generation tasks.
Do LLMs Know Tool Irrelevance? Demystifying Structural Alignment Bias in Tool Invocations (2026.acl-long)

Copied to clipboard

Challenge: Large language models (LLMs) have impressive capabilities in utilizing external tools, but in practice, they are often exposed to tools that are irrelevant to the user’s query, in which case the desired behavior is to refrain from invocations.
Approach: They propose a new dataset that decouples structural alignment from semantic relevance and propose rebalancing strategies that effectively mitigates structural alignment bias.
Outcome: The proposed approach effectively mitigates structural alignment bias without degrading general tool-use capabilities.
Open-Topic False Information Detection on Social Networks with Contrastive Adversarial Learning (2022.emnlp-main)

Copied to clipboard

Challenge: Existing models for false information detection on social networks are too harsh for actual social networks that contain both seen and unseen topics simultaneously.
Approach: They propose an open-topic scenario that assumes that all test data topics are seen or unseen by the model, but which is too harsh for actual social networks that contain both seen and unseened topics simultaneously.
Outcome: The proposed model improves on two benchmark datasets and a variety of graph neural networks on two social networks and shows that it is more accurate than existing models.
When Backdoors Speak: Understanding LLM Backdoor Attacks Through Model-Generated Explanations (2025.acl-long)

Copied to clipboard

Challenge: Recent studies have shown that Large Language Models (LLMs) are susceptible to backdoor attacks, where triggers embedded in poisoned data can maliciously alter LLMs’ behaviors.
Approach: They propose to leverage LLMs' generative capabilities to generate human-readable explanations for their decisions, enabling direct comparisons between explanations of clean and poisoned data.
Outcome: The proposed model produces coherent explanations for clean inputs but logically flawed explanations on poisoned data.
Always the Best Fit: Adaptive Domain Gap Filling from Causal Perspective for Few-Shot Relation Extraction (2023.findings-emnlp)

Copied to clipboard

Challenge: Existing approaches to cross-domain relation extraction have been limited by domains . data bias between domains can be difficult to fill, especially in few-shot scenarios .
Approach: They propose a framework to bridge the semantic gap caused by data bias between domains . they use syntactic structure, label distribution, and entities to calculate causal effects .
Outcome: The proposed framework fills the domain gap and yields better results on the few-shot task.
Musical Score Understanding Benchmark: Evaluating Large Language Models’ Comprehension of Complete Musical Scores (2026.acl-long)

Copied to clipboard

Challenge: Existing benchmarks for musical score understanding are narrow in scope, focusing on isolated fragments, short excerpts, or multiple-choice formulations, rather than supporting holistic reasoning over entire scores.
Approach: They propose a benchmark for score-level musical understanding across textual and visual modalities.
Outcome: The musical score understanding benchmark contains 1,800 question-answer pairs from works by Bach, Beethoven, Chopin, Debussy, and others.
UP5: Unbiased Foundation Model for Fairness-aware Recommendation (2024.eacl-long)

Copied to clipboard

Challenge: Large Language Models (LLMs) are gaining a foothold in Recommender Systems (RS) but there is growing concern that LLMs perpetuate stereotypes and may result in unfair recommendations.
Approach: They propose a counterfactually-fair-prompt method for LLM-based recommendation that is based on unbiased foundation mOdels.
Outcome: The proposed method achieves better recommendation performance with a high level of fairness on two real-world datasets.
Lattice Transformer for Speech Translation (P19-1)

Copied to clipboard

Challenge: Recent advances in sequence modeling have highlighted the strengths of the transformer architecture.
Approach: They propose a general lattice transformer for speech translation where the input is the output of the automatic speech recognition (ASR) they propose 'controllable' lattica attention mechanism to consume latent representations.
Outcome: The proposed model outperforms baseline and lattice LSTM on the Chinese-English translation task.
Improving Personalized Explanation Generation through Visualization (2022.acl-long)

Copied to clipboard

Challenge: Existing explainable recommendation models generate repetitive sentences for different items or empty sentences with insufficient details.
Approach: They propose a visual-enhanced approach to generate rating scores and text explanations using visualization generation and text–image matching discrimination.
Outcome: The proposed approach improves both the text quality and the diversity and explainability of the generated explanations.
Supplement Generation Training for Enhancing Agentic Task Performance (2026.findings-acl)

Copied to clipboard

Challenge: Training large foundation models for agentic tasks is impractical due to high computational costs, long iteration cycles, and rapid obsolescence as new models are released.
Approach: They propose a method that trains a small LLM to generate supplemental text that helps the larger LLM solve the task more effectively.
Outcome: The proposed approach decouples task-specific optimization from large foundation models . it achieves consistent and significant performance gains across diverse tasks and models - all without gradient access to the actor model.
Toward Machine Translation Literacy: How Lay Users Perceive and Rely on Imperfect Translations (2025.emnlp-main)

Copied to clipboard

Challenge: Using machine translation tools for everyday tasks is becoming more commonplace, but a lack of evaluation strategies and alternatives can cause users to over-rely on it.
Approach: They propose to use MT evaluation techniques to promote MT quality and MT literacy among its users.
Outcome: The findings highlight the need for evaluation and NLP explanation techniques to promote MT quality and MT literacy among its users.
VideoCuRL: Video Curriculum Reinforcement Learning with Orthogonal Difficulty Decomposition (2026.acl-long)

Copied to clipboard

Challenge: Reinforcement Learning (RL) is crucial for Video-LLMs with complex spatiotemporal reasoning.
Approach: They propose a framework that decomposes difficulty into two axes in video understanding . they employ efficient, training-free proxies to map data onto a 2D curriculum grid .
Outcome: The proposed framework surpasses strong RL baselines on reasoning and perception tasks.
MusiLingo: Bridging Music and Text with Pre-trained Language Models for Music Captioning and Query Response (2024.findings-naacl)

Copied to clipboard

Challenge: Large Language Models have shown immense potential in multimodal applications, but convergence between textual and musical domains remains unexplored.
Approach: They propose a system that aligns music representations with a frozen LLM . they train the system on an extensive music caption dataset and fine-tune it with instructional data .
Outcome: The proposed system bridges the gap between music audio and textual contexts by combining music captions with a frozen model . it performs well in generating music caption and composing music-related Q&A pairs . the proposed system is available for free download at http://www.musilingo.com/ .
MARS-Bench: A Multi-turn Athletic Real-world Scenario Benchmark for Dialogue Evaluation (2025.findings-emnlp)

Copied to clipboard

Challenge: Large Language Models (LLMs) have been widely adopted in real-world dialogue applications, but their robustness is criticized all along.
Approach: They propose to use play-by-play text commentary to build a multi-turn athletic real-world scenario dialogue benchmark to evaluate three critical aspects of multi-turned conversations: ultra multi- turn, interactive multi-twist, and cross-turn tasks.
Outcome: The proposed benchmarks outperform open-source LLMs on three critical aspects of multi-turn conversations: ultra multi-turned, interactive multi- turn, and cross-turn tasks.
EA-Agent: A Structured Multi-Step Reasoning Agent for Entity Alignment (2026.acl-long)

Copied to clipboard

Challenge: Entity alignment (EA) aims to identify entities across different knowledge graphs (KGs) that refer to the same real-world object.
Approach: They propose to use large language models to integrate semantic knowledge into EA to identify entities across different knowledge graphs that refer to the same object.
Outcome: The proposed agent outperforms existing methods and achieves state-of-the-art performance on three benchmark datasets.
Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation (2024.emnlp-main)

Copied to clipboard

Challenge: Existing methods for instruction data selection have limitations such as relying on fragile external APIs, being affected by biases in GPT models, or reducing the diversity of the selected instruction dataset.
Approach: They propose an industrial-friendly, expert-aligned and diversity-preserved instruction data selection method: Clustering and Ranking (CaR).
Outcome: The proposed method outperforms Alpaca's existing methods by 32.1% in GPT-4 evaluations.
Bypassing Neural Evaluations for Fast Audio Editing via Adaptive Trajectory Extrapolation (2026.findings-acl)

Copied to clipboard

Challenge: Recent advances in audio diffusion models have significantly improved text-to-audio editing via inversion techniques, but these models typically rely on dense, fixed-step sampling trajectories to maintain structural integrity.
Approach: They propose a model-agnostic Adaptive Trajectory Extrapolation framework that accelerates inversion-based editing process by dynamically evaluating only the most critical generative phases.
Outcome: The proposed framework achieves a 3.9 speedup with negligible loss in fidelity.
MMTE: Corpus and Metrics for Evaluating Machine Translation Quality of Metaphorical Language (2024.emnlp-main)

Copied to clipboard

Challenge: Existing evaluation methods focus on fluency and factual reliability, while neglecting figurative quality.
Approach: They propose a set of human evaluation metrics focused on the translation of figurative language and a parallel metaphor corpus generated by post-editing.
Outcome: The proposed evaluation protocol estimates four aspects of MT: Metaphorical Equivalence, Emotion, Authenticity, and Quality.
HiRoPE: Length Extrapolation for Code Models Using Hierarchical Position (2024.acl-long)

Copied to clipboard

Challenge: Existing LLMs are constrained by their pre-trained context lengths, leading to performance issues . elucidating this limitation, we propose a training-free solution to the context length limitation in LLM applications .
Approach: They propose a method that integrates hierarchical rotary position embedding into LLMs without extra training costs.
Outcome: The proposed method improves performance on language modeling and long code completion tasks.
Low-Bit Quantization Favors Undertrained LLMs (2025.acl-long)

Copied to clipboard

Challenge: Larger models or those trained on fewer tokens exhibit less quantization-induced degradation (QiD), while smaller, well-trained models face significant performance losses.
Approach: They propose to use QiD to measure an LLM’s training levels and determine the number of training tokens required for fully training LLMs of various sizes.
Outcome: The proposed scaling laws can predict the quantization performance of different-sized LLMs trained with tokens.
On the Emotion Understanding of Synthesized Speech (2026.acl-long)

Copied to clipboard

Challenge: Existing models for emotion understanding do not capture fundamental features of synthesized speech.
Approach: They evaluate emotion recognition models on synthesized speech using SER models and generative models.
Outcome: The proposed model can't generalize to synthesized speech because of speech token prediction . generative models tend to infer emotion from textual semantics while ignoring paralinguistic cues.
CLaMP 2: Multimodal Music Information Retrieval Across 101 Languages Using Large Language Models (2025.findings-naacl)

Copied to clipboard

Challenge: Current music information retrieval systems struggle to meet linguistic diversity challenges . current systems struggle with text queries in non-English languages .
Approach: They propose a music information retrieval system that supports both ABC notation and MIDI . CLaMP 2 includes a multilingual text encoder and a multiple-modal music encoder .
Outcome: The proposed system achieves state-of-the-art results in multilingual semantic search and music classification across modalities.
CIF-Bench: A Chinese Instruction-Following Benchmark for Evaluating the Generalizability of Large Language Models (2024.findings-acl)

Copied to clipboard

Challenge: a recent study shows that large language models have limited generalization in low-resource languages like Chinese.
Approach: They propose to evaluate the zero-shot generalizability of large language models to the Chinese language . they release only half of the dataset publicly, with the remainder kept private .
Outcome: The Chinese Instruction-Following Benchmark evaluates the generalizability of LLMs to the Chinese language.
TReMu: Towards Neuro-Symbolic Temporal Reasoning for LLM-Agents with Memory in Multi-Session Dialogues (2025.findings-acl)

Copied to clipboard

Challenge: Temporal reasoning in multi-session dialogues presents a significant challenge which has been under-studied in previous temporal reasoning benchmarks.
Approach: They propose to augment LoCoMo dialogues and create multi-choice QAs to construct a temporal reasoning evaluation task and a framework to enhance temporal thinking capabilities of LLM-agents.
Outcome: The proposed framework significantly improves temporal reasoning performance compared to baseline methods, raising from 29.83 on GPT-4o via standard prompting to 77.67 via the proposed framework.
Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction (2020.emnlp-main)

Copied to clipboard

Challenge: Existing methods to improve the efficiency of GEC are not efficient enough for GEC.
Approach: They propose a language-independent approach to improve the efficiency of GEC by dividing the task into two subtasks: ESD and ESC.
Outcome: The proposed approach performs comparably to conventional seq2seq approaches in English and Chinese GEC benchmarks with less than 50% time cost for inference.
Don’t Miss the Potential Customers! Retrieving Similar Ads to Improve User Targeting (2021.findings-emnlp)

Copied to clipboard

Challenge: a method for user targeting is developed to identify online users to whom an ad should be targeted.
Approach: They propose a method for automatic augmentation of positive and negative clickthrough data for user targeting models.
Outcome: The proposed method can increase positive and negative instances of positive training instances on two datasets.
PRIME: A Process-Outcome Alignment Benchmark for Verifiable Reasoning in Mathematics and Engineering (2026.acl-long)

Copied to clipboard

Challenge: Current outcome-centric verification paradigms neglect potential errors in the derivation process.
Approach: They propose a process-aware RLVR training paradigm utilizing verifiers selected via **PRIME**.
Outcome: The proposed approach outperforms the baseline verification paradigm on AIME24, AIME25, and Beyond-AIME models.
VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generation (2024.emnlp-main)

Copied to clipboard

Challenge: Existing video metrics are lagging behind in providing reliable scores over generated videos due to lack of large-scale human-annotated dataset.
Approach: They propose to use VideoFeedback to train a human-annotated multi-aspect score over 37.6K synthesized videos from 11 existing video generative models.
Outcome: The proposed model outperforms the prior best metrics by 50 points in the test.
Refining Corpora from a Model Calibration Perspective for Chinese Spelling Correction (2024.findings-acl)

Copied to clipboard

Challenge: Chinese Spelling Correction (CSC) lacks large-scale high-quality corpora due to labor-intensive labeling of spelling errors in real-life writing or typing scenarios.
Approach: They propose to use OCR/ASR-based generation to refine Chinese Spelling Correction models on random replacement-based corpora and filter them based on prediction confidence.
Outcome: The proposed model outperforms existing models on three widely-used benchmarks while significantly alleviating over-correction.
Rethinking Multiple-Choice Questions for RLVR: Unlocking Potential via Distractor Design (2026.findings-acl)

Copied to clipboard

Challenge: Existing approaches to RLVR use multiple-choice questions as verifiable rewards . however, not all tasks provide reliable verification .
Approach: They propose a framework that actively constructs high-quality distractors to block elimination shortcuts and promote deep reasoning.
Outcome: The proposed method significantly improves reasoning capabilities of Large Language Models.
MdEval: Massively Multilingual Code Debugging (2026.findings-acl)

Copied to clipboard

Challenge: Existing benchmarks primarily focus on Python and are limited in terms of language diversity.
Approach: They propose a multilingual debugging benchmark that includes 3.9K test samples of 20 programming languages and introduces the debug instruction corpora MdEval-Instruct by injecting bugs into the correct multilingual queries and solutions.
Outcome: The proposed benchmark includes 3.9K test samples of 20 programming languages and covers the automated program repair task, bug localization task, and bug identification task.
MeMoTune: A Measure and Moment-Driven Fine-Tuning Framework for Quantized Large Language Models (2025.findings-acl)

Copied to clipboard

Challenge: Existing methods combine quantization with parameter-efficient fine-tuning but fail to meet practical performance requirements.
Approach: They propose a measure and moment approach to optimize objective function for superior fine-tuning results by scaling the update process through a gradient.
Outcome: The proposed framework outperforms state-of-the-art methods on tasks like text generation, summarization, and understanding.
Natural Language Embedded Programs for Hybrid Language Symbolic Reasoning (2024.findings-naacl)

Copied to clipboard

Challenge: Existing methods for surfacing symbolic reasoning capabilities are limited to narrow tasks . arithmetic computations are unnatural to perform in pure language space, and hence present difficulties for LLMs.
Approach: They propose a natural language embedded program framework for solving symbolic reasoning tasks.
Outcome: The proposed framework improves on strong baselines across math and symbolic reasoning, text classification, question answering, and instruction following tasks.
Structural Information Preserving for Graph-to-Text Generation (2020.acl-main)

Copied to clipboard

Challenge: Existing models that mess up or drop the core structural information of input graphs are lacking in graph-to-text generation.
Approach: They propose to leverage richer training signals to guide a graph-to-text generation model by focusing on autoencoding losses and back-propagating the losses to better calibrate the model.
Outcome: Experiments on two benchmarks show the proposed model over a state-of-the-art model . two types of autoencoding losses are used to back-propagate the model based on multitask training .
Benchmarking Long-Context Language Models on Long Code Understanding (2025.acl-long)

Copied to clipboard

Challenge: Currently, long-context language models are limited by the lack of a rigorous evaluation framework for long code understanding.
Approach: They propose to use a long code understanding benchmark LongCodeU to evaluate LCLMs' long code comprehension ability for practical applications.
Outcome: The proposed benchmarks show that current LCLMs are limited in their long code understanding ability, particularly when the long code length is greater than 32K, falling far short of their claimed 128K to 1M context windows.
CriticLean: Critic-Guided Reinforcement Learning for Mathematical Formalization (2026.acl-long)

Copied to clipboard

Challenge: Existing approaches to formalizing mathematical statements face limitations in accuracy, especially in the context of complex, highlevel problems that involve sophisticated mathematical reasoning.
Approach: They propose a CriticLean framework that elevates the role of the critic from a passive validator to an active learning component and introduce a benchmark to measure models’ ability to distinguish semantically correct from incorrect formalizations.
Outcome: The proposed framework outperforms open- and closed-source benchmarks and shows that it significantly outperformed existing models.
ChatMusician: Understanding and Generating Music Intrinsically with LLM (2024.findings-acl)

Copied to clipboard

Challenge: Despite LLMs' impressive capabilities in musical knowledge, music reasoning remains an unsolved task.
Approach: They propose an open-source large language model (LLM) that integrates intrinsic musical abilities into LLaMA2 and GPT-3.5.
Outcome: The proposed model can understand and generate music with a pure text tokenizer without external multi-modal neural structures or tokenizers.
MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark (2025.acl-long)

Copied to clipboard

Challenge: Recent advances in multimodal large language models have led to progress in tackling complex reasoning tasks that combine textual and visual information.
Approach: They introduce a robust version of the Massive Multi-discipline Multimodal Understanding and Reasoning (MMMU) benchmark.
Outcome: The proposed model performs lower on MMMU-Pro than on the previous benchmark, ranging from 16.8% to 26.9%.
Long-Short Term Masking Transformer: A Simple but Effective Baseline for Document-level Neural Machine Translation (2020.emnlp-main)

Copied to clipboard

Challenge: Recent advances in deep learning have led to significant improvement of document-level neural machine translation (NMT).
Approach: They propose a long-short term masking self-attention on top of the standard transformer to capture the long-range dependence and reduce the propagation of errors.
Outcome: The proposed model captures the long-range dependence and reduces errors on two publicly available document-level datasets.
VaseVQA: Multimodal Agent and Benchmark for Ancient Greek Pottery (2026.findings-eacl)

Copied to clipboard

Challenge: MLLMs that use domain-specific data are limited in understanding cultural heritage artifacts such as ancient Greek pottery . supervised fine-tuning improves adaptation to domain knowledge, but it struggles with deeper reasoning tasks.
Approach: They propose a visual question-answer tool that augments SFT with reinforcement learning using verifiable rewards.
Outcome: The proposed model outperforms baseline models on reasoning-intensive questions on ancient Greek pottery.
Characterizing the Impacts of Instances on Robustness (2023.findings-acl)

Copied to clipboard

Challenge: Existing defense approaches focus on developing new model structures or training algorithms, but they do little to tap the potential of training instances.
Approach: They propose a method that can distinguish between robust and non-robust instances according to the model’s sensitivity to perturbations on individual instances during training.
Outcome: The proposed method can distinguish between robust and non-robust instances according to the model’s sensitivity to perturbations on individual instances during training.
LLM-Powered Test Case Generation for Detecting Bugs in Plausible Programs (2025.acl-long)

Copied to clipboard

Challenge: TrickCatcher generates test cases that pass existing tests yet contain bugs . a recent study found that tricky bugs are not detected by test suites .
Approach: They propose an LLM-powered approach to generating test cases for uncovering bugs in plausible programs . they use a PUT and specification to generate program variants, an input generator and an Llm to construct test inputs .
Outcome: The proposed approach achieves recall, precision, and F1 scores that are 1.80, 2.65, and 1.66 . trickCatcher generates program variants based on the program under test and its specification .
MAPRO: Recasting Multi-Agent Prompt Optimization as Maximum a Posteriori Inference (2026.findings-eacl)

Copied to clipboard

Challenge: Large language models (LLMs) have demonstrated remarkable capabilities across diverse tasks.
Approach: They propose a framework that optimizes MAS prompts as a maximum a posteriori problem and then iteratively updates agent prompts.
Outcome: The proposed framework surpasses manual and automated benchmarks in multiple tasks and provides general guidelines for building more reliable and principled multi-agent systems in the future.
SummaCoz: A Dataset for Improving the Interpretability of Factual Consistency Detection for Summarization (2024.findings-emnlp)

Copied to clipboard

Challenge: Summarization is an important application of Large Language Models.
Approach: They integrate human-annotated and model-generated natural language explanations to elucidate how a summary deviates and becomes inconsistent with its source article.
Outcome: The proposed model provides rationales for its judgments and improves its accuracy significantly.
ReCaLL: Membership Inference via Relative Conditional Log-Likelihoods (2024.emnlp-main)

Copied to clipboard

Challenge: ReCaLL (Relative Conditional Log-Likelihood) is a membership inference attack that can detect LLMs’ pretraining data by leveraging their conditional language modeling capabilities.
Approach: They propose a membership inference attack to detect LLMs’ pretraining data by leveraging their conditional language modeling capabilities.
Outcome: The proposed model achieves state-of-the-art performance on the WikiMIA dataset, even with random and synthetic prefixes, and can be further improved using an ensemble approach.
MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria (2025.naacl-long)

Copied to clipboard

Challenge: Existing evaluation methodologies for multimodal large language models are limited in evaluating objective queries without considering real-world user experiences.
Approach: They propose to evaluate multimodal large language models with per-sample criteria using potent MLLM as the judge.
Outcome: The proposed evaluation paradigm shows that it can be used to evaluate multimodal large language models with per-sample criteria.
Dialogue Medical Information Extraction with Medical-Item Graph and Dialogue-Status Enriched Representation (2023.findings-emnlp)

Copied to clipboard

Challenge: Existing approaches ignore relationships between medical items and statuses in the multi-turn doctor-patient dialogue.
Approach: They propose a task to extract structured medical information from free text dialogues . they propose 'Dialogue Medical Information Extraction' to model relationships between items .
Outcome: The proposed model outperforms previous models and achieves state-of-the-art performance on the public benchmark data set.
Can MLLMs Understand the Deep Implication Behind Chinese Images? (2025.acl-long)

Copied to clipboard

Challenge: MLLMs perform poorly on traditional culture images, indicating limitations in understanding high-level semantics and lacking a deep knowledge base of Chinese traditional culture.
Approach: They propose to use Chinese images to assess MLLMs' higher-order perception and understanding of Chinese visual content.
Outcome: The proposed model incorporates images that represent Chinese traditional culture, such as famous Chinese traditional paintings, to ensure the authenticity of the Chinese context.
Mind the (DH) Gap! A Contrast in Risky Choices Between Reasoning and Conversational LLMs (2026.acl-long)

Copied to clipboard

Challenge: Large language models are increasingly used in decision support systems and workflows . traditional computational paradigms for decision-making under uncertainty choose an option that maximizes expected utility or payoff .
Approach: They compare large language models as decision support systems and agentic workflows . they find that LLMs cluster into reasoning models and conversational models .
Outcome: The proposed models differ in their ability to perform tasks and their ability in a human-like way.
CMDAG: A Chinese Metaphor Dataset with Annotated Grounds as CoT for Boosting Metaphor Generation (2024.lrec-main)

Copied to clipboard

Challenge: Metaphors are a prominent linguistic device in human language and literature, as they add color, imagery, and emphasis to enhance effective communication.
Approach: They propose a large-scale high quality annotated Chinese Metaphor Corpus . they use a set of guidelines to ensure the accuracy and consistency of their annotations .
Outcome: The proposed corpus generates metaphors that resonate more with real-world intuition.
UNIKIE-BENCH: Benchmarking Large Multimodal Models for Key Information Extraction in Visual Documents (2026.acl-long)

Copied to clipboard

Challenge: Recent Large Multimodal Models (LMMs) have shown promising potential for performing end-to-end KIE directly from document images.
Approach: They propose a benchmark to evaluate the performance of Large Multimodal Models (LMMs) using a constrained-category KIE track and an open-categorical KIE Track.
Outcome: Experiments on 15 state-of-the-art LMMs show performance degradation under diverse schema definitions, long-tail key fields, and complex layouts, along with pronounced performance disparities across different document types and scenarios.
Focused-DPO: Enhancing Code Generation Through Focused Preference Optimization on Error-Prone Points (2025.findings-acl)

Copied to clipboard

Challenge: Current code generation models produce errors concentrated at specific error-prone points, affecting accuracy of code.
Approach: They propose a framework that focuses preference optimization on error-prone areas . focused-DPO improves the accuracy and reliability of code generation by reducing common errors .
Outcome: The proposed framework improves code generation by focusing on error-prone areas.
Evaluating Evaluation Metrics – The Mirage of Hallucination Detection (2025.findings-emnlp)

Copied to clipboard

Challenge: a large-scale empirical evaluation of hallucination detection metrics is conducted . hallucinosity is a significant obstacle to the reliability and widespread adoption of language models .
Approach: They conduct large-scale empirical evaluation of hallucination detection metrics . they compare hallucinian language models, language models and decoding methods .
Outcome: The results show that the evaluations of hallucination detection metrics fail to align with human judgments, they say . they also show that evaluations with LLM-based evaluation yield the best overall results .
LawBench: Benchmarking Legal Knowledge of Large Language Models (2024.emnlp-main)

Copied to clipboard

Challenge: LegalBench evaluated 20 LLMs in 162 legal tasks in 20 countries and jurisdictions.
Approach: They present a comprehensive evaluation of 21 popular Large Language Models and the first comparative analysis of the empirical results.
Outcome: The proposed benchmarks are based on the Bloom’s cognitive taxonomy and are compared to 21 popular LLMs.
MIO: A Foundation Model on Multimodal Tokens (2025.emnlp-main)

Copied to clipboard

Challenge: Existing models lack multimodal understanding capabilities, resulting in closed-source model that does not support multimodal interleaved sequences.
Approach: They propose a foundation model built on multimodal tokens capable of understanding and generating speech, text, images, and videos in an end-to-end, autoregressive manner.
Outcome: The proposed model is able to understand speech, text, images, and videos in an end-to-end, autoregressive manner.
Unraveling the Mystery: Defending Against Jailbreak Attacks Via Unearthing Real Intention (2025.coling-main)

Copied to clipboard

Challenge: Large Language Models (LLMs) are increasingly vulnerable to elusive and implicit intentions, causing security risks and compromising user experience.
Approach: They propose a method to detect and mitigate implicit jailbreak attacks using LLMs by unearthing real intentions and a greedy gradient-based algorithm to remove the least important parts of a sentence.
Outcome: The proposed method reduces attacks success rate and Harmful Score while maintaining overall model performance.
Automatic Grammatical Error Correction for Sequence-to-sequence Text Generation: An Empirical Study (P19-1)

Copied to clipboard

Challenge: Sequence-to-sequence (seq2sequ) models have a weakness: they cannot always generate sentences without grammatical errors.
Approach: They propose to use automatic grammatical error correction to improve seq2seq models . they conduct experiments on machine translation, formality style transfer, sentence compression and simplification .
Outcome: The proposed system can improve grammaticality of generated text and improve formal style tasks.
CapOnImage: Context-driven Dense-Captioning on Image (2022.emnlp-main)

Copied to clipboard

Challenge: Existing image captioning systems generate narrative captions for images, which are spatially detached from the image in presentation.
Approach: They propose a task called captioning on image which generatesense captions at different locations of the image based on contextual information.
Outcome: The proposed model achieves the best results in both captioning accuracy and diversity aspects.
LLM-SLM Collaborative Framework of Idiomatic Expression Generation (2026.acl-long)

Copied to clipboard

Challenge: Existing methods for idiomatic expression generation lack parallel data and manual annotations.
Approach: They propose an iterative LLM-SLM collaborative framework that replaces human supervision for idiomatic expression data generation.
Outcome: The proposed framework outperforms DeepSeek-R1 in Chinese Idiom Polishing with a 25.2% improvement in accuracy.
M3TQA: Massively Multilingual Multitask Table Question Answering (2026.findings-acl)

Copied to clipboard

Challenge: Existing multilingual table benchmarks suffer from geolinguistic imbalance - overrepresenting certain languages and lacking sufficient scale for rigorous cross-lingual analysis.
Approach: They propose a framework for massively multilingual table question answering that includes tables expanded to 97 languages from Chinese and English sources.
Outcome: Experiments on state-of-the-art LLMs show that synthetically generated training data significantly boosts performance, especially for low-resource languages.
Sample-Efficient Human Evaluation of Large Language Models via Maximum Discrepancy Competition (2025.acl-long)

Copied to clipboard

Challenge: Existing methods for evaluation of large language models are inefficient and inefficient due to inaccuracy of standard metrics in human perception of text quality and inefficiency in sampling informative test examples.
Approach: They propose a sample-efficient human evaluation method for large language models based on the principle of MAximum Discrepancy (MAD) competition.
Outcome: The proposed method achieves the “golden” ranking of LLMs with a minimum set of input instructions, which in turn reveal their relative strengths and weaknesses.
COIG-P: A High-Quality and Large-Scale Chinese Preference Dataset for Alignment with Human Values (2026.findings-eacl)

Copied to clipboard

Challenge: Existing Chinese preference datasets suffer from limited scale, restricted domain coverage, and insufficiently rigorous data validation.
Approach: They propose an LLM-based data annotation pipeline with no human intervention to annotate Chinese preference datasets.
Outcome: The proposed pipeline outperforms existing Chinese preference datasets on AlignBench and Chinese Reward Benchmark.
Inverse-Q*: Token Level Reinforcement Learning for Aligning Large Language Models Without Preference Data (2024.findings-emnlp)

Copied to clipboard

Challenge: Reinforcement Learning from Human Feedback (RLHF) relies on complex methodologies like Proximal Policy Optimization (PPO) that require extensive hyper-parameter tuning and pose challenges in sample efficiency and stability.
Approach: They propose an innovative framework that leverages direct preference optimization techniques but extends them by estimating the conditionally optimal policy directly from the model’s responses.
Outcome: The proposed framework matches and exceeds the effectiveness of Proximal Policy Optimization (PPO) in terms of convergence speed and alignment of model responses with human preferences.
Enhancing Code Generation Performance of Smaller Models by Distilling the Reasoning Ability of LLMs (2024.lrec-main)

Copied to clipboard

Challenge: Large Language Models (LLMs) have made significant advances in code generation through the ‘Chain-of-Thought’ prompting technique.
Approach: They propose a framework which aims to transfer LLMs’ reasoning capabilities to smaller models through distillation.
Outcome: The proposed framework improves the smaller model's code generation performance by over 130% on the APPS benchmark.
Targeted Exploration via Unified Entropy Control for Reinforcement Learning (2026.findings-acl)

Copied to clipboard

Challenge: Existing methods for group relative policy optimization suffer from entropy collapse . Existing exploration methods introduce additional bias or variance during exploration, making it difficult to maintain stability.
Approach: They propose a framework that provides targeted mechanisms for exploration and stabilization.
Outcome: The proposed framework expands search space on difficult prompts while preventing entropy growth uncontrollably.
SciMMIR: Benchmarking Scientific Multi-modal Information Retrieval (2024.findings-acl)

Copied to clipboard

Challenge: Multi-modal information retrieval (MMIR) is a rapidly evolving field . current benchmarks for image-text pairings overlook the scientific domain .
Approach: They develop a scientific domain-specific MMIR benchmark to evaluate image-text pairings using open-access research paper corpora.
Outcome: The proposed benchmarks are based on 530K image-text pairs extracted from scientific documents with detailed captions.
E-ViC: Reasoning Beyond Text via Embodied Visual Chain for Spatial Intelligence (2026.acl-long)

Copied to clipboard

Challenge: Existing Vision-Language Models (VLMs) lack spatial reasoning, despite text-based CoTs . e-ViC reframes spatial intelligence as a verifiable, tool-using capability, argues a new study.
Approach: They propose a framework that moves reasoning beyond text into the visual domain . they ground reasoning in pixel-level interactions to enable human-like "look-and-confirm" strategies .
Outcome: The proposed framework outperforms existing Vision-Language Models with an average gain of 10.1%.
Adapting Unsupervised Syntactic Parsing Methodology for Discourse Dependency Parsing (2021.acl-long)

Copied to clipboard

Challenge: Discourse dependency parsing is a task that requires a large amount of training data, but there is little research on it.
Approach: They propose to adapt unsupervised syntactic dependency parsing methods for unsupervised discourse dependency parses using unlabeled training data.
Outcome: The proposed methods outperform existing methods in semi-supervised and supervised settings and outperformed existing methods.
Aligning Generative Language Models with Human Values (2022.findings-naacl)

Copied to clipboard

Challenge: Existing methods for learning human values do not consider contextual and abstract nature of human values.
Approach: They propose a reinforcement learning based method that embeds human values judgements into each step of language generation.
Outcome: The proposed method improves on human values judgements and shows higher alignment performance.
K-Level Reasoning: Establishing Higher Order Beliefs in Large Language Models for Strategic Reasoning (2025.naacl-long)

Copied to clipboard

Challenge: Strategic reasoning requires Large Language Model (LLM) agents to adapt their strategies dynamically in multi-agent environments.
Approach: They propose a framework that enables Large Language Models to achieve varying levels of strategic depth by recursive mechanisms that allow agents to form higher order beliefs about others' beliefs.
Outcome: The proposed framework enables LLMs to achieve varying levels of strategic depth, allowing agents to form higher order beliefs—beliefs about others’ beliefs.
OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models (2025.acl-long)

Copied to clipboard

Challenge: Code LLMs lack reproducible data pipelines and training protocols for reproducible advancements in code intelligence.
Approach: They propose a top-tier code LLM that releases model weights and inference code . reproducible data pipelines, rigorous experimental ablation results and training protocols are included .
Outcome: The proposed model achieves comparable performance to leading models and serves as an "open cookbook" reproducible training data, rigorous experimental ablation results, and detailed training protocols are also included in the model.
KoCo-Bench: Can Large Language Models Leverage Domain Knowledge in Software Development? (2026.acl-long)

Copied to clipboard

Challenge: Existing domain-specific code benchmarks focus on assessing what knowledge LLMs possess rather than how they acquire and apply new knowledge.
Approach: They propose a benchmark to evaluate domain specialization methods in real-world software development.
Outcome: KOCO-bench is a new benchmark for evaluating domain specialization methods in real-world software development.
InternLM-Law: An Open-Sourced Chinese Legal Large Language Model (2025.coling-main)

Copied to clipboard

Challenge: InternLM-Law is a large language model (LLM) tailored for addressing diverse legal tasks related to Chinese laws.
Approach: They introduce a large language model (LLM) tailored for addressing diverse legal tasks related to Chinese laws.
Outcome: The proposed model performs better than existing models in a variety of legal tasks related to Chinese laws.
Fusion Makes Perfection: An Efficient Multi-Grained Matching Approach for Zero-Shot Relation Extraction (2024.naacl-short)

Copied to clipboard

Challenge: Existing methods to extract unseen relations require laborious manual annotation . a new approach uses fine-grained matching to reduce manual annotation cost .
Approach: They propose an efficient multi-grained matching approach that uses virtual entity matching to reduce manual annotation cost.
Outcome: The proposed approach outperforms the state-of-the-art methods and achieves inference efficiency and accuracy in zero-shot relation extraction tasks.
MelTrim: Coarse-to-Fine Data Pruning for Speech Classification (2026.findings-acl)

Copied to clipboard

Challenge: Unlike image or text classification, speech classification tasks are particularly challenging due to the difficulty in capturing the acoustic, semantic, and contextual representations.
Approach: They propose a dataset pruning method that coarsely filters redundant samples using DBSCAN clustering on Mel-Frequency Cepstral Coefficients (MFCC) features.
Outcome: The proposed method achieves 49.5% improvement in WA on the MEAD dataset and 41.9% reduction in EER on speaker identification tasks.
MMRA: A Benchmark for Evaluating Multi-Granularity and Multi-Image Relational Association Capabilities in Large Visual Language Models (2026.findings-eacl)

Copied to clipboard

Challenge: Current multimodal benchmarks focus on facts within individual images, but neglect associative relations among multiple images.
Approach: They propose a multi-image relational association task and a MMRA benchmark to evaluate LVLMs.
Outcome: The proposed benchmarks show that entity-level multi-image perception tasks pose greater challenges than image-level tasks.
Learn Beyond The Answer: Training Language Models with Reflection for Mathematical Reasoning (2024.emnlp-main)

Copied to clipboard

Challenge: Existing studies focus on *broadening* the training set with data augmentation techniques to maximize such benefits.
Approach: They propose a method that embeds problem reflection into each training instance.
Outcome: The proposed method enhances performance in standard and complex scenarios that require reflective thinking.
OAgents: An Empirical Study of Building Effective Agents (2025.findings-emnlp)

Copied to clipboard

Challenge: a recent study shows that agent research practices are far from standard, rigorous . lack of a standard evaluation protocol makes previous works not reproducible, authors say .
Approach: They conduct an empirical study on the GAIA benchmark to investigate agent design choices . they find that lack of a standard evaluation protocol makes previous works not reproducible .
Outcome: The proposed framework achieves state-of-the-art performance among open-source projects.
Genre Separation Network with Adversarial Training for Cross-genre Relation Extraction (D18-1)

Copied to clipboard

Challenge: Existing methods to extract genre-specific and genre-agnostic features require great human effort.
Approach: They propose to use two encoders to explicitly extract genre-specific and genre-agnostic features.
Outcome: The proposed approach outperforms the state-of-the-art by 1.7% on three distinct genres.
ConceptMath: A Bilingual Concept-wise Benchmark for Measuring Mathematical Reasoning of Large Language Models (2024.findings-acl)

Copied to clipboard

Challenge: ConceptMath evaluates concept-wise mathematical reasoning of Large Language Models (LLMs) Existing benchmarks that evaluate general mathematical reasoning with an average accuracy fail to probe the fine-grained failure modes of mathematical reasoning on specific datasets.
Approach: They introduce a bilingual, fine-grained benchmark that evaluates concept-wise mathematical reasoning of Large Language Models.
Outcome: The proposed benchmarks evaluate concept-wise mathematical reasoning of Large Language Models with concept-based accuracies.
Using Context-to-Vector with Graph Retrofitting to Improve Word Embeddings (2022.acl-long)

Copied to clipboard

Challenge: Contextualized embeddings are expensive and resource-demanding, hence environmentally unfriendly.
Approach: They propose a method to convert contextualized embeddings from pre-trained models into static embeddables using synonym knowledge and weighted vector distribution.
Outcome: The proposed method outperforms baseline embeddings by a large margin through extrinsic and intrinsic tasks.
ProtoCycle: Reflective Tool-Augmented Planning for Text-Guided Protein Design (2026.findings-acl)

Copied to clipboard

Challenge: Recent deep generative models have already shown encouraging * Equal contribution.
Approach: They propose to use generic instruction-tuned LLMs as direct text-to-sequence generators to achieve this goal.
Outcome: Recent studies show that reflection improves sequence quality and alignment while maintaining competitive foldability.
Unsupervised Multi-Granularity Summarization (2022.findings-emnlp)

Copied to clipboard

Challenge: Experimental results confirm the substantial superiority of GranuSum on multi-granularity summarization over strong baselines.
Approach: They propose to rank events by their salience and annotate a benchmark for GranuSum that contains multiple summaries at different granularities for each document cluster.
Outcome: The proposed framework is capable of producing multi-granular summaries in unsupervised manner over strong baselines.
COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning (2025.findings-naacl)

Copied to clipboard

Challenge: Existing datasets for Chinese instruction tuning are not well-aligned with Chinese users’ interaction patterns.
Approach: They propose to use Chinese instruction tuning datasets to improve instruction fine-tuning for Chinese users.
Outcome: The proposed dataset shows that Chinese models achieve competitive performance in diverse benchmarks.
CodeDPO: Aligning Code Models with Self Generated and Verified Source Code (2025.acl-long)

Copied to clipboard

Challenge: Existing training methods for code generation do not improve code correctness and efficiency.
Approach: They propose a framework that integrates preference learning into code generation to improve code correctness and efficiency.
Outcome: The proposed framework improves code correctness and efficiency by integrating preference learning into code generation.
M2RC-EVAL: Massively Multilingual Repository-level Code Completion Evaluation (2025.acl-long)

Copied to clipboard

Challenge: Existing repository-level code completion benchmarks focus on a limited number of languages . existing benchmarks report overall average scores of different languages ignoring fine-grained abilities .
Approach: They propose to use repository-level code completion benchmarks to evaluate general code intelligence abilities across languages for existing code Large Language Models.
Outcome: The proposed benchmarks improve the code completion abilities of existing LLMs by using two types of annotations on the parsed syntax tree.
CoTJudger: A Graph-Driven Framework for Automatic Evaluation of Chain-of-Thought Efficiency and Redundancy in LRMs (2026.findings-acl)

Copied to clipboard

Challenge: Existing evaluations emphasize final accuracy or coarse token counts, and lack automated tools to separate essential logic from structural redundancy.
Approach: They propose a graph-driven framework that quantifies reasoning efficiency by converting free-form CoTs into directed dependency graphs and extracting the Shortest Effective Path needed to reach a correct solution.
Outcome: Evaluating 21 LRMs, the proposed framework quantifies reasoning efficiency by converting free-form CoTs into directed dependency graphs and extracting the Shortest Effective Path (SEP) needed to reach a correct solution.
Easy Guided Decoding in Providing Suggestions for Interactive Machine Translation (2023.acl-long)

Copied to clipboard

Challenge: In order to improve translation efficiency, human translators perform post-editing on machine translations to correct errors.
Approach: They propose to use the parameterized objective function of neural machine translation to deal with the TS problem without additional training.
Outcome: The proposed method improves translation quality by 10.6 BLEU and reduces time overhead by 63.4% on benchmark datasets.
LIME: Less Is More for MLLM Evaluation (2025.findings-acl)

Copied to clipboard

Challenge: Existing MLLM benchmarks and unified evaluation frameworks cannot accurately and efficiently reflect the ability of MLMLs.
Approach: They propose a semi-automated benchmark curated using a pipeline that filters out uninformative samples and eliminates answer leakage by focusing on tasks that require image-based understanding.
Outcome: The proposed benchmark reduces the number of samples by 76% and evaluation time by 77% while it can more effectively distinguish different models’ abilities.
Bi-Tuning with Collaborative Information for Controllable LLM-based Sequential Recommendation (2025.acl-long)

Copied to clipboard

Challenge: Existing approaches to optimize sequential recommendation systems rely on item ID sequences, but they lack collaborative knowledge and limited controllability.
Approach: They propose a simple bi-tuning framework with collaborative information for controllable Large Language Model-based Sequential Recommendation (Laser) they incorporate learnable virtual tokens at prefix and suffix of input text to adapt LLMs with collaborative knowledge .
Outcome: The proposed framework outperforms state-of-the-art recommendations on real-world datasets.
On-Policy Self-Distillation for Efficient Diffusion Language Models with Early-Stage Calibration (2026.findings-acl)

Copied to clipboard

Challenge: Recent studies have demonstrated that masked diffusion models (MDMs) can surpass autoregressive models (ARMs) in various tasks.
Approach: They propose a method to calibrate early token predictions without demonstration data by distilling an unnormalized target distribution into the original model.
Outcome: Experiments on math, planning, and RLHF tasks show that COPSD improves both effectiveness and efficiency, and further enhances performance when combined with supervised fine-tuning.
RL-PLUS: Countering Capability Boundary Collapse of LLMs in Reinforcement Learning with Hybrid-policy Optimization (2026.acl-long)

Copied to clipboard

Challenge: Reinforcement Learning with Verifiable Reward (RLVR) has significantly advanced the complex reasoning abilities of Large Language Models (LLMs).
Approach: They propose a hybrid-policy optimization approach that synergizes internal exploitation with external data to achieve stronger reasoning capabilities.
Outcome: The proposed approach achieves state-of-the-art performance on six math reasoning benchmarks and superior performance on out-of distribution reasoning tasks.
RAP: Efficient Text-Video Retrieval with Sparse-and-Correlated Adapter (2024.findings-acl)

Copied to clipboard

Challenge: Text-Video Retrieval (TVR) aims to align relevant video content with natural language queries.
Approach: They propose to conduct efficient text-video Retrieval with a salient-and-correlated AdaPter . they propose a low-rank modulation module to refine per-image features from frozen CLIP backbone .
Outcome: Experiments on four TVR datasets show that the proposed method performs better than other methods.
MORE-3S:Multimodal-based Offline Reinforcement Learning with Shared Semantic Spaces (2024.lrec-main)

Copied to clipboard

Challenge: Existing approaches to offline reinforcement learning (RL) focus on learning value functions or policy gradients, but they view it as a sequence modeling task.
Approach: They propose a method that integrates multimodal and pre-trained language models to transform offline reinforcement learning into a supervised learning task by integrating state information derived from images and action-related data obtained from text.
Outcome: The proposed approach outperforms baselines on Atari and OpenAI Gym environments while promoting long-term strategic thinking.
SAMULE: Self-Learning Agents Enhanced by Multi-level Reflection (2025.emnlp-main)

Copied to clipboard

Challenge: Modern AI agents rely on Large Language Models (LLMs) as their reasoning engines, but they still face the challenge of generating meaningful reflections due to inadequate error analysis and a reliance on rare successful trajectories.
Approach: They propose a framework for self-learning agents powered by a retrospective language model that generates reflections during inference.
Outcome: The proposed framework outperforms reflection-based baselines on three challenging benchmarks.
Can Large Language Models Detect Errors in Long Chain-of-Thought Reasoning? (2025.acl-long)

Copied to clipboard

Challenge: Recent advances in o1-like models have generated long Chain-of-Thought reasoning steps to improve the reasoning abilities of existing Large Language Models (LLMs).
Approach: They propose a DeltaBench to analyze the quality and effectiveness of o1-like models and measure their ability to detect errors in long COT reasoning.
Outcome: The proposed model can detect errors in long COT reasoning.
OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement (2024.findings-acl)

Copied to clipboard

Challenge: OpenCodeInterpreter-33B provides a high level of performance for code generation, executing, and iterative refinement.
Approach: They propose a family of open-source code systems for generating, executing, and iteratively refining code.
Outcome: The OpenCodeInterpreter-33B performs well on humanEval, MBPP, and EvalPlus benchmarks.
Navigating the OverKill in Large Language Models (2024.acl-long)

Copied to clipboard

Challenge: Recent studies have highlighted a tendency among large language models to refuse to answer benign queries.
Approach: They propose a model-agnostic approach to reduce excessive attention to harmful words like ‘kill’ and a method to decode the next-token predictions by contrastive decoding.
Outcome: The proposed approach reduces the refusal rate by 20% while having little impact on safety.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations