Papers by Yang Cui

76 papers
Parameter-free and Accessible Prompt Learning to Enhance Adversarial Robustness for Pre-trained Vision-Language Models (2025.naacl-long)

Copied to clipboard

Challenge: Large pre-trained Vision-Language Models (VLMs) have revolutionized downstream vision-language tasks including classification, object detection, and segmentation.
Approach: They propose to search for text prompts at the word level rather than optimizing continuous textual embeddings to boost adversarial robustness.
Outcome: Experiments show that the proposed method outperforms hand-engineered prompts with average gains of +4.9% and +5.8%.
ReflectDiffu: Reflect between Emotion-intent Contagion and Mimicry for Empathetic Response Generation via a RL-Diffusion Framework (2025.acl-long)

Copied to clipboard

Challenge: Existing models for empathetic dialogue generation neglect the intricate interplay between emotion and intent, leading to suboptimal controllability of empathy.
Approach: They propose a framework that integrates emotion contagion and intent mimicry to enhance empathetic response generation.
Outcome: The proposed framework outperforms existing models in relevance, controllability, and informativeness.
Composable Text Controls in Latent Space with ODEs (2023.emnlp-main)

Copied to clipboard

Challenge: Existing approaches to composable text operations often require plug-and-play . a single LM can perform arbitrary text operation composition in the latent space .
Approach: They propose an efficient approach for composable text operations in the latent space of text . they connect pretrained LMs to the laten space and adapt them to the space .
Outcome: The proposed approach improves on existing methods in the latent space of text.
FuxiTranyu: A Multilingual Large Language Model Trained with Balanced Data (2024.emnlp-industry)

Copied to clipboard

Challenge: Large language models exhibit significant performance discrepancies between high- and low-resource languages.
Approach: They present an open-source multilingual LLM with 8 billion parameters and a multilingual instruction dataset.
Outcome: The proposed model achieves consistent multilingual representations across languages.
Gradient-based Intra-attention Pruning on Pre-trained Language Models (2023.acl-long)

Copied to clipboard

Challenge: Pre-trained language models are computationally expensive and slow in inference due to their large sizes.
Approach: They propose a structured pruning method which combines pruning with knowledge distillation to yield highly effective models.
Outcome: The proposed method outperforms other pruning methods in sparsity regimes while maintaining 93% 99% performance.
Enhancing Grammatical Error Correction Systems with Explanations (2023.acl-long)

Copied to clipboard

Challenge: To help language learners better understand why the GEC system makes a correction, the causes of errors and the corresponding error types are two key factors.
Approach: They propose to annotate large dataset with evidence words and grammatical error types to help language learners better understand corrections.
Outcome: The proposed model can be validated by human evaluation and can be used to help second-language learners decide whether to accept a correction suggestion and understand the associated grammar rule.
FB-Bench: A Fine-Grained Multi-Task Benchmark for Evaluating LLMs’ Responsiveness to Human Feedback (2025.emnlp-main)

Copied to clipboard

Challenge: Existing research focuses on benchmarking LLMs in single-turn dialogues, neglecting the nuanced nature of human feedback within real-world usage scenarios.
Approach: They propose a fine-grained, multi-task benchmark designed to evaluate LLMs’ responsiveness to human feedback under real-world usage scenarios in Chinese.
Outcome: The proposed benchmarks show that human feedback can significantly impact LLMs’ responsiveness in real-world usage scenarios.
Evaluating Open-Domain Dialogues in Latent Space with Next Sentence Prediction and Mutual Information (2023.acl-long)

Copied to clipboard

Challenge: Existing evaluation methods for open-domain dialogues are difficult due to the one-to-many issue of the open- domain dialogues.
Approach: They propose a learning-based automatic evaluation metric which can robustly evaluate open-domain dialogues by augmenting CVAEs with a Next Sentence Prediction objective and employing Mutual Information to model the semantic similarity of text in the latent space.
Outcome: The proposed method can evaluate open-domain dialogues on two open- domain dialogue datasets.
CodeArena: Evaluating and Aligning CodeLLMs on Human Preference (2025.emnlp-main)

Copied to clipboard

Challenge: Code large language models (codeLLMs) focus on synthesizing the correct code snippet, ignoring the alignment with human preferences.
Approach: They propose a benchmark code-based on 40 categories and 44 programming languages to emulate real-world coding tasks.
Outcome: The proposed benchmarks show that open-source code LLMs perform better than open-sourced ones.
Turning the Tide: Repository-based Code Reflection (2025.findings-emnlp)

Copied to clipboard

Challenge: Code large language models (LLMs) enhance programming by understanding and generating code across languages.
Approach: a new benchmark evaluates code understanding and generation in repositories using code large language models.
Outcome: The proposed model improves code understanding and generation in repositories by evaluating 1,888 test cases across 6 programming languages.
CRPO: Confidence-Reward Driven Preference Optimization for Machine Translation (2025.findings-acl)

Copied to clipboard

Challenge: Large language models (LLMs) have shown great potential in natural language processing tasks, but their application to machine translation remains challenging due to pretraining on predominantly English-centric datasets.
Approach: They propose a method that combines reward scores with model confidence to improve model selection for fine-tuning.
Outcome: The proposed method outperforms existing methods in translation accuracy and data efficiency.
PV2TEA: Patching Visual Modality to Textual-Established Information Extraction (2023.findings-acl)

Copied to clipboard

Challenge: Empirical results show up to 11.74% absolute (20.97% relative) increase over unimodal baselines.
Approach: They propose to patch the visual modality to the textual-established attribute in- formation extractor.
Outcome: Empirical results show up to 11.74% absolute (29.9% relative) increase over unimodal baselines.
Direct Value Optimization: Improving Chain-of-Thought Reasoning in LLMs with Refined Values (2025.emnlp-main)

Copied to clipboard

Challenge: Existing offline preference optimization methods rely on preference labels to optimize large language models.
Approach: They propose an offline method for enhancing large language models in reasoning tasks that utilizes value signals at individual reasoning steps.
Outcome: The proposed framework outperforms offline preference optimization techniques by 4% to 6% on math reasoning, commonsense reasoning, and coding tasks.
FinMaster: A Holistic Benchmark for Full-Pipeline Financial Management with Large Language Models (2026.findings-acl)

Copied to clipboard

Challenge: Existing benchmarks lack domain-specific data, realistic workflow-level task design, and standardized workflow- level evaluation.
Approach: a new benchmark evaluates large language models on financial management workflows . the global financial services market is projected to grow to $37 trillion by 2027 .
Outcome: a new benchmark for large language models on financial management workflows reveals critical capability gaps . accuracy drops from 90% on basic tasks to 40% on complex scenarios requiring multi-step reasoning . the global financial services market reached $25.8 trillion in 2022 and is projected to grow to $37 trillion by 2027 .
TextPruner: A Model Pruning Toolkit for Pre-Trained Language Models (2022.acl-demo)

Copied to clipboard

Challenge: Large pre-trained language models have been used for many NLP tasks but computational resources are limited.
Approach: They propose an open-source model pruning toolkit for pre-trained language models . they propose a self-supervised pruning method that can be applied without labeled data.
Outcome: The proposed pruning method reduces model size without retraining the model and speeds up inference speed on the common CPU and GPU devices.
Improving Alignment in LVLMs with Debiased Self-Judgment (2025.findings-emnlp)

Copied to clipboard

Challenge: Existing methods for aligning LVLMs rely on external datasets, human annotations or complex post-processing.
Approach: They propose a method that generates a debiased self-judgment score for LVLMs . this self-evaluation metric is created internally by the model without external resources .
Outcome: The proposed approach outperforms existing methods in reducing hallucinations and safety concerns.
IDOL: Indicator-oriented Logic Pre-training for Logical Reasoning (2023.findings-acl)

Copied to clipboard

Challenge: Existing systems for logical reasoning have surpassed the average performance of humans in many tasks like SQuAD but there is still a long way to go when it comes to logical reasoning.
Approach: They propose an InDicator-Oriented Logic Pre-training task which logically strengthens pre-trained models with the help of 6 types of logical indicators and a logicalally rich dataset.
Outcome: The proposed task achieves state-of-the-art on ReClor and LogiQA, the two most representative benchmarks in logical reasoning MRC.
Investigating Non-local Features for Neural Constituency Parsing (2022.acl-long)

Copied to clipboard

Challenge: Constituency parsers have been able to achieve competitive performance by using local features.
Approach: They propose to inject non-local features into the training process of a local span-based parser by predicting constituent n-gram non-local patterns and ensuring consistency between constituents and local constituents.
Outcome: The proposed method outperforms the self-attentive parser in multi-lingual and zero-shot cross-domain settings.
MathSight: A Benchmark Exploring Have Vision-Language Models Really Seen in University-Level Mathematical Reasoning? (2026.acl-long)

Copied to clipboard

Challenge: Existing benchmarks rarely isolate how much visual information contributes to reasoning . a growing collection of benchmarks has catalyzed rapid progress in multimodal reasoning - but how much it contributes remains unclear .
Approach: They propose a university-level multimodal mathematical reasoning benchmark to quantify the effect of visual input.
Outcome: The proposed benchmark disentangles and quantifies the effect of visual input on multimodal reasoning models.
LASA: Language-Agnostic Semantic Alignment at the Semantic Bottleneck for LLM Safety (2026.acl-long)

Copied to clipboard

Challenge: Large language models (LLMs) have demonstrated better safety performance in high-resource languages than in low-resourced languages.
Approach: They propose language-agnostic semantic alignment (LASA) which anchors safety alignment directly in semantic bottlenecks.
Outcome: The proposed approach significantly improves safety across all languages: average attack success rate drops from 24.7% to 2.8% on LLaMA-3.1-8B-Instruct and remains within 3–4% across Qwen2.5 and Qwend3 Instruct models (7B–32B).
Inspecting Unification of Encoding and Matching with Transformer: A Case Study of Machine Reading Comprehension (D19-58)

Copied to clipboard

Challenge: Experimental results show that unified model outperforms other models that treat encoding and matching separately.
Approach: They evaluate a unified model with Transformer layers for machine reading comprehension . they find that the model learns different modeling strategies compared with previous models .
Outcome: The unified model outperforms models with Transformer layers on the machine reading comprehension task.
Towards Database-Free Text-to-SQL Evaluation: A Graph-Based Metric for Functional Correctness (2025.coling-main)

Copied to clipboard

Challenge: Existing metrics for evaluating functional correctness of SQL queries are prone to false positives due to inadequately prepared test databases.
Approach: They propose a graph-based metric that uses a relational operator tree to extract rich semantic information from the logical execution plan of SQL queries and embed it into a diagram.
Outcome: The proposed method eliminates the need for extensive test database preparation and performs graph matching on unseen SQL queries.
Adversarial Training for Machine Reading Comprehension with Virtual Embeddings (2021.starsem-1)

Copied to clipboard

Challenge: Neural networks are vulnerable to adversarial examples that have been mixed with certain perturbations.
Approach: They propose a novel adversarial training method that perturbs the embedding matrix instead of word vectors to differentiate the roles of passages and questions.
Outcome: The proposed method is effective universally and further improves the performance of MRC tasks.
MIND: From Passive Mimicry to Active Reasoning through Capability-Aware Multi-Perspective CoT Distillation (2026.acl-long)

Copied to clipboard

Challenge: Existing approaches restrict students to following a single golden rationale and treat different reasoning paths independently, causing suboptimal performance.
Approach: They propose a capability-adaptive framework that transitions distillation from passive mimicry to active cognitive construction and employ a feedback-driven inertia calibration mechanism to align supervision with the student’s current adaptability.
Outcome: Experiments show that the proposed framework achieves state-of-the-art performance on both in-distribution and out-of distribution benchmarks.
Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal (2024.acl-long)

Copied to clipboard

Challenge: Existing methods to train LLMs on previous training data are not feasible in real-world applications because of catastrophic forgetting.
Approach: They propose a framework that uses the LLM to generate synthetic instances for rehearsal and refine the instance outputs based on the synthetic inputs.
Outcome: The proposed framework achieves superior or comparable performance compared to conventional rehearsal-based approaches while being more data-efficient.
How Should We Enhance the Safety of Large Reasoning Models: An Empirical Study (2026.acl-long)

Copied to clipboard

Challenge: Large Reasoning Models have achieved remarkable success on reasoning-intensive tasks, but their enhanced reasoning capabilities do not translate to improved safety performance.
Approach: They propose to use supervised fine tuning to enhance the safety of Large Reasoning Models.
Outcome: The proposed method improves the safety of large reasoning models on reasoning-intensive tasks.
Guiding not Forcing: Enhancing the Transferability of Jailbreaking Attacks on LLMs via Removing Superfluous Constraints (2025.acl-long)

Copied to clipboard

Challenge: Jailbreaking attacks can effectively induce unsafe behaviors in Large Language Models (LLMs).
Approach: They propose a conceptual framework to elucidate transferability of gradient-based jailbreaking methods . they identify superfluous constraints as significant barriers to improved transferability .
Outcome: The proposed method increases the overall transfer attack success rate (T-ASR) across target and source models with varying safety levels from 18.4% to 50.3% while improving stability and controllability of jailbreak behaviors.
Video-LLaVA: Learning United Visual Representation by Alignment Before Projection (2024.emnlp-main)

Copied to clipboard

Challenge: Existing approaches to visual-language understanding lack unified tokenization for images and videos . lack of unified visual representations makes it difficult to learn multi-modal interactions from poor projection layers.
Approach: They propose to unify visual representation into the language feature space to advance the foundational LLM towards a unified LVLM.
Outcome: The proposed model outperforms Video-ChatGPT on image benchmarks and on 9 image benchmark benchmarks.
LINKAGE: Listwise Ranking among Varied-Quality References for Non-Factoid QA Evaluation via LLMs (2024.findings-emnlp)

Copied to clipboard

Challenge: Non-factoid (NF) question answering is challenging to evaluate due to diverse potential answers and no objective criterion.
Approach: They propose a listwise NFQA evaluation approach that uses Large Language Models to rank candidate answers in a descending list of reference answers sorted by descending quality.
Outcome: The proposed method has higher correlations with human annotations than standard methods.
Augur: Modeling Covariate Causal Associations in Time Series via Large Language Models (2026.acl-long)

Copied to clipboard

Challenge: Large language models (LLMs) have emerged as a promising avenue for time series forecasting . existing approaches face limitations such as marginalized role in model architectures and lack of interpretability.
Approach: They propose a framework that exploits LLM causal reasoning to discover and use directed causal associations among covariates.
Outcome: The proposed model improves predictive accuracy while yielding transparent, traceable reasoning about variable interactions.
CLUE: A Chinese Language Understanding Evaluation Benchmark (2020.coling-main)

Copied to clipboard

Challenge: Existing language evaluation benchmarks for English are limited to English . lack of such benchmarks makes it difficult to replicate success in other languages .
Approach: They introduce a large-scale Chinese language understanding evaluation benchmark . the benchmark uses a set of current state-of-the-art pre-trained Chinese models .
Outcome: The first large-scale Chinese Language Understanding Evaluation (CLUE) benchmark is released . the benchmark evaluates models across a wide range of tasks on original Chinese text . existing language evaluation benchmarks are mostly limited to English .
CINO: A Chinese Minority Pre-trained Language Model (2022.coling-1)

Copied to clipboard

Challenge: Existing multilingual pre-trained language models do not perform well on some low-resource languages.
Approach: They propose a multilingual pre-trained language model for Chinese minority languages . they collect documents from Wikipedia and construct two classification datasets .
Outcome: The proposed model outperforms baseline models on various classification tasks.
Template-Based Named Entity Recognition Using BART (2021.findings-acl)

Copied to clipboard

Challenge: Existing methods for fewshot NER do not make full use of knowledge transfer in NER model parameters.
Approach: They propose a template-based method for NER that treats NER as a language model ranking problem in a sequence-to-sequence framework.
Outcome: The proposed method achieves 92.55% F1 score on the CoNLL03 task and significantly better than fine-tuning BERT 10.88%, 15.34%, and 11.73% F1 scores on the MIT Movie, the ATIS, and the MATLAB task.
Exploring Multimodal Challenges in Toxic Chinese Detection: Taxonomy, Benchmark, and Findings (2025.findings-acl)

Copied to clipboard

Challenge: Recent studies show that character substitutions in toxic Chinese text can confuse state-of-the-art LLMs.
Approach: They propose a taxonomy of 3 perturbation strategies and 8 specific approaches in Chinese text to assess if they can detect perturbed Chinese toxic contents.
Outcome: The proposed model can detect perturbed Chinese text with 8 different approaches . the proposed model is compared with 9 other LLMs from the US and China .
Graph-Reward-SQL: Execution-Free Reinforcement Learning for Text-to-SQL via Graph Matching and Stepwise Reward (2025.findings-emnlp)

Copied to clipboard

Challenge: Existing methods to enhance performance of large language models (LLMs) on Text-to-SQL tasks rely on execution-based or LLM-based reward models.
Approach: They propose a reward model framework for RL-based Text-to-SQL that employs the GMNScore outcome reward model.
Outcome: The proposed reward model outperforms existing reward models on standard benchmarks including Spider and BIRD.
MedTem2.0: Prompt-based Temporal Classification of Treatment Events from Discharge Summaries (2023.acl-srw)

Copied to clipboard

Challenge: Clinical texts contain important temporal information, such as medication start and end dates, appointment dates, and diagnosis dates.
Approach: They propose to use prompt-based learning and fine-tuning to classify temporal relations between treatments and hospitalisation periods in discharge summaries.
Outcome: The proposed method identifies whether a treatment was administered between the time of admission and discharge from the hospital.
A Sentence Cloze Dataset for Chinese Machine Reading Comprehension (2020.coling-main)

Copied to clipboard

Challenge: Using cloze-style reading comprehension, Chinese machine reading comprehension datasets are becoming more and more popular . a new task is proposed to fill the right candidate sentence into the passage with several blanks .
Approach: They propose a Chinese task to fill the right candidate sentence into a passage with blanks . they build a dataset to evaluate the difficulty of the task and make fake candidates .
Outcome: The proposed task fills the right candidate sentence into the passage with blanks . the proposed dataset contains over 100K blanks within over 10K passages based on Chinese narrative stories .
TextBrewer: An Open-Source Knowledge Distillation Toolkit for Natural Language Processing (2020.acl-demos)

Copied to clipboard

Challenge: Large pre-trained language models have hundreds of millions of parameters and take several gigabytes of memory to train and inference.
Approach: They propose an open-source knowledge distillation toolkit designed for natural language processing that provides a set of predefined distillation methods and can be extended with custom code.
Outcome: The proposed method is comparable with or even higher than the public distilled BERT models with similar numbers of parameters.
UniGeM: Unifying Data Selection and Mixing via Geometric Exploration and Mining (2026.findings-acl)

Copied to clipboard

Challenge: Large Language Models (LLMs) scaling is limited by data quality and domain mixing and instance selection are two separate problems.
Approach: They propose a framework that unifies mixing and selection without training proxy models or relying on external reference datasets.
Outcome: The proposed framework achieves 2.0 data efficiency over a random baseline and further improves overall performance compared to SOTA methods in reasoning-heavy evaluations and multilingual generalization.
GigaSpeech 2: An Evolving, Large-Scale and Multi-domain ASR Corpus for Low-Resource Languages with Automated Crawling, Transcription and Refinement (2025.acl-long)

Copied to clipboard

Challenge: GigaSpeech 2 is a large-scale, multi-domain, multilingual speech recognition corpus for low-resource languages.
Approach: They propose a large-scale, multi-domain, multilingual speech recognition corpus for low-resource languages and an automated pipeline for data crawling, transcription, and label refinement.
Outcome: The proposed corpus reduces the word error rate for Thai, Indonesian, and Vietnamese on a realistic YouTube test set by 25% to 40% compared to Whisper large-v3.
InferPilot: Autonomous Inference Attacks Against ML Services With LLM-Based Agents (2026.findings-acl)

Copied to clipboard

Challenge: Inference attacks are important for assessing model's robustness, but their implementation and parameters are challenging for non-experts.
Approach: They propose an autonomous agent capable of conducting inference attacks without human intervention.
Outcome: The proposed agent achieves a 100.0% task completion rate and near-expert attack performance with an average token cost of only 0.627 per run.
Cross-domain Generalization for AMR Parsing (2022.emnlp-main)

Copied to clipboard

Challenge: Abstract Meaning Representation (AMR) parsing aims to predict an AMR graph from textual input.
Approach: They evaluate five representative AMR parsers on five domains and analyze challenges to cross-domain parsing.
Outcome: The proposed method reduces the domain distribution divergence of text and AMR features on two out-of-domain sets.
LayoutLMv2: Multi-modal Pre-training for Visually-rich Document Understanding (2021.acl-long)

Copied to clipboard

Challenge: Existing pre-training tasks for text and layout are effective in visually-rich document understanding tasks.
Approach: They propose to combine pre-training tasks with a multi-modal model to model interaction between text, layout and image in a single multi-module framework.
Outcome: The proposed model outperforms LayoutLM by a large margin on visual-rich document understanding tasks.
CLeVeR: Multi-modal Contrastive Learning for Vulnerability Code Representation (2025.findings-acl)

Copied to clipboard

Challenge: Existing methods for detecting code capture the overall semantics of the code rather than its intrinsic vulnerability-specific semantics.
Approach: They propose an approach that leverages contrastive learning to generate precise vulnerability code representations under the supervision of vulnerability descriptions.
Outcome: The proposed approach outperforms state-of-the-art methods in vulnerability detection tasks by 11.85% and 13.61%.
Refining Sample Embeddings with Relation Prototypes to Enhance Continual Relation Extraction (2021.acl-long)

Copied to clipboard

Challenge: Existing methods to extract relationships from texts depend on memory size and replay these memorized samples in subsequent tasks.
Approach: They propose to use a model to extract relations between entities from texts where the samples of different relations are delivered into the model continuously.
Outcome: The proposed model outperforms the state-of-the-art models and avoids catastrophic forgetting.
Don’t Take It Literally: An Edit-Invariant Sequence Loss for Text Generation (2022.naacl-main)

Copied to clipboard

Challenge: Neural text generation models are typically trained by maximizing log-likelihood with the sequence cross entropy (CE) loss.
Approach: They propose an Edit-Invariant Sequence Loss method which computes the matching loss of a target sequence with all n-grams in the generated sequence.
Outcome: The proposed method outperforms the common CE loss and strong baselines on a wide range of tasks.
V-GameGym: Visual Game Generation for Code Large Language Models (2026.findings-acl)

Copied to clipboard

Challenge: Existing code-related benchmarks focus on single modality rather than visual game development.
Approach: They propose a multimodal benchmark for evaluating code large language models in visual game generation that integrates a clustering-based curation methodology and a pipeline for visual code synthesis.
Outcome: The proposed framework assesses code generation and visual game generation using a sandbox environment.
LVP-M3: Language-aware Visual Prompt for Multilingual Multimodal Machine Translation (2022.emnlp-main)

Copied to clipboard

Challenge: Recent advances struggle to train a separate model for each language pair, which is costly and unaffordable when the number of languages increases in the real world.
Approach: They propose to train different MMT models to support translations between different languages.
Outcome: The proposed model is able to handle the above issues by providing a shared semantic space for multiple languages.
FactMix: Using a Few Labeled In-domain Examples to Generalize to Cross-domain Named Entity Recognition (2022.coling-1)

Copied to clipboard

Challenge: Existing approaches for few-shot Named Entity Recognition (NER) are evaluated mainly under in-domain settings, but little is known about how these models perform in cross-domain NER using labeled in- domain examples.
Approach: They propose to use a rationale-centric data augmentation method to improve model generalization ability by allowing model to learn from a few labeled examples in a new target domain.
Outcome: The proposed method improves the performance of cross-domain NER tasks compared to the counterfactual data augmentation and prompt-tuning methods.
FineState-Bench: Benchmarking State-Conditioned Grounding for Fine-grained GUI State Setting (2026.findings-acl)

Copied to clipboard

Challenge: FineState-Bench evaluates whether an agent can correctly ground an instruction to the intended UI control and reach the exact target state.
Approach: They propose a benchmark that evaluates whether an agent can correctly ground an instruction to the intended UI control and reach the exact target state.
Outcome: The proposed benchmark evaluates whether an agent can ground an instruction to the intended UI control and reach the exact target state.
Challenges to Open-Domain Constituency Parsing (2022.findings-acl)

Copied to clipboard

Challenge: Existing findings on cross-domain constituency parsing are only made on a limited number of domains.
Approach: They manually annotate a high-quality constituency treebank containing five domains and analyze challenges to open-domain constituency parsing using a set of linguistic features.
Outcome: The proposed model significantly improves the performance of the proposed model on the domain-variant features.
CFBench: A Comprehensive Constraints-Following Benchmark for LLMs (2025.acl-long)

Copied to clipboard

Challenge: Existing evaluations of Large Language Models (LLMs) focus on fragmented constraints or narrow scenarios, but they overlook the comprehensiveness and authenticity of constraints from the user’s perspective.
Approach: They propose a Chinese Comprehensive Constraints Following Benchmark for LLMs that compiles constraints from real-world instructions and constructs a systematic framework for constraint types.
Outcome: The proposed framework integrates multi-dimensional assessment criteria with requirement prioritization, covering various perspectives of constraints, instructions, and requirement fulfillment.
XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation (2020.emnlp-main)

Copied to clipboard

Challenge: XGLUE provides a benchmark dataset to train large-scale cross-lingual pre-trained models . XCLUE provides 11 diversified tasks that cover both understanding and generation scenarios .
Approach: They introduce a new benchmark dataset to train large-scale cross-lingual pre-trained models using multilingual and bilingual corpora.
Outcome: The proposed dataset is labeled in English and includes only natural language understanding tasks.
Improving Autoformalization Using Direct Dependency Retrieval (2026.acl-long)

Copied to clipboard

Challenge: Existing methods for hallucinate formal dependencies lack scalability and precision to leverage ever-growing public datasets.
Approach: They propose a retrieval-augmented framework based on Direct Dependency Retrieval to generate formal dependencies from natural-language mathematical descriptions and verify their existence via an efficient Suffix Array Check (SAC).
Outcome: The proposed framework outperforms state-of-the-art methods in retrieval precision and recall and can be used to validate formal representations in a public dataset.
EcomScriptBench: A Multi-task Benchmark for E-commerce Script Planning via Step-wise Intention-Driven Product Association (2025.acl-long)

Copied to clipboard

Challenge: Goal-oriented script planning is used by humans to plan for typical activities . however, this capability remains underexplored due to several challenges .
Approach: They propose a framework that enables product-enriched scripts by associating products with each step based on the semantic similarity between the actions and their purchase intentions.
Outcome: The proposed framework can generate product-enriched scripts from 2.4 million scripts . human annotations are conducted to provide gold labels for a sampled subset .
Qwen2.5-xCoder: Multi-Agent Collaboration for Multilingual Code Instruction Tuning (2025.acl-long)

Copied to clipboard

Challenge: Existing methods to train code LLMs view each programming language in isolation . experimental results show that Qwen2.5-xCoder can bridge the gap between different programming languages .
Approach: They propose a framework that allows agents to collaborate to enhance multilingual instruction tuning for code LLMs.
Outcome: Experimental results show that Qwen2.5-xCoder can transfer knowledge efficiently and effectively between languages.
Evaluating Character Understanding of Large Language Models via Character Profiling from Fictional Works (2024.emnlp-main)

Copied to clipboard

Challenge: Recent advances in large language models (LLMs) have catalyzed numerous AI applications, among which role-playing agents (RPAs) are particularly popular.
Approach: They propose to evaluate LLMs' character understanding capability via the character profiling task, i.e., summarizing character profiles from corresponding materials, a widely adopted yet understudied practice for RPA development.
Outcome: The proposed model outperforms existing models and literature summarization methods and proves its ability to understand fictional characters in downstream tasks.
MAGE: Machine-generated Text Detection in the Wild (2024.acl-long)

Copied to clipboard

Challenge: Existing research has focused on evaluating detection methods for specific domains or language models.
Approach: They build a testbed to detect texts from diverse human writings and LLMs using different detection methods.
Outcome: Empirical results show that the top performing detector can identify 84.12% out-of-domain texts generated by a new LLM, indicating the feasibility for application scenarios.
Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Models (2024.findings-acl)

Copied to clipboard

Challenge: Clinical natural language processing (NLP) is a subfield that requires the extraction, analysis, and interpretation of unstructured clinical text.
Approach: They propose a model which infuses knowledge into clinical text generation with LLMs for clinical NLP tasks.
Outcome: The proposed model improves performance across 8 clinical NLP tasks and 18 datasets by 7.7%-8.7% on average.
Benchmarking Robustness of Machine Reading Comprehension Models (2021.findings-acl)

Copied to clipboard

Challenge: Existing benchmarks only evaluate models' robustness under test-time perturbations or adversarial attacks.
Approach: They propose a model-agnostic benchmark to evaluate models' robustness under adversarial attacks.
Outcome: The proposed model-agnostic benchmark evaluates models under four different types of adversarial attacks.
Speech Discrete Tokens or Continuous Features? A Comparative Analysis for Spoken Language Understanding in SpeechLLMs (2025.emnlp-main)

Copied to clipboard

Challenge: Speech Large Language Models (SpeechLLMs) have emerged as dominant speech processing approaches.
Approach: They compare self-supervised learning-based discrete and continuous features . they compare performance across six spoken language understanding-related tasks .
Outcome: The proposed models outperform discrete tokens and continuous features in six spoken language understanding-related tasks.
Rethinking Assessments of Prompt Injection Attacks (2026.findings-acl)

Copied to clipboard

Challenge: Prompt injection attacks are recognized as one of the primary risks faced by LLM-integrated applications in recent years.
Approach: They evaluate prompt injection attacks on LLM-integrated applications across 37 target tasks, 185 injected tasks, 21 attack instructions, and 143,745 queries.
Outcome: The proposed framework provides a solid foundation for assessing vulnerabilities in LLM-integrated applications and evaluating the efficacy of defensive strategies.
GUITester: Enabling GUI Agents for Exploratory Defect Discovery (2026.findings-acl)

Copied to clipboard

Challenge: Exploratory GUI testing is essential for software quality but suffers from high manual costs.
Approach: They propose a framework that decouples navigation from verification via two modules . they propose 143 tasks and a GUITestBench benchmark that features 26 defects .
Outcome: The proposed framework outperforms state-of-the-art benchmarks in 143 tasks and 26 defects.
What Have We Achieved on Text Summarization? (2020.emnlp-main)

Copied to clipboard

Challenge: Existing methods for text summarization have been investigated, but there are still gaps between them and human professionals.
Approach: They analyze 8 major sources of errors on 10 representative summarization models manually.
Outcome: Aiming to gain more understanding of summarization systems with respect to their strengths and limitations on a fine-grained syntactic and semantic level, we use 8 major sources of errors on 10 representative summarizing models.
MirrorCAPTCHA: Wild CAPTCHA, Wild Distribution, Wild Web-based Platform Meet Multimodal LLM Agents (2026.acl-long)

Copied to clipboard

Challenge: Existing agent benchmarks fail to evaluate an agent's real-world capacity to handle CAPTCHA . Existing benchmarks ignore this practical challenge, failing to evaluate agents' ability to handle complex visual CAPTchas.
Approach: They propose a benchmark annotated with Weighted Pass Rate and a new metric to measure agent's ability to handle CAPTCHA.
Outcome: The proposed benchmark outperforms current state-of-the-art closed-source models on mirrorCAPTCHA and achieves 9.4% higher average weighted pass rate and 2.13% higher average Completion degree.
New Terms, New Toxicity: Consensus-based Chinese Neologism Toxicity Detection via Search-Augmented LLMs (2026.acl-long)

Copied to clipboard

Challenge: Neologisms can foster new linguistic consensus by stabilizing shared meanings and usage in common communicative norms.
Approach: They propose a taxonomy that captures the origins and consensus-verification criteria of toxic neologisms . they propose 'SeTox' framework that integrates real-time web context for naeologim detection .
Outcome: The proposed framework outperforms large-scale models in detecting neologism toxicity.
MidPO: Dual Preference Optimization for Safety and Helpfulness in Large Language Models via a Mixture of Experts Framework (2025.findings-emnlp)

Copied to clipboard

Challenge: Recent studies address safety-constrained online and offline preferences optimizations, but offline methods perform poorly in adaptively balancing safety and helpfulness.
Approach: They propose a mixture of experts framework for safety-helpfulness dual Preference Optimization . they combine a single-preference enhanced direct preference optimization approach with a dynamic routing mechanism .
Outcome: The proposed framework outperforms state-of-the-art methods in safety and helpfulness.
Neuro-Symbolic Integration Brings Causal and Reliable Reasoning Proofs (2025.findings-naacl)

Copied to clipboard

Challenge: a new framework for complex reasoning with LLMs is developed to improve reasoning proof accuracy and interpretability.
Approach: They propose to use LLMs to generate search logs that can be interpreted into human-readable reasoning proofs.
Outcome: The proposed framework improves reasoning accuracy but lacks interpretability due to black-box nature of the solvers.
From Completion to Editing: Unlocking Context-Aware Code Infilling via Search-and-Replace Instruction Tuning (2026.acl-long)

Copied to clipboard

Challenge: Fill-in-the-Middle (FIM) models suffer from performance degradation and prohibitive latency.
Approach: They propose a search-and-replace infilling framework that integrates agentic verification and editing into a single-pass inference process.
Outcome: The proposed framework harmonizes completion tasks with the instruction-following priors of Chat LLMs, extending the paradigm from static infilling to dynamic context-aware editing.
Not All Preference Pairs Are Created Equal: A Recipe for Annotation-Efficient Iterative Preference Learning (2024.findings-emnlp)

Copied to clipboard

Challenge: Large language models (LLMs) have shown remarkable capabilities to understand and generate human languages, supporting applications such as question answering, coding, and psychological counseling.
Approach: They propose strategies to save annotation budgets while achieving competitive or even better performances for iterative preference learning.
Outcome: The proposed methods save annotation budgets while achieving better performance.
Detecting Customer Complaint Escalation with Recurrent Neural Networks and Manually-Engineered Features (N19-2)

Copied to clipboard

Challenge: e-commerce companies often have the option of escalating complaints by filing grievances with a government authority . this is detrimental to an ecommerce company, but this problem is challenging to solve by integrating recurrent neural networks with manually-engineered features.
Approach: They propose a model that integrates recurrent neural networks with manually-engineered features to identify cases where the customer expresses such an intent.
Outcome: The proposed model outperforms baseline models and provides better recall and triage for specialized agents.
The Best of Both Worlds: Combining Parallel and Sequential Inference Scaling via Aggregation Fine-Tuning (2026.findings-acl)

Copied to clipboard

Challenge: Empirical results show that AFT-trained models achieve substantial gains with test-time scaling.
Approach: They introduce a supervised fine-tuning paradigm where models synthesize multiple draft responses into a single, refined answer.
Outcome: Empirical results show that AFT-trained models outperform baseline models while eliminating external guidance.
CodeV: Issue Resolving with Visual Data (2025.findings-acl)

Copied to clipboard

Challenge: Large Language Models (LLMs) have expanded to more complex repository-level tasks.
Approach: They propose a first approach to leveraging visual data to enhance the issue-resolving capabilities of Large Language Models (LLMs) they demonstrate the effectiveness of CodeV and provide valuable insights into leveraging visualization to resolve GitHub issues.
Outcome: The proposed approach improves the issue-resolving capabilities of Large Language Models (LLMs) by using visual data.
Making the Best Use of Review Summary for Sentiment Analysis (2020.coling-main)

Copied to clipboard

Challenge: Existing methods for sentiment analysis of user reviews are limited to a few examples.
Approach: They propose a hierarchically-refined attention model that exploits the sentimental distribution of a review and its corresponding summary.
Outcome: The proposed model can make better use of user-written summaries for review sentiment analysis and is more effective compared to existing methods when the user summary is replaced with summary generated by an automatic summarization system.
BERT-BC: A Unified Alignment and Interaction Model over Hierarchical BERT for Response Selection (2024.lrec-main)

Copied to clipboard

Challenge: Recent performance boosting for dialogue response selection task achieved by Cross-Encoder based models is limited and the learned models have poor generalization capability in realistic scenarios.
Approach: They propose a model that combines the representation-based Bi-Encoder and interaction-based Cross-Encoding to achieve better semantic representation.
Outcome: The proposed model can achieve state-of-the-art performance on three benchmark datasets for multi-turn response selection.
CTAP for Chinese:A Linguistic Complexity Feature Automatic Calculation Platform (2022.lrec-1)

Copied to clipboard

Challenge: Existing tools to analyze linguistic complexity are limited and different because of different research purposes.
Approach: They propose to integrate Chinese component into CTAP to analyze linguistic complexity . they propose to use 196 linguistic complex indexes to calculate linguistic characteristics .
Outcome: The proposed indexes are compared with three linguistic complexity tools for Chinese . the proposed index sets include four levels of 196 linguistic complex indexe .

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations