Papers by Zheng Ye

48 papers
Bridge Video and Text with Cascade Syntactic Structure (C18-1)

Copied to clipboard

Challenge: Using LSTM-CSS, we construct basic syntactic structure by completing syntastic structure.
Approach: They propose a video captioning approach that progressively completes syntactic structure by a conditional random field to construct basic syntaktic structure.
Outcome: The proposed method produces natural sentences with 42.3% and 28.5% accuracy compared to state-of-the-art methods.
Optimizing Code Retrieval: High-Quality and Scalable Dataset Annotation through Large Language Models (2024.emnlp-main)

Copied to clipboard

Challenge: Existing methods for code retrieval struggle to balance scalability and annotation quality.
Approach: They propose a method that integrates functions called within the repository and information on third-party APIs to enhance the annotation context.
Outcome: The proposed method improves the annotation context by incorporating functions called within the repository and information on third-party API functionalities.
CLEME2.0: Towards Interpretable Evaluation by Disentangling Edits for Grammatical Error Correction (2025.acl-long)

Copied to clipboard

Challenge: Existing studies have focused on the interpretability of Grammatical Error Correction (GEC) evaluation metrics, but the interpretabilty of these metrics has been neglected.
Approach: They propose a reference-based metric that describes four aspects of GEC systems: hit-correction, wrong-corrections, under-correcties, and over-corrects.
Outcome: The proposed metric reveals critical qualities and locates drawbacks of GEC systems.
Experience-driven Multi-turn Reinforcement Learning for GUI Agents (2026.acl-long)

Copied to clipboard

Challenge: GUI agents have demonstrated remarkable progress in automating complex user interface interactions . training such agents for long-horizon tasks remains challenging due to limited rewards and prohibitive costs.
Approach: They propose a method that leverages expert trajectories as environment experiences for on-policy multi-turn training.
Outcome: The proposed method achieves significant gains over the base model with 1K public trajectories as RL experiences . it achieves competitive performance against strong baselines such as UI-TARS-7B and GPT-4o .
GMSA: Enhancing Context Compression via Group Merging and Layer Semantic Alignment (2026.acl-long)

Copied to clipboard

Challenge: Large Language Models (LLMs) have achieved remarkable performance across NLP tasks . however, in long-context scenarios, they face high computational cost and information redundancy.
Approach: They propose an encoder-decoder context compression framework that generates a compact sequence of soft tokens for downstream tasks.
Outcome: Experiments show that GMSA outperforms baselines on multiple long-context question answering and summarization benchmarks while maintaining low end-to-end latency.
Zero-shot Text Classification via Reinforced Self-training (2020.acl-main)

Copied to clipboard

Challenge: Existing methods to learn from unlabeled data are difficult for zero-shot text classification tasks.
Approach: They propose a self-training based method to efficiently leverage unlabeled data.
Outcome: The proposed method significantly outperforms existing methods in zero-shot text classification tasks on benchmarks and a real-world e-commerce dataset.
LoopCoder: Scaling Code Intelligence via Looped Language Models (2026.findings-acl)

Copied to clipboard

Challenge: Large language models have mastered syntax-level code generation, but complex algorithmic reasoning remains a challenge.
Approach: They propose a recurrent inductive bias that aligns with the recursive nature of programming logic.
Outcome: The proposed model achieves comparable performance to standard dense models with more parameters.
Improving Discriminative Capability of Reward Models in RLHF Using Contrastive Learning (2024.emnlp-main)

Copied to clipboard

Challenge: Current methods rely on ranking losses to teach reward model to assess preferences, but they are susceptible to noise and ambiguous data, often failing to deeply understand human intentions.
Approach: They propose a method that incorporates contrastive learning into the reward modeling process to enhance generalization and stabilize the reinforcement learning training process.
Outcome: The proposed method enhances generalization of the reward model, stabilizes the reinforcement learning training process, and improves the final alignment with human preferences.
CLEME: Debiasing Multi-reference Evaluation for Grammatical Error Correction (2023.emnlp-main)

Copied to clipboard

Challenge: Evaluating the performance of Grammatical Error Correction systems is a challenging task due to its subjectivity.
Approach: They propose a method to evaluate GEC systems in multi-reference evaluation setting . they use consistent edit boundaries to eliminate bias caused by inconsistent edit boundaries .
Outcome: The proposed evaluation metric eliminates bias caused by inconsistent edit boundaries on six English reference sets.
DARM: Distribution-Aware Reward Modeling by Alleviating Biases from Low Preference-Context Dependency Data (2026.acl-long)

Copied to clipboard

Challenge: Existing methods for training reward models are vulnerable to context neglect and degraded accuracy.
Approach: They propose distribution-aware reward modeling that augments the RM objective with a conditional mutual information regularizer that maximizes context and the predicted reward conditioned on the response.
Outcome: The proposed model improves performance in RLHF and improves accuracy in other settings.
TransferTOD: A Generalizable Chinese Multi-Domain Task-Oriented Dialogue System with Transfer Capabilities (2024.emnlp-main)

Copied to clipboard

Challenge: Current datasets cater to user-led systems and are limited to predefined specific scenarios and slots.
Approach: They propose to use a Chinese dialogue dataset to train a model that authentically simulates human-computer dialogues in 30 popular life service scenarios.
Outcome: The proposed model achieves a joint accuracy of 75.09% in out-of-domain evaluations . it also achieves notable abilities in slot filling and questioning .
MAssistant: A Personal Knowledge Assistant for MOOC Learners (D19-3)

Copied to clipboard

Challenge: Massive Open Online Courses (MOOCs) have experienced a rapid development since 2012 . many MOOC platforms have been launched, including Coursera1 , edX2 , and Udacity3 etc.
Approach: They present a personal knowledge assistant system called MAssistant for MOOC learners . MAsistants has a large-scale concept graph built from open data . it also provides a browser extension which interacts with users during video lectures .
Outcome: The proposed system helps users trace the concepts they have learned in MOOCs, and to build their own concept graphs.
Empowering Large Language Model for Continual Video Question Answering with Collaborative Prompting (2024.emnlp-main)

Copied to clipboard

Challenge: Existing VideoQA models struggle to adapt to new questions or tasks posed by newly available content.
Approach: They propose a continual learning framework that fine-tunes a large language model for a sequence of tasks and integrates specific question constraint prompting, knowledge acquisition prompting and visual temporal awareness prompting.
Outcome: The proposed model achieves 55.14% accuracy on both NExT-QA and DramaQA datasets and 71.24% accuracy for DramaQA.
KidsArtBench: Multi-Dimensional Children’s Art Evaluation with Attribute-Aware MLLMs (2026.eacl-long)

Copied to clipboard

Challenge: Multimodal Large Language Models (MLLMs) show impressive capabilities across visual–language tasks, but their capacity to evaluate artistic expression remains limited.
Approach: They propose an attribute-specific multi-LoRA approach where each attribute corresponds to a distinct evaluation dimension in the scoring rubric.
Outcome: The proposed approach increases correlation from 0.468 to 0.653 on Qwen2.5-VL-7B, with the largest gains on perceptual dimensions and narrowed gaps on higher-order attributes.
MMAC: A Multilingual, Multimodal Alignment Framework for Cultural Grounding Evaluation (2026.acl-long)

Copied to clipboard

Challenge: Existing models lack cultural alignment across modalities and languages . a new framework to assess cultural awareness across linguistics and languages is needed .
Approach: They propose a framework that integrates tri-modally aligned cultural benchmarks and a five-dimensional evaluation protocol to assess cross-country awareness disparities.
Outcome: The proposed framework assesses cultural awareness disparities across modalities and languages . it is the first dataset aligned at the input level across text, image, and speech .
A Frustratingly Easy Plug-and-Play Detection-and-Reasoning Module for Chinese Spelling Check (2023.findings-emnlp)

Copied to clipboard

Challenge: Recent years, Chinese Spelling Check (CSC) has been greatly improved by designing task-specific pre-training methods or introducing auxiliary tasks.
Approach: They propose to decompose Chinese Spelling Check into detection, reasoning, and searching subtasks and to train a module that is compatible with existing CSC models.
Outcome: The proposed module can be trained for one model and benefit other models.
Towards Hierarchical Multi-Step Reward Models for Enhanced Reasoning in Large Language Models (2026.findings-acl)

Copied to clipboard

Challenge: Existing Process Reward Models (PRMs) are vulnerable to reward hacking and require expensive, large-scale annotation of reasoning steps.
Approach: They propose a reward model approach which evaluates both individual and consecutive reasoning steps from fine-grained and coarse-grounded level.
Outcome: Empirical results show that the proposed model performs better than existing PRMs and is more robust than existing models.
emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representation (2024.findings-acl)

Copied to clipboard

Challenge: Existing models for speech emotion recognition are not suitable for emotional tasks.
Approach: They propose a universal speech emotion representation model that is pre-trained on open-source emotion data.
Outcome: euphoria2vec outperforms state-of-the-art models and emotion specialist models . it shows consistent improvements among 10 different languages of speech emotion recognition datasets .
AgentThink: A Unified Framework for Tool-Augmented Chain-of-Thought Reasoning in Vision-Language Models for Autonomous Driving (2025.findings-emnlp)

Copied to clipboard

Challenge: Vision-Language Models struggle with hallucinations, inefficient reasoning, and limited real-world validation hinders accurate perception and robust step-by-step reasoning.
Approach: AgentThink integrates Chain-of-Thought reasoning with dynamic, agent-style tool invocation for autonomous driving tasks.
Outcome: Experiments on the DriveLMM-o1 benchmark show AgentThink significantly boosts overall reasoning scores by 53.91% and enhances answer accuracy by 33.54% .
TRIGO: Benchmarking Formal Mathematical Proof Reduction for Generative Language Models (2023.emnlp-main)

Copied to clipboard

Challenge: Automated theorem proving (ATP) benchmarks focus on symbolic inference but rarely involve understanding complex number combination reasoning.
Approach: They propose a benchmark that requires a model to reduce a trigonometric expression with step-by-step proof and evaluates a generative LM’s reasoning ability on formulas and ability to manipulate, group, and factor number terms.
Outcome: The proposed benchmark evaluates a generative LM’s reasoning ability on formulas and ability to manipulate, group, and factor number terms.
UNComp: Can Matrix Entropy Uncover Sparsity? — A Compressor Design from an Uncertainty-Aware Perspective (2025.emnlp-main)

Copied to clipboard

Challenge: Deploying large language models (LLMs) for long-context inference remains challenging due to their substantial memory and computational demands.
Approach: They propose an uncertainty-aware framework that leverages truncated matrix entropy to identify areas of low information content.
Outcome: The proposed framework reduces the KV cache size to 4.74% of the original and achieves a 6% speedup.
Beyond Boundaries: Learning a Universal Entity Taxonomy across Datasets and Languages for Open Named Entity Recognition (2025.coling-main)

Copied to clipboard

Challenge: Current Large Language Models struggle with complex entity taxonomies in open domains and lack NER capabilities.
Approach: They propose a dataset to guide LLMs' generalization in Open NER under a universal entity taxonomy.
Outcome: The proposed model outperforms GPT-4 in 3 out-of-domain benchmarks across 15 datasets and 6 languages.
Rethinking the Roles of Large Language Models in Chinese Grammatical Error Correction (2025.acl-industry)

Copied to clipboard

Challenge: Recent studies have shown that Large Language Models’ performance as correctors on Chinese Grammatical Error Correction (CGEC) remains unsatisfactory due to the challenging nature of the task.
Approach: They propose a training framework EXAM that uses LLMs as explainers to enhance CGEC small models and a novel evaluation method SEE that utilizes LLM as evaluators to bring more reasonable evaluations.
Outcome: The proposed methods improve the performance of LLMs on Chinese Grammatical Error Correction (CGEC) task.
Towards Quantifiable Dialogue Coherence Evaluation (2021.acl-long)

Copied to clipboard

Challenge: Existing automatic dialogue coherence evaluation metrics are expensive and high-latency, which cannot meet the requirements of a dialogue system.
Approach: They propose a framework to train a quantifiable dialogue coherence metric that can reflect actual human rating standards.
Outcome: Experimental results show that the model trained by QuantiDCE presents stronger correlations with human judgements than the other state-of-the-art metrics.
Landmark-Guided Cross-Speaker Lip Reading with Mutual Information Regularization (2024.lrec-main)

Copied to clipboard

Challenge: Lip reading is a process of interpreting silent speech from visual lip movements . but lip reading in cross-speaker scenarios poses a challenging problem due to inter-speech variability .
Approach: They propose to exploit lip landmark-guided visual clues instead of mouth-cropped images as input features.
Outcome: Experimental results show that the proposed approach reduces speaker-specific appearance characteristics in cross-speaker scenarios.
DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environments (2025.emnlp-main)

Copied to clipboard

Challenge: Large Language Models (LLMs) with web search capabilities show significant potential for deep research.
Approach: They introduce a framework for end-to-end training of LLM-based deep research agents . they implement a specialized multi-agent architecture where browsing agents extract relevant information from various webpage structures.
Outcome: The proposed framework improves on open-domain research tasks by 28.9 points over prompt engineering and 7.2 points over RAG-based RL agents.
Sentient Agent as a Judge: Evaluating Higher-Order Social Cognition in Large Language Models (2026.findings-acl)

Copied to clipboard

Challenge: Large language models (LLMs) have evolved from statistical sequence predictors to sophisticated autonomous agents capable of reasoning, planning, and sustaining multi-turn conversa-tions.
Approach: They propose a system that instantiates a "Sentient Agent" that simulates human-like emotional changes and inner thoughts to provide a more realistic evaluation of the model in multi-turn conversations.
Outcome: The proposed framework measures the agent's higher-order social cognition in multi-turn conversations.
HICD: Hallucination-Inducing via Attention Dispersion for Contrastive Decoding to Mitigate Hallucinations in Large Language Models (2025.findings-acl)

Copied to clipboard

Challenge: Large Language Models (LLMs) often generate hallucinations, producing outputs that are contextually inaccurate or factually incorrect.
Approach: They propose a method that selects attention heads crucial to the model's prediction as inducing heads and induces hallucinations by dispersing attention of these inducers.
Outcome: The proposed method significantly improves performance on tasks requiring contextual faithfulness, reading comprehension, and question answering.
Training-free LLM Merging for Multi-task Learning (2025.acl-long)

Copied to clipboard

Challenge: Large Language Models (LLMs) have demonstrated exceptional capabilities across diverse natural language processing tasks.
Approach: They propose a training-free method for unifying different specialized LLMs into a single model using model-wise and layer-wise pruning and scaling.
Outcome: The proposed method outperforms existing merging techniques and surpasses models fine-tuned on combined datasets in most scenarios.
Everyone is unique: Towards Behaviorally Heterogeneous Negotiation Dialogue Systems for Debt Collection (2026.acl-long)

Copied to clipboard

Challenge: Existing models that assume users to be static, rational agents with fixed preferences fail to capture rich behavioral heterogeneity in real-world debt collection scenarios.
Approach: They propose a public persona-enriched debt collection benchmark that highlights behavioral heterogeneity in negotiation.
Outcome: The proposed benchmark outperforms existing models in realistic scenarios using 16 state-of-the-art LLMs.
CausalMACE: Causality Empowered Multi-Agents in Minecraft Cooperative Tasks (2025.findings-emnlp)

Copied to clipboard

Challenge: Existing work on multi-agent collaborative tasks in Minecraft is limited due to inefficiency and limited fault tolerance.
Approach: They propose a framework that incorporates causality to manage dependencies among subtasks.
Outcome: The proposed framework achieves state-of-the-art performance in multi-agent cooperative tasks of Minecraft.
Can LLMs Simulate L2-English Dialogue? An Information-Theoretic Analysis of L1-Dependent Biases (2025.acl-long)

Copied to clipboard

Challenge: Large Language Models (LLMs) can simulate non-native-like English use observed in human second language (L2) learners interfered with by their native first language (N1) knowledge.
Approach: They use large language models to simulate non-native-like English use observed in human second language (L2) learners, and then compare their outputs to real L2 learner data.
Outcome: The proposed models replicate L1-dependent patterns observed in human second language (L2) learners, with distinct influences from various languages.
Towards Imperceptible Document Manipulations against Neural Ranking Models (2023.findings-acl)

Copied to clipboard

Challenge: Current approaches to detect vulnerabilities in neural ranking models often introduce noticeable errors and require a well-imitated surrogate NRM to guarantee the attack effect.
Approach: They propose a framework called Imperceptible DocumEnt Manipulation to produce adversarial documents that are less noticeable to both algorithms and humans.
Outcome: The proposed framework outperforms strong baselines while maintaining fluency and correctness of the target documents.
MixEdit: Revisiting Data Augmentation and Beyond for Grammatical Error Correction (2023.findings-emnlp)

Copied to clipboard

Challenge: Various data augmentation strategies have been proposed to improve GEC models . high-quality parallel data for GEC is not as widely available .
Approach: They propose a data augmentation approach that strategically augments real data by generating pseudo data.
Outcome: The proposed approach significantly improves GEC models on English and Chinese datasets.
GRADE: Automatic Graph-Enhanced Coherence Metric for Evaluating Open-Domain Dialogue Systems (2020.emnlp-main)

Copied to clipboard

Challenge: Existing evaluation metrics only consider surface features or utterance-level semantics, without explicitly considering the fine-grained topic transition dynamics of dialogue flows.
Approach: They propose a graph-enhanced evaluation metric GRADE to evaluate dialogue coherence . GRADE incorporates utterance-level contextualized representations and fine-grained topic-level graph representations to improve communication logic.
Outcome: The proposed evaluation metric outperforms state-of-the-art metrics on measuring diverse dialogue models in terms of Pearson and Spearman correlations with human judgments.
UniRAG: Unified Query Understanding Method for Retrieval Augmented Generation (2025.acl-long)

Copied to clipboard

Challenge: Existing query augmentation methods face knowledge update lag and hallucinations in large language models (LLMs) Existing methods face two key challenges: (1) separation of query augmented and encoding tasks, which hinders information sharing and introduces cumulative errors; (2) difficulty of selecting optimal augmentation strategy for different scenarios.
Approach: They propose a unified framework for query understanding in RAG that integrates internal and external knowledge to enhance query augmentation and encoding tasks.
Outcome: The proposed framework outperforms traditional query augmentation methods in five knowledge-intensive benchmark tasks in both closed and open domain question answering.
A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutions (2026.acl-long)

Copied to clipboard

Challenge: Existing research on reinforcement learning for LLMs under data scarcity has not been unified.
Approach: They propose a top-up hierarchical framework built around three complementary perspectives: data-centric, training-centric and framework-centric.
Outcome: The proposed framework provides a clear conceptual foundation for understanding the design space of data-efficient RL for large language models and to guide researchers working in this emerging area.
ICA-Proto: Iterative Cross Alignment Prototypical Network for Incremental Few-Shot Relation Classification (2023.findings-eacl)

Copied to clipboard

Challenge: Existing methods for relation classification suffer from the scarcity of manually annotated data.
Approach: They propose a novel relation classification model that incorporates query representation into the encoding of novel prototypes and utilizes iteratively to achieve more interaction.
Outcome: The proposed model outperforms the state-of-the-art model on two benchmark datasets.
CL2GEC: A Multi-Discipline Benchmark for Continual Learning in Chinese Literature Grammatical Error Correction (2026.acl-long)

Copied to clipboard

Challenge: Existing CGEC benchmarks for multi-disciplinary writing are limited . continual learning (CL) is a promising solution to handle domain-specific linguistic variation and prevent catastrophic forgetting.
Approach: They propose a Chinese Literature Continual Learning benchmark to evaluate adaptive CGEC across disciplines.
Outcome: The proposed benchmark includes 10,000 human-annotated sentences spanning 10 disciplines, each exhibiting distinct linguistic styles and error patterns.
DopplerBAS: Binaural Audio Synthesis Addressing Doppler Effect (2023.findings-acl)

Copied to clipboard

Challenge: Existing methods for binaural audio synthesis are limited in phase estimation, which is crucial for spatial hearing.
Approach: They propose a method to explicitly address the Doppler effect of the moving speaker . it calculates the radial relative velocity of the speaker in spherical coordinates .
Outcome: The proposed method improves the representative WarpNet and BinauralGrad backbones in phase error metric and reaches a new state of the art (SOTA) it is compared with the current method which is limited in phase estimation .
Position: LLMs Can be Good Tutors in English Education (2025.emnlp-main)

Copied to clipboard

Challenge: Recent efforts to integrate large language models into English education lack adaptability to language learning.
Approach: They argue that large language models can be effective tutors in English education . they encourage interdisciplinary research to explore these roles, fostering innovation and risks .
Outcome: The proposed models can play three critical roles: 1) as data enhancers, 2) as task predictors, 3) as agents, enabling personalized and inclusive education.
mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections (2022.emnlp-main)

Copied to clipboard

Challenge: Existing pre-trained vision-language models suffer from inefficiency and linguistic signal overwhelmed by long visual sequences in cross-modal alignment.
Approach: They propose a vision-language foundation model with cross-modal skip-connections that can be pre-trained end-to-end on large-scale image-text pairs with both discriminative and generative objectives.
Outcome: The proposed model achieves state-of-the-art results on a wide range of vision-language downstream tasks, including image captioning, image-text retrieval, visual grounding and visual question answering.
RealBehavior: A Framework for Faithfully Characterizing Foundation Models’ Human-like Behavior Mechanisms (2023.findings-emnlp)

Copied to clipboard

Challenge: Existing studies on human-like behaviors in foundation models do not verify their faithfulness . a simple application of psychological tools cannot faithfully characterize all human-type behaviors .
Approach: They propose a framework to characterize humanoid behaviors in foundation models . they argue that a simple application of psychological tools cannot faithfully characterize all human-like behaviors .
Outcome: The proposed framework assesses the faithfulness of results based on reproducibility, internal consistency, and generalizability.
Towards Mitigating Modality Bias in Vision-Language Models for Temporal Action Localization (2026.acl-long)

Copied to clipboard

Challenge: Existing vision-language models overemphasize linguistic priors, leading to modality bias.
Approach: They propose a vision-language aggregation framework that mitigates modality bias in TAL by preserving vision as the dominant signal while adaptively exploiting language only when beneficial.
Outcome: Experiments on THUMOS14 show that the proposed model outperforms state-of-the-art models by up to 3.2% mAP.
Beyond Surface Features: Advancing Medical Vision-Language Alignment via Dynamic Evidence-Guided Preference Optimization (2026.acl-long)

Copied to clipboard

Challenge: Existing preference-based methods for medical large vision-Language Models face limitations in medical settings . existing methods are limited by overfitting to superficial cues and pseudo convergence of the preference signal.
Approach: They propose a framework that enables evidence-aware and adaptive preference learning for Med-LVLMs.
Outcome: The proposed framework improves evidence-aware and adaptive preference learning for Med-LVLMs.
TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing (2021.acl-demo)

Copied to clipboard

Challenge: Existing approaches to textual robustness evaluation focus on slightly modifying the input data, which maintains the original meaning and results in a different prediction.
Approach: They propose a multilingual robustness evaluation toolkit for NLP that integrates universal text transformations, task-specific transformations and adversarial attack.
Outcome: The toolkit includes universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analyses.
The Stackelberg Speaker: Optimizing Persuasive Communication in Social Deduction Games (2026.acl-long)

Copied to clipboard

Challenge: Existing approaches focus on information processing and strategy selection, overlooking the significance of persuasive communication in social deduction games.
Approach: They propose a reinforcement learning framework that trains agents to optimize influential utterances for persuasive impact by formalizing turn-based dialogue as a Stackelberg competition .
Outcome: The proposed framework outperforms baselines across four social deduction benchmarks and shows that it is effective in persuasive communication.
LAMB: A Training-Free Method to Enhance the Long-Context Understanding of SSMs via Attention-Guided Token Filtering (2025.acl-short)

Copied to clipboard

Challenge: Recent work attributes performance degradation to an exponential decay in hidden-state memory.
Approach: They propose a token filtering strategy that is training-free and attention-guided . they propose 'LAMB' to preserve critical tokens during inference .
Outcome: The proposed token filtering improves long-context performance by 30.35% over state-of-the-art methods on benchmarks.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations