Papers by Ye Lu

47 papers
Optimizing Code Retrieval: High-Quality and Scalable Dataset Annotation through Large Language Models (2024.emnlp-main)

Copied to clipboard

Challenge: Existing methods for code retrieval struggle to balance scalability and annotation quality.
Approach: They propose a method that integrates functions called within the repository and information on third-party APIs to enhance the annotation context.
Outcome: The proposed method improves the annotation context by incorporating functions called within the repository and information on third-party API functionalities.
MusicAgent: An AI Agent for Music Understanding and Generation with Large Language Models (2023.emnlp-demo)

Copied to clipboard

Challenge: MusicAgent integrates numerous music-related tools and an autonomous workflow to address user requirements.
Approach: a new system is built to integrate music-related tools and an autonomous workflow . the system is based on large language models (LLMs) that can be used to organize and decompose requests .
Outcome: the proposed system integrates numerous music-related tools and an autonomous workflow to address user requirements.
FinToolSyn: A forward synthesis Framework for Financial Tool-Use Dialogue Data with Dynamic Tool Retrieval (2026.findings-acl)

Copied to clipboard

Challenge: Existing data synthesis methods rely on static tools to generate queries . this approach fails to capture the implicit, event-driven nature of real-world needs .
Approach: They propose a forward synthesis framework to generate high-quality financial dialogues . they construct a repository of 43,066 tools and synthesize over 148k dialogue instances .
Outcome: Experiments show that models trained on FinToolSyn achieve a 21.06% improvement . the framework is designed to generate high-quality financial dialogues .
ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition (2024.acl-long)

Copied to clipboard

Challenge: Experiments show that ChunkAttention can speed up the self-attention kernel by 3.2-4.8 compared to the start-of-the-art implementation.
Approach: They propose a prefix-aware self-attention module that can detect matching prompt prefixes across multiple requests and share their key/value tensors in memory at runtime.
Outcome: The proposed module can speed up the self-attention kernel by 3.2-4.8 compared to the start-of-the-art implementation, with the length of the system prompt ranging from 1024 to 4096.
AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments (2026.acl-long)

Copied to clipboard

Challenge: Existing benchmarks evaluate agents in simplified, idealized settings, relying on pre-packaged tool interfaces, overlooking critical steps, and assume inputs are clean and fully specified.
Approach: They propose a framework that evaluates language agents in simplified, idealized settings . they show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 .
Outcome: Experiments on 15 proprietary and open-source models show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 .
Experience-driven Multi-turn Reinforcement Learning for GUI Agents (2026.acl-long)

Copied to clipboard

Challenge: GUI agents have demonstrated remarkable progress in automating complex user interface interactions . training such agents for long-horizon tasks remains challenging due to limited rewards and prohibitive costs.
Approach: They propose a method that leverages expert trajectories as environment experiences for on-policy multi-turn training.
Outcome: The proposed method achieves significant gains over the base model with 1K public trajectories as RL experiences . it achieves competitive performance against strong baselines such as UI-TARS-7B and GPT-4o .
GMSA: Enhancing Context Compression via Group Merging and Layer Semantic Alignment (2026.acl-long)

Copied to clipboard

Challenge: Large Language Models (LLMs) have achieved remarkable performance across NLP tasks . however, in long-context scenarios, they face high computational cost and information redundancy.
Approach: They propose an encoder-decoder context compression framework that generates a compact sequence of soft tokens for downstream tasks.
Outcome: Experiments show that GMSA outperforms baselines on multiple long-context question answering and summarization benchmarks while maintaining low end-to-end latency.
RepoAgent: An LLM-Powered Open-Source Framework for Repository-level Code Documentation Generation (2024.emnlp-demo)

Copied to clipboard

Challenge: Xia et al., 2018) demonstrate that a large language model can generate and maintain high-quality code documentation.
Approach: They propose a large language model powered open-source framework for generating, maintaining, and updating code documentation.
Outcome: The proposed framework generates high-quality documentation for the entire project.
Improving Discriminative Capability of Reward Models in RLHF Using Contrastive Learning (2024.emnlp-main)

Copied to clipboard

Challenge: Current methods rely on ranking losses to teach reward model to assess preferences, but they are susceptible to noise and ambiguous data, often failing to deeply understand human intentions.
Approach: They propose a method that incorporates contrastive learning into the reward modeling process to enhance generalization and stabilize the reinforcement learning training process.
Outcome: The proposed method enhances generalization of the reward model, stabilizes the reinforcement learning training process, and improves the final alignment with human preferences.
Interpretable Composition Attribution Enhancement for Visio-linguistic Compositional Understanding (2024.emnlp-main)

Copied to clipboard

Challenge: Despite promising progress, vision-language models still exhibit significant challenges in understanding visio-linguistic concepts beyond object terms.
Approach: They propose a framework that encourages the model to pay greater attention to composition words denoting relationships and attributes within the text.
Outcome: The proposed framework improves the ability to discern intricate details and construct more sophisticated interpretations of combined visual and linguistic elements.
Semi-Supervised Learning for Neural Keyphrase Generation (D18-1)

Copied to clipboard

Challenge: Existing models for keyphrase generation only use labeled data, which is limited to resource-rich domains.
Approach: They propose semi-supervised keyphrase generation methods by leveraging labeled data and large-scale unlabeled samples for learning.
Outcome: The proposed methods outperform state-of-the-art models trained with labeled data and large-scale unlabeled samples for learning.
Visual Evidence Prompting Mitigates Hallucinations in Large Vision-Language Models (2025.acl-long)

Copied to clipboard

Challenge: LVLMs have shown impressive progress by integrating visual perception with linguistic understanding to produce contextually grounded outputs.
Approach: They propose a visual evidence prompting method to mitigate hallucinations in large vision-language models by using small visual models to complement them.
Outcome: The proposed method reduces hallucinations by reducing false activation and enhancing correct ones.
AnaloBench: Benchmarking the Identification of Abstract and Long-context Analogies (2024.emnlp-main)

Copied to clipboard

Challenge: Analogical reasoning is an important part of human communication, says a new study . a benchmark to determine analogical reasoning ability in language models is needed .
Approach: They propose to benchmark analogical reasoning ability in language models by collecting 340 analogies from human writings.
Outcome: The proposed benchmark aims to determine analogical reasoning ability in language models.
MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answering (2025.findings-acl)

Copied to clipboard

Challenge: Text-Centric Visual Question Answering (TEC-VQA) is a text-centric visual task understanding tool.
Approach: They introduce a benchmark that features human expert annotations across 9 languages . they prioritize the text in question-answer pairs while disregarding visual text in images .
Outcome: The proposed benchmarks prioritize the text in question-answer pairs while disregarding visual text in images.
Empowering Large Language Model for Continual Video Question Answering with Collaborative Prompting (2024.emnlp-main)

Copied to clipboard

Challenge: Existing VideoQA models struggle to adapt to new questions or tasks posed by newly available content.
Approach: They propose a continual learning framework that fine-tunes a large language model for a sequence of tasks and integrates specific question constraint prompting, knowledge acquisition prompting and visual temporal awareness prompting.
Outcome: The proposed model achieves 55.14% accuracy on both NExT-QA and DramaQA datasets and 71.24% accuracy for DramaQA.
Jointly Learning Semantic Parser and Natural Language Generator via Dual Information Maximization (P19-1)

Copied to clipboard

Challenge: Semantic parsing aims to transform natural language utterances into formal meaning representations (MRs) whereas an NL generator achieves the reverse, the two tasks are often studied separately.
Approach: They propose a method of dual information maximization to regularize the learning process by matching the joint distributions of p and q of NLs.
Outcome: The proposed method empirically maximizes the variational lower bounds of expected joint distributions of NL and MRs.
Unlocking Multilingual Reasoning Capability of LLMs and LVLMs through Representation Engineering (2026.acl-long)

Copied to clipboard

Challenge: Existing approaches to enhance multilingual reasoning capabilities rely on costly multilingual training or employ prompting with external translation tools.
Approach: They propose a training-free inference-time method to enhance multilingual reasoning capabilities via Representation Engineering without additional training data or tools.
Outcome: The proposed method outperforms existing methods on four reasoning benchmarks in English and Thai and Swahili.
EmoHarbor: Evaluating Personalized Emotional Support by Simulating the User’s Internal World (2026.acl-long)

Copied to clipboard

Challenge: EmoHarbor is an evaluation framework that rewards generic empathetic responses but fails to assess whether the support is genuinely personalized to users’ unique psychological profiles and contextual needs.
Approach: They propose an automated evaluation framework that adopts a User-as-a-Judge paradigm by simulating the user's inner world.
Outcome: The proposed framework decomposes users' internal processes into three specialized roles and defines 10 evaluation dimensions of personalized support quality.
A Fast and High-quality Text-to-Speech Method with Compressed Auxiliary Corpus and Limited Target Speaker Corpus (2024.lrec-main)

Copied to clipboard

Challenge: Existing methods to generate high-quality speech with limited target speaker corpus require extensive training data.
Approach: They propose an auxiliary corpus compression algorithm that reduces the training cost while the naturalness of synthesized speech is not significantly degraded.
Outcome: The proposed method significantly reduces training costs while maintaining the naturalness of synthesized speech.
CLAIM: Mitigating Multilingual Object Hallucination in Large Vision-Language Models with Cross-Lingual Attention Intervention (2025.acl-long)

Copied to clipboard

Challenge: Large Vision-Language Models (LVLMs) have impressive multimodal abilities but remain prone to multilingual object hallucination.
Approach: They propose a cross-lingual attention intervention method to mitigate multilingual object hallucination in LVLMs by aligning attention patterns.
Outcome: The proposed method improves 13.56% (up to 30%) on the POPE and 21.75% on the hallucination subsets across languages.
UNComp: Can Matrix Entropy Uncover Sparsity? — A Compressor Design from an Uncertainty-Aware Perspective (2025.emnlp-main)

Copied to clipboard

Challenge: Deploying large language models (LLMs) for long-context inference remains challenging due to their substantial memory and computational demands.
Approach: They propose an uncertainty-aware framework that leverages truncated matrix entropy to identify areas of low information content.
Outcome: The proposed framework reduces the KV cache size to 4.74% of the original and achieves a 6% speedup.
A Bounding Box is Worth One Token - Interleaving Layout and Text in a Large Language Model for Document Understanding (2025.findings-acl)

Copied to clipboard

Challenge: Existing methods for integrating spatial layouts with text have limitations . existing methods produce overly long text sequences or lack autoregressive traits of LLMs .
Approach: They introduce Interleaving Layout and Text in a Large Language Model (LayTextLLM) they use OCR-derived text and spatial layouts to integrate with LLMs for document understanding .
Outcome: The proposed model shows an increase in performance in KIE and VQA tasks.
From Generic Empathy to Personalized Emotional Support: A Self-Evolution Framework for User Preference Alignment (2025.findings-emnlp)

Copied to clipboard

Challenge: Large Language Models (LLMs) show great potential for expressing empathy, but often deliver generic responses that fail to address users’ specific needs.
Approach: They propose a self-evolution framework to help LLMs improve their responses to better align with users’ implicit preferences concerning personality, emotional state, and specific context.
Outcome: The proposed model significantly improves the model's performance in emotional support, reducing unhelpful responses and minimizing discrepancies between user preferences and model outputs.
RealHiTBench: A Comprehensive Realistic Hierarchical Table Benchmark for Evaluating LLM-Based Table Analysis (2025.findings-acl)

Copied to clipboard

Challenge: Existing benchmarks for large language models focus on simple, flat table structures.
Approach: They propose a benchmark to evaluate the performance of both Large Language Models and Multimodal LLMs across a variety of input formats for complex tabular data, including LaTeX, HTML, and PNG.
Outcome: The proposed benchmark evaluates the performance of LLMs and Multimodal LLM models across a variety of input formats for complex tabular data, including LaTeX, HTML, and PNG.
MolErr2Fix: Benchmarking LLM Trustworthiness in Chemistry via Modular Error Detection, Localization, Explanation, and Correction (2025.emnlp-main)

Copied to clipboard

Challenge: Large Language Models (LLMs) have shown growing potential in molecular sciences, but they often produce chemically inaccurate descriptions and struggle to recognize or justify potential errors.
Approach: They propose a benchmark to assess LLMs on error detection and correction in molecular descriptions.
Outcome: The proposed benchmark targets LLMs on error detection and correction in molecular descriptions.
Towards Quantifiable Dialogue Coherence Evaluation (2021.acl-long)

Copied to clipboard

Challenge: Existing automatic dialogue coherence evaluation metrics are expensive and high-latency, which cannot meet the requirements of a dialogue system.
Approach: They propose a framework to train a quantifiable dialogue coherence metric that can reflect actual human rating standards.
Outcome: Experimental results show that the model trained by QuantiDCE presents stronger correlations with human judgements than the other state-of-the-art metrics.
News2vec: News Network Embedding with Subnode Information (D19-1)

Copied to clipboard

Challenge: Existing approaches to embed news as vectors do not integrate features and inter-textual knowledge of news.
Approach: They propose a model that integrates news features and inter-textual knowledge into a dense vector representation.
Outcome: The proposed model can be used to represent news as a dense vector . it is compared with existing models on stock movement prediction and news recommendation tasks .
SweetieChat: A Strategy-Enhanced Role-playing Framework for Diverse Scenarios Handling Emotional Support Agent (2025.coling-main)

Copied to clipboard

Challenge: Large Language Models (LLMs) have demonstrated promising potential in providing empathetic support during interactions, but their responses are often verbose or overly formulaic, failing to adequately address the diverse emotional support needs of real-world scenarios.
Approach: They propose a strategy-enhanced role-playing framework that emulates real-world interactions and a dataset that is used to develop an emotional support agent.
Outcome: The proposed framework emulates real-world interactions and promotes a broader range of dialogues and Emotional Support Agent training.
DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environments (2025.emnlp-main)

Copied to clipboard

Challenge: Large Language Models (LLMs) with web search capabilities show significant potential for deep research.
Approach: They introduce a framework for end-to-end training of LLM-based deep research agents . they implement a specialized multi-agent architecture where browsing agents extract relevant information from various webpage structures.
Outcome: The proposed framework improves on open-domain research tasks by 28.9 points over prompt engineering and 7.2 points over RAG-based RL agents.
Spec-o3: A Tool-Augmented Vision-Language Agent for Rare Celestial Object Candidate Vetting via Automated Spectral Inspection (2026.acl-long)

Copied to clipboard

Challenge: Spec-o3 is a tool-augmented vision-language agent that performs astronomer-aligned spectral inspection.
Approach: They propose a tool-augmented vision-language agent that performs astronomer-aligned spectral inspection via interleaved multimodal chain-of-thought reasoning.
Outcome: Spec-o3 outperforms traditional visual inspection methods on rare-object inspection tasks.
MoDification: Mixture of Depths Made Easy (2025.naacl-long)

Copied to clipboard

Challenge: Long-context efficiency is a trending topic in large language model (LLM) serving.
Approach: They propose a method to combine long-context efficiency and mixture of depths to bring down both latency and memory.
Outcome: The proposed method achieves 1.2 speedup in latency and 1.8 reduction in memory compared to original LLMs especially in long-context applications.
LLaMA Pro: Progressive LLaMA with Block Expansion (2024.acl-long)

Copied to clipboard

Challenge: Existing studies have demonstrated that pre-trained LLMs are limited in certain domains, such as programming, mathematics, biomedical, or finance.
Approach: They propose a new post-pretraining method with an expansion of Transformer blocks to tune the expanded blocks using only new corpus, efficiently and effectively improving the model’s knowledge while mitigating forgetting.
Outcome: The proposed model outperforms existing models in programming and math and its instruction-following counterpart LLaMA Pro-8.3B in general tasks, programming, and mathematics.
Detection of Propaganda Using Logistic Regression (D19-50)

Copied to clipboard

Challenge: Various propaganda techniques are used to manipulate peoples perspectives to foster a predetermined agenda.
Approach: They propose a Logistic Regression-based tool that automatically classifies whether a sentence is propagandistic or not.
Outcome: The proposed tool outperforms the baseline on linguistic and semantic features.
Culture-Aware Machine Translation in Large Language Models: Benchmarking and Investigation (2026.acl-long)

Copied to clipboard

Challenge: Existing studies on large language models focus on literal-level translation quality, such as adequacy and fluency.
Approach: They propose a Culture-Aware Novel-Driven Parallel Dataset for Machine Translation and a multi-dimensional evaluation framework for assessing cultural translation quality.
Outcome: The proposed model improves evaluation reliability in LLM-as-a-judge scenarios under culture-aware constraints.
ToW: Thoughts of Words Improve Reasoning in Large Language Models (2025.naacl-long)

Copied to clipboard

Challenge: Unlike other data augmentation methods, thoughts of words (TOW) views next-word prediction as a core reasoning task and injects fine-grained thoughts into pre-training texts.
Approach: They propose a training-time data-augmentation method called thoughts of words (TOW) that injects fine-grained thoughts directly into a next-word prediction task and teaches the model to understand how the observed next word is related to previous contexts.
Outcome: The proposed method reduces model hallucination by 10% and improves reasoning performance by 7% to 9% on average.
x1: Learning to Think Adaptively Across Languages and Cultures (2026.findings-acl)

Copied to clipboard

Challenge: Existing large language models (LLMs) ignore this diversity by reasoning in a single dominant language.
Approach: They propose a family of reasoning models that can adaptively reason in an advantageous language on a per-instance basis.
Outcome: The proposed model can reason in a single dominant language on a per-instance basis.
CogDual: Enhancing Dual Cognition of LLMs via Reinforcement Learning with Implicit Rule-Based Rewards (2025.emnlp-main)

Copied to clipboard

Challenge: Existing approaches to role-playing language models rely on prompt engineering or supervised fine-tuning to emulate character behaviors but neglect the underlying cognitive mechanisms driving these behaviors.
Approach: They propose a novel RPLA adopting a cognize-then-respond reasoning paradigm that leverages dual cognition for more contextually grounded and psychologically coherent responses.
Outcome: The proposed RPLA outperforms baselines and generalizes effectively across diverse role-playing tasks.
Language Model Adaption for Reinforcement Learning with Natural Language Action Space (2024.acl-long)

Copied to clipboard

Challenge: Previous research has focused on reducing the size of the natural language action space due to the combinatorial nature of the language.
Approach: They propose mutual-information regularized policy optimization to reduce the action space by dynamically adjusting the prior provided by the pretrained model.
Outcome: The proposed method improves monotonically on the mutual-information regularized RL objective.
Interpret and Improve In-Context Learning via the Lens of Input-Label Mappings (2025.acl-long)

Copied to clipboard

Challenge: Large language models excel at downstream NLP tasks through in-context learning . however, the internal mechanisms behind ICL remain under-explored .
Approach: They propose a PC patching approach to identify modules where input-label mappings function . they observe and verify that key heads utilize input-labeled mappings to generate target labels for new queries.
Outcome: The proposed approach detects modules where input-label mappings function . it also detects that key heads use the mappings to generate labels for new queries .
RedOne: Revealing Domain-specific LLM Post-Training in Social Networking Services (2025.emnlp-industry)

Copied to clipboard

Challenge: Social networking services (SNS) have experienced rapid growth, which has proposed significant challenges for platform content management and interaction quality improvement.
Approach: They propose a domain-specific LLM to break the performance bottleneck of single-task baselines and establish a comprehensive foundation for social networking services.
Outcome: The proposed model achieves an average improvement of 14.02% across 8 major tasks and 7.56% in bilingual evaluation benchmark, compared with baseline models.
Towards Mitigating Modality Bias in Vision-Language Models for Temporal Action Localization (2026.acl-long)

Copied to clipboard

Challenge: Existing vision-language models overemphasize linguistic priors, leading to modality bias.
Approach: They propose a vision-language aggregation framework that mitigates modality bias in TAL by preserving vision as the dominant signal while adaptively exploiting language only when beneficial.
Outcome: Experiments on THUMOS14 show that the proposed model outperforms state-of-the-art models by up to 3.2% mAP.
CC-Tuning: A Cross-Lingual Connection Mechanism for Improving Joint Multilingual Supervised Fine-Tuning (2025.acl-long)

Copied to clipboard

Challenge: Existing fine-tuning approaches that focus on English-centric training corpora often introduce implicit cross-lingual alignment, overlooking the potential for more profound, latent-level cross-linguistic interactions.
Approach: They propose a multilingual fine-tuning paradigm that explicitly establishes a cross-lingual connection mechanism at the latent level.
Outcome: The proposed model outperforms vanilla SFT and offers a strong latent-level alternative to data-level augmentation methods.
QA‐LIGN: Aligning LLMs through Constitutionally Decomposed QA (2025.findings-emnlp)

Copied to clipboard

Challenge: QA-LIGN decomposes monolithic rewards into interpretable principle-specific evaluations . scalar rewards obscure which objectives drive the training signal .
Approach: a new method decomposes monolithic rewards into interpretable principle-specific evaluations . QA-LIGN reduces attack success rates by up to 68.7% while maintaining a 0.67% false refusal rate .
Outcome: QA-LIGN reduces attack success rates by up to 68.7% while maintaining a 0.67% false refusal rate . the results outperform DPO and GRPO with state-of-the-art reward models given equivalent training .
iPET: An Interactive Emotional Companion Dialogue System with LLM-Powered Virtual Pet World Simulation (2025.acl-demo)

Copied to clipboard

Challenge: Existing approaches to role-playing emotional companion products lack sustained personalization and contextual adaptability, limiting their effectiveness in real-world settings.
Approach: They propose a virtual pet agent that can enhance user engagement through rich, dynamic pet behaviors and interactions tailored to individual preferences.
Outcome: The proposed system has been deployed in a real-world, non-commercial product for 200 days and has demonstrated its effectiveness in practical applications.
TED-TTS: Training-Free Intra-Utterance Emotion and Duration Control for Text-to-Speech Synthesis (2026.acl-long)

Copied to clipboard

Challenge: Existing controllable Text-to-Speech methods limited to inter-utterance-level control . utterance expressiveness remains a challenge in building human-like TTS synthesis systems .
Approach: They propose a training-free controllable framework for pretrained zero-shot TTS to enable intra-utterance emotion and duration expression.
Outcome: The proposed framework achieves state-of-the-art intra-utterance consistency while maintaining baseline-level speech quality.
TrendFact: A Benchmark Towards Hotspot Perception in Automatic Fact-Checking (2026.acl-long)

Copied to clipboard

Challenge: Existing benchmarks lack social metadata and evaluation framework to meet this urgent evaluation needs.
Approach: They propose a benchmark capable of evaluating HPA and three fact-checking tasks.
Outcome: The proposed framework improves HPA and computational efficiency for RLM-driven systems.
Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub (2025.acl-long)

Copied to clipboard

Challenge: Existing approaches lack flexibility to address diverse and ever-evolving user queries in open domains.
Approach: They propose to evaluate LLMs on open-domain knowledge that requires tools to solve diverse and ever-evolving user queries.
Outcome: The proposed system outperforms baselines in the open domain task-solving benchmark.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations