Papers by Yiming Yan
DynaEval: Unifying Turn and Dialogue Level Evaluation (2021.acl-long)
Copied to clipboard
Chen Zhang, Yiming Chen, Luis Fernando D’Haro, Yan Zhang, Thomas Friedrichs, Grandee Lee, Haizhou Li
| Challenge: | Existing evaluation metrics focus on the turn-level quality of a dialogue . a unified framework that holistically considers the quality of the entire dialogue is needed . |
| Approach: | They propose a unified automatic evaluation framework which holistically considers the quality of the entire dialogue. |
| Outcome: | The proposed framework outperforms the state-of-the-art dialogue coherence model and correlates strongly with human judgements across multiple evaluation aspects at both turn and dialogue level. |
Generate, Discriminate and Contrast: A Semi-Supervised Sentence Representation Learning Framework (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing supervised sentence embedding techniques rely on expensive human-annotated sentence pairs as the supervised signals. |
| Approach: | They propose a semi-supervised sentence embedding framework that leverages large-scale unlabeled data. |
| Outcome: | The proposed framework surpasses state-of-the-art methods on four domain adaptation tasks. |
RAM-SD: Retrieval-Augmented Multi-agent framework for Sarcasm Detection (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to sarcastic detection use a uniform reasoning strategy . existing approaches lack a framework to deal with the diverse analytical demands of sarcasm . |
| Approach: | They propose a Retrieval-Augmented Multi-Agent framework for Sarcasm Detection . the framework provides transparent and interpretable reasoning traces . |
| Outcome: | The proposed framework outperforms existing methods on four benchmarks and outperformed the strong GPT-4o+CoC baseline. |
Understanding the Dark Side of LLMs’ Intrinsic Self-Correction (2025.acl-long)
Copied to clipboard
Qingjie Zhang, Di Wang, Haoting Qian, Yiming Li, Tianwei Zhang, Minlie Huang, Ke Xu, Hewu Li, Liu Yan, Han Qiu
| Challenge: | Recent studies show that LLMs’ intrinsic self-correction fails without oracle labels as feedback. |
| Approach: | They propose to use one simple task and three complex tasks with state-of-the-art LLMs like ChatGPT, Llama, and DeepSeek to interpret LLM's intrinsic self-correction. |
| Outcome: | The proposed methods reveal the dark side of LLMs’ intrinsic self-correction for different tasks, especially for those failure cases. |
MIND: Towards Immersive Psychological Healing with Multi-Agent Inner Dialogue (2025.findings-emnlp)
Copied to clipboard
Yujia Chen, Changsong Li, Yiming Wang, Tianjie Ju, Qingqing Xiao, Nan Zhang, Zifan Kong, Peng Wang, Binyu Yan
| Challenge: | Mental health issues are worsening in today’s competitive society, such as depression and anxiety. |
| Approach: | They propose a multi-agent inner dialogue paradigm that provides more immersive psychological healing environments. |
| Outcome: | The proposed paradigm provides more immersive psychological healing environments. |
Analyzing and Evaluating Faithfulness in Dialogue Summarization (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on faithfulness of text summarization have not been conducted on abstractive summarizing. |
| Approach: | They propose a method to evaluate faithfulness of dialogue summarization models by multi-choice questions. |
| Outcome: | The proposed method can facilitate the development of dialogue summarization systems. |
Evo-Attacker: Memory-Augmented Reinforcement Learning for Long-Horizon Tool Attacks on LLM-MAS (2026.acl-long)
Copied to clipboard
| Challenge: | Existing tool attacks are limited by domain specificity or fixed and static templates. |
| Approach: | They propose an attack-based memory-augmented reinforcement learning process that constructs a dynamic attack memory and employs deliberative reasoning to retrieve adversarial patterns. |
| Outcome: | Evo-Attacker outperforms baselines in the long-horizon credit assignment challenge. |
DA-Code: Agent Data Science Code Generation Benchmark for Large Language Models (2024.emnlp-main)
Copied to clipboard
Yiming Huang, Jianwen Luo, Yan Yu, Yitong Zhang, Fangyu Lei, Yifan Wei, Shizhu He, Lifu Huang, Xiao Liu, Jun Zhao, Kang Liu
| Challenge: | DA-Code is a code generation benchmark designed to assess LLMs on agent-based data science tasks. |
| Approach: | They propose a code generation benchmark specifically designed for LLMs on agent-based data science tasks. |
| Outcome: | The benchmark performs better than existing frameworks, but lacks accuracy . it is based on real-world data, and includes examples that cover a wide range of tasks . |
Revisiting Self-training for Few-shot Learning of Language Model (2021.emnlp-main)
Copied to clipboard
| Challenge: | Unlabeled data are useful for few-shot learning of language models. |
| Approach: | They propose a prompt-based few-shot learner that uses unlabeled data to fine-tune language models. |
| Outcome: | The proposed approach outperforms state-of-the-art models on six sentence classification and six sentence-pair classification benchmarking tasks. |
BLEURT Has Universal Translations: An Analysis of Automatic Metrics by Minimum Risk Training (2023.acl-long)
Copied to clipboard
| Challenge: | Automated metrics are used for machine translation, but they are often considered to be black boxes with potential biases that are difficult to detect. |
| Approach: | They analyze automatic metrics from the perspective of their guidance for machine translation training. |
| Outcome: | The proposed measures improve the performance of machine translation models. |