Papers by Dakuo Wang
Are Human Explanations Always Helpful? Towards Objective Evaluation of Human Natural Language Explanations (2023.acl-long)
Copied to clipboard
| Challenge: | Human-annotated labels and explanations are critical for training explainable NLP models. |
| Approach: | They propose a metric that measures the usefulness of an explanation for model performance at both fine-tuning and inference. |
| Outcome: | The proposed metric can evaluate the quality of human-annotated explanations, while Simulatability falls short. |
Extracting Multiple-Relations in One-Pass with Pre-Trained Transformers (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to extract multiple relations from a paragraph require multiple passes over the paragraph. |
| Approach: | They propose a method to extract multiple relations from a paragraph by encoding the paragraph only once. |
| Outcome: | The proposed approach can perform state-of-the-art on the benchmark ACE 2005. |
D2S: Document-to-Slide Generation Via Query-Based Text Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing research efforts to automate the document-to-slide generation process face a critical challenge: no publicly available dataset for training and benchmarking. |
| Approach: | They propose a dataset SciDuet that gathers papers and their corresponding slides from recent years’ NLP and ML conferences. |
| Outcome: | The proposed system outperforms state-of-the-art summarization baselines on both automated ROUGE metrics and qualitative human evaluation. |
Label Sleuth: From Unlabeled Text to a Classifier in a Few Hours (2022.emnlp-demos)
Copied to clipboard
Eyal Shnarch, Alon Halfon, Ariel Gera, Marina Danilevsky, Yannis Katsis, Leshem Choshen, Martin Santillan Cooper, Dina Epelboim, Zheng Zhang, Dakuo Wang
| Challenge: | Label Sleuth is an open source system for labeling and creating text classifiers which does not require coding skills nor machine learning knowledge. |
| Approach: | *Label Sleuth* is an open source system for labeling and creating text classifiers which does not require coding skills nor machine learning knowledge. |
| Outcome: | *Label Sleuth* is an open source system for labeling and creating text classifiers. |
It is AI’s Turn to Ask Humans a Question: Question-Answer Pair Generation for Children’s Story Books (2022.acl-long)
Copied to clipboard
| Challenge: | Existing question answering (QA) techniques are created mainly to answer questions asked by humans, but in educational applications, teachers often need to decide what questions to ask . |
| Approach: | They propose to use a fairytale-themed storybook as input to generate QA pairs that can test a student's comprehension skills. |
| Outcome: | The proposed system outperforms state-of-the-art QAG baseline systems and builds an interactive story-telling application for the future real-world deployment. |
Out-of-Domain Detection for Low-Resource Text Classification Tasks (D19-1)
Copied to clipboard
| Challenge: | Existing methods for OOD detection and ID classification tasks require massive amounts of ID labeled data and no OOD labeles. |
| Approach: | They propose to use OOD-resistant Prototypical Network to detect OOD cases with limited in-domain (ID) training data to solve this task. |
| Outcome: | The proposed solution outperforms state-of-the-art methods in zero-shot OOD detection task while maintaining a competitive performance on ID classification task. |
HAConvGNN: Hierarchical Attention Based Convolutional Graph Neural Network for Code Documentation Generation in Jupyter Notebooks (2021.findings-emnlp)
Copied to clipboard
| Challenge: | In computational notebooks, one documentation in a markdown cell often corresponds to multiple code cells, and these code cells have an inherent structure. |
| Approach: | They propose a new task of code documentation generation for computational notebooks that uses hierarchical attention mechanism to consider code cells and code tokens information when generating documentation. |
| Outcome: | The proposed model outperforms baseline models on a corpus constructed from well-documented Kaggle notebooks. |
Fantastic Questions and Where to Find Them: FairytaleQA – An Authentic Dataset for Narrative Comprehension (2022.acl-long)
Copied to clipboard
Ying Xu, Dakuo Wang, Mo Yu, Daniel Ritchie, Bingsheng Yao, Tongshuang Wu, Zheng Zhang, Toby Li, Nora Bradford, Branda Sun, Tran Hoang, Yisi Sang, Yufang Hou, Xiaojuan Ma, Diyi Yang, Nanyun Peng, Zhou Yu, Mark Warschauer
| Challenge: | Existing QA datasets rarely distinguish fine-grained reading skills, such as the understanding of varying narrative elements. |
| Approach: | They propose to use FairytaleQA to generate 10,580 questions based on 278 children-friendly stories to assess model's fine-grained learning skills. |
| Outcome: | The proposed dataset consists of 10,580 questions derived from 278 children-friendly stories, covering seven types of narrative elements or relations. |
StorySparkQA: Expert-Annotated QA Pairs with Real-World Knowledge for Children’s Story-Based Learning (2024.emnlp-main)
Copied to clipboard
Jiaju Chen, Yuxuan Lu, Shao Zhang, Bingsheng Yao, Yuanzhe Dong, Ying Xu, Yunyao Li, Qianwen Wang, Dakuo Wang, Yuling Sun
| Challenge: | Existing story reading systems fail to capture the nuances of how education experts think when conducting interactive story reading activities. |
| Approach: | They propose to use existing question-answering (QA) datasets to capture experts' annotations and thinking process to construct a story-based annotation framework. |
| Outcome: | The proposed framework captures experts’ annotations and thinking process and can be used to generate 5, 868 expert-annotated QA pairs with real-world knowledge. |
Beyond Labels: Empowering Human Annotators with Natural Language Explanations through a Novel Active-Learning Architecture (2023.findings-emnlp)
Copied to clipboard
Bingsheng Yao, Ishan Jindal, Lucian Popa, Yannis Katsis, Sayan Ghosh, Lihong He, Yuxuan Lu, Shashank Srivastava, Yunyao Li, James Hendler, Dakuo Wang
| Challenge: | Existing low-resource learning techniques focus on label annotation while neglecting the natural language explanation of a data point. |
| Approach: | They propose a novel architecture that leverages an explanation-generation model to produce explanations guided by human explanations and a prediction model that utilizes generated explanations toward prediction faithfully. |
| Outcome: | The proposed architecture produces explanations guided by human explanations, a prediction model that utilizes generated explanations toward prediction faithfully, and a data diversity-based AL sampling strategy that benefits from the explanation annotations. |
Trajectory2Task: Training Robust Tool-Calling Agents with Synthesized Yet Verifiable Data for Complex User Intents (2026.acl-long)
Copied to clipboard
Ziyi Wang, Yuxuan Lu, Yimeng Zhang, Pei Chen, Ziwei Dong, Jing Huang, Jiri Gesi, Xianfeng Tang, Chen Luo, Qun Liu, Yisi Sang, Hanqing Lu, Manling Li, Jin Lai, Dakuo Wang
| Challenge: | Tool-calling agents are increasingly deployed in real-world customer-facing workflows . but most studies on tool-callers focus on idealized settings with general, fixed, and well-specified tasks. |
| Approach: | They propose a tool-calling agent-based data pipeline that converts trajectories into user-facing tasks with controlled intent adaptations. |
| Outcome: | The proposed pipeline can be used to study tool use under three scenarios. |
LLMs for Now, Fine-Tuning for Later: An Ensemble Approach to Data Drift in Domain-Specific Tasks (2026.acl-srw)
Copied to clipboard
Yuxuan Lu, Bingsheng Yao, Shao Zhang, Yisi Sang, Yun Wang, Hansu Gu, Peng Zhang, Tun Lu, Toby Jia-Jun Li, Dakuo Wang
| Challenge: | Deploying machine learning models in domain-specific scenarios is challenged by data drift and the scarcity of expert annotations. |
| Approach: | They propose a system that combines an LLM, an AL-assisted compact model and an automatic switch module to assist the active learning process. |
| Outcome: | The proposed system achieves 96–98% switch accuracy and outperforms both models used alone. |
Large Language Models with Temporal Reasoning for Longitudinal Clinical Summarization and Prediction (2025.findings-emnlp)
Copied to clipboard
Maya Kruse, Shiyue Hu, Nicholas Derby, Yifu Wu, Samantha Stonbraker, Bingsheng Yao, Dakuo Wang, Elizabeth M. Goldberg, Yanjun Gao
| Challenge: | Recent advances in large language models have shown potential in clinical text summarization, but their ability to handle long patient trajectories with multi-modal data spread across time remains underexplored. |
| Approach: | They evaluate open-source large language models, their Retrieval Augmented Generation variants and chain-of-thought prompting on long-context clinical summarization and prediction. |
| Outcome: | The proposed models can synthesize structured and unstructured EHR data while reasoning over temporal coherence. |
Can LLM Agents Simulate Multi-Turn Human Behavior? Evidence from Real Online Customer Behavior Data (2026.acl-long)
Copied to clipboard
Yuxuan Lu, Jing Huang, Yan Han, Bingsheng Yao, Sisong Bei, Yaochen Xie, Yisi Sang, Qi He, Dakuo Wang
| Challenge: | Recent research shows that LLM Agents can generate “believable” human behaviors via prompt-only methods, leaving open questions of whether they can accurately generate step-by-step actions in multi-turn interaction tasks. |
| Approach: | They propose to use shopping data to evaluate LLMs' ability to accurately generate step-by-step actions in a multi-turn interaction task. |
| Outcome: | The proposed model achieves 17.26% action generation accuracy and 33.86% F1 score on final purchase prediction, representing improvements of 5.4% and 13.85% over baselines. |
OPeRA: A Dataset of Observation, Persona, Rationale, and Action for Evaluating LLMs on Human Online Shopping Behavior Simulation (2026.acl-long)
Copied to clipboard
Ziyi Wang, Yuxuan Lu, Wenbo Li, Amirali Amini, Bo Sun, Yakov Bart, Weimin Lyu, Jiri Gesi, Tian Wang, Jing Huang, Yu Su, Upol Ehsan, Malihe Alikhani, Toby Jia-Jun Li, Lydia Chilton, Dakuo Wang
| Challenge: | evaluating LLMs' ability to mimic real user behavior remains an open challenge due to the lack of high-quality, publicly available datasets that capture both the observable actions and the internal reasoning of an actual user. |
| Approach: | They propose a dataset of Observation, Persona, Rationale, and Action collected from real human participants during online shopping sessions. |
| Outcome: | The proposed dataset is the first to evaluate how well current LLMs can accurately simulate the next web action of a specific user. |
Agentic Conversational Search with Contextualized Reasoning via Reinforcement Learning (2026.findings-acl)
Copied to clipboard
Fengran Mo, Yifan Gao, Sha Li, Hansi Zeng, Xin Liu, Zhaoxuan Tan, Xian Li, Jianshu Chen, Dakuo Wang, Meng Jiang
| Challenge: | Existing studies focus on single-turn scenarios, which might lack the ability to handle multi-turn interactions. |
| Approach: | They propose a conversational agent that interleaves search and reasoning across turns and provides tailored rewards towards evolving user goals. |
| Outcome: | The proposed agent interleaves search and reasoning across turns, enabling exploratory and adaptive behaviors learned through reinforcement learning (RL) training with tailored rewards towards evolving user goals. |
GEMv2: Multilingual NLG Benchmarking in a Single Line of Code (2022.emnlp-demos)
Copied to clipboard
Sebastian Gehrmann, Abhik Bhattacharjee, Abinaya Mahendiran, Alex Wang, Alexandros Papangelis, Aman Madaan, Angelina Mcmillan-major, Anna Shvets, Ashish Upadhyay, Bernd Bohnet, Bingsheng Yao, Bryan Wilie, Chandra Bhagavatula, Chaobin You, Craig Thomson, Cristina Garbacea, Dakuo Wang, Daniel Deutsch, Deyi Xiong, Di Jin, Dimitra Gkatzia, Dragomir Radev, Elizabeth Clark, Esin Durmus, Faisal Ladhak, Filip Ginter, Genta Indra Winata, Hendrik Strobelt, Hiroaki Hayashi, Jekaterina Novikova, Jenna Kanerva, Jenny Chim, Jiawei Zhou, Jordan Clive, Joshua Maynez, João Sedoc, Juraj Juraska, Kaustubh Dhole, Khyathi Raghavi Chandu, Laura Perez Beltrachini, Leonardo F . R. Ribeiro, Lewis Tunstall, Li Zhang, Mahim Pushkarna, Mathias Creutz, Michael White, Mihir Sanjay Kale, Moussa Kamal Eddine, Nico Daheim, Nishant Subramani, Ondrej Dusek, Paul Pu Liang, Pawan Sasanka Ammanamanchi, Qi Zhu, Ratish Puduppully, Reno Kriz, Rifat Shahriyar, Ronald Cardenas, Saad Mahamood, Salomey Osei, Samuel Cahyawijaya, Sanja Štajner, Sebastien Montella, Shailza Jolly, Simon Mille, Tahmid Hasan, Tianhao Shen, Tosin Adewumi, Vikas Raunak, Vipul Raheja, Vitaly Nikolaev, Vivian Tsai, Yacine Jernite, Ying Xu, Yisi Sang, Yixin Liu, Yufang Hou
| Challenge: | Evaluations in machine learning rarely use the latest metrics, datasets, or human evaluation in favor of remaining compatible with prior work. |
| Approach: | They propose to use the Generation, Evaluation, and Metrics Benchmark to integrate new evaluation methods into existing evaluations. |
| Outcome: | The proposed evaluation infrastructure bridges the gap between the advantages of leaderboards and in-depth and evolving evaluations by allowing model developers to benefit from each other's work. |
Towards a Progression-Aware Autonomous Dialogue Agent (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent advances in large-scale language modeling and generation have enabled the creation of dialogue agents that exhibit human-like responses in a wide range of conversational scenarios. |
| Approach: | They propose a framework in which dialogue agents can evaluate the progression of a conversation toward or away from desired outcomes and use this signal to inform planning for subsequent responses. |
| Outcome: | The proposed framework evaluates the progression of a conversation toward or away from desired outcomes and uses this signal to inform planning for subsequent responses. |
Educational Question Generation of Children Storybooks via Question Type Distribution Learning and Event-centric Summarization (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods to generate educational questions of fairytales or storybooks are difficult to implement due to adults lacking the skills or time to integrate such interactive opportunities. |
| Approach: | They propose a question generation method that first learns the question type distribution of an input story paragraph, and then summarizes salient events which can be used to generate high-cognitive-demand questions. |
| Outcome: | The proposed method performs well on automatic and human evaluation metrics on a newly proposed educational question-answering dataset FairytaleQA. |
Context-Aware Conversation Thread Detection in Multi-Party Chat (D19-1)
Copied to clipboard
| Challenge: | In multi-party chat, it is common for multiple conversations to occur concurrently . a new model that automatically disentangles conversation threads is proposed . |
| Approach: | They propose a Context-Aware Thread Detection model that automatically disentangles conversation threads in chat logs. |
| Outcome: | The proposed model outperforms state-of-the-art models on four real-world chat logs. |
A Word is Worth A Thousand Dollars: Adversarial Attack on Tweets Fools Stock Prediction (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing models are vulnerable to adversarial attacks, but their vulnerability is underexplored. |
| Approach: | They propose to concatenate a perturbed but semantically similar tweet into a model that fools stock prediction models. |
| Outcome: | The proposed method achieves consistent success rates and causes significant monetary loss in trading simulation by simply concatenating a perturbed but semantically similar tweet. |
More Samples or More Prompts? Exploring Effective Few-Shot In-Context Learning for LLMs with In-Context Sampling (2024.findings-naacl)
Copied to clipboard
Bingsheng Yao, Guiming Chen, Ruishi Zou, Yuxuan Lu, Jiachen Li, Shao Zhang, Yisi Sang, Sijia Liu, James Hendler, Dakuo Wang
| Challenge: | Existing studies on LLM prompting focus on selecting a better set of data samples inside one single prompt input, but why not design and leverage multiple ICL prompts together to further improve the LLM’s performance? |
| Approach: | They propose a low-resource LLM prompting technique to optimize the construction of multiple ICL prompt inputs to produce confident predictions. |
| Outcome: | The proposed technique can produce confident predictions by optimizing the construction of multiple ICL prompt inputs on four NLI datasets and one QA dataset. |
MBTI Personality Prediction for Fictional Characters Using Movie Scripts (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing NLP models cannot predict character's personality types based on text classifications . character comprehension is the cornerstone of understanding stories in psychology and education. |
| Approach: | They propose a benchmark to predict movie character's MBTI or Big 5 personality types based on the narratives of the character. |
| Outcome: | The proposed model outperforms existing models in the task and is more accurate than random guesses. |
Multi-Agent-as-Judge: Aligning LLM-Agent-Based Automated Evaluation with Multi-Dimensional Human Evaluation (2026.acl-long)
Copied to clipboard
Jiaju Chen, Yuxuan Lu, Xiaojie Wang, Huimin Zeng, Jing Huang, Jiri Gesi, Ying Xu, Bingsheng Yao, Dakuo Wang
| Challenge: | Existing "LLM-as-a-judge" evaluation frameworks are limited by persona descriptions and are not generalizable to other tasks. |
| Approach: | They propose a framework that can automatically construct multiple evaluator personas with distinct dimensions from relevant text documents and instantiate LLM agents with the persona. |
| Outcome: | The proposed framework can believably simulate human evaluators . it extracts stakeholders' diverse perspectives from the provided research papers and constructs personas for the agents . |
‘Don’t Get Too Technical with Me’: A Discourse Structure-Based Framework for Automatic Science Journalism (2023.emnlp-main)
Copied to clipboard
| Challenge: | Science journalism is the production of journalistic content covering scientific topics that are not covered in the scientific literature. |
| Approach: | They propose to use a dataset to generate a scientific paper's tuples, a summary snippet and a novel technical framework to integrate a paper' s discourse structure with its metadata to guide generation. |
| Outcome: | The proposed system outperforms baseline methods in elaborating a content plan meaningful for the target audience, simplifying the information selected, and producing a coherent final report in a layman’s style. |
Are Fairy Tales Fair? Analyzing Gender Bias in Temporal Narrative Event Chains of Children’s Fairy Tales (2023.acl-long)
Copied to clipboard
| Challenge: | Social biases and stereotypes are embedded in our culture through their presence in our stories. |
| Approach: | They propose a computational pipeline that automatically extracts a story’s temporal narrative verb-based event chain for each of its characters as well as character attributes such as gender. |
| Outcome: | The proposed framework extracts a story’s verb-based event chain for each of its characters as well as character attributes such as gender. |
On-Policy Self-Distillation for Efficient Diffusion Language Models with Early-Stage Calibration (2026.findings-acl)
Copied to clipboard
Huaisheng Zhu, MingYu Liu, Junze Liu, Zhen Ge, Tian Wang, Jiri Gesi, Dakuo Wang, Weiqi Zhang, Houyu Zhang, Yufan Guo, Xian Li, Bing Yin, Sujay Sanghavi
| Challenge: | Recent studies have demonstrated that masked diffusion models (MDMs) can surpass autoregressive models (ARMs) in various tasks. |
| Approach: | They propose a method to calibrate early token predictions without demonstration data by distilling an unnormalized target distribution into the original model. |
| Outcome: | Experiments on math, planning, and RLHF tasks show that COPSD improves both effectiveness and efficiency, and further enhances performance when combined with supervised fine-tuning. |
Towards a Design Guideline for RPA Evaluation: A Survey of Large Language Model-Based Role-Playing Agents (2025.findings-acl)
Copied to clipboard
| Challenge: | Role-Playing Agents (RPAs) are increasingly popular due to diverse task requirements and agent designs. |
| Approach: | They propose an evidence-based evaluation design guideline for LLM-based RPAs based on agent attributes, task attributes, and evaluation metrics. |
| Outcome: | The proposed evaluation design guideline is based on a systematic review of 1,676 papers published between Jan. 2021 and Dec. 2024. |