Papers by Dan Hao
Task-Oriented Dialogue as Dataflow Synthesis (2020.tacl-1)
Copied to clipboard
Jacob Andreas, John Bufe, David Burkett, Charles Chen, Josh Clausman, Jean Crawford, Kate Crim, Jordan DeLoach, Leah Dorner, Jason Eisner, Hao Fang, Alan Guo, David Hall, Kristin Hayes, Kellie Hill, Diana Ho, Wendy Iwaszuk, Smriti Jha, Dan Klein, Jayant Krishnamurthy, Theo Lanman, Percy Liang, Christopher H. Lin, Ilya Lintsbakh, Andy McGovern, Aleksandr Nisnevich, Adam Pauls, Dmitrij Petters, Brent Read, Dan Roth, Subhro Roy, Jesse Rusak, Beth Short, Div Slomin, Ben Snyder, Stephon Striplin, Yu Su, Zachary Tellman, Sam Thomson, Andrei Vorobev, Izabela Witoszko, Jason Wolfe, Abby Wray, Yuchen Zhang, Alexander Zotov
| Challenge: | Existing approaches to task-oriented dialogue represent dialogue state as a dataflow graph . microsoft's SMCalFlow dataset features complex dialogues about events, weather, places, and people . |
| Approach: | They propose a dataflow graph-based dialogue agent that maps each user utterance to a program that extends this graph. |
| Outcome: | The proposed framework improves representability and predictability in natural dialogues . it uses dataflow graphs and metacomputation to map user intents to a program . |
ReEval: Automatic Hallucination Evaluation for Retrieval-Augmented Large Language Models via Transferable Adversarial Attacks (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing static benchmarks do not guarantee that models can use the provided evidence for answering, which is essential to avoid hallucination when the required knowledge is new or private. |
| Approach: | They propose to automatically perturb existing static one for dynamic evaluation by using a chatGPT framework and a set of open-domain QA datasets. |
| Outcome: | The proposed framework generates new test cases on two open-domain QA datasets and is human-readable and useful to trigger hallucination in LLMs. |
The Whole Truth and Nothing But the Truth: Faithful and Controllable Dialogue Response Generation with Dataflow Transduction and Constrained Decoding (2023.findings-acl)
Copied to clipboard
Hao Fang, Anusha Balakrishnan, Harsh Jhamtani, John Bufe, Jean Crawford, Jayant Krishnamurthy, Adam Pauls, Jason Eisner, Jacob Andreas, Dan Klein
| Challenge: | In a task-oriented dialogue system, response generation is a conditional language model, but effective dialogue agents must balance fluent generation with stricter constraints. |
| Approach: | They propose a rule-based content selection model that transduces a dialogue agent’s actions and their results into context-free grammars representing the space of contextually acceptable responses. |
| Outcome: | The proposed architecture outperforms both rule-based and learned approaches in human evaluations of fluency, relevance, and truthfulness. |
Improving Temporal Relation Extraction with a Globally Acquired Statistical Resource (N18-1)
Copied to clipboard
| Challenge: | Existing temporal extraction systems that extract temporal relations can be improved by using a resource that provides prior knowledge of the temporal order that events usually follow. |
| Approach: | They propose to use a probabilistic knowledge base acquired in the news domain to extract temporal relations between events from the New York Times articles over a 20-year span. |
| Outcome: | The proposed system and resource are both publicly available. |
AndroidLab: Training and Systematic Benchmarking of Android Autonomous Agents (2025.acl-long)
Copied to clipboard
Yifan Xu, Xiao Liu, Xueqiao Sun, Siyi Cheng, Hao Yu, Hanyu Lai, Shudan Zhang, Dan Zhang, Jie Tang, Yuxiao Dong
| Challenge: | Existing studies on Android agents lack systematic research on open-source and closed-source models. |
| Approach: | They propose a framework for Android agents that includes an operation environment and a reproducible benchmark. |
| Outcome: | The proposed framework lifts the success rate of open-source LLMs and LMMs from 4.59% to 21.50% for LLM and 1.93% to 13.28% for LMM. |
TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questions (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current machine reading comprehension benchmarks have no questions that test temporal phenomena . a new study studies reading comprehension for temporal relations . |
| Approach: | They propose a reading comprehension benchmark built on news snippets and 21k human-generated questions querying temporal relationships. |
| Outcome: | The new reading comprehension benchmark TORQUE achieves an exact-match score of 51% on the test set . the benchmark is built on 3.2k news snippets with 21k human-generated questions . |
FCGCL: Fine- and Coarse-Granularity Contrastive Learning for Speech Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to perform implicit knowledge transfer from machine translation to ST model are difficult because of the task complexity and data scarcity. |
| Approach: | They recommend a method which conducts explicit knowledge transfer from MT to ST model by fine and coarse granularity contrastive learning. |
| Outcome: | The proposed method improves the performance of the end-to-end speech translation model on all 8 languages. |
A Multi-Axis Annotation Scheme for Event Temporal Relations (P18-1)
Copied to clipboard
| Challenge: | Existing temporal relation (TempRel) annotation schemes have low inter-annotator agreements even between experts, suggesting that the current annotation task needs a better definition. |
| Approach: | They propose to annotate temporal relation (TempRel) annotation schemes based on event start-points instead of a conventional 60’s-80’s model. |
| Outcome: | The proposed model improves IAA from the conventional 60’s to 80’s and can be used by crowdsourcing to alleviate labor intensity. |
MetaMixSpeech: Meta Task Augmentation for Low-Resource Speech Recognition (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Meta-learning has proven to be a powerful paradigm for improving speech recognition performance . however, multilingual meta learning also faces challenges such as task overfitting and learner overfit . |
| Approach: | a new method is proposed to augment meta-training tasks with "more data" the method incorporates both support and query augmentations . |
| Outcome: | The proposed method achieves a 6.35% improvement in the word error rate on FLEURS and Common Voice datasets. |
Meta-Adapter for Self-Supervised Speech Models: A Solution to Low-Resource Speech Recognition Challenges (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing self-supervised learning models can learn latent representations from large amounts of unlabeled data, but they are expensive to fine-tune. |
| Approach: | They develop a meta-adapter to obtain meta-initialized parameters for self-supervised models . meta-Adapters show better generalization and extensibility than traditional pretraining methods . |
| Outcome: | Experiments on common voice and FLEURS datasets show Meta-Adapter performs better on low-resource languages . authors show it can be used on 12 low-source languages, but it requires huge computational resources . |
TRACE: Evaluating Execution Efficiency of LLM-Based Code Translation (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have improved the functional correctness of code translation, but execution efficiency remains overlooked. |
| Approach: | They propose a benchmark to explicitly assess execution efficiency in LLM-translated code. |
| Outcome: | The proposed benchmark identifies that execution efficiency is an essential dimension of code translation . the results highlight that correctness and efficiency are often misaligned . |
Joint Reasoning for Temporal and Causal Relations (P18-1)
Copied to clipboard
| Challenge: | a cause must occur earlier than its effect, temporal and causal relations are closely related . a joint inference framework is developed for studying temporal, causal relations . |
| Approach: | They propose a joint inference framework for temporal and causal relations . they use constraints inherent in time and causality to enforce constraints . |
| Outcome: | The proposed framework improves extraction of temporal and causal relations from text. |
A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasoners (2024.emnlp-main)
Copied to clipboard
Bowen Jiang, Yangxinyu Xie, Zhuoqun Hao, Xiaomeng Wang, Tanwi Mallick, Weijie Su, Camillo Taylor, Dan Roth
| Challenge: | a new hypothesis-testing framework is developed to assess whether large language models possess genuine reasoning abilities or primarily depend on token bias. |
| Approach: | They propose a framework to assess whether large language models have genuine reasoning abilities or primarily depend on token bias. |
| Outcome: | The proposed framework outlines a list of hypotheses where token biases are readily identifiable . the results suggest that most LLMs still struggle with logical reasoning . |
IntelliCockpitBench: A Comprehensive Benchmark to Evaluate VLMs for Intelligent Cockpit (2025.findings-acl)
Copied to clipboard
| Challenge: | Visual Question Answering (VQA) is a key task in vehicular systems. |
| Approach: | They propose a benchmark that encompasses diverse automotive scenarios . they use images from front, side, and rear cameras, various road types, weather conditions, and interior views . |
| Outcome: | The proposed benchmark includes images from front, side, and rear cameras, various road types, weather conditions, and interior views. |