Papers by Ray Mooney
CaT-Bench: Benchmarking Language Model Understanding of Causal and Temporal Dependencies in Plans (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on reasoning in plans focus on classical problems, simulated environments, or restricted language such as PDDL, but real-world plans cannot be tested to test for correctness and reliability. |
| Approach: | They propose a benchmark question that tests whether a step must necessarily occur before or after another in cooking recipe plans. |
| Outcome: | The proposed question-driven evaluation shows that SOTA LLMs are underwhelming and biased towards predicting dependence more often, but the best F1 result is 0.73. |
Do Images Speak Louder than Words? Investigating the Effect of Textual Misinformation in VLMs (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing studies have shown that Vision-Language Models have robust multimodal reasoning capabilities, but their robustness against textual misinformation remains under-explored. |
| Approach: | They propose to use visual-question-answering (VQA) prompts to generate persuasive prompts that deliberately conflict with visual evidence to test their models. |
| Outcome: | The proposed framework shows that models are vulnerable to misleading prompts, and show an average performance drop of over 48.2% after only one round of persuasive conversation. |
When is Tree Search Useful for LLM Planning? It Depends on the Discriminator (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods to build language agents that can plan efficiently and accurately have not met the needs of advanced planning methods to achieve such improvements. |
| Approach: | They propose to use iterative correction and tree search to solve multi-step problems in a language agent framework with three components: a generator, a discriminator, and a planning method. |
| Outcome: | The proposed methods improve performance on two tasks, text-to-SQL parsing and mathematical reasoning, while using discriminators with 90% accuracy. |
Learning to Describe Solutions for Bug Reports Based on Developer Discussions (2022.findings-acl)
Copied to clipboard
| Challenge: | Software bugs in open-source projects are reported through issue tracking systems like GitHub Issues. |
| Approach: | They propose a method for generating a natural language description of a bug by synthesizing relevant content within the discussion. |
| Outcome: | The proposed system generates a natural language description of the solution by synthesizing relevant content within the discussion. |
Multimodal Contextualized Semantic Parsing from Speech (2024.acl-long)
Copied to clipboard
| Challenge: | Towards this goal, we introduce Semantic Parsing in Contextual Environments (SPICE) task designed to enhance artificial agents’ contextual awareness by integrating multimodal inputs with prior contexts. |
| Approach: | They introduce a task designed to enhance artificial agents’ contextual awareness by integrating multimodal inputs with prior contexts. |
| Outcome: | The proposed task is based on the VG-SPICE dataset and the Audio-Vision Dialogue Scene Parser (AViD-SP) it allows agents to maintain their contextual state within a structured, dense information framework that is scalable and interpretable . |
Impact of Evaluation Methodologies on Code Summarization (2022.acl-long)
Copied to clipboard
| Challenge: | Existing evaluation methodologies for code summarization tasks do not consider timestamps of code and comments. |
| Approach: | They propose a time-segmented evaluation methodology for code summarization that considers timestamps of code and comments during evaluation. |
| Outcome: | The proposed evaluation methodology compares with other evaluation methodologies that have been widely used. |