Papers by Hardy Hardy
HighRES: Highlight-based Reference-less Evaluation of Summarization (P19-1)
Copied to clipboard
| Challenge: | Existing methods for summarizing documents are inconsistent due to the difficulty of manual evaluation. |
| Approach: | They propose a method where summaries are evaluated by multiple annotators against the source document via manually highlighted salient content. |
| Outcome: | The proposed method improves inter-annotator agreement while highlighting differences among systems. |
Interpretability from the Ground Up: Stakeholder-Centric Design of Automated Scoring in Educational Assessments (2026.findings-acl)
Copied to clipboard
| Challenge: | Despite increasing demand for transparency and interpretability, the field has yet to develop a widely accepted solution for interpretable automated scoring to be used in large-scale real-world assessments. |
| Approach: | They propose to develop four principles of interpretability targeted at assessment stakeholder groups to address the need for transparency and interpretability in automated scoring. |
| Outcome: | The proposed framework outperforms many uninterpretable scoring methods in terms of scoring accuracy and is, on average, within 0.06 QWK of the uninterprétable SOTA. |
PATIENT-𝜓: Using Large Language Models to Simulate Patients for Training Mental Health Professionals (2024.emnlp-main)
Copied to clipboard
Ruiyi Wang, Stephanie Milani, Jamie Chiu, Jiayin Zhi, Shaun Eack, Travis Labrum, Samuel Murphy, Nev Jones, Kate Hardy, Hong Shen, Fei Fang, Zhiyu Chen
| Challenge: | Mental illness remains one of the most critical public health issues. |
| Approach: | They propose a patient simulation framework for cognitive behavior therapy training that uses large language models to act as a simulated therapy patient. |
| Outcome: | The proposed framework improves the skill acquisition and confidence of mental health trainees beyond textbooks, videos, and role-play with non-patients. |
MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria (2025.naacl-long)
Copied to clipboard
Wentao Ge, Shunian Chen, Hardy Chen, Nuo Chen, Junying Chen, Zhihong Chen, Wenya Xie, Shuo Yan, ChenghaoZhu ChenghaoZhu, Ziyue Lin, Dingjie Song, Xidong Wang, Anningzhe Gao, Zhang Zhiyi, Jianquan Li, Xiang Wan, Benyou Wang
| Challenge: | Existing evaluation methodologies for multimodal large language models are limited in evaluating objective queries without considering real-world user experiences. |
| Approach: | They propose to evaluate multimodal large language models with per-sample criteria using potent MLLM as the judge. |
| Outcome: | The proposed evaluation paradigm shows that it can be used to evaluate multimodal large language models with per-sample criteria. |
Knowledge without Wisdom: Measuring Misalignment between LLMs and Intended Impact (2026.acl-long)
Copied to clipboard
| Challenge: | a recent study shows that large language models excel on benchmarks that operationalize knowledge. |
| Approach: | They compare LLM alignment on benchmarks, downstream tasks and intended impact . they find that inter-model behaviors on disparate tasks correlate higher than expert human behaviors on target tasks . |
| Outcome: | The proposed methods show that LLMs perform poorly on learning tasks . the results show that they are poorly aligned with downstream measures of teaching quality . |
Reasoning While Asking: Transforming Reasoning Large Language Models from Passive Solvers to Proactive Inquirers (2026.acl-long)
Copied to clipboard
| Challenge: | Existing reasoning-oriented LLMs lack a blind self-thinking paradigm . current models fail to recognize when their reasoning is underinformed or based on ambiguous user instructions . |
| Approach: | They propose a new reasoning paradigm that transforms LLMs into proactive inquirers that interleave reasoning with clarification. |
| Outcome: | The proposed model outperforms baseline models on mathematical reasoning, code generation, and document editing. |
ASTPrompter: Preference-Aligned Automated Language Model Red-Teaming to Generate Low-Perplexity Unsafe Prompts (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing red-teaming approaches prioritize high attack success rate, resulting in high-perplexity prompts. |
| Approach: | a new method uses contrastive preference learning to train an attacker to maintain low perplexity while achieving a high attack success rate. |
| Outcome: | ASTPrompter achieves 5.1 times higher attack success rate on Llama-8.1B . low-perplexity attacks are more difficult to filter and more likely to arise during benign usage . |
ViLBench: A Suite for Vision-Language Process Reward Modeling (2025.emnlp-main)
Copied to clipboard
| Challenge: | Despite its advantages, evaluation on PRMs remains less explored, especially in the multimodal domain. |
| Approach: | They propose to benchmark vision large language models as output reward models and process reward models as process-supervised reward models. |
| Outcome: | The proposed model outperforms both ORM and PRM on vision-language benchmarks and achieves an average improvement of 3.3% over standard CoT and up to 2.5% over its untrained counterpart on ViLBench. |
Guided Neural Language Generation for Abstractive Summarization using Abstract Meaning Representation (D18-1)
Copied to clipboard
| Challenge: | Recent work on abstractive summarization has made progress with neural encoder-decoder architectures, but these models lack explicit semantic modeling of the source document and its summary. |
| Approach: | They extend previous work on abstractive summarization using Abstract Meaning Representation (AMR) with a neural language generation stage which they guide using the source document. |
| Outcome: | The proposed approach improves summarization performance by 7.4 and 10.5 points in ROUGE-2 using gold standard AMR parses and parses obtained from an off-the-shelf parser respectively. |
“All that Glitters”: Techniques for Evaluations with Unreliable Model and Human Annotations (2025.findings-naacl)
Copied to clipboard
| Challenge: | Using standard metrics in the presence of poor labels masks label and model quality . evaluation techniques accounting for unreliable labels reveal important flaws, including spurious correlations and nonrandom racial biases . |
| Approach: | They analyze human labels, GPT model ratings, and transformer encoder model ratings . they show that standard metrics in the presence of poor labels mask label and model quality . |
| Outcome: | The proposed methods mask label and model quality even in the presence of poor models. |