Papers by Lawrence Hunter
CRAFT Shared Tasks 2019 Overview — Integrated Structure, Semantics, and Coreference (D19-57)
Copied to clipboard
William Baumgartner, Michael Bada, Sampo Pyysalo, Manuel R. Ciosici, Negacy Hailu, Harrison Pielke-Lombardo, Michael Regan, Lawrence Hunter
| Challenge: | CRAFT corpus provides a unique foundation for integrating natural language processing (NLP) tasks involving structure, semantics, and coreference. |
| Approach: | They propose to use the CRAFT corpus to evaluate three fundamental language processing tasks over full-text biomedical articles. |
| Outcome: | The CRAFT corpus provides a unique foundation for integrating natural language processing tasks involving structure, semantics, and coreference. |
Comparing Template-based and Template-free Language Model Probing (2024.eacl-long)
Copied to clipboard
| Challenge: | Template-based and template-based approaches rank models differently except for the top domain-specific models. |
| Approach: | They evaluate 16 different cloze-task language model probing approaches on 10 probing English datasets to answer questions about model rankings and absolute scores. |
| Outcome: | The results show that the template-based and template-free approaches rank models differently except for the top domain-specific models. |
It Is Not About What You Say, It Is About How You Say It: A Surprisingly Simple Approach for Improving Reading Comprehension (2024.findings-acl)
Copied to clipboard
| Challenge: | Experimenting with 9 large language models across 3 datasets, emphasizing the context yields superior results compared to question emphasis. |
| Approach: | They ask: How does the order of inputs affect model performance? |
| Outcome: | Experiments with 9 large language models show that emphasizing the question and context improves model performance. |
MALAMUTE: A Multilingual, Highly-granular, Template-free, Education-based Probing Dataset (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing cloze-style benchmarks for language models lack specific, granular areas of knowledge and often rely on templates that can bias models. |
| Approach: | They propose a multilingual, template-free, and highly granular probing dataset comprising expert-written, peer-reviewed probes from 71 university-level textbooks across three languages. |
| Outcome: | The proposed dataset covers eight domains, each with up to 14 subdomains, further broken down into concepts and concept-based prompts. |
Desiderata For The Context Use Of Question Answering Systems (2024.eacl-long)
Copied to clipboard
| Challenge: | Prior work has uncovered a set of common problems in state-of-the-art context-based question answering systems, such as a lack of attention to the context when it conflicts with a model’s parametric knowledge and a loss of consistency with their answers. |
| Approach: | They propose to examine the desiderata for context-based question answering systems and then compare them to a set of prior work. |
| Outcome: | The proposed models are based on 15 datasets and evaluated on 5 datasets. |