Papers by Michael Daniel
How Much Does Attention Actually Attend? Questioning the Importance of Attention in Pretrained Transformers (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained language models use the attention mechanism to contextualize input inputs . but, we find that it is not as important as thought for pretrained models . |
| Approach: | They propose a probing method that replaces input-dependent attention matrices with constant ones. |
| Outcome: | The proposed method improves performance of pretrained language models without input-dependent attention. |
Toward Implicit Reference in Dialog: A Survey of Methods and Data (2022.aacl-main)
Copied to clipboard
| Challenge: | In natural language, speakers often leave out information that is understood by the other party through the shared context. |
| Approach: | They propose to use omitted entities as implicit references in dialogs to improve language processing. |
| Outcome: | The proposed method is based on a set of experiments which show that the proposed method has a high level of accuracy and is a success. |
CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data (2026.acl-long)
Copied to clipboard
Pedro Ortiz Suarez, Laurie Burchell, Catherine Arnett, Rafael Mosquera, Sara Hincapié Monsalve, Thom Vaughan, Damian Stewart, Malte Ostendorff, Idris Abdulmumin, Vukosi Marivate, Shamsuddeen Hassan Muhammad, Atnafu Lambebo Tonja, Hend Al-Khalifa, Nadia Ghezaiel Hammouda, Verrah Akinyi Otiende, Tack Hwa Wong, Jakhongir Saydaliev, Melika Nobakhtian, Muhammad Ravi Shulthan Habibi, Chalamalasetti Kranti, Carol Muchemi, Khang Nguyen, Faisal Muhammad Adam, Luis Frentzen Salim, Reem Alqifari, Cynthia Jayne Amol, Joseph Marvin Imperial, Ilker Kesen, Ahmad Mustafid, Pavel Stepachev, Leshem Choshen, David Anugraha, Hamada Nayel, Seid Muhie Yimam, Vallerie Alexandra Putra, My Chiffon Nguyen, Azmine Toushik Wasi, Gouthami Vadithya, Rob Van Der Goot, Lanwenn ar C’horr, Karan Dua, Andrew Yates, Mithil Bangera, Yeshil Bangera, Hitesh Laxmichand Patel, Shu Okabe, Fenal Ashokbhai Ilasariya, Dmitry Gaynullin, Genta Indra Winata, Yiyuan Li, Juan Pablo Martínez, Amit Agarwal, Ikhlasul Akmal Hanif, Raia Abu Ahmad, Esther Adenuga, Filbert Aurelian Tjiaranata, Weerayut Buaphet, Michael Anugraha, Sowmya Vajjala, Benjamin L Rice, Azril Hafizi Amirudin, Jesujoba Oluwadara Alabi, Srikant Panda, Yassine Toughrai, Bruhan Kyomuhendo, Daniel Ruffinelli, null Akshata, Manuel Goulão, Ej Zhou, Ingrid Gabriela Franco Ramirez, Cristina Aggazzotti, Konstantin Dobler, Jun Kevin, Quentin Pagès, Nicholas Andrews, Nuhu Ibrahim, Mattes Ruckdeschel, Amr Keleg, Mike Zhang, Casper Rufaro Muziri, Saron Samuel, Sotaro Takeshita, Kun Kerdthaisong, Luca Foppiano, Rasul Dent, Tommaso Green, Ahmad Mustapha Wali, Kamohelo Makaaka, Vicky Feliren, Inshirah Idris, Hande Celikkanat, Abdulhamid Abubakar, Jean Maillard, Benoît Sagot, Thibault Clérice, Kenton Murray, Sarah K. K. Luger
| Challenge: | Language identification (LID) is a fundamental step in curating multilingual corpora. |
| Approach: | They introduce CommonLID, a community-driven, human-annotated LID benchmark for the web domain, covering 109 languages. |
| Outcome: | The proposed benchmark covers 109 languages and shows that existing evaluations overestimate accuracy for many languages in the web domain. |
Let’s Think Frame by Frame with VIP: A Video Infilling and Prediction Dataset for Evaluating Video Chain-of-Thought (2023.emnlp-main)
Copied to clipboard
Vaishnavi Himakunthala, Andy Ouyang, Daniel Rose, Ryan He, Alex Mei, Yujie Lu, Chinmay Sonar, Michael Saxon, William Wang
| Challenge: | Existing studies show vision-language systems can reason about images using natural language, but their capacity for video reasoning remains underexplored. |
| Approach: | They propose to frame video reasoning as the sequential understanding of a small number of keyframes, thereby leveraging the power and robustness of vision-language systems' capacity to reason about images using natural language. |
| Outcome: | The proposed models can generate multiple intermediate keyframes and predict future keyframe, and they perform poorly on GPT-4, GPT-3, and VICUNA. |
Synthetic QA Corpora Generation with Roundtrip Consistency (P19-1)
Copied to clipboard
| Challenge: | Existing methods for generating synthetic question answering corpora are not suitable for QA, but can be constructed from widely available natural text. |
| Approach: | They propose a method for generating synthetic question answering corpora by combining question generation and answer extraction models and filtering the results to ensure roundtrip consistency. |
| Outcome: | The proposed model achieves exact match and F1 at less than 0.1% and 0.4% from human performance on SQuAD2 and NQ. |
Medical Adaptation of Large Language and Vision-Language Models: Are We Making Progress? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Several studies claim that domain-adaptive pretraining improves performance on downstream medical tasks. |
| Approach: | They compare medical LLMs and VLMs against their corresponding base models . they find that medical Lms outperform their base models in 12.1% of cases . |
| Outcome: | The proposed models outperform their base models on medical questions and tasks in 12.1% of cases and reach a tie in 49.8% of cases. |
The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models (2022.emnlp-main)
Copied to clipboard
Eldar Kurtic, Daniel Campos, Tuan Nguyen, Elias Frantar, Mark Kurtz, Benjamin Fineran, Michael Goin, Dan Alistarh
| Challenge: | Pre-trained Transformer models provide robust language representations which can be specialized on various tasks. |
| Approach: | They propose an efficient pruning method based on approximate second-order information that allows pruning weight blocks to be used for pruning. |
| Outcome: | The proposed method is the first to be applied at the BERT scale and significantly pushes the boundaries of the current sparse models with respect to all metrics: model size, inference speed and task accuracy. |
Lightweight Transformers for Conversational AI (2022.naacl-industry)
Copied to clipboard
| Challenge: | Commercial dialogue systems typically require a small footprint and fast execution time, but recent trends are in the other direction, resulting in difficulties in model deployment. |
| Approach: | They build Transformer-based Language Models from scratch on large corpora of conversational data and compare their performance against BERT and other strong baselines on dialogue probing tasks. |
| Outcome: | The proposed model outperforms existing models on dialogue probing tasks and can be fine-tuned on a single consumer GPU card. |
Hard to Be Heard: Phoneme-Level ASR Analysis of Phonologically Complex, Low-Resource Endangered Languages (2026.findings-acl)
Copied to clipboard
| Challenge: | a phoneme-level analysis of automatic speech recognition (ASR) is performed on two low-resource, typologically complex East Caucasian languages. |
| Approach: | They propose a phoneme-level analysis of automatic speech recognition for two East Caucasian languages, Archi and Rutul. |
| Outcome: | The proposed model improves on existing models and improves in low-resource settings. |
Rewarding Coreference Resolvers for Being Consistent with World Knowledge (D19-1)
Copied to clipboard
Rahul Aralikatte, Heather Lent, Ana Valeria Gonzalez, Daniel Hershcovich, Chen Qiu, Anders Sandholm, Michael Ringaard, Anders Søgaard
| Challenge: | Unresolved coreference is a bottleneck for relation extraction systems . a state-of-the-art system may be able to infer the relation using distributional information about the phrase the Sunshine State, but is likely to have limited evidence for the decision that it is coreferential with Florida rather than with Skynyrd. |
| Approach: | They propose to forward coreference input to relation extraction system and reward them for producing triples that are found in knowledge bases. |
| Outcome: | The proposed approach improves over the state-of-the-art by forwarding their input to a relation extraction system and rewarding resolvers for producing triples that are found in knowledge bases. |
Looking Beyond the Surface: A Challenge Set for Reading Comprehension over Multiple Sentences (N18-1)
Copied to clipboard
| Challenge: | Using a dataset of 6,500+ questions, we found that human solvers achieved an F1-score of 88.1%. |
| Approach: | They propose a reading comprehension challenge in which questions can only be answered by taking into account information from multiple sentences. |
| Outcome: | The proposed reading comprehension challenge is based on a reading comprehension dataset with 6,500+ questions and 1000+ paragraphs across 7 domains. |
QED: A Framework and Dataset for Explanations in Question Answering (2021.tacl-1)
Copied to clipboard
Matthew Lamm, Jennimaria Palomaki, Chris Alberti, Daniel Andor, Eunsol Choi, Livio Baldini Soares, Michael Collins
| Challenge: | Existing question answering systems provide no explanation of reasoning that leads to answer . linguistically informed, extensible framework provides explanations in question answering . |
| Approach: | They propose a linguistically informed, extensible framework for explanations in question answering . they propose an expert-annotated dataset of QED explanations built upon a subset of the Natural Questions dataset . |
| Outcome: | The proposed framework improves the ability of untrained raters to spot errors in QA datasets. |
Finding the SWEET Spot: Analysis and Improvement of Adaptive Inference in Low Resource Settings (2023.acl-long)
Copied to clipboard
| Challenge: | Pre-trained Transformer-based language models such as BERT, DeBERTa, and GPT3 have become the go-to tool in NLP. |
| Approach: | They propose an Early-Exit fine-tuning method that assigns each classifier its own set of unique model weights, not updated by other classifiers. |
| Outcome: | The proposed method outperforms Early-Exit and Multi-Model at fast speeds while maintaining comparable scores to Early- Exit at slow speeds. |
GEMv2: Multilingual NLG Benchmarking in a Single Line of Code (2022.emnlp-demos)
Copied to clipboard
Sebastian Gehrmann, Abhik Bhattacharjee, Abinaya Mahendiran, Alex Wang, Alexandros Papangelis, Aman Madaan, Angelina Mcmillan-major, Anna Shvets, Ashish Upadhyay, Bernd Bohnet, Bingsheng Yao, Bryan Wilie, Chandra Bhagavatula, Chaobin You, Craig Thomson, Cristina Garbacea, Dakuo Wang, Daniel Deutsch, Deyi Xiong, Di Jin, Dimitra Gkatzia, Dragomir Radev, Elizabeth Clark, Esin Durmus, Faisal Ladhak, Filip Ginter, Genta Indra Winata, Hendrik Strobelt, Hiroaki Hayashi, Jekaterina Novikova, Jenna Kanerva, Jenny Chim, Jiawei Zhou, Jordan Clive, Joshua Maynez, João Sedoc, Juraj Juraska, Kaustubh Dhole, Khyathi Raghavi Chandu, Laura Perez Beltrachini, Leonardo F . R. Ribeiro, Lewis Tunstall, Li Zhang, Mahim Pushkarna, Mathias Creutz, Michael White, Mihir Sanjay Kale, Moussa Kamal Eddine, Nico Daheim, Nishant Subramani, Ondrej Dusek, Paul Pu Liang, Pawan Sasanka Ammanamanchi, Qi Zhu, Ratish Puduppully, Reno Kriz, Rifat Shahriyar, Ronald Cardenas, Saad Mahamood, Salomey Osei, Samuel Cahyawijaya, Sanja Štajner, Sebastien Montella, Shailza Jolly, Simon Mille, Tahmid Hasan, Tianhao Shen, Tosin Adewumi, Vikas Raunak, Vipul Raheja, Vitaly Nikolaev, Vivian Tsai, Yacine Jernite, Ying Xu, Yisi Sang, Yixin Liu, Yufang Hou
| Challenge: | Evaluations in machine learning rarely use the latest metrics, datasets, or human evaluation in favor of remaining compatible with prior work. |
| Approach: | They propose to use the Generation, Evaluation, and Metrics Benchmark to integrate new evaluation methods into existing evaluations. |
| Outcome: | The proposed evaluation infrastructure bridges the gap between the advantages of leaderboards and in-depth and evolving evaluations by allowing model developers to benefit from each other's work. |
ADVISER: A Toolkit for Developing Multi-modal, Multi-domain and Socially-engaged Conversational Agents (2020.acl-demos)
Copied to clipboard
Chia-Yu Li, Daniel Ortega, Dirk Väth, Florian Lux, Lindsey Vanderlyn, Maximilian Schmidt, Michael Neumann, Moritz Völkel, Pavel Denisov, Sabrina Jenne, Zorica Kacarevic, Ngoc Thang Vu
| Challenge: | Existing toolkits for developing dialog systems are limited to core components and do not support multi-modal processing and social signals. |
| Approach: | They propose to use ADVISER to develop multi-modal dialog agents using multi-text and social signals. |
| Outcome: | The proposed toolkit is flexible, easy to use, and easy to extend for linguists and cognitive scientists, thereby providing a flexible platform for collaborative research. |
Intent Discovery for Enterprise Virtual Assistants: Applications of Utterance Embedding and Clustering to Intent Mining (2022.naacl-industry)
Copied to clipboard
| Challenge: | Existing approaches to clustering unlabeled utterances are based on transformerbased sentence embedding methods. |
| Approach: | They propose a semantic embedding approach that can be leveraged to identify clusters of utterances that correspond to unhandled intents. |
| Outcome: | The proposed approach can identify clusters of utterances that correspond to unhandled intents from a large collection of enterprise virtual assistant data using a multi-task softmax loss. |
Selectively Answering Ambiguous Questions (2023.emnlp-main)
Copied to clipboard
| Challenge: | Prior work has focused on the case in which the question is clear and the answer is unambiguous but possibly unknown. |
| Approach: | They propose to use a sampled set of questions to calibrate answers to ambiguous questions with varying model scales. |
| Outcome: | The results show that sampling-based confidence scores help calibrate answers to relatively unambiguous questions, with more dramatic improvements on ambiguous ones. |
Med-PRM: Medical Reasoning Models with Stepwise, Guideline-verified Process Rewards (2025.emnlp-main)
Copied to clipboard
Jaehoon Yun, Jiwoong Sohn, Jungwoo Park, Hyunjae Kim, Xiangru Tang, Daniel Shao, Yong Hoe Koo, Ko Minhyeok, Qingyu Chen, Mark Gerstein, Michael Moor, Jaewoo Kang
| Challenge: | Large language models have shown promise in clinical decision making, but current approaches struggle to localize and correct reasoning errors at specific steps of the reasoning process. |
| Approach: | They propose a process reward modeling framework that leverages retrieval-augmented generation to verify each reasoning step against established medical knowledge bases. |
| Outcome: | The proposed model improves on five medical QA benchmarks and two open-ended diagnostic tasks by 13.50% on MedQA. |
LexGLUE: A Benchmark Dataset for Legal Language Understanding in English (2022.acl-long)
Copied to clipboard
Ilias Chalkidis, Abhik Jana, Dirk Hartung, Michael Bommarito, Ion Androutsopoulos, Daniel Katz, Nikolaos Aletras
| Challenge: | Laws and their interpretations, legal arguments and agreements are typically expressed in writing. |
| Approach: | They propose a benchmark to evaluate model performance across legal NLU tasks . they also evaluate several generic and legal-oriented models . |
| Outcome: | The proposed model performs better across multiple tasks than previous models. |