Papers by Takuma Udagawa
Robust ASR Error Correction with Conservative Data Filtering (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Error correction (EC) based on large language models is an emerging technology to enhance the performance of automatic speech recognition systems. |
| Approach: | They propose to pair large set of ASR hypotheses with gold references to improve linguistic acceptability over sources and be inferable from available context. |
| Outcome: | The proposed approach significantly reduces overcorrection and improves quality in out-of-domain (OOD) settings. |
Bias Analysis and Mitigation through Protected Attribute Detection and Regard Classification (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models acquire general knowledge from pretraining but pretraining data contain undesirable social biases which can be perpetuated or even amplified by LLMs. |
| Approach: | They propose an efficient yet effective annotation pipeline to investigate social biases in pretraining data. |
| Outcome: | The proposed pipeline investigates social biases in the pretraining corpus using protected attribute detection and regard classification. |
Maintaining Common Ground in Dynamic Environments (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing task settings focus on creating and maintaining common ground under static contexts and ignore their dynamic aspects. |
| Approach: | They propose a task setting to study the ability of creating and maintaining common ground in dynamic environments. |
| Outcome: | The proposed task setting enables fine-grained evaluation and analysis of various dialogue systems. |
A Comparative Analysis of Task-Agnostic Distillation Methods for Compressing Transformer Language Models (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Large language models are often inefficient for real-world deployment due to expensive inference costs. |
| Approach: | They propose to use knowledge distillation to transfer the knowledge of the original model to a smaller, more efficient student model. |
| Outcome: | The proposed method is the best for multi-lingual and multilingual student architectures. |
A Linguistic Analysis of Visually Grounded Dialogues Based on Spatial Expressions (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for visually grounded dialogues often contain undesirable biases and lack sophisticated linguistic analyses, making it difficult to understand how well they recognize their precise linguistic structures. |
| Approach: | They propose a framework for studying fine-grained language understanding in visually grounded dialogues by using a common grounding dataset which contains minimal bias by design. |
| Outcome: | The proposed framework can reveal both strengths and weaknesses of baseline models in essential levels of detail. |
Sentence Identification with BOS and EOS Label Combinations (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for preprocessing sentences only use the end of the sentence (EOS) however, real-world texts often contain non-sentential units (NSUs) such as metadata, sentence fragments, etc. |
| Approach: | They propose a task of sentence identification where the goal is to identify SUs while excluding NSUs in a given text. |
| Outcome: | The proposed method outperforms baselines which only use EOS labels on the sentence identification task. |
INDUS: Effective and Efficient Language Models for Scientific Applications (2024.emnlp-industry)
Copied to clipboard
Bishwaranjan Bhattacharjee, Aashka Trivedi, Masayasu Muraoka, Muthukumaran Ramasubramanian, Takuma Udagawa, Iksha Gurung, Nishan Pantha, Rong Zhang, Bharath Dandala, Rahul Ramachandran, Manil Maskey, Kaylin Bugbee, Michael Little, Elizabeth Fancher, Irina Gerasimov, Armin Mehrabian, Lauren Sanders, Sylvain Costes, Sergi Blanco-Cuaresma, Kelly Lockhart, Thomas Allen, Felix Grezes, Megan Ansdell, Alberto Accomazzi, Yousef El-Kurdi, Davis Wertheimer, Birgit Pfitzmann, Cesar Berrospi Ramis, Michele Dolfi, Rafael Lima, Panagiotis Vagenas, S. Mukkavilli, Peter Staar, Sanaz Vahidinia, Ryan McGranaghan, Tsengdar Lee
| Challenge: | Large language models trained on general domain corpora showed remarkable results on natural language processing tasks. |
| Approach: | They develop a suite of large language models trained on general domain corpora that address NLP tasks and smaller versions of them created using knowledge distillation. |
| Outcome: | The proposed models outperform general-purpose and domain-specific encoders on new and existing tasks and in industrial settings. |