Papers by Akiko Aizawa
Copied to clipboard
| Challenge: | Existing multi-hop reading comprehension datasets have reasoning shortcuts that can be used to answer comparison questions without performing multi- hop reasoning. |
| Approach: | They propose a dataset with three probing tasks in addition to the main question . they then evaluate the model's ability to understand date information . |
| Outcome: | The proposed model performs well in date comparison and number subtraction tasks. |
Copied to clipboard
| Challenge: | Recent research suggests using Large Language Models (LLMs) to automate the annotation process, reducing these costs while maintaining data quality. |
| Approach: | They propose to use Large Language Models to automate annotation process and train classifiers on large datasets. |
| Outcome: | The proposed model outperforms all of the annotator LLMs on two media bias benchmark datasets (BABE and BASIL) while maintaining data quality. |
Copied to clipboard
| Challenge: | Empirical studies show that supervised learning is extremely effective in in-domain datasets and models trained on SuperDialseg can achieve good generalization ability on out-of-domain data. |
| Approach: | They propose a supervised definition of dialogue segmentation points using document-grounded dialogues and a large-scale supervised dataset called SuperDialseg. |
| Outcome: | The proposed model can achieve good generalization ability on out-of-domain data. |
Copied to clipboard
| Challenge: | Existing studies on multilingual fine-tuning with a fixed set of languages lack dynamic adaptability to new languages. |
| Approach: | They propose a modular fine-tuning pipeline that enables dynamic language adaptation for LLMs by first training English-centric adapters for each language separately and then merging them for arbitrary-direction translation. |
| Outcome: | The proposed pipeline achieves 86% performance over traditional fine-tuning on four languages, while training only 0.1% parameters and relying on English as a bridge language without catastrophic forgetting. |
Copied to clipboard
| Challenge: | Existing studies on the detection and aggregation of media bias lack a gold standard data set and high context dependencies. |
| Approach: | They propose to use a data set to identify media bias by word and sentence level . they propose to train a model to detect bias-inducing sentences in news articles automatically . |
| Outcome: | The proposed model outperforms existing methods on a large corpus of labels on the word and sentence level. |
Copied to clipboard
| Challenge: | empirical results show that our model significantly outperforms all existing models on four benchmark datasets. |
| Approach: | They propose a novel attention-based relational graph convolutional neural network to exploit syntactic information over dependency graphs. |
| Outcome: | The proposed model outperforms existing models on four benchmark datasets. |
Copied to clipboard
| Challenge: | Generally speaking, the meanings of math symbols are not necessarily constant, and the same symbol is used in multiple meanings. |
| Approach: | They annotated 15 papers with the meanings of math symbols and found they can be grounding . they developed a special annotation tool to help them identify the meaning of each symbol . |
| Outcome: | The constructed dataset shows that the meanings of symbols can be ground with a high agreement . the authors developed a special annotation tool to analyze the data . |
Copied to clipboard
| Challenge: | Existing models that use English and local languages have a multilingual gap . a language-informed co-reasoning framework can be used to improve multilingual reasoning . |
| Approach: | They propose a language-informed co-reasoning framework that elicits parallel English and local-language reasoning and abstracts them into structured concepts. |
| Outcome: | Experiments show that Med-CoReasoner improves multilingual reasoning performance by 5% . the framework produces clinically sound and culturally grounded reasoning traces . |
Copied to clipboard
| Challenge: | Existing evaluation methods for large language models lack robustness and accuracy in medical contexts. |
| Approach: | They propose an evaluation framework for medical LLMs that measures disagreement between model-generated responses and established medical ground truths. |
| Outcome: | The proposed evaluation framework captures accuracy and reliability in medical settings. |
Copied to clipboard
| Challenge: | Existing image captioning approaches treat image-caption pairs indistinctly without considering the differences in their learning difficulties. |
| Approach: | They propose a pretrained vision–language model that measures cross-modal similarity and a model that uses cross-module similarity to measure the difficulty of captioning. |
| Outcome: | The proposed model achieves superior performance and competitive convergence speed to baselines without incurring additional training costs. |
Copied to clipboard
| Challenge: | Existing studies have utilized underlying reasoning (UR) tasks in multi-hop question answering datasets to explain the predicted answers and evaluate models' reasoning abilities. |
| Approach: | They analyze UR tasks in QA datasets to determine their effectiveness . they find that UR task is helpful in preventing reasoning shortcuts . |
| Outcome: | The proposed model improves QA performance, reasoning shortcuts, and robustness on adversarial questions. |
Copied to clipboard
| Challenge: | Existing computer-based writing assistance systems do not help non-native English speakers find better expressions than those they already know . existing FE dictionaries are limited in example sentences and are difficult to predict the category labels from user input, forcing users to manually designate the category when searching. |
| Approach: | They propose a framework for semantic searches of formulaic expressions and a method to leverage existing dictionaries and domain sentence corpora. |
| Outcome: | The proposed method leverages existing dictionaries and domain sentence corpora to find expressions that are not accurate. |
Copied to clipboard
| Challenge: | Autoregressive LLMs perform well on relational tasks that require linking entities via relational words, but it is unclear whether they learn the logical semantics of such relations or whether left-to-right order bias is involved. |
| Approach: | They propose a framework that generates text from symmetric/inverse triples and trains autoregressive models from scratch. |
| Outcome: | The proposed framework generates text from symmetric/inverse triples, trains autoregressive models from scratch, and evaluates memorization, logical inference, and in-context generalization to unseen entities. |
Copied to clipboard
| Challenge: | Existing methods for creating vision-and-language models involve structural modifications and V&L pre-training. |
| Approach: | They propose to extend a language model through structural modifications and V&L pre-training to make it inherit the capability of natural language understanding from the original language model. |
| Outcome: | The proposed method improves performance of vision-and-language models by extending pre-trained models with the same pre-training. |
Copied to clipboard
| Challenge: | Formulaic expressions (FEs) are used in scientific papers. |
| Approach: | They propose to use top–down approach to assign CF labels to sentences and then extract FEs from them using a CF-labelled sentence dataset. |
| Outcome: | The proposed method can be used to build FE databases of disciplines that are different from the training data. |
Copied to clipboard
| Challenge: | Large Multimodal Models (LMMs) have demonstrated strong performance in English, but their effectiveness in Japanese remains limited due to the lack of high-quality training data. |
| Approach: | They propose a pipeline that leverages pretrained models to extract image-text pairs from PDFs . they use layout analysis, OCR, and vision-language pairing to enrich the training data . |
| Outcome: | The proposed pipeline extracts image-text pairs from Japanese PDFs, eliminating manual annotations. |
Copied to clipboard
| Challenge: | Debiasing language models from unwanted behaviors in natural language understanding datasets is a topic with increasing interest in the NLP community. |
| Approach: | They propose a method to debiase language models from unwanted behaviors in NLU tasks by identifying pruning masks that can be applied to a finetuned model. |
| Outcome: | The proposed method shows superior performance and performance over standard methods. |
Copied to clipboard
| Challenge: | Existing large language models (LLMs) focus on general domains, with fewer advancements in Japanese biomedical LLMs. |
| Approach: | They propose a benchmark for Japanese large language models with eight LLMs across four categories and 20 Japanese biomedical datasets for comparison. |
| Outcome: | The proposed benchmark includes eight LLMs across four categories and 20 Japanese biomedical datasets across five tasks. |
Copied to clipboard
| Challenge: | Existing methods for dialog agent training lack a robust action space for entangled information, which can cause bias and deviate from natural human language. |
| Approach: | They propose phrase-level action reinforcement learning which allows the model to alter the sentence structure and content with the sequential action selection. |
| Outcome: | The proposed model achieves competitive results with state-of-the-art models on the MultiWOZ dataset, indicating that it is effective for solving task-oriented dialogs. |
Copied to clipboard
| Challenge: | Identifying the types of contributions an article makes can help readers grasp its significance. |
| Approach: | They propose to use a typology to categorize articles by their contributions to improve review quality and fairness. |
| Outcome: | The ACL Rolling Review (ARR) introduced a typology requiring authors to specify their contributions to improve review quality and fairness. |
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) are used to produce examples sentences targeting L2 learners. |
| Approach: | They propose to use pre-trained language models to produce diverse examples of Japanese sentences that are aligned with learners’ proficiency levels. |
| Outcome: | The proposed method is adaptable to other languages with minor adjustments. |
Copied to clipboard
| Challenge: | Existing evaluation metrics are not designed to cope with this flexibility. |
| Approach: | They propose to group the qualities into three groups to obtain a single metric called USL-H. |
| Outcome: | The proposed metric achieves good correlations with human judgment and maintains its configurability towards different aspects and metrics. |
Copied to clipboard
| Challenge: | Existing task settings focus on creating and maintaining common ground under static contexts and ignore their dynamic aspects. |
| Approach: | They propose a task setting to study the ability of creating and maintaining common ground in dynamic environments. |
| Outcome: | The proposed task setting enables fine-grained evaluation and analysis of various dialogue systems. |
Copied to clipboard
| Challenge: | Formulaic expressions are used by authors of scientific papers because they convey specific communicative functions in the rhetorical structure of papers. |
| Approach: | They created a manually annotated dataset to detect formulaic expressions in sentences using a seed list of labelled formulaic words. |
| Outcome: | The proposed dataset can detect communicative functions in sentences using a seed list of labelled expressions from scholarly papers in the ACL Anthology. |
Copied to clipboard
| Challenge: | predicting the final label alone is insufficient and offers limited interpretability. |
| Approach: | They propose to reframe table–text alignment as an explanation task requiring models to identify the table cells essential for claim verification. |
| Outcome: | The proposed taxonomy improves claim verification performance and most LLMs fail to recover human-aligned rationales, suggesting that their predictions do not stem from faithful reasoning. |
Copied to clipboard
| Challenge: | Sequence-to-sequence models have been used to generate keyphrases, but it is unclear whether they are reliable enough for document retrieval. |
| Approach: | They propose a framework for extrinsic evaluation that allows for a better understanding of the limitations of keyphrase generation models. |
| Outcome: | The proposed models improve retrieval performance by supplementing documents with keyphrases that are not present in the source text and generalizing models across domains. |
Copied to clipboard
| Challenge: | Existing auto-regressive large language models (LLMs) are primarily trained using documents from general domains. |
| Approach: | They propose to use citation network to improve the pre-training of auto-regressive large language models (LLMs) in the biomedical domain. |
| Outcome: | Empirical studies show that the proposed method improves both the intra-sample and inter-sammple referring abilities of auto-regressive large language models in the biomedical domain. |
Copied to clipboard
| Challenge: | Referring expression comprehension is the ability to locate language to an object in an image. |
| Approach: | They propose a neural network architecture that modulates visual features with varying sizes of receptive field by linguistic features and a visual dialogue game to generate referring expression comprehension. |
| Outcome: | The proposed architecture is able to modulate visual features with varying sizes of receptive field by linguistic features on two visual dialogue games. |
Copied to clipboard
| Challenge: | Existing methods to produce readable sentence compression are based on machine learning or syntactic tree-based approaches. |
| Approach: | They propose a language-model-based evaluator for deletion-based sentence compression . they propose deleting operations on source sentences to obtain best target compression based on the proposed model . |
| Outcome: | The proposed model generates more readable compression comparable to strong baselines. |
Copied to clipboard
| Challenge: | Multi-sentence compression aims to generate a grammatical but reduced compression from multiple input sentences while retaining key information. |
| Approach: | They propose a neural rewriter for multi-sentence compression that does not need any parallel corpus. |
| Outcome: | Empirical studies show that the proposed approach achieves comparable results upon automatic evaluation and improves the grammaticality of compression based on human evaluation. |
Copied to clipboard
| Challenge: | Existing data augmentation methods for reading comprehension lack robustness to challenge sets whose distribution is different from that of training sets. |
| Approach: | They propose a question-answer pair generation method that generates multiple diverse QA pairs from a paragraph to mitigate this problem. |
| Outcome: | The proposed model improves the accuracy of 12 challenge sets and the in-distribution accuracy. |
Copied to clipboard
| Challenge: | Existing models for visually grounded dialogues often contain undesirable biases and lack sophisticated linguistic analyses, making it difficult to understand how well they recognize their precise linguistic structures. |
| Approach: | They propose a framework for studying fine-grained language understanding in visually grounded dialogues by using a common grounding dataset which contains minimal bias by design. |
| Outcome: | The proposed framework can reveal both strengths and weaknesses of baseline models in essential levels of detail. |
Copied to clipboard
| Challenge: | Deletion-based sentence compression has made significant progress in the english language . however, there is a lack of large-scale and high-quality parallel corpus for the Chinese language to train an efficient system. |
| Approach: | They propose to construct a Chinese corpus with 151k pairs of sentences and train extractive and generative neural compression models on the constructed corpus. |
| Outcome: | The proposed method generates high-quality compressed sentences on automatic and human evaluation metrics compared with baselines. |
Copied to clipboard
| Challenge: | naive language models treat numerals as string tokens, resulting in difficulty in acquiring commonsense . kNN-LM is an extension of pre-trained neural LMs with the k-nearest neighbor (kNN) search . |
| Approach: | They apply k-nearest neighbor LM to a masked numeral prediction task . they found it is effective for fine-grained predictions of numerals from context . |
| Outcome: | The retrieval-based method is effective for fine-grained numeral prediction from context . it improves accuracy for the OOV numerals, the study shows . |
Copied to clipboard
| Challenge: | Previously, it was common to disregard ambiguity as noise or as a sign of poor quality data. |
| Approach: | They propose to train on the estimated label distribution of annotators in a NLI task . they use a learning loss based on this ambiguity distribution instead of gold-labels . |
| Outcome: | The proposed training method reduces divergence scores on a trial dataset . the study shows that targeting the ambiguity distribution can improve performance . |
Copied to clipboard
| Challenge: | Multilingual domain adaptation (ML-DA) enables large language models to acquire domain knowledge across languages. |
| Approach: | They propose an adaptive evaluation method that constructs multiple-choice QA datasets from the same bilingual domain corpus used for training. |
| Outcome: | The proposed method constructs multiple-choice QA datasets from the same bilingual domain corpus used for training, thereby enabling direct analysis of multilingual knowledge acquisition. |
Copied to clipboard
| Challenge: | Existing language models are difficult to detect numerical errors because of their finite set of tokens. |
| Approach: | They use a benchmark dataset to classify numerical errors using automatically generated numerical errors and investigate their ability to detect errors. |
| Outcome: | The proposed model performs well in the numerical error detection task, but not as accurate as humans. |
Copied to clipboard
| Challenge: | Existing methods to induce circular emotion representations in language models are limited . elucidates trade-offs involved in applying circumplex models to deep learning architectures . |
| Approach: | They propose a method to induce circular emotion representations within language models via contrastive learning on a hypersphere. |
| Outcome: | The proposed method underperforms in high-dimensional settings and fine-grained classification. |
Copied to clipboard
| Challenge: | a task is underway to create a dependency tree bank for the Ainu language in the scheme of Universal Dependencies (UD). |
| Approach: | They propose to create a dependency tree bank for the Ainu language in the scheme of Universal Dependencies (UD) their mini-lexicon is encoded under the W3C OntoLex specification with UD and UniMorph features with the system-friendly JSON-LD format and is bearable to future extensions. |
| Outcome: | The proposed tree bank contains 10,000 word tokens and is small enough to be used as a base annotation for the next step. |
Copied to clipboard
| Challenge: | MRC is a task that tests the ability of a machine to read and understand unstructured text. |
| Approach: | They propose a theoretical basis for the design of MRC datasets based on psychology and psychometrics and propose shortcut-proof questions and explanations as a part of the task design. |
| Outcome: | The proposed datasets should evaluate the model's ability to understand context-dependent situations and ensure substantive validity by shortcut-proof questions and explanation as a part of the task design. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) demonstrates remarkable zero-shot annotation tasks . but, they struggle with the specialized conventions of gold-standard benchmarks . |
| Approach: | They propose to reuse and refine annotation guidelines as an alignment mechanism . they propose to use iterative moderation framework to simulate early phases of annotation projects . |
| Outcome: | The proposed framework shows a good potential in effectively refining guidelines, but there is room for improvement. |
Copied to clipboard
| Challenge: | Existing multi-hop question answering datasets do not provide a complete explanation for the reasoning process from the question to the answer. |
| Approach: | They propose a multi-hop question answering dataset that uses structured and unstructured data to test reasoning skills. |
| Outcome: | The proposed dataset ensures multi-hop reasoning while being challenging for multi-models. |
Copied to clipboard
| Challenge: | Existing frameworks for analyzing text embedding models are limited. |
| Approach: | They propose a framework that uses lightweight poolers to analyze STS, PI, and Triplet datasets. |
| Outcome: | The proposed framework shows that the model captures semantic differences between sentences and is consistent across datasets. |
Copied to clipboard
| Challenge: | Recent studies have shown that questions require a deeper understanding of language to answer beyond using superficial cues. |
| Approach: | They propose to use simple heuristics to split MRC datasets into easy and hard subsets and manually annotate questions from each subset with validity and reasoning skills to investigate which skills explain the difference between easy and harder questions. |
| Outcome: | The proposed model performs better for hard and easy questions than for easy questions. |
Copied to clipboard
| Challenge: | low-resource language corpora in professional domains like medicine hinder cross-lingual domain adaptation of pre-trained large language models. |
| Approach: | They examine how linguistic features affect performance on a Japanese–English medical knowledge benchmark. |
| Outcome: | The proposed model can leverage English-language resources in medical domains while ensuring sufficient coverage of language-specific expressions in a target language. |
Copied to clipboard
| Challenge: | Recent language models have succeeded in tasks that do not require contextual reasoning, such as size comparison and prediction of event frequency. |
| Approach: | They propose to use a dataset to measure physical commonsense to infer size relationships between objects in contexts that fit physical common sense. |
| Outcome: | The proposed dataset consists of contexts that fit physical commonsense and those that don't . results show that language models fail to use verbs and make incorrect judgments . |
Copied to clipboard
| Challenge: | Existing methods for keyphrase generation are limited to resource-rich languages. |
| Approach: | They propose to extract silver-standard keyphrases from citation contexts to create synthetic labeled data for domain adaptation. |
| Outcome: | The proposed method produces significant and consistent improvements over baselines across three domains. |
Copied to clipboard
| Challenge: | Existing approaches to media bias detection lack generalizability, resulting in limited generalizarability. |
| Approach: | They propose a large-scale multi-task pre-training approach specifically tailored for media bias detection that can be used to train 59 bias-related tasks. |
| Outcome: | The proposed approach outperforms existing methods on the BABE dataset with a relative improvement of 3.3% F1-score. |