Papers by Kentaro Inui

Distance-Free Modeling of Multi-Predicate Interactions in End-to-End Japanese Predicate-Argument Structure Analysis (C18-1)

Challenge:	Language models (LMs) have been used in cognitive modeling and engineering studies to simulate human cognitive load during reading.
Approach:	They propose to constrain LMs' context access to improve their simulation of human reading behavior by incorporating syntactic biases into their context access.
Outcome:	The proposed model improves the simulation of human reading behavior by incorporating syntactic biases into their context access.

Copied to clipboard

Yuichiroh Matsubayashi, Kentaro Inui

Challenge:	Existing models for analyzing PASs in Japanese are lacking in identifying elliptical arguments.
Approach:	They propose to extend the input and last layers of a bidirectional recurrent neural network model to capture the potential interactions among multiple PASs.
Outcome:	The proposed models improve prediction accuracy on a benchmark corpus and achieve state-of-the-art on standardized corpus.

Cross-Lingual Learning-to-Rank with Shared Representations (N18-2)

Copied to clipboard

Shota Sasaki, Shuo Sun, Shigehiko Schamoni, Kevin Duh, Kentaro Inui

Representation Learning

Challenge:	Cross-lingual information retrieval (CLIR) is a document retrieval task where the documents are written in a language different from that of the user's query.
Approach:	They propose a large-scale dataset derived from Wikipedia to support CLIR research in 25 languages.
Outcome:	The proposed model can improve the results of Swahili-English CLIR in Japanese and Japanese.

Can LMs Store and Retrieve 1-to-N Relational Knowledge? (2023.acl-srw)

Copied to clipboard

Haruki Nagasawa, Benjamin Heinzerling, Kazuma Kokuta, Kentaro Inui

Monotonic Representation of Numeric Attributes in Language Models (2024.acl-short)

Challenge:	Pretraining language models on large amounts of text has made it difficult to store and retrieve world knowledge.
Approach:	They propose to view pretrained language models as knowledge bases by examining their ability to store and retrieve world knowledge.
Outcome:	The proposed model can store and retrieve world knowledge with high accuracy, but it is not clear how accurately it can handle 1-to-N relational knowledge.

Copied to clipboard

Benjamin Heinzerling, Kentaro Inui

Unsupervised Learning of Discourse-Aware Text Representation for Essay Scoring (P19-2)

Challenge:	Language models (LMs) can express factual knowledge involving numeric properties such as Karl Popper was born in 1902, but how this information is encoded in the model’s internal representations is not understood well.
Approach:	They propose a method for finding and editing representations of numeric properties such as Karl Popper’s birth year.
Outcome:	The proposed method can express an increasingly late birthyear by patching activations along a “birthyear” direction.

Copied to clipboard

Farjana Sultana Mim, Naoya Inoue, Paul Reisert, Hiroki Ouchi, Kentaro Inui

Responsible & Trustworthy NLP

Challenge:	Existing document embedding approaches focus on capturing sequences of words in documents . however, some document classification and regression tasks need to consider discourse structure of text .
Approach:	They propose an unsupervised approach to capture discourse structure in terms of coherence and cohesion for document embedding that does not require expensive parsers or annotation.
Outcome:	The proposed method improves essay Organization scoring and Argument Strength scoring.

Topicalization in Language Models: A Case Study on Japanese (2022.coling-1)

Copied to clipboard

Riki Fujihara, Tatsuki Kuribayashi, Kaori Abe, Ryoko Tokuhisa, Kentaro Inui

Filtering Noisy Dialogue Corpora by Connectivity and Content Relatedness (2020.emnlp-main)

Challenge:	a recent study has shown that neural language models can capture discourse-level preferences in text generation . a particular aspect of discourse is the topic-comment structure .
Approach:	They analyze whether neural language models can capture discourse-level preferences in text generation . they use Japanese language and crowdsourced human topicalization judgment data .
Outcome:	The proposed model can capture human-like generalizations in discourse-level linguistic aspects.

Copied to clipboard

Reina Akama, Sho Yokoi, Jun Suzuki, Kentaro Inui

A Melody-Conditioned Lyrics Language Model (N18-1)

Challenge:	Large-scale dialogue datasets contain a non-negligible number of unacceptable utterance pairs . previous studies have identified such flaws and reported that the corpus is noisy .
Approach:	They propose a method for scoring the quality of utterance pairs based on their connectivity and relatedness.
Outcome:	The proposed method has a good correlation with human judgment of dialogue quality and is applied to training data filtered by the proposed method.

Copied to clipboard

Kento Watanabe, Yuichiroh Matsubayashi, Satoru Fukayama, Masataka Goto, Kentaro Inui, Tomoyasu Nakano

When Choosing Plausible Alternatives, Clever Hans can be Clever (D19-60)

Challenge:	Existing models for lyrics generation are insufficient to capture relationship between lyrics and melody.
Approach:	They propose a data-driven language model that generates entire lyrics for a given melody.
Outcome:	The proposed model generates fluent lyrics while maintaining compatibility between lyrics and melodies.

Copied to clipboard

Pride Kavumba, Naoya Inoue, Benjamin Heinzerling, Keshav Singh, Paul Reisert, Kentaro Inui

Disentangling the Effects of Unlearning in Measuring Parametric Faithfulness of Chain-of-Thought (2026.acl-srw)

Challenge:	Pretrained language models have shown large improvements in the commonsense reasoning benchmark COPA, but recent work has identified superficial cues in benchmark datasets which are predictive of the correct answer.
Approach:	They propose an extension of COPA that does not suffer from easy-to-exploit single token cues and exploits them.
Outcome:	The proposed extension of COPA does not suffer from easy-to-exploit single token cues.

Copied to clipboard

Ryo Mitsuhashi, Gaku Morio, Ayana Niwa, Masahiro Kaneko, Kentaro Inui, Terufumi Morishita, Yuta Koreeda, Yasuhiro Sogawa

Transductive Learning of Neural Language Models for Syntactic and Semantic Analysis (D19-1)

Challenge:	Chain-of-Thought (CoT) has been debated as a model's faithfulness to internal reasoning process.
Approach:	They propose to use unlearning to measure parametric faithfulness of models by adjusting for unintended artifacts of unlearning.
Outcome:	The proposed metric accounts for the unintended artifacts of unlearning and shows that it is non-negligible.

Copied to clipboard

Hiroki Ouchi, Jun Suzuki, Kentaro Inui

SyGNS: A Systematic Generalization Testbed Based on Natural Language Semantics (2021.findings-acl)

Challenge:	despite its practical advantages, transductive learning is underexplored in natural language processing . despite the simplicity of the technique, it is understudied in natural languages .
Approach:	They conduct an empirical study of transductive learning for neural models . they fine-tune language models on an unlabeled test set to obtain test-set-specific word representations.
Outcome:	The proposed method improves state-of-the-art neural models in syntactic and semantic tasks.

Copied to clipboard

Hitomi Yanaka, Koji Mineshima, Kentaro Inui

Challenge:	Existing models have limitations to generalize to diverse semantic phenomena, and it is unclear whether they can capture compositional meanings.
Approach:	They propose a systematic generalization testbed based on Natural language semantics to map natural language sentences to multiple meaning representations.
Outcome:	The proposed model can generalize to unseen combinations of quantifiers, negations, and modifiers, but not to the others.

An Empirical Study of Incorporating Pseudo Data into Grammatical Error Correction (D19-1)

Copied to clipboard

Shun Kiyono, Jun Suzuki, Masato Mita, Tomoya Mizumoto, Kentaro Inui

LLMs Faithfully and Iteratively Compute Answers During CoT: A Systematic Analysis With Multi-step Arithmetics (2026.findings-eacl)

Challenge:	incorporating pseudo data in the training of grammatical error correction models has been a key factor in improving performance of such models.
Approach:	They investigate the choice of how pseudo data should be generated or used in a grammatical error correction model and show that the results are state-of-the-art.
Outcome:	The proposed method achieves state-of-the-art on the CoNLL-2014 test set and the official test set of the BEA-2019 shared task without making any modifications to the model architecture.

Copied to clipboard

Keito Kudo, Yoichi Aoki, Tatsuki Kuribayashi, Shusaku Sone, Masaya Taniguchi, Ana Brassard, Keisuke Sakaguchi, Kentaro Inui

Modeling Event Salience in Narratives via Barthes’ Cardinal Functions (2020.coling-main)

Challenge:	Specifically, we examine when the LLMs’ answer is (pre)determined, especially before the CoT begins or after, and how strongly the information from CoT specifically has a causal effect on the final answer.
Approach:	They examine when the LLMs’ answer is (pre)determined, especially before the CoT begins or after, and how strongly the information from CoT specifically has a causal effect on the final answer.
Outcome:	The proposed model can generate reasoning chains while generating the reasoning chain on the fly.

Copied to clipboard

Takaki Otake, Sho Yokoi, Naoya Inoue, Ryo Takahashi, Tatsuki Kuribayashi, Kentaro Inui

Pseudo Zero Pronoun Resolution Improves Zero Anaphora Resolution (2021.emnlp-main)

Challenge:	Existing methods for estimating event salience without annotations are prohibitively costly because they require annotators to understand the concept of event salientity.
Approach:	They propose to use Barthes’ definition of event salience to compute event salientity without annotations by using a pre-trained language model.
Outcome:	The proposed methods outperform baseline methods on folktales with event salience annotation and fine-tuned language model is key factor in improving the methods.

Copied to clipboard

Ryuto Konno, Shun Kiyono, Yuichiroh Matsubayashi, Hiroki Ouchi, Kentaro Inui

Unsupervised Learning of Style-sensitive Word Vectors (P18-2)

Challenge:	Masked language models have contributed to drastic performance improvements with regard to zero anaphora resolution (ZAR).
Approach:	They propose a pretraining task that trains MLMs on anaphoric relations with explicit supervision and a finetuning method that remedies a notorious discrepancy.
Outcome:	The proposed method improves zero anaphora resolution in Japanese ZAR . it uses a pretrain task and finetuning task to correct the discrepancy .

Copied to clipboard

Reina Akama, Kento Watanabe, Sho Yokoi, Sosuke Kobayashi, Kentaro Inui

Prompting for explanations improves Adversarial NLI. Is this true? {Yes} it is {true} because {it weakens superficial cues} (2023.findings-eacl)

Challenge:	Existing studies on what is said and how it is said focus on stylistic variations . lack of objective definitions is a major difficulty in studying style .
Approach:	They propose to extend the continuous bag of words embedding model to learn style-sensitive word vectors using a wider context window.
Outcome:	The proposed extensions contribute to the acquisition of style-sensitive word embeddings.

Copied to clipboard

Pride Kavumba, Ana Brassard, Benjamin Heinzerling, Kentaro Inui

Responsible & Trustworthy NLP

Challenge:	Explanation prompts are used to generate an explanation for a given input . they are also used to improve model performance on adversarial benchmarks .
Approach:	They propose to use explanation prompts to generate an explanation that supports a label . they argue that prompting for explanations weakens superficial cues .
Outcome:	The proposed explanation prompts outperform label-only prompts on adversarial benchmarks.

Langsmith: An Interactive Academic Text Revision System (2020.emnlp-demos)

Copied to clipboard

Takumi Ito, Tatsuki Kuribayashi, Masatoshi Hidaka, Jun Suzuki, Kentaro Inui

Challenge:	Currently, diversity and inclusion initiatives in the academic community are encouraged . however, writing papers in English can be a daunting task .
Approach:	They propose a system that helps non-native English speakers to write papers in English . the system can suggest fluent, academic-style sentences based on their rough, incomplete phrases or sentences .
Outcome:	The proposed system can help non-native English speakers write papers in English . the system can suggest fluent, academic-style sentences based on their rough sentences .

Predicting Stances from Social Media Posts using Factorization Machines (C18-1)

Copied to clipboard

Akira Sasaki, Kazuaki Hanawa, Naoaki Okazaki, Kentaro Inui

Challenge:	Social media provide platforms to express, discuss, and shape opinions about events and issues in the real world.
Approach:	They propose to use factorization machines to model user preferences toward topics from social media data to predict whether a given text/user is in favor (agree), against (disagreer), or neutral toward a target topic.
Outcome:	The proposed method can predict stances of silent users based on their stance toward other topics and the social media posts of the user.

Exploring Methods for Generating Feedback Comments for Writing Learning (2021.emnlp-main)

Copied to clipboard

Kazuaki Hanawa, Ryo Nagata, Kentaro Inui

Information Retrieval

Challenge:	Existing methods for generating explanatory notes for language learners are inadequate . nagata et al. demonstrates that neural-retrieval-based methods can generate feedback comments for preposition use .
Approach:	They investigate three different methods for generating feedback comments for preposition use . grammatical and writing items can also be used to generate feedback comments .
Outcome:	The proposed methods outperform neural-retrieval-based methods in generating feedback comments for preposition use.

COPA-SSE: Semi-structured Explanations for Commonsense Reasoning (2022.lrec-1)

Copied to clipboard

Ana Brassard, Benjamin Heinzerling, Pride Kavumba, Kentaro Inui

Weight-based Analysis of Detokenization in Language Models: Understanding the First Stage of Inference Without Inference (2025.findings-naacl)

Challenge:	Semi-structured explanations for Choice of Plausible Alternatives (COPA-SSE) are a crowdsourced dataset of 9,747 common sense explanations .
Approach:	They propose a semi-structured approach to explain Choice of Plausible Alternatives questions using a crowdsourced dataset of 9,747 common sense explanations with ConceptNet relations but freely written concepts.
Outcome:	The proposed explanations are geared towards commonsense reasoners operating on knowledge graphs and serve as a starting point for improving such systems.

Copied to clipboard

Go Kamoda, Benjamin Heinzerling, Tatsuro Inaba, Keito Kudo, Keisuke Sakaguchi, Kentaro Inui

Instance-Based Learning of Span Representations: A Case Study through Named Entity Recognition (2020.acl-main)

Challenge:	Existing evidence for the stages-of-inference hypothesis is that early layers of language models map their subword tokenized input to more meaningful representations that form the model’s “inner vocabulary”.
Approach:	They propose an analytical decomposition of first-layer attention in language models that quantifies the relative contributions of position-related, token-related and mixed effects.
Outcome:	The proposed analysis yields interpretable terms that quantify the relative contributions of position-related, token-related and mixed effects.

Copied to clipboard

Hiroki Ouchi, Jun Suzuki, Sosuke Kobayashi, Sho Yokoi, Tatsuki Kuribayashi, Ryuto Konno, Kentaro Inui

Responsible & Trustworthy NLP

Challenge:	Recent neural networks can induce good span feature representations and achieve high performance in structured prediction tasks.
Approach:	They propose an instance-based learning method that learns similarities between spans . they aim to build models that have high interpretability without sacrificing performance .
Outcome:	The proposed method improves interpretability without sacrificing performance.

Transformer Language Models Handle Word Frequency in Prediction Head (2023.findings-acl)

Copied to clipboard

Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi, Kentaro Inui

Empirical Investigation of Neural Symbolic Reasoning Strategies (2023.findings-eacl)

Challenge:	Prediction head is a crucial component of Transformer language models. Despite its direct impact on prediction, its characteristics have been overlooked in previous analyses.
Approach:	They examine the inner workings of the prediction head, specifically the bias parameters, and quantify the effect of controlling their frequency biases on text generation.
Outcome:	The prediction head is a crucial component of the Transformer language models.

Copied to clipboard

Yoichi Aoki, Keito Kudo, Tatsuki Kuribayashi, Ana Brassard, Masashi Yoshikawa, Keisuke Sakaguchi, Kentaro Inui

Target-Guided Open-Domain Conversation Planning (2022.coling-1)

Challenge:	Neural reasoning accuracy improves when generating intermediate reasoning steps.
Approach:	They decompose the reasoning strategy w.r.t. step granularity and chaining strategy.
Outcome:	The proposed reasoning strategy significantly affects performance in a symbolic reasoning dataset.

Copied to clipboard

Yosuke Kishinami, Reina Akama, Shiki Sato, Ryoko Tokuhisa, Jun Suzuki, Kentaro Inui

Identification of Multiple Logical Interpretations in Counter-Arguments (2025.emnlp-main)

Challenge:	Existing studies on goal-oriented conversational tasks lack planning . prior studies on this topic have focused on generating proactive behavior in agents .
Approach:	They propose a task to evaluate whether neural conversational agents have goal-oriented conversation planning abilities.
Outcome:	The proposed task evaluates whether neural conversational agents have goal-oriented conversation planning abilities.

Copied to clipboard

Wenzhi Wang, Paul Reisert, Shoichi Naito, Naoya Inoue, Machi Shimmei, Surawat Pothong, Jungmin Choi, Kentaro Inui

Explainability & Interpretability in NLP

Challenge:	Counter-arguments (CAs) are a good way to improve learners' critical thinking skills . however, it is difficult to provide every learner tailored feedback due to limited human resources and heavy workloads.
Approach:	They propose to annotate a dataset of 134 CAs annotated with 13 logical predicate questions and train a model with Reinforcement Learning with Verifiable Rewards to identify multiple logical interpretations.
Outcome:	The proposed model performs on par with larger proprietary models.

Improving Evidence Detection by Leveraging Warrants (D19-66)

Copied to clipboard

Keshav Singh, Paul Reisert, Naoya Inoue, Pride Kavumba, Kentaro Inui

Cross-stitching Text and Knowledge Graph Encoders for Distantly Supervised Relation Extraction (2022.emnlp-main)

Challenge:	Existing methods for extracting warrants from a corpus of arguments are lacking in argument detection.
Approach:	They propose to extract multiple warrants from an existing corpus of arguments and then aggregate them . they show that the method needs to be improved, but that it can still improve evidence detection.
Outcome:	The proposed method can improve the performance of evidence detection by analyzing arguments and aggregating them.

Copied to clipboard

Qin Dai, Benjamin Heinzerling, Kentaro Inui

An Empirical Study of Span Representations in Argumentation Structure Parsing (P19-1)

Challenge:	Existing bi-encoder architectures do not allow any sharing between text and knowledge graphs . john sutter: experimental results show that enabling full interaction yields strong improvements.
Approach:	They propose cross-stitch bi-encoders that allow full interaction between text and KG . they say the amount of sharing is dynamically controlled via cross-attention-based gates .
Outcome:	Experimental results show that bi-encoder architectures yield strong improvements . cross-stitch mechanism allows sharing and updating representations between two encoders .

Copied to clipboard

Tatsuki Kuribayashi, Hiroki Ouchi, Naoya Inoue, Paul Reisert, Toshinori Miyoshi, Jun Suzuki, Kentaro Inui

Representation Learning

Challenge:	Argumentation structure parsing (ASP) is a task of identifying argumentation structures in argumentative text.
Approach:	They propose to exploit neural network-based span representations for ASP to improve performance . they also propose task-dependent extensions for a parser that can be used to parse arguments .
Outcome:	The proposed model outperforms neural network-based approaches for argumentation structure parsing (ASP) it also provides some challenging types of instances to be parsed.

Summarize-then-Answer: Generating Concise Explanations for Multi-hop Reading Comprehension (2021.emnlp-main)

Copied to clipboard

Naoya Inoue, Harsh Trivedi, Steven Sinha, Niranjan Balasubramanian, Kentaro Inui

Machine Reading Comprehension

Challenge:	Recent approaches to multi-hop Reading Comprehension (RC) have greatly improved its explainability, models ability to explain their own answers.
Approach:	They propose to generate a question-focused abstractive summary of input paragraphs and feed it to an RC system.
Outcome:	The proposed explanation generates more compact explanations than an extractive explainer with limited supervision while maintaining sufficiency.

Sycophancy Hides Linearly in the Attention Heads (2026.eacl-long)

Copied to clipboard

Rifo Ahmad Genadi, Munachiso Samuel Nwadike, Nurdaulet Mukhituly, Tatsuya Hiraoka, Hilal AlQuabeh, Kentaro Inui

Challenge:	Using TruthfulQA as the base dataset, we find that probes trained on TruthfulQ transfer effectively to other factual QA benchmarks.
Approach:	They train linear probes across the residual stream, multilayer perceptron, and attention layers to analyze where sycophancy signals emerge.
Outcome:	The proposed model can be used to steer truthfulness and toxicity behaviors.

Do Neural Models Learn Systematicity of Monotonicity Inference in Natural Language? (2020.acl-main)

Copied to clipboard

Hitomi Yanaka, Koji Mineshima, Daisuke Bekki, Kentaro Inui

Why Mean Pooling Works: Quantifying Second-Order Collapse in Text Embeddings (2026.acl-long)

Challenge:	Despite the success of language models using neural networks, it remains unclear to what extent neural models have the generalization ability to perform inferences.
Approach:	They propose a method to evaluate whether neural models can learn systematicity of monotonicity inference in natural language.
Outcome:	The proposed method shows that neural models can perform inferences on unseen combinations of lexical and logical phenomena when syntactic structures are similar between training and test sets.

Copied to clipboard

Tomomasa Hara, Hiroto Kurita, Masaaki Imaizumi, Kentaro Inui, Sho Yokoi

Select and Attend: Towards Controllable Content Selection in Text Generation (D19-1)

Challenge:	Text embeddings are used across a wide range of NLP tasks, including retrieval-augmented generation.
Approach:	They propose a metric to quantify the collapse induced by mean pooling and a simple metric for measuring how often it occurs in real models and texts.
Outcome:	The proposed metric measures how often the collapse occurs in real models and texts.

Copied to clipboard

Xiaoyu Shen, Jun Suzuki, Kentaro Inui, Hui Su, Dietrich Klakow, Satoshi Sekine

Text Generation

Challenge:	Recent neural network models conflate content selection and surface realization into a black-box architecture, resulting in content to be described in text cannot be explicitly controlled.
Approach:	They propose to decouple content selection from the decoder to allow finer-grained control over the generation.
Outcome:	The proposed model can be trained end-to-end without human annotations and achieves promising results in data-totext and headline generation tasks.

Cross-Corpora Evaluation and Analysis of Grammatical Error Correction Models — Is Single-Corpus Evaluation Enough? (N19-1)

Copied to clipboard

Masato Mita, Tomoya Mizumoto, Masahiro Kaneko, Ryo Nagata, Kentaro Inui

Interpretable and Compositional Relation Learning by Joint Training with an Autoencoder (P18-1)

Challenge:	Existing studies have evaluated grammatical error correction models on a single corpus, but the evaluation is incomplete because the task difficulty varies depending on the corpus and conditions such as proficiency levels of the writers and essay topics.
Approach:	They evaluate the performance of several GEC models against various learner corpora and compare their rankings against the corpus.
Outcome:	The evaluation of several models against learner corpora shows that the models’ rankings vary depending on the corpus, indicating that single-corpus evaluation is insufficient for GEC models.

Copied to clipboard

Ryo Takahashi, Ran Tian, Kentaro Inui

Encoder-Decoder Models Can Benefit from Pre-trained Masked Language Models in Grammatical Error Correction (2020.acl-main)

Challenge:	Embedding models for entities and relations are useful for recovering missing facts in knowledge bases.
Approach:	They propose a dimension reduction technique by training relations jointly with an autoencoder to capture compositional constraints.
Outcome:	The proposed model improves on Knowledge Base Completion tasks with a significantly higher mean rank and better compositional training.

Copied to clipboard

Masahiro Kaneko, Masato Mita, Shun Kiyono, Jun Suzuki, Kentaro Inui

Evaluating Dialogue Generation Systems via Response Selection (2020.acl-main)

Challenge:	Existing methods for incorporating a masked language model into an EncDec model have potential drawbacks when applied to GEC.
Approach:	They propose to incorporate a pre-trained masked language model (MLM) into an encoder-decoder model for grammatical error correction.
Outcome:	The proposed method achieves state-of-the-art on BEA-2019 and CoNLL-2014 benchmarks.

Copied to clipboard

Shiki Sato, Reina Akama, Hiroki Ouchi, Jun Suzuki, Kentaro Inui

A Large Collection of Model-generated Contradictory Responses for Consistency-aware Dialogue Systems (2024.findings-acl)

Challenge:	Existing automatic evaluation metrics for open-domain dialogue systems correlate poorly with human evaluation.
Approach:	They propose to construct response selection test sets with well-chosen false candidates to evaluate response generation systems via response selection.
Outcome:	The proposed method correlates with human evaluation better than widely used metrics such as BLEU.

Copied to clipboard

Shiki Sato, Reina Akama, Jun Suzuki, Kentaro Inui

R4C: A Benchmark for Evaluating RC Systems to Get the Right Answer for the Right Reason (2020.acl-main)

Challenge:	Recent large-scale neural response generation models (RGMs) have made significant progress but still struggle to generate semantically appropriate responses.
Approach:	They build a large dataset of model-generated contradictions for the first time and analyze the results to gain valuable insights into their characteristics.
Outcome:	The proposed dataset significantly improves the performance of data-driven contradiction suppression methods.

Copied to clipboard

Naoya Inoue, Pontus Stenetorp, Kentaro Inui

Preventing Critical Scoring Errors in Short Answer Scoring with Confidence Estimation (2020.acl-srw)

Challenge:	Recent studies have revealed that reading comprehension (RC) systems learn to exploit annotation artifacts and other biases in current datasets.
Approach:	They propose a task that requires giving answers and derivations to evaluate RC systems' internal reasoning.
Outcome:	The proposed framework annotates 4.6k questions with 3 reference derivations and shows that it is reliable and compares with existing benchmarks.

Copied to clipboard

Hiroaki Funayama, Shota Sasaki, Yuichiroh Matsubayashi, Tomoya Mizumoto, Jun Suzuki, Masato Mita, Kentaro Inui

Challenge:	Recent Short Answer Scoring systems use Quadratic Weighted Kappa (QWK) but it is unsatisfactory when measuring their effectiveness in actual usage.
Approach:	They propose a task formulation of Short Answer Scoring (SAS) that matches actual usage and extracts as many scoring predictions that are not critical scoring errors (CSEs).
Outcome:	The proposed system predicts scores with zero critical scoring errors (CSEs) for 50% of test data at maximum by filtering out low-reliability predictions on the basis of a certain confidence estimation.

Two Training Strategies for Improving Relation Extraction over Universal Graph (2021.eacl-main)

Copied to clipboard

Qin Dai, Naoya Inoue, Ryo Takahashi, Kentaro Inui

Do Deep Neural Networks Capture Compositionality in Arithmetic Reasoning? (2023.eacl-main)

Challenge:	Existing methods for Distantly Supervised Relation Extraction (DS-RE) with a UG may lead to degradation in performance.
Approach:	They propose to use a Universal Graph (UG) to train a distantly supervised relation extraction model.
Outcome:	The proposed training strategies on biomedical and NYT10 datasets prove the robustness of the proposed methods and achieve state-of-the-art results.

Copied to clipboard

Keito Kudo, Yoichi Aoki, Tatsuki Kuribayashi, Ana Brassard, Masashi Yoshikawa, Keisuke Sakaguchi, Kentaro Inui

To Drop or Not to Drop? Predicting Argument Ellipsis Judgments: A Case Study in Japanese (2024.lrec-main)

Challenge:	Using a pre-trained dataset, we examine how well recent neural models capture compositionality in symbolic reasoning tasks.
Approach:	They propose a skill tree on compositionality that defines hierarchical levels of complexity along with three compositionality dimensions: systematicity, productivity, and substitutivity.
Outcome:	The proposed model struggled most with systematicity, performing poorly even with relatively simple compositions.

Copied to clipboard

Yukiko Ishizuki, Tatsuki Kuribayashi, Yuichiroh Matsubayashi, Ryohei Sasano, Kentaro Inui

Challenge:	Speakers sometimes omit certain arguments of a predicate in a sentence; such omission is especially frequent in pro-drop languages.
Approach:	They collect large-scale human annotations of whether and why a particular argument should be omitted across over 2,000 data points in Japanese, a prototypical pro-drop language.
Outcome:	The proposed model can explain why certain arguments are omitted in Japanese, a prototypical pro-drop language.

Incorporating Residual and Normalization Layers into Analysis of Masked Language Models (2021.emnlp-main)

Copied to clipboard

Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi, Kentaro Inui

Flee the Flaw: Annotating the Underlying Logic of Fallacious Arguments Through Templates and Slot-filling (2024.emnlp-main)

Challenge:	Transformer architecture is composed of multi-head attention, which has been extensively analyzed.
Approach:	They extended the scope of the analysis of Transformers from solely the attention patterns to the whole attention block, i.e., multi-head attention, residual connection, and layer normalization.
Outcome:	The proposed method incorporates the whole attention block, i.e., multi-head attention, residual connection, and layer normalization into the analysis.

Copied to clipboard

Irfan Robbani, Paul Reisert, Surawat Pothong, Naoya Inoue, Camélia Guerraoui, Wenzhi Wang, Shoichi Naito, Jungmin Choi, Kentaro Inui

Challenge:	Prior work on quality assessment has focused on numerical scoring and fallacy type-labeling tasks, without aiming to analyze fallacy logic structures.
Approach:	They propose four sets of explainable templates for common informal logical fallacies designed to explicate a fallacy’s implicit logic.
Outcome:	The proposed models achieve a high agreement score and reasonable coverage 83% on 400 fallacious arguments and state-of-the-art language models struggle with detecting fallacy templates (0.47 accuracy).

PheMT: A Phenomenon-wise Dataset for Machine Translation Robustness on User-Generated Contents (2020.coling-main)

Copied to clipboard

Ryo Fujii, Masato Mita, Kaori Abe, Kazuaki Hanawa, Makoto Morishita, Jun Suzuki, Kentaro Inui

Text Generation

Challenge:	Existing studies suggest that Neural Machine Translation still struggles with certain kinds of input with considerable noise, such as User-Generated Contents (UGC) on the Internet.
Approach:	They propose to evaluate the robustness of Neural Machine Translation models against specific linguistic phenomena in Japanese-English translation.
Outcome:	The proposed model can handle user-generated content (UGC) on the Internet, but it is difficult to translate clean inputs.

Investigating the Effectiveness of Multiple Expert Models Collaboration (2023.findings-emnlp)

Copied to clipboard

Ikumi Ito, Takumi Ito, Jun Suzuki, Kentaro Inui

Machine Translation

Challenge:	Using multi-domain MT, we compare the performance of a single model with a multi-expert model in a fair condition.
Approach:	They propose to combine a multi-domain machine translation model with a aggregation strategy to investigate their results.
Outcome:	The proposed approach outperforms the current multi-domain model and aggregation methods in a fair condition on multiple domain datasets.

Attention is Not Only a Weight: Analyzing Transformers with Vector Norms (2020.emnlp-main)

Copied to clipboard

Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi, Kentaro Inui

Language Models as an Alternative Evaluator of Word Order Hypotheses: A Case Study in Japanese (2020.acl-main)

Challenge:	Attention is a key component of Transformers, which have achieved considerable success in natural language processing.
Approach:	They propose to integrate attention weights and the norm of transformed input vectors into a norm-based analysis that incorporates the norm.
Outcome:	The proposed analysis shows that attention weights alone determine the output of attention and that reasonable word alignment can be extracted from attention mechanisms of Transformers.

Copied to clipboard

Tatsuki Kuribayashi, Takumi Ito, Jun Suzuki, Kentaro Inui

IRAC: A Domain-Specific Annotated Corpus of Implicit Reasoning in Arguments (2022.lrec-1)

Challenge:	a method using neural language models (LMs) for analyzing the word order of language is currently lacking.
Approach:	They propose a method using neural language models to analyze the word order in Japanese . they test whether there is a parallel between LMs and human word order preference .
Outcome:	The proposed method is validated by comparing it with other linguistic studies.

Copied to clipboard

Keshav Singh, Naoya Inoue, Farjana Sultana Mim, Shoichi Naito, Kentaro Inui

How Well Do Vision Models Encode Diagram Attributes? (2024.acl-srw)

Challenge:	Using crowdsourcing, we show that models trained with domain-specific implicit reasonings outperform domain-general models in both automatic and human evaluations.
Approach:	They propose to create a domain-specific corpus of implicit reasonings annotated for a wide range of arguments and use it to generate models.
Outcome:	The proposed corpus outperforms domain-general models in automatic and human evaluations.

Copied to clipboard

Haruto Yoshida, Keito Kudo, Yoichi Aoki, Ryota Tanaka, Itsumi Saito, Keisuke Sakaguchi, Kentaro Inui

Multimodality

Challenge:	Experimental results show vision models struggle to identify diagram attributes such as node colors and shapes, along with edge colors and connection patterns.
Approach:	They evaluated vision models and retrieving diagrams using text queries to determine how well they recognize diagram attributes and edge connection patterns.
Outcome:	The models can recognize node colors, shapes, and edge colors, but struggle to identify differences in edge connection patterns that play a pivotal role in the semantics of diagrams.

Test-time Augmentation for Factual Probing (2023.findings-emnlp)

Copied to clipboard

Go Kamoda, Benjamin Heinzerling, Keisuke Sakaguchi, Kentaro Inui

Representational Analysis of Binding in Language Models (2024.emnlp-main)

Challenge:	Existing methods to improve factual probing are relation-specific and do not generalize to unseen relation types.
Approach:	They propose to use test-time augmentation to augment and ensemble prompts at test time to reduce sensitivity to prompt variations.
Outcome:	The proposed method improves model confidence, but for other models, it leads to degradation.

Copied to clipboard

Qin Dai, Benjamin Heinzerling, Kentaro Inui

Library-Like Behavior In Language Models is Enhanced by Self-Referencing Causal Cycles (2025.acl-long)

Challenge:	Existing research has shown that LMs use a concept called Binding ID (BI) to mark entity-attribute pairs, but have not captured the information from entity activations.
Approach:	They propose to localize the Binding ID mechanism by localizing BI information in LMs by encoding it in a low-rank subspace.
Outcome:	The proposed model can infer attributes for a given entity from a container .

Copied to clipboard

Munachiso S Nwadike, Zangir Iklassov, Toluwani Aremu, Tatsuya Hiraoka, Benjamin Heinzerling, Velibor Bojkovic, Hilal AlQuabeh, Martin Takáč, Kentaro Inui

Transformer-based Lexically Constrained Headline Generation (2021.emnlp-main)

Challenge:	Existing models that use sequential data can bypass the limitations of unidirectional causality.
Approach:	They propose a mechanism that enables large language models to bypass unidirectional causality . they propose 'cycle tokens' that enable recall of preceding tokens from succeeding ones .
Outcome:	The proposed model bypasses the limitations of unidirectional causality by enabling recall of preceding contexts.

Copied to clipboard

Kosuke Yamada, Yuta Hitomi, Hideaki Tamori, Ryohei Sasano, Naoaki Okazaki, Kentaro Inui, Koichi Takeda

Japanese-English Sentence Translation Exercises Dataset for Automatic Grading (2024.eacl-srw)

Challenge:	Existing automatic headline generation methods cannot include a given phrase in the generated headline.
Approach:	They propose a Transformer-based method that guarantees to include a given phrase in a generated headline.
Outcome:	The proposed method achieves ROUGE scores comparable to previous methods with Japanese news corpus.

Copied to clipboard

Naoki Miura, Hiroaki Funayama, Seiya Kikuchi, Yuichiroh Matsubayashi, Yuya Iwase, Kentaro Inui

Spelling-out is not Straightforward: LLMs’ Capability of Tokenization from Token to Characters (2025.findings-emnlp)

Challenge:	Sentence translation exercises (STEs) are used in the early stages of L2 language learning.
Approach:	They propose to formalize the task of automatic assessment of Sentence Translation Exercises (STEs) they use a dataset of 21 questions and 3,498 student responses to test the framework .
Outcome:	The proposed task is based on a dataset of 21,498 student responses between Japanese and English and compares them to baseline models with fine-tuned BERT and GPT-3.5 with few-shot learning.

Copied to clipboard

Tatsuya Hiraoka, Kentaro Inui

A Self-Refinement Strategy for Noise Reduction in Grammatical Error Correction (2020.findings-emnlp)

Challenge:	Large language models (LLMs) can spell out tokens character by character with high accuracy, yet struggle with more complex character-level tasks.
Approach:	They examine how large language models internally represent character-level information during the spelling-out process.
Outcome:	The embedding layer does not fully encode character-level information, especially beyond the first character.

Copied to clipboard

Masato Mita, Shun Kiyono, Masahiro Kaneko, Jun Suzuki, Kentaro Inui

LPAttack: A Feasible Annotation Scheme for Capturing Logic Pattern of Attacks in Arguments (2022.lrec-1)

Challenge:	Existing approaches for grammatical error correction (GEC) rely on supervised learning with manually created datasets.
Approach:	They propose to denoise GEC datasets by leveraging prediction consistency of existing models.
Outcome:	The proposed method outperforms baseline methods on CoNLL-2014, JFLEG, and BEA-2019 benchmarks.

Copied to clipboard

Farjana Sultana Mim, Naoya Inoue, Shoichi Naito, Keshav Singh, Kentaro Inui

Robustness in NLP

Challenge:	Argumentation plays a central role in human communication, where refuting or attacking others’ arguments is a common persuasion strategy.
Approach:	They propose a novel annotation scheme that captures common modes and complex rhetorical moves in attacks along with the implicit presuppositions and value judgments.
Outcome:	The proposed scheme shows moderate agreement between the two annotations, indicating that human annotation is feasible.

Repetition Neurons: How Do Language Models Produce Repetitions? (2025.naacl-short)

Copied to clipboard

Tatsuya Hiraoka, Kentaro Inui

The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspaces (2025.naacl-short)

Challenge:	Existing studies on text generation with LLMs focus on attention heads, framing repetition as a key mechanism for in-context learning.
Approach:	They introduce repetition neurons that are regarded as "skill neurons" responsible for the repetition problem in text generation tasks.
Outcome:	The authors identify repetition neurons as "skill neurons" that perceive repetition as a task to copy the previous context repeatedly, similar to in-context learning.

Copied to clipboard

Ahmed Oumar El-Shangiti, Tatsuya Hiraoka, Hilal AlQuabeh, Benjamin Heinzerling, Kentaro Inui

SPIRIT: Patching Speech Language Models against Jailbreak Attacks (2025.emnlp-main)

Challenge:	Existing studies have focused on simple factual recall, but we have not explored how this is used in more complex queries.
Approach:	They propose to identify low-dimensional subspaces which encode numerical attributes associated with entities in comparison prompts.
Outcome:	The proposed model can answer numeric comparison questions using a low-dimensional subspace of theembedding space.

Copied to clipboard

Amirbek Djanibekov, Nurdaulet Mukhituly, Kentaro Inui, Hanan Aldarmaki, Nils Lukas

Robustness in NLP

Challenge:	Speech language models (SLMs) enable natural interactions via spoken instructions, which more effectively capture user intent by detecting nuances in speech.
Approach:	They propose post-hoc patching defenses to intervene during inference by modifying the SLM’s activations that improve robustness up to 99% with negligible impact on utility and without any re-training.
Outcome:	The proposed defenses improve robustness up to 99% with negligible impact on utility and (ii) without any re-training.

How a Bilingual LM Becomes Bilingual: Tracing Internal Representations with Sparse Autoencoders (2025.findings-emnlp)

Copied to clipboard

Tatsuro Inaba, Go Kamoda, Kentaro Inui, Masaru Isonuma, Yusuke Miyao, Yohei Oseki, Yu Takagi, Benjamin Heinzerling

Instance-Based Neural Dependency Parsing (2021.tacl-1)

Challenge:	Using sparse autoencoders, we explore how bilingual language models develop complex internal representations.
Approach:	They employ sparse autoencoders to analyze bilingual language models' internal representations.
Outcome:	The proposed method integrates decomposed representations from a fully trained model into a mid-training model.

Copied to clipboard

Hiroki Ouchi, Jun Suzuki, Sosuke Kobayashi, Sho Yokoi, Tatsuki Kuribayashi, Masashi Yoshikawa, Kentaro Inui

Pointwise HSIC: A Linear-Time Kernelized Co-occurrence Norm for Sparse Linguistic Expressions (D18-1)

Challenge:	Existing models that use instance-based inference for dependency parsing are difficult to understand for humans.
Approach:	They develop neural models that adopt an interpretable inference process for dependency parsing.
Outcome:	The proposed models achieve competitive accuracy with standard neural models and have plausibility of instance-based explanations.

Copied to clipboard

Sho Yokoi, Sosuke Kobayashi, Kenji Fukumizu, Jun Suzuki, Kentaro Inui

Challenge:	Empirically, PHSIC is learned thousands of times faster than an RNN-based PMI while outperforming PMI in accuracy.
Approach:	They propose a new kernel-based co-occurrence measure that can be applied to sparse linguistic expressions with a very short learning time.
Outcome:	The proposed measure can be applied to sparse linguistic expressions with a very short learning time, and is called the pointwise HSIC.

Subword-based Compact Reconstruction of Word Embeddings (N19-1)

Copied to clipboard

Shota Sasaki, Jun Suzuki, Kentaro Inui

Lower Perplexity is Not Always Human-Like (2021.acl-long)

Challenge:	Existing word-based word embeddings are based on subword information and memory-shared embeddables.
Approach:	They propose a method for reconstructing pre-trained word embeddings using subword information using memory-shared embedds and a variant of the key-value-query self-attention mechanism.
Outcome:	The proposed method can imitate well-trained word embeddings in a small fixed space while preventing quality degradation across several linguistic benchmark datasets.

Copied to clipboard

Tatsuki Kuribayashi, Yohei Oseki, Takumi Ito, Ryo Yoshida, Masayuki Asahara, Kentaro Inui

Rectifying Belief Space via Unlearning to Harness LLMs’ Reasoning (2025.findings-acl)

Challenge:	Existing efforts to build human-like computational models have focused on English . a cross-lingual evaluation is needed to build such models, but current research has focused on Japanese .
Approach:	They re-examine an established generalization that lower perplexity is not always human-like in Japanese . they propose a cross-lingual evaluation to build human-type computational models .
Outcome:	The proposed model lacks universality and lower perplexity is not always human-like . the results suggest a cross-lingual evaluation will be necessary to build human-type models .

Copied to clipboard

Ayana Niwa, Masahiro Kaneko, Kentaro Inui

TYPIC: A Corpus of Template-Based Diagnostic Comments on Argumentation (2022.lrec-1)

Challenge:	Large Language Models (LLMs) exhibit sophisticated reasoning yet still generate incorrect answers.
Approach:	They propose a belief space rectification framework that suppresses spurious beliefs and enhances true ones to reduce erroneous reasoning and generalization.
Outcome:	The proposed framework reduces erroneous reasoning and improves generalization on three QA datasets and three LLMs.

Copied to clipboard

Shoichi Naito, Shintaro Sawada, Chihiro Nakagawa, Naoya Inoue, Kenshi Yamaguchi, Iori Shimizu, Farjana Sultana Mim, Keshav Singh, Kentaro Inui

Challenge:	Argumentation and debate are effective tools for developing critical thinking skills, but it requires a lot of time and effort.
Approach:	They propose to automate the process of giving diagnostic comments to students . they define criteria for a template set that can be used to evaluate the model .
Outcome:	The proposed model can be used to evaluate arguments and evaluate them in real time.

TEASPN: Framework and Protocol for Integrated Writing Assistance Environments (D19-3)

Copied to clipboard

Masato Hagiwara, Takumi Ito, Tatsuki Kuribayashi, Jun Suzuki, Kentaro Inui

Cell-Based Representation of Relational Binding in Language Models (2026.acl-long)

Challenge:	TEASPN is an open-source protocol for integrated writing assistance environments . authors propose that developers and researchers can integrate the latest developments in natural language processing with low cost.
Approach:	They propose a protocol and framework for integrating writing aids with writing software.
Outcome:	The proposed protocol standardizes the way writing software communicates with servers that implement such technologies, allowing developers and researchers to integrate the latest developments in natural language processing (NLP) with low cost.

Copied to clipboard

Qin Dai, Benjamin Heinzerling, Kentaro Inui

LLMs Can Compensate for Deficiencies in Visual Representations (2025.findings-emnlp)

Challenge:	Recent work has found evidence that Large Language Models (LLMs) are able to track entities across discourse . however, the mechanism by which they bind entities, relations, and attributes remains unclear .
Approach:	They propose a low-dimensional cell-based binding representation for relational binding . they also show that context-specific CBR representations are related by translation vectors .
Outcome:	The proposed model encodes a low-dimensional cell-based binding representation (CBR) a translation vector in activation space enables cross-context transfer, the study shows .

Copied to clipboard

Sho Takishita, Jay Gala, Abdelrahman Mohamed, Kentaro Inui, Yova Kementchedjhieva

Word Rotator’s Distance (2020.emnlp-main)

Challenge:	a strong language backbone in vision-language models compensates for weak visual features by contextualizing or enriching them.
Approach:	They investigate whether strong language backbone compensates for weak visual features . they use CLIP-based vision encoders to perform controlled self-attention ablations .
Outcome:	The proposed model compensates for weak visual features by contextualizing or enriching them.

Copied to clipboard

Sho Yokoi, Ryo Takahashi, Reina Akama, Jun Suzuki, Kentaro Inui

Representation Learning

Challenge:	Existing approaches to measure textual similarity are inconsistent with the word alignment and are empirically inferior to the simple cosine similarity between general-purpose sentence vectors.
Approach:	They propose to decouple word vectors into their norm and direction and then grow the norm and directions of word vector.
Outcome:	The proposed methods outperform alignment-based approaches on several benchmarks and strong baselines on the semantic textual similarity task.

An Empirical Study of Contextual Data Augmentation for Japanese Zero Anaphora Resolution (2020.coling-main)

Copied to clipboard

Ryuto Konno, Yuichiroh Matsubayashi, Shun Kiyono, Hiroki Ouchi, Ryo Takahashi, Kentaro Inui

Exploring Transitivity in Neural NLI Models through Veridicality (2021.eacl-main)

Challenge:	Existing methods to augment labeled data are limited by the scarcity of labeles . a method called contextual data augmentation (CDA) can be used to augment labels .
Approach:	They propose a data augmentation method that generates labeled training instances using a pretrained language model.
Outcome:	The proposed method can improve the quality of augmented training data compared to the conventional method.

Copied to clipboard

Hitomi Yanaka, Koji Mineshima, Kentaro Inui

Challenge:	Despite recent success of deep neural networks in natural language processing, the extent to which they can demonstrate human-like generalization capacities remains unclear.
Approach:	They propose an analysis method to evaluate whether models can draw inferences composed of veridical inference and arbitrary inference types.
Outcome:	The proposed model performs poorly on transitivity inference tasks, suggesting it lacks generalization capacity for drawing composite inferences from training examples.

Embeddings of Label Components for Sequence Labeling: A Case Study of Fine-grained Named Entity Recognition (2020.acl-srw)

Copied to clipboard

Takuma Kato, Kaori Abe, Hiroki Ouchi, Shumpei Miyawaki, Jun Suzuki, Kentaro Inui

Contrastive Learning-based Sentence Encoders Implicitly Weight Informative Words (2023.findings-emnlp)

Challenge:	In general, the labels used in sequence labeling consist of different types of elements.
Approach:	They propose to integrate label component information as embeddings into sequence labeling models.
Outcome:	The proposed method improves on English and Japanese fine-grained named entity recognition on low-frequency labels.

Copied to clipboard

Hiroto Kurita, Goro Kobayashi, Sho Yokoi, Kentaro Inui

On Entity Identification in Language Models (2025.findings-acl)

Challenge:	Embedding a sentence into a point in a highdimensional continuous space plays a foundational role in the natural language processing.
Approach:	They propose to use contrastive loss to fine-tune sentences by inverse word frequency . they also show that more informative words receive greater weight than less informative ones .
Outcome:	The proposed method improves the performance of sentence embeddings by weighing them based on information-theoretic quantities.

Copied to clipboard

Masaki Sakata, Benjamin Heinzerling, Sho Yokoi, Takumi Ito, Kentaro Inui

What Makes Reading Comprehension Questions Easier? (D18-1)

Challenge:	Existing work has shed light on the internal mechanisms of language models that can recall factual knowledge composed of entities and relations.
Approach:	They propose a framework analogous to clustering quality metrics to analyze the correspondence between entities and their mentions.
Outcome:	The proposed framework is analogous to clustering quality metrics.

Copied to clipboard

Saku Sugawara, Kentaro Inui, Satoshi Sekine, Akiko Aizawa

SHAPE: Shifted Absolute Position Embedding for Transformers (2021.emnlp-main)

Challenge:	Recent studies have shown that questions require a deeper understanding of language to answer beyond using superficial cues.
Approach:	They propose to use simple heuristics to split MRC datasets into easy and hard subsets and manually annotate questions from each subset with validity and reasoning skills to investigate which skills explain the difference between easy and harder questions.
Outcome:	The proposed model performs better for hard and easy questions than for easy questions.

Copied to clipboard

Shun Kiyono, Sosuke Kobayashi, Jun Suzuki, Kentaro Inui

Designing Logic Pattern Templates for Counter-Argument Logical Structure Analysis (2024.findings-emnlp)

Challenge:	Existing position representations suffer from a lack of generalization to test data with unseen lengths or high computational cost.
Approach:	They propose to achieve shift invariance by randomly shifting absolute positions during training by a SHAPE algorithm that is empirically comparable to its counterpart.
Outcome:	The proposed method outperforms existing representations on sequence-to-sequence tasks due to extrapolation, i.e., the ability to generalize to sequences that are longer than those observed during training.

Copied to clipboard

Shoichi Naito, Wenzhi Wang, Paul Reisert, Naoya Inoue, Camélia Guerraoui, Kenshi Yamaguchi, Jungmin Choi, Irfan Robbani, Surawat Pothong, Kentaro Inui

First Heuristic Then Rational: Dynamic Use of Heuristics in Language Model Reasoning (2024.emnlp-main)

Challenge:	Despite their effectiveness, the logical attack structure of counterarguments remains unexplored due to its complexity.
Approach:	They propose a task to analyze logical attack structure of counterarguments in relation to their corresponding opponent argument using 10 new CA logic patterns.
Outcome:	The proposed task achieves high annotator agreement and coverage and high coverage on a dataset of 778 CAs.

Copied to clipboard

Yoichi Aoki, Keito Kudo, Tatsuki Kuribayashi, Shusaku Sone, Masaya Taniguchi, Keisuke Sakaguchi, Kentaro Inui

Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queries (2021.eacl-main)

Challenge:	Explicit multi-step reasoning is widely adopted to improve the performance of language models.
Approach:	They propose a systematic reasoning strategy that LMs use to solve multi-step reasoning tasks.
Outcome:	The proposed strategy improves the performance of language models by combining heuristics with rational strategies.

Copied to clipboard

Benjamin Heinzerling, Kentaro Inui

Learning to Learn to be Right for the Right Reasons (2021.naacl-main)

Challenge:	Pretrained language models have been suggested as an alternative or complement to structured knowledge bases . however, this paradigm has only been considered in a very limited setting .
Approach:	They propose a paradigm that allows LMs to store a large number of entities . they propose LM-as-KB paradigm which allows querying stored facts .
Outcome:	The proposed paradigm allows handling 21k entities whose name is found in common LM vocabularies . the proposed paradigm has only been considered in a very limited setting .

Copied to clipboard

Pride Kavumba, Benjamin Heinzerling, Ana Brassard, Kentaro Inui

Inject Rubrics into Short Answer Grading System (D19-61)

Challenge:	Recent work shows that models trained on held-out data perform poorly on hard instances . previous methods have resorted to manual methods of encouraging models not to overfit to superficial cues .
Approach:	They propose to explicitly learn a model that does well on both easy and hard tests . they use Choice of Plausible Alternatives and Commonsense Explanation to evaluate the model .
Outcome:	The proposed model performs well on easy and hard tests with superficial cues but performs poorly on hard ones without superficial cuings.

Copied to clipboard

Tianqi Wang, Naoya Inoue, Hiroki Ouchi, Tomoya Mizumoto, Kentaro Inui

Challenge:	Short Answer Grading (SAG) is a task of scoring students’ answers in examinations. Existing SAG systems only predict scores based on the answers, but they ignore important evaluation criteria such as rubrics.
Approach:	They propose to inject rubrics into SAG models by introducing word-level attention mechanism into the model to locate information in each answer that are highly related to the score.
Outcome:	The proposed model outperforms the state-of-the-art model on the widely used ASAP-SAS dataset under low-resource settings.

Iterative Span Selection: Self-Emergence of Resolving Orders in Semantic Role Labeling (2022.coling-1)

Copied to clipboard

Shuhei Kurita, Hiroki Ouchi, Kentaro Inui, Satoshi Sekine