Papers by Michihiro Yasunaga
Robust Multilingual Part-of-Speech Tagging via Adversarial Training (N18-1)
Copied to clipboard
| Challenge: | Adversarial training (AT) is a powerful regularization method for neural networks, aiming to achieve robustness to input perturbations. |
| Approach: | They propose and analyze a neural POS tagging model that exploits adversarial training by training on unmodified and adversarials. |
| Outcome: | The proposed model improves overall tagging accuracy and prevents over-fitting in low resource languages and boosts tabbing accuracy for rare / unseen words. |
LinkBERT: Pretraining Language Models with Document Links (2022.acl-long)
Copied to clipboard
| Challenge: | Existing language model pretraining methods do not capture dependencies or knowledge that span across documents. |
| Approach: | They propose a language model pretraining method that leverages links between documents . they use masked language modeling and document relation prediction to model LMs . |
| Outcome: | The proposed method outperforms existing methods on downstream tasks across two domains. |
Is ChatGPT a General-Purpose Natural Language Processing Task Solver? (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in scale have enabled large language models to perform NLP tasks zero-shot . however, it is not known whether ChatGPT can serve as a generalist model that can perform many NLP jobs zero- shot. |
| Approach: | They empirically evaluate ChatGPT's zero-shot learning ability on 20 popular NLP datasets . they find it performs well on many tasks favoring reasoning abilities . |
| Outcome: | The proposed model can perform many NLP tasks zero-shot without adaptation on downstream data. |
SParC: Cross-Domain Semantic Parsing in Context (P19-1)
Copied to clipboard
Tao Yu, Rui Zhang, Michihiro Yasunaga, Yi Chern Tan, Xi Victoria Lin, Suyi Li, Heyang Er, Irene Li, Bo Pang, Tao Chen, Emily Ji, Shreya Dixit, David Proctor, Sungrok Shim, Jonathan Kraft, Vincent Zhang, Caiming Xiong, Richard Socher, Dragomir Radev
| Challenge: | Xu et al., 2017): a dataset for cross-domain semantic parsing in context with 4,298 question sequences. |
| Approach: | They present a dataset for cross-domainSemanticParsing inContext that consists of 4,298 coherent question sequences. |
| Outcome: | The proposed dataset demonstrates that it has greater semantic diversity and can be generalized to unseen domains due to its cross-domain nature and the unseened databases at test time. |
Complex Reasoning in Natural Language (2023.acl-tutorials)
Copied to clipboard
| Challenge: | Recent research shows that pretrained language models are often brittle for complex reasoning tasks. |
| Approach: | They propose to use pre-trained language models to teach machines to reason over texts . they will review recent promising approaches to tackling complex reasoning tasks . |
| Outcome: | This tutorial reviews promising approaches to complex reasoning tasks . it reviews the methods that can be used to augment models with robustness . |
SyntaxSQLNet: Syntax Tree Networks for Complex and Cross-Domain Text-to-SQL Task (D18-1)
Copied to clipboard
| Challenge: | Existing studies in text-to-SQL do not require generating complex SQL queries with multiple clauses or sub-queries. |
| Approach: | They propose a syntax tree network to address the complex text-to-SQL generation task. |
| Outcome: | The proposed model outperforms the current state-of-the-art model by 9.5% on a large text-to-SQL corpus. |
LM-Critic: Language Models for Unsupervised Grammatical Error Correction (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work casts GEC as a translation problem using encoder-decoder models to map bad (ungrammatical) sentences into good (grammatically) sentences. |
| Approach: | They propose to use a pretrained language model to define an LM-Critic that judges a sentence to be grammatical if the LM assigns it a higher probability than its local perturbations. |
| Outcome: | The proposed method outperforms existing methods in both the unsupervised and supervised setting. |
REPLUG: Retrieval-Augmented Black-Box Language Models (2024.naacl-long)
Copied to clipboard
Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Richard James, Mike Lewis, Luke Zettlemoyer, Wen-tau Yih
| Challenge: | Existing retrieval-augmented language models require access to internal representations to enhance performance. |
| Approach: | They introduce a retrieval-augmented language modeling framework that treats the language model as a black box and augments it with a tuneable retrieval model. |
| Outcome: | The proposed framework improves performance on language modeling tasks by 6.3% and 5.1%. |
Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models (2023.findings-acl)
Copied to clipboard
Yuhui Zhang, Michihiro Yasunaga, Zhengping Zhou, Jeff Z. HaoChen, James Zou, Percy Liang, Serena Yeung
| Challenge: | Recent studies show that some tasks exhibit inverse scaling, or U-shaped scaling, where the performance degrades as models are scaled up. |
| Approach: | They propose a task that asks questions with negation to show positive scaling . they hypothesize that solving NeQA depends on question answering and negation understanding . |
| Outcome: | The proposed task can exhibit inverse scaling, U-shaped scaling, or positive scaling, and the scaling trends shift as the task is more powerful. |
Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task (D18-1)
Copied to clipboard
Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, Dragomir Radev
| Challenge: | Existing datasets for semantic parsing are too small in terms of number of programs for training modern data-intensive models. |
| Approach: | They propose a large-scale complex and cross-domain semantic parsing task for a database . they use a dataset with 10,181 questions and 5,693 unique complex SQL queries . |
| Outcome: | The proposed task is different from previous tasks because it uses the same database and program . the best model achieves only 9.7% exact matching accuracy on a database split setting. |
How to Make LMs Strong Node Classifiers? (2026.findings-eacl)
Copied to clipboard
Zhe Xu, Kaveh Hassani, Si Zhang, Hanqing Zeng, Michihiro Yasunaga, Limei Wang, Dongqi Fu, Ning Yao, Bo Long, Hanghang Tong
| Challenge: | Language Models (LMs) are increasingly challenging the dominance of domain-specific models, such as Graph Neural Networks (GNNs) and Graph Transformers (GTs). |
| Approach: | They propose a novel approach that empowers off-the-shelf LMs to achieve performance comparable to state-of-the art (SOTA) GNNs on node classification tasks without requiring any architectural modifications. |
| Outcome: | The proposed approach outperforms existing GNNs on node classification tasks and is open-source upon publication. |
UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models (2022.emnlp-main)
Copied to clipboard
Tianbao Xie, Chen Henry Wu, Peng Shi, Ruiqi Zhong, Torsten Scholak, Michihiro Yasunaga, Chien-Sheng Wu, Ming Zhong, Pengcheng Yin, Sida I. Wang, Victor Zhong, Bailin Wang, Chengzu Li, Connor Boyle, Ansong Ni, Ziyu Yao, Dragomir Radev, Caiming Xiong, Lingpeng Kong, Rui Zhang, Noah A. Smith, Luke Zettlemoyer, Tao Yu
| Challenge: | Structured knowledge grounding (SKG) uses structured knowledge to complete user requests . since inputs and outputs of SKG tasks are heterogeneous, they have been studied separately . |
| Approach: | They propose a framework that unifies 21 SKG tasks into a text-to-text format . they use unifiedSKG to benchmark T5 with different sizes . |
| Outcome: | The proposed framework unifies 21 SKG tasks into a text-to-text format . it achieves state-of-the-art performance on almost all of the 21 tasks, the authors show . |
Neural Coreference Resolution with Deep Biaffine Attention by Joint Mention Detection and Mention Clustering (P18-2)
Copied to clipboard
| Challenge: | Existing methods for identifying and clustering mentions in text are complex and require heuristics to solve. |
| Approach: | They propose to use a biaffine attention model to get antecedent scores for each possible mention and optimize mention detection and mention clustering accuracy given the mention cluster labels. |
| Outcome: | The proposed model achieves the state-of-the-art performance on the CoNLL-2012 shared task English test set. |
QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing question answering systems lack the ability to access relevant knowledge and reason over it. |
| Approach: | They propose a model that uses KGs to identify relevant knowledge in QA contexts and perform joint reasoning over them. |
| Outcome: | The proposed model improves on the CommonsenseQA and OpenBookQA datasets and performs interpretable and structured reasoning. |
reWordBench: Benchmarking and Improving the Robustness of Reward Models with Transformed Inputs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing reward models have a high performance on benchmarks, but performance degradation is often due to overfitting. |
| Approach: | They propose to explicitly train reward models to assign similar scores to paraphrases to improve their robustness. |
| Outcome: | The proposed model reduces degradation by half for the Chat Hard subset in RewardBench. |
CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases (D19-1)
Copied to clipboard
Tao Yu, Rui Zhang, Heyang Er, Suyi Li, Eric Xue, Bo Pang, Xi Victoria Lin, Yi Chern Tan, Tianze Shi, Zihan Li, Youxuan Jiang, Michihiro Yasunaga, Sungrok Shim, Tao Chen, Alexander Fabbri, Zifan Li, Luyao Chen, Yuwen Zhang, Shreya Dixit, Vincent Zhang, Caiming Xiong, Richard Socher, Walter Lasecki, Dragomir Radev
| Challenge: | CoSQL is a corpus for building cross-domain, general-purpose database querying dialogue systems. |
| Approach: | They present a corpus for building cross-domain, general-purpose database querying dialogue systems . they use a Wizard-of-Oz collection of 3k turns plus 10k+ annotated SQL queries . |
| Outcome: | The proposed corpus is based on a Wizard-of-Oz dataset of 3k dialogues querying 200 complex DBs spanning 138 domains. |