Papers by Kazuma Hashimoto
Accelerated Reinforcement Learning for Sentence Generation by Vocabulary Prediction (N19-1)
Copied to clipboard
| Challenge: | In reinforcement learning-based sentence generation, the large action space is often too computa-tionally demanding to be used with large training data. |
| Approach: | They propose to reduce the action space by using dynamic vocabulary prediction to generate a fixed-size small vocabulary for each input to generate its target sentence. |
| Outcome: | The proposed method achieves faster reinforcement learning (2.7x faster) with less GPU memory (2.3x less) and more rewards with fewer iterations of supervised pre-training. |
Take One Step at a Time to Know Incremental Utility of Demonstration: An Analysis on Reranking for Few-Shot In-Context Learning (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent advances of Large Language Models (LLMs) have been pushing the field of Natural Language Processing (NLP) to the next level in many different aspects. |
| Approach: | They propose a novel labeling method which estimates how much incremental knowledge is brought into LLMs by a demonstration. |
| Outcome: | The proposed method estimates how much incremental knowledge is brought into the LLMs by a demonstration. |
OneAligner: Zero-shot Cross-lingual Transfer with One Rich-Resource Language Pair for Low-Resource Sentence Retrieval (2022.findings-acl)
Copied to clipboard
| Challenge: | a new model for parallel sentence retrieval can be used to align parallel sentences in multilingual corpora . a faithful aligner can help narrow down the candidate pool without having to deal with an enormous search space . |
| Approach: | They propose a model that can be trained on only one language pair and transfers to low-resource languages with negligible degradation in performance. |
| Outcome: | The proposed model outperforms the previous model on the Tateoba dataset by 8.0 points in accuracy and using less than 0.6% of their parallel data. |
Transforming Sequence Tagging Into A Seq2Seq Task (2022.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained, large, generative language models have had great success in a wide range of sequence tagging and structured prediction tasks. |
| Approach: | They propose to use a new format for casting input text sentences and their output labels into the input and target of a Seq2Seq model and introduce it to test their hypothesis. |
| Outcome: | The proposed format shows to be both simpler and more effective and devoid of hallucination. |
Simple Data Augmentation with the Mask Token Improves Domain Adaptation for Dialog Act Tagging (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on DA tagging focus on human-human social conversations, which is less applicable for task-oriented setting. |
| Approach: | They propose a controllable mechanism that augments text input by leveraging the pre-trained Mask token from BERT model. |
| Outcome: | The proposed mechanism augments text input by leveraging the pre-trained Mask token from BERT model. |
Modeling Multi-hop Question Answering as Single Sequence Prediction (2022.acl-long)
Copied to clipboard
| Challenge: | Existing generative question answering models that leverage passage retrieval with a pre-trained transformer are not effective for multihop QA. |
| Approach: | They propose a generative approach that explicitly models the reasoning process to resolve the answer for multi-hop questions by encoding cross-passage interactions. |
| Outcome: | The proposed model improves on two multi-hop QA datasets and is interpretable. |
Focused Attention Improves Document-Grounded Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | Document grounded generation is the task of using the information provided in a document to improve text generation. |
| Approach: | They propose two new document grounded generation tasks that use information provided in a document to improve text generation. |
| Outcome: | The proposed models outperform existing methods on automated and human evaluation for closeness to reference and relevance to the document. |
Dense Hierarchical Retrieval for Open-domain Question Answering (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Dense neural text retrieval has achieved promising results on open-domain Question Answering (QA) current dense retrievers require splitting documents into short passages that usually contain local, partial and sometimes biased context, and may yield inaccurate and misleading hidden representations, thus deteriorating the final retrieval result. |
| Approach: | They propose a hierarchical framework which can generate accurate dense representations of passages by utilizing both macroscopic semantics in the document and microscopic specific to each passage. |
| Outcome: | The proposed framework significantly outperforms the original dense passage retriever and helps an end-to-end QA system outperfect the strong baselines on multiple open-domain QA benchmarks. |
RNG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answering (2022.acl-long)
Copied to clipboard
| Challenge: | Existing KBQA approaches struggle with generalization of unseen KB schema items . Rank-and-generate approach solves coverage issue with strong generalization . |
| Approach: | They propose a Rank-and-Generate approach for KBQA that uses a generation model to generalize to unseen KB schema items. |
| Outcome: | The proposed approach outperforms the prior state-of-the-art on GrailQA and WebQSP datasets. |
[CASPI] Causal-aware Safe Policy Improvement for Task-oriented Dialogue (2022.acl-long)
Copied to clipboard
| Challenge: | Recent advances in off-policy reinforcement learning methods that use offline data as against a simulator have proven to be sample efficient. |
| Approach: | They propose a batch-RL framework for ToD policy learning: Causal-aware Safe Policy Improvement (CASPI) that uses a mechanism to learn fine-grained reward that captures intention behind human response and offers guarantee on dialogue policy’s performance against a baseline. |
| Outcome: | The proposed framework outperforms the current state of the art on an end-to-end dialogue task using a multiwoz2.0 dataset. |
Find or Classify? Dual Strategy for Slot-Value Predictions on Multi-Domain Dialog State Tracking (2020.starsem-1)
Copied to clipboard
| Challenge: | Existing methods for dialog state tracking are ontology-based and ontologie-free . however, it is not clear enough which slots are better handled by either of the two methods . |
| Approach: | They propose a dual-strategy model that integrates both ontology-based and ontological-free methods. |
| Outcome: | The proposed model outperforms the existing model on noisy and cleaner datasets. |
Discriminative Nearest Neighbor Few-Shot Intent Detection by Transferring Natural Language Inference (2020.emnlp-main)
Copied to clipboard
Jianguo Zhang, Kazuma Hashimoto, Wenhao Liu, Chien-Sheng Wu, Yao Wan, Philip Yu, Richard Socher, Caiming Xiong
| Challenge: | Existing work on few-shot intent classification without OOS has focused on the few-shot intent classification with out-of-scope intents. |
| Approach: | They propose to use BERT-style pairwise encoding to train a binary classifier that estimates the best matched training example for a user input. |
| Outcome: | The proposed approach achieves more stable and accurate in-domain and OOS detection accuracy than RoBERTa-based classifiers and embedding-based nearest neighbor approaches. |
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control (2022.findings-naacl)
Copied to clipboard
| Challenge: | Abstractive summarization systems have been shown to be more prone to unfaithful facts . 30% of summaries generated by pre-trained language models suffer from hallucination . |
| Approach: | They propose a method to remedy entity-level extrinsic hallucinations with Entity Coverage Control . they first compute entity coverage precision and prepend the corresponding control code . a further fine-tuning is performed to unlock zero-shot summarization . |
| Outcome: | The proposed method leads to more faithful and salient abstractive summarization in fine-tuning and zero-shot settings. |