Papers by Nitish Joshi
Cross-Lingual Training for Automatic Question Generation (P19-1)
Copied to clipboard
| Challenge: | Automatic question generation is a challenging problem in natural language understanding . manual curating a dataset of comparable size for a new language is tedious and expensive. |
| Approach: | They propose to reuse available large QG dataset in a secondary language to learn a QG model for a primary language. |
| Outcome: | The proposed model outperforms baseline models in Hindi and Chinese. |
LLMs Are Prone to Fallacies in Causal Inference (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that causal facts can be extracted from LLMs through prompting . but it is unclear if this success is limited to explicitly-mentioned causal facts in pretraining data . |
| Approach: | They fine tune LLMs on synthetic data and test whether they can infer causal relations . they find that LLM can correctly deduce absence of causal relations from temporal and spatial relations if order is randomized . |
| Outcome: | The proposed model outperforms existing methods on causal inference tasks. |
Explore, Propose, and Assemble: An Interpretable Model for Multi-Hop Reading Comprehension (P19-1)
Copied to clipboard
| Challenge: | Existing models for multi-hop reading comprehension only require a single-hop reasoning, meaning that the evidence needed to answer the question is scattered in a set of supporting documents. |
| Approach: | They propose an interpretable 3-module system called Explore-Propose-Assemble reader (EPAr) that explores and connects relevant information from multiple documents in order to answer a question about the context. |
| Outcome: | The proposed model approximates coarse-to-fine-grained comprehension behavior of human readers when facing multiple long documents. |
Measuring Inductive Biases of In-Context Learning with Underspecified Demonstrations (2023.acl-long)
Copied to clipboard
| Challenge: | In-context learning is an important paradigm for adapting large language models to new tasks . but the generalization behavior of ICL remains poorly understood . |
| Approach: | They characterize the feature biases of large language models by constructing underspecified demonstrations . they find that LLMs exhibit clear feature bias, and they evaluate interventions . |
| Outcome: | The proposed model prefers the "default" task features over distractor features more often than the base model. |
QuALITY: Question Answering with Long Input Texts, Yes! (2022.naacl-main)
Copied to clipboard
Richard Yuanzhe Pang, Alicia Parrish, Nitish Joshi, Nikita Nangia, Jason Phang, Angelica Chen, Vishakh Padmakumar, Johnny Ma, Jana Thompson, He He, Samuel Bowman
| Challenge: | Existing models for natural language understanding are limited to processing only a few hundred words at a time. |
| Approach: | They propose a dataset with context passages in English that have an average length of 5,000 tokens. |
| Outcome: | a new dataset with long-text comprehension questions is used to test models on long-document comprehension . the questions are validated by contributors who have read the entire passage, not just excerpts . only half of the questions can be answered by annotators working under tight time constraints . |
An Investigation of the (In)effectiveness of Counterfactually Augmented Data (2022.acl-long)
Copied to clipboard
| Challenge: | Pretrained language models tend to rely on spurious correlations and generalize poorly to out-of-distribution (OOD) data. |
| Approach: | They propose to use counterfactually-augmented data (CAD) to identify robust features that are invariant under distribution shift to train models for OOD generalization. |
| Outcome: | The proposed model can learn robust features that are invariant under distribution shifts, but lacks spurious correlations, and may exacerbate existing correlations. |
Personas as a Way to Model Truthfulness in Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models are trained on vast amounts of text from the internet, which contains factual and misleading information. |
| Approach: | They hypothesize that the pretraining data is generated by groups of (un)truthful agents whose outputs share common features and form a (un-truthfully persona) this allows the model to separate truth from falsehoods and controls the truthfulness of its generation. |
| Outcome: | The proposed model can infer truth from falsehoods by finetuning its model on a set of facts and finetuned it on unseen topics. |
Are All Spurious Features in Natural Language Alike? An Analysis through a Causal Lens (2022.emnlp-main)
Copied to clipboard
| Challenge: | 'spurious correlations' have been used in NLP to informally denote any undesirable feature-label correlations. |
| Approach: | They formalize this distinction using a causal model and probabilities of necessity and sufficiency, which delineates causal relations between a feature and a label. |
| Outcome: | The proposed model is invariant to the feature, but not sufficient for prediction. |