Papers by Harsh Trivedi
DeFormer: Decomposing Pre-trained Transformers for Faster Question Answering (2020.acl-main)
Copied to clipboard
| Challenge: | DeFormer is a transformer-based QA model that uses input-wide self-attention at all layers, causing them to be slow and memory-intensive. |
| Approach: | They propose a transformer which substitutes the full self-attention with question-wide and passage-wide self- attentions in the lower layers. |
| Outcome: | The proposed model can be used to speed up QA by over 4.3x and with simple distillation-based losses they incur only a 1% drop in accuracy. |
MoNaCo: More Natural and Complex Questions for Reasoning Across Dozens of Documents (2026.tacl-1)
Copied to clipboard
Tomer Wolfson, Harsh Trivedi, Mor Geva, Yoav Goldberg, Dan Roth, Tushar Khot, Ashish Sabharwal, Reut Tsarfaty
| Challenge: | Automated agents powered by large language models are becoming more ingrained into how people seek information . but evaluation benchmarks for LLMs rarely feature natural questions that are time-consuming . a new benchmark, MoNaCo, aims to address this gap by eliciting and manually answering time-wasting questions . |
| Approach: | They propose a benchmark of 1,315 natural and time-consuming questions that require dozens of intermediate steps to solve. |
| Outcome: | MoNaCo benchmarks achieve at least 61.2% F1 in real-world time-consuming questions hampered by low recall and hallucinations . Frontier LLMs evaluated on MoN achieving at least 61% F1, harmed by low memory and halluzinations. |
IrEne-viz: Visualizing Energy Consumption of Transformer Models (2021.emnlp-demo)
Copied to clipboard
Yash Kumar Lal, Reetu Singh, Harsh Trivedi, Qingqing Cao, Aruna Balasubramanian, Niranjan Balasubramanian
| Challenge: | IrEne is an energy prediction system that accurately predicts inference energy consumption of transformer-based NLP models. |
| Approach: | They present an online platform for visualizing and exploring energy consumption of transformer-based NLP models. |
| Outcome: | The proposed system predicts energy consumption of transformer-based models and their components. |
Summarize-then-Answer: Generating Concise Explanations for Multi-hop Reading Comprehension (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent approaches to multi-hop Reading Comprehension (RC) have greatly improved its explainability, models ability to explain their own answers. |
| Approach: | They propose to generate a question-focused abstractive summary of input paragraphs and feed it to an RC system. |
| Outcome: | The proposed explanation generates more compact explanations than an extractive explainer with limited supervision while maintaining sufficiency. |
Is Multihop QA in DiRe Condition? Measuring and Reducing Disconnected Reasoning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models exploit dataset artifacts to produce correct answers without connecting information across multiple facts. |
| Approach: | They formalize disconnected reasoning across subsets of supporting facts to reduce disconnected reasoning . they propose an automatic transformation of existing datasets that reduces disconnected reasoning. |
| Outcome: | The proposed model-agnostic probe reduces disconnected reasoning in a reading comprehension setting. |
What Ingredients Make for an Effective Crowdsourcing Protocol for Difficult NLU Data Collection Tasks? (2021.acl-long)
Copied to clipboard
| Challenge: | Despite the importance of datasets for natural language understanding, there has been little attention on crowdsourcing methods for collecting datasets. |
| Approach: | They compare the effectiveness of crowdsourcing methods for boosting NLU example difficulty with training crowdworkers instead of expert judgments. |
| Outcome: | The proposed method is ineffective for boosting NLU example difficulty, but it is not effective for training crowdworkers and qualifying workers based on expert judgments. |
Repurposing Entailment for Multi-Hop Question Answering Tasks (N19-1)
Copied to clipboard
| Challenge: | Existing approaches to use entailment models for question answering are limited . large scale datasets are typically framed at a sentence level, whereas question answering requires verifying whether multiple sentences, taken together as a premise, entitle a hypothesis. |
| Approach: | They propose a general architecture that can use entailment models for multi-hop QA tasks. |
| Outcome: | The proposed model outperforms QA models trained on target datasets and the OpenAI transformer models. |
♫ MuSiQue: Multihop Questions via Single-hop Question Composition (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing multihop reasoning benchmarks are largely solvable via shortcuts . a bottom–up approach allows us to create a multihop QA dataset that requires proper multihop thinking. |
| Approach: | They propose a bottom–up approach that selects composable pairs of single-hop questions that are connected and adds stringent filters to the construction process. |
| Outcome: | The proposed approach creates a multihop question answering dataset with 25K 2–4 hop questions. |
Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions (2023.acl-long)
Copied to clipboard
| Challenge: | Large language models generate natural language reasoning steps or Chains-of-Thoughts when prompted appropriately. |
| Approach: | They propose a new approach that interleaves retrieval with steps (sentences) in a CoT and uses retrieved results to improve CoT. |
| Outcome: | The proposed approach improves retrieval and downstream QA significantly on four datasets. |
IrEne: Interpretable Energy Prediction for Transformers (2021.acl-long)
Copied to clipboard
| Challenge: | Existing software-based energy measurements of NLP models are not accurate because they do not consider the complex interactions between energy consumption and model execution. |
| Approach: | They propose an interpretable and extensible energy prediction system that predicts inference energy consumption of Transformer-based NLP models. |
| Outcome: | The proposed system predicts inference energy consumption of transformer models with an error of under 7% compared to the ground truth. |
Teaching Broad Reasoning Skills for Multi-Step QA by Generating Hard Contexts (2022.emnlp-main)
Copied to clipboard
| Challenge: | Question-answering datasets require a broad set of reasoning skills. |
| Approach: | They use QDMR representations to programmatically create hard-to-cheat synthetic contexts for real questions in multi-step reasoning datasets. |
| Outcome: | The proposed model improves performance by 13 F1 points on 4 multi-step QA datasets and 21 points on more complex questions. |