Papers by Srini Iyer
FiD-Ex: Improving Sequence-to-Sequence Models for Extractive Rationale Generation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained sequence to sequence models are effective in making and generating NL explanations, but they have many shortcomings. |
| Approach: | They propose a model that uses sentence markers to eliminate explanation fabrication . they use fusion-in-decoder architecture to handle long input contexts . |
| Outcome: | The proposed model significantly improves on the ERASER explainability benchmark. |
Improving In-Context Few-Shot Learning via Self-Supervised Training (2022.naacl-main)
Copied to clipboard
Mingda Chen, Jingfei Du, Ramakanth Pasunuru, Todor Mihaylov, Srini Iyer, Veselin Stoyanov, Zornitsa Kozareva
| Challenge: | Existing approaches to improve in-context few-shot learning are pretraining and downstream fewshot evaluation. |
| Approach: | They propose to use self-supervision as an intermediate training stage between pretraining and downstream fewshot usage to train models to perform in-context few shot learning. |
| Outcome: | The proposed model outperforms baseline models on two benchmarks. |
ACUEval: Fine-grained Hallucination Evaluation and Correction for Abstractive Summarization (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent-proposed evaluation metrics for large language models have a preference-bias . however, such metrics often lack interpretability and only offer a single score . |
| Approach: | They propose a metric that leverages the power of large language models to perform two sub-tasks: decomposing summaries into atomic content units and validating them against the source document. |
| Outcome: | The proposed metric improves faithfulness scores on three summarization evaluation benchmarks by 3% compared to the next-best metric. |
Instruction-tuned Language Models are Better Knowledge Learners (2024.acl-long)
Copied to clipboard
Zhengbao Jiang, Zhiqing Sun, Weijia Shi, Pedro Rodriguez, Chunting Zhou, Graham Neubig, Xi Lin, Wen-tau Yih, Srini Iyer
| Challenge: | Large language models store factual knowledge in parameters, but it can become outdated as the work evolves . pre-instruction-tuning improves ability of LLMs to absorb knowledge from new documents . |
| Approach: | They propose a method that instruction-tunes on questions prior to training on documents . they propose to use QA pairs to update factual knowledge of large language models . |
| Outcome: | The proposed method outperforms instruction-tuning on documents by 17.8%. |
Byte Latent Transformer: Patches Scale Better Than Tokens (2025.acl-long)
Copied to clipboard
Artidoro Pagnoni, Ramakanth Pasunuru, Pedro Rodriguez, John Nguyen, Benjamin Muller, Margaret Li, Chunting Zhou, Lili Yu, Jason E Weston, Luke Zettlemoyer, Gargi Ghosh, Mike Lewis, Ari Holtzman, Srini Iyer
| Challenge: | Existing large language models (LLMs) are trained on bytes, except for tokenization, which groups bytes into a static set of tokens. |
| Approach: | They propose a new byte-level LLM architecture that encodes bytes into dynamically sized patches, which serve as the primary units of computation. |
| Outcome: | The proposed architecture matches tokenization-based models with improvements in inference efficiency and robustness. |
AnswerSumm: A Manually-Curated Dataset and Pipeline for Answer Summarization (2022.naacl-main)
Copied to clipboard
| Challenge: | Community Question Answering (CQA) fora lack a dataset to produce answer summarizations . a novel dataset of 4,631 CQA threads is used to generate answer summaries . |
| Approach: | They propose a dataset of 4,631 CQA threads for answer summarization curated by professional linguists. |
| Outcome: | The proposed approach boosts summarization performance according to automatic evaluation. |
Demystifying Prompts in Language Models via Perplexity Estimation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Language models can be prompted to perform a wide variety of tasks with zero- and few-shot learning. |
| Approach: | They propose a method to automatically extend a small seed set of manually written prompts by paraphrasing with GPT3 and backtranslation. |
| Outcome: | The proposed method extends a small seed set of manually written prompts by paraphrasing with GPT3 and backtranslation. |