Papers by Saket Sharma
Systematic Evaluation of Long-Context LLMs on Financial Concepts (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Long-context large language models (LC LLMs) are promising for tasks with long context windows . however, their ability to reliably utilize their growing context windows remains under investigation . |
| Approach: | They evaluate the performance of long-context large language models using a real-world financial news dataset. |
| Outcome: | The proposed models exhibit brittleness at longer context lengths even for simple tasks, the authors show . they advocate for more rigorous evaluation of LC LLMs by employing holistic metrics such as F1 (rather than recall) |
When and how to paraphrase for named entity recognition? (2023.acl-long)
Copied to clipboard
Saket Sharma, Aviral Joshi, Yiyun Zhao, Namrata Mukhija, Hanoz Bhathena, Prateek Singh, Sashank Santhanam
| Challenge: | Named entity recognition (NER) is a key component underpinning many industrial pipelines for a variety of downstream applications. |
| Approach: | They propose to use back translation to annotate entity spans in generations and propose a paraphraser with a larger dataset. |
| Outcome: | The proposed method improves NER performance across different datasets with gold annotations and paraphrasing strength. |