Papers by Pragya Srivastava
Evaluating LLMs’ Mathematical Reasoning in Financial Document Question Answering (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models excel in natural language understanding, but their capability for complex mathematical reasoning with a hybrid of structured tables and unstructured text remain uncertain. |
| Approach: | They propose a prompting technique tailored to semi-structured documents that matches or outperforms baselines performance while providing a nuanced understanding of LLMs' abilities. |
| Outcome: | The proposed prompting technique outperforms baseline prompting techniques while providing a nuanced understanding of LLMs' abilities. |
NICE: To Optimize In-Context Examples or Not? (2024.acl-long)
Copied to clipboard
| Challenge: | Recent work shows that in-context learning and optimization of in-const examples (ICE) can improve the accuracy of large language models on a wide range of tasks. |
| Approach: | They propose a task-specific metric called Normalized Invariability to Choice of Examples (NICE) metric measures the learnability of tasks from a given instruction and provides a heuristic to decide whether to optimize ICE for a new task. |
| Outcome: | The proposed metric predicts the utility of optimizing ICE for a given task compared to random ICE. |