Papers by Ellen Zhang
On Evaluating the Integration of Reasoning and Action in LLM Agents with Database Question Answering (2024.findings-naacl)
Copied to clipboard
| Challenge: | a new study evaluates how Large Language Models interact with a SQL interpreter . the model is limited in context and is stochastic, making it less suited for tasks requiring high precision and extensive computations. |
| Approach: | They propose and evaluate two interaction strategies to evaluate how LLMs interact with a SQL interpreter. |
| Outcome: | The proposed framework improves the accuracy and reliability of the evaluations. |
Enhancing Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategies (2023.findings-emnlp)
Copied to clipboard
Linyong Nan, Yilun Zhao, Weijin Zou, Narutatsu Ri, Jaesung Tae, Ellen Zhang, Arman Cohan, Dragomir Radev
| Challenge: | In-context learning (ICL) is a new approach to natural language processing tasks that rely on large language models to make predictions based on context . recent studies have shown that neural symbolic design is the preferred choice for question answering systems because of its limited working memory and unreliable long-term memory. |
| Approach: | They propose to extend in-context learning to question answering tasks that utilize structured knowledge sources and to explore various prompt design strategies for employing LLMs. |
| Outcome: | The proposed approach outperforms the state-of-the-art system by 2.5 points and the best fine-tuned system by 5.1 points on the Spider dataset. |