Papers by Catherine Finegan-Dollak
Improving Text-to-SQL Evaluation Methodology (P18-1)
Copied to clipboard
Catherine Finegan-Dollak, Jonathan K. Kummerfeld, Li Zhang, Karthik Ramanathan, Sesh Sadasivam, Rui Zhang, Dragomir Radev
| Challenge: | Current evaluations of text-to-SQL systems are limited by the way they divide data into training and test sets. |
| Approach: | They propose to standardize and improve existing and new text-to-SQL datasets . they propose a template-based slot-filling baseline that cannot generalize to new queries . |
| Outcome: | The proposed system is competitive with prior work on multiple datasets and can be used on training and test sets. |
Label Noise in Context (2020.acl-demos)
Copied to clipboard
| Challenge: | Label noise—incorrectly or ambiguously labeled training examples—can negatively impact model performance. |
| Approach: | They propose a noise-detection method that uses an example's neighborhood within the training set to reduce false positives and provide an explanation as to why the ex ample was flagged as noise. |
| Outcome: | The proposed method outperforms the state-of-the-art on precision and F0.5-score on short-text classification datasets. |
Grounded, or a Good Guesser? A Per-Question Balanced Dataset to Separate Blind from Grounded Models for Embodied Question Answering (2025.acl-short)
Copied to clipboard
Miles Shelton, Nate Wingerd, Kritim K Rijal, Ayush Garg, Adelina Gutic, Brett Barnes, Catherine Finegan-Dollak
| Challenge: | Embodied question answering (EQA) is based on using perception and action in an environment to answer natural language questions. |
| Approach: | They propose a "per-question balanced" EQA dataset that uses two different environments to ground a model's answers in its environment. |
| Outcome: | The proposed model performs better than chance on the PQB-EQA benchmark, showing that it does not require the model to use perception, let alone to act in its environment to find the answer. |
GVdoc - Graph-based Visual DOcument Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | Visual document classifiers have shown impressive performance on in-distribution test sets, but they have a hard time classifying out-of-difference examples. |
| Approach: | They propose a graph-based document classification model that generates a document graph based on its layout and trains a neural network to learn node and graph embeddings. |
| Outcome: | The proposed model outperforms state-of-the-art models on out-of distribution data while maintaining comparable performance on in-distribution data. |
Effective Crowdsourcing for a New Type of Summarization Task (N18-2)
Copied to clipboard
| Challenge: | Currently, summarization research focuses on summarizing the entire text, but in practice, readers are often interested in only one aspect of the document or conversation. |
| Approach: | They propose a new task where the goal is to summarize a particular aspect of a document. |
| Outcome: | The proposed task is based on a crowdsourced data collection workflow that allows users to collect high-quality summaries. |