Papers by Marjorie Freedman
Perhaps PTLMs Should Go to School – A Task to Assess Open Book and Closed Book QA (2021.emnlp-main)
Copied to clipboard
| Challenge: | Taking the exam closed book, but having read the textbook, yields at best minor improvement (56%), suggesting that the PTLM may not have “understood” the textbook (or perhaps misundersttoo the questions). |
| Approach: | They propose to use pre-trained language models to answer questions from introductory college textbooks and hundreds of true/false statements based on review questions written by the authors. |
| Outcome: | The proposed task includes two college-level introductory texts in the social sciences (American Government 2e) and humanities (U.S. History). |
RECAP: Retrieval-Enhanced Context-Aware Prefix Encoder for Personalized Dialogue Response Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to personalized dialogue generation rely on dialogue data paired with user traits, profiles or persona description sentences. |
| Approach: | They propose a hierarchical transformer retriever trained on dialogue domain data to perform personalized retrieval and a context-aware prefix encoder that fuses the retrieved information to the decoder more effectively. |
| Outcome: | The proposed model generates more fluent and personalized responses under a suite of human and automatic metrics and is superior to state-of-the-art baselines on English Reddit conversations. |
Machine-Assisted Script Curation (2021.naacl-demos)
Copied to clipboard
Manuel Ciosici, Joseph Cummings, Mitchell DeHaven, Alex Hedges, Yash Kankanampati, Dong-Ho Lee, Ralph Weischedel, Marjorie Freedman
| Challenge: | Scripts have been of interest for encoding procedural knowledge and understanding stories for over 40 years . narrative descriptions often omit common knowledge . |
| Approach: | They propose a machine-aided script creator that automates script creation with suggestions for event types, links to Wikidata, and sub-events that may have been forgotten. |
| Outcome: | The proposed system automates portions of the script creation process with suggestions for event types, links to Wikidata, and sub-events that may have been forgotten. |
ACQUIRED: A Dataset for Answering Counterfactual Questions In Real-Life Videos (2023.emnlp-main)
Copied to clipboard
Te-Lin Wu, Zi-Yi Dou, Qingyuan Hu, Yu Hou, Nischal Chandra, Marjorie Freedman, Ralph Weischedel, Nanyun Peng
| Challenge: | despite its importance, there are few datasets that cover multimodal counterfactual reasoning . a dataset focusing on this area is limited because of its limited coverage over synthetic environments . |
| Approach: | They develop a video question answering dataset that provides questions on multimodal reasoning . they ask questions about counterfactual hypotheses over visual events . |
| Outcome: | The proposed dataset shows a significant performance gap between models and humans . it provides questions that span physical, social, and temporal dimensions . |
Understanding Multimodal Procedural Knowledge by Sequencing Multimodal Instructional Manuals (2022.acl-long)
Copied to clipboard
| Challenge: | Current machine learning methods are incapable of efficiently utilizing multimodal information. |
| Approach: | They propose to use text-and-image alignment to improve machine learning's performance on multimodal event sequencing. |
| Outcome: | The proposed models perform significantly worse than humans on multimodal event sequencing than humans. |
GAIA: A Fine-grained Multimedia Knowledge Extraction System (2020.acl-demos)
Copied to clipboard
Manling Li, Alireza Zareian, Ying Lin, Xiaoman Pan, Spencer Whitehead, Brian Chen, Bo Wu, Heng Ji, Shih-Fu Chang, Clare Voss, Daniel Napierski, Marjorie Freedman
| Challenge: | Open source knowledge extraction tools are used for many real-world applications, but there is no comprehensive system for KE. |
| Approach: | They propose a multimedia knowledge extraction system that takes multimedia data from various sources and languages as input and creates a coherent, structured knowledge base. |
| Outcome: | The system achieves top performance at the recent NIST TAC SM-KBP2019 evaluation. |
When ACE met KBP: End-to-End Evaluation of Knowledge Base Population with Component-level Annotation (L18-1)
Copied to clipboard
| Challenge: | Automating constructing a Knowledge Base from unstructured text is a goal of natural language processing. |
| Approach: | They propose a method to evaluate a Knowledge Base population from unstructured text . they propose bootstrap resampling to provide statistical significance to the results . |
| Outcome: | The proposed method uses component-level annotations to evaluate Cold Start KBP . it also uses bootstrap resampling to provide statistical significance to the results reported . |
Remember what you did so you know what to do next (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have shown large language models (LLMs) to be poor fit for a simulated robot to achieve 30 classes of goals. |
| Approach: | They use the 6B parameter GPT-J language model to create a plan for a simulated robot to achieve 30 classes of goals in ScienceWorld. |
| Outcome: | The proposed model outperforms the state-of-the-art by a factor of 1.4 when training on as many prior steps as will fit, and the results are 2.3x better than the state of the-art. |
SARAL: A Low-Resource Cross-Lingual Domain-Focused Information Retrieval System for Effective Rapid Document Triage (P19-3)
Copied to clipboard
Elizabeth Boschee, Joel Barry, Jayadev Billa, Marjorie Freedman, Thamme Gowda, Constantine Lignos, Chester Palen-Michel, Michael Pust, Banriskhem Kayang Khonglah, Srikanth Madikeri, Jonathan May, Scott Miller
| Challenge: | a new cross-lingual information retrieval system for low-resource languages is available in less-frequently-taught languages . a multilingual system can search for relevant information in a haystack of documents in swahili or Somali . human-driven approaches to this problem are complicated in 'low-resourced' languages aaron sagar: "the key role played by humans in triaging results is complicated" |
| Approach: | They propose an end-to-end cross-lingual information retrieval system for low-resource languages . the system enables English speakers to search foreign language repositories using English queries . it summarizes the retrieved documents in English with respect to a particular information need . |
| Outcome: | The proposed system achieves top performance in the most recent IARPA MATERIAL CLIR+summarization evaluations. |