Papers by James Pustejovsky
ChainNet: Structured Metaphor and Metonymy in WordNet (2024.lrec-main)
Copied to clipboard
| Challenge: | In a typical lexicon, word senses are encoded as a list, without inter-sense relations. |
| Approach: | They propose a lexical resource which explicitly identifies the senses of a word's senses by expressing how they are derived from one another. |
| Outcome: | The proposed resource expresses how senses in the Open English Wordnet are derived from one another. |
Assessing the Efficacy of Clinical Sentiment Analysis and Topic Extraction in Psychiatric Readmission Risk Prediction (D19-62)
Copied to clipboard
Elena Alvarez-Mellado, Eben Holderness, Nicholas Miller, Fyonn Dhang, Philip Cawkwell, Kirsten Bolton, James Pustejovsky, Mei-Hua Hall
| Challenge: | Previously, readmission risk classifications rely on structured information, such as sociodemographic data, comorbidity codes and physiological variables. |
| Approach: | They propose to incorporate additional clinically interpretable NLP-based features such as topic extraction and clinical sentiment analysis to predict early readmission risk in psychiatry patients. |
| Outcome: | The proposed model incorporates topic extraction and clinical sentiment analysis to predict early readmission risk in psychiatry patients. |
Bridging the LAPPS Grid and CLARIN (L18-1)
Copied to clipboard
Erhard Hinrichs, Nancy Ide, James Pustejovsky, Jan Hajič, Marie Hinrichs, Mohammad Fazleh Elahi, Keith Suderman, Marc Verhagen, Kyeongmin Rim, Pavel Straňák, Jozef Mišutka
| Challenge: | The LAPPS-CLARIN project is creating a "trust network" between the Language Applications Grid and WebLicht workflow engine . the goal is to allow users on one side of the bridge to gain appropriately authenticated access to the other . |
| Approach: | The LAPPS-CLARIN project is creating a "trust network" between the Language Applications Grid and WebLicht workflow engine hosted by the CLARIN-D Center in Tübingen. |
| Outcome: | The LAPPS-CLARIN project is creating a "trust network" between the Language Applications (LAPPS) Grid and the WebLicht workflow engine hosted by the CLARIN-D Center in Tübingen. |
An Evaluation Framework for Multimodal Interaction (L18-1)
Copied to clipboard
| Challenge: | a framework for evaluating multimodal interactions is presented . it leverages the semantics of language and gesture to assess mutual understanding . consistent evaluation is required to test areas where the system needs improvement . |
| Approach: | They propose a framework for evaluating interactions between human and virtual agent . they use VoxML as a platform to model interactions using natural language and gesture . |
| Outcome: | The proposed framework assesses the level of mutual understanding and ease of communication between human and computer agents in a blocks world scenario. |
Reproducing Neural Ensemble Classifier for Semantic Relation Extraction inScientific Papers (2020.lrec-1)
Copied to clipboard
| Challenge: | Replicability and reproducibility are core ideas of modern scientific methods. |
| Approach: | They describe challenges encountered in reproducing the results of a top performing system in computational linguistics. |
| Outcome: | The proposed system was able to reproduce the results of a task 7 in the domain of natural language processing and computational linguistics. |
Building a Broad Infrastructure for Uniform Meaning Representations (2024.lrec-main)
Copied to clipboard
Julia Bonn, Matthew J. Buchholz, Jayeol Chun, Andrew Cowell, William Croft, Lukas Denk, Sijia Ge, Jan Hajič, Kenneth Lai, James H. Martin, Skatje Myers, Alexis Palmer, Martha Palmer, Claire Benet Post, James Pustejovsky, Kristine Stenzel, Haibo Sun, Zdeňka Urešová, Rosa Vallejos, Jens E. L. Van Gysel, Meagan Vigus, Nianwen Xue, Jin Zhao
| Challenge: | This paper reports the first release of the UMR data set for six languages . it includes annotations for six different languages that vary greatly in terms of their linguistic properties and resource availability. |
| Approach: | They report the first release of the UMR data set for six languages . they describe on-going efforts to enlarge the data set and extend it to other languages - including Navajo, Navájo, and Sanapaná . |
| Outcome: | The first release of the UMR data set includes annotations for six languages . the language dataset is available for free and can be extended to other languages if needed . |
Enhanced Noun-Noun Compound Interpretation through Textual Enrichment (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent benchmarks frame Noun-Noun Compound Interpretation as a multiple-choice question . but, it still faces key limitations: vague relation descriptions as options and inability to handle polysemous compounds. |
| Approach: | They propose a textual enrichment framework that parses relations into eventoriented descriptions . the framework explicitly surfaces the hidden event connecting head and modifier . |
| Outcome: | The proposed framework yields consistently higher accuracy across three LLM families. |
Spatial and Temporal Language Understanding: Representation, Reasoning, and Grounding (2024.naacl-tutorials)
Copied to clipboard
| Challenge: | This tutorial provides an overview of cutting edge research on spatial and temporal language understanding. |
| Approach: | This tutorial provides an overview of cutting edge research on spatial and temporal language understanding. |
| Outcome: | This tutorial provides an overview of cutting edge research on spatial and temporal language understanding. |
Abstract Meaning Representation for Gesture (2022.lrec-1)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) is an annotated graphbased representation that expresses the meaning of a sentence in terms of its predicate-argument structure. |
| Approach: | They propose an extension to Abstract Meaning Representation (AMR) that captures the meaning of gesture. |
| Outcome: | The proposed model is more challenging than standard AMR while integrating meaningful elements unique to gesture. |
Interchange Formats for Visualization: LIF and MMIF (2020.lrec-1)
Copied to clipboard
| Challenge: | In this paper, we discuss the enhanced data visualization capabilities enabled by interoperating computational linguistics and natural language processing (NLP) applications. |
| Approach: | They propose to use interchange formats to enable enhanced data visualization . they propose to combine CL tools with openly available visualization tools . |
| Outcome: | The proposed formats can be used to create visualizations and manipulate annotations in multiple ways. |
Evaluating Retrieval for Multi-domain Scientific Publications (2022.lrec-1)
Copied to clipboard
Nancy Ide, Keith Suderman, Jingxuan Tu, Marc Verhagen, Shanan Peters, Ian Ross, John Lawson, Andrew Borg, James Pustejovsky
| Challenge: | a new framework for retrieval and mining of scientific publications is being developed . the AskMe retrieval engine is a bridge between xDD's publication database and the LAPPS Grid suite of NLP tools. |
| Approach: | They evaluate AskMe retrievalengine using BEIR benchmark datasets . they aim to determine when and why certain approaches perform well on in-domain and out-of-domain data. |
| Outcome: | The AskMe retrieval engine performs well on both in-domain and out-of-domain data. |
Towards an ISO Standard for the Annotation of Quantification (L18-1)
Copied to clipboard
| Challenge: | Quantification occurs in every sentence of written text or spoken discourse because application of a predicate to one or more sets of objects gives rise to questions of relative scope, of cardinality, and of distribution (or 'distributivity') of the predicacy over the sets of arguments. |
| Approach: | They propose an approach to the annotation of quantification that is being developed as part of an effort by the International Organisation for Standardisation ISO to define interoperable semantic annotation schemes. |
| Outcome: | The proposed scheme includes both count and mass NP quantifiers, as well as NPs with syntactically and semantically complex heads with internal quantification and scoping structures. |
Representation, Learning and Reasoning on Spatial Language for Downstream NLP Tasks (2020.emnlp-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we discuss the cutting-edge research results and existing challenges related to spatial language understanding including semantic annotations, existing corpora, symbolic and sub-symbolic representations, qualitative spatial reasoning, spatial common sense, deep and structured learning models. |
| Approach: | This tutorial presents cutting-edge research results and current challenges related to spatial language understanding including semantic annotations, existing corpora, symbolic and sub-symbolic representations, qualitative spatial reasoning, spatial common sense, deep and structured learning models. |
| Outcome: | This paper reviews the cutting-edge research results and current challenges related to spatial language understanding including semantic annotations, existing corpora, symbolic and sub-symbolic representations, qualitative spatial reasoning, spatial common sense, deep and structured learning models. |
A Formal Analysis of Multimodal Referring Strategies Under Common Ground (2020.lrec-1)
Copied to clipboard
| Challenge: | a recent study has focused on multimodality in the CL/NLP community, but it has not been widely studied. |
| Approach: | They propose to analyze mixed-modality definite referring expressions using gestures and linguistic descriptions. |
| Outcome: | The proposed models can predict viewer judgment of referring expressions and generate more natural and informative expressions. |
Beyond Benchmarks: Building a Richer Cross-Document Event Coreference Dataset with Decontextualization (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing datasets for Cross-Document Event Coreference (CDEC) are small and lacking diversity. |
| Approach: | They propose a new approach leveraging large language models to decontextualize event mentions by simplifying the document-level annotation task to sentence pairs with enriched context. |
| Outcome: | The proposed approach improves the quality of the dataset and generalizability of the model. |
Improving Neural Metaphor Detection with Visual Datasets (2020.lrec-1)
Copied to clipboard
| Challenge: | a new method for metaphor detection uses text from visual datasets to identify words . a metaphor is a complex interaction between two terms, creating an "implicationcomplex" |
| Approach: | They propose a technique for sampling text from visual datasets to create a visibility word embedding. |
| Outcome: | The proposed method improves on previous approaches that use more complex neural networks and richer linguistic features for verb classification. |
COVID-19 Literature Knowledge Graph Construction and Drug Repurposing Report Generation (2021.naacl-demos)
Copied to clipboard
Qingyun Wang, Manling Li, Xuan Wang, Nikolaus Parulian, Guangxing Han, Jiawei Ma, Jingxuan Tu, Ying Lin, Ranran Haoran Zhang, Weili Liu, Aabhas Chauhan, Yingjun Guan, Bangzheng Li, Ruisong Li, Xiangchen Song, Yi Fung, Heng Ji, Jiawei Han, Shih-Fu Chang, James Pustejovsky, Jasmine Rah, David Liem, Ahmed ELsayed, Martha Palmer, Clare Voss, Cynthia Schneider, Boyan Onyshkevych
| Challenge: | a new framework to digest relevant biomedical knowledge is needed to combat COVID-19 . quantity of research results is a bottleneck, and false information promoted in publications . |
| Approach: | a team of researchers has developed a framework to extract multimedia knowledge elements from scientific literature to combat COVID-19. |
| Outcome: | a new framework extracts fine-grained multimedia knowledge elements from scientific literature . it provides detailed contextual sentences, subfigures, and knowledge subgraphs as evidence . the framework is based on a case study of drug repurposing . |
Exploration and Discovery of the COVID-19 Literature through Semantic Visualization (2021.naacl-srw)
Copied to clipboard
| Challenge: | Existing semantic visualization methods are limited in finding connections between corpora targeting a specific topic. |
| Approach: | They propose to use semantic visualization to explore large datasets of complex networks by exploiting the semantics of the relations in them. |
| Outcome: | The proposed method can enable exploration and discovery over large datasets of complex networks by exploiting the semantics of the relations in them. |
Common Ground Tracking in Multimodal Dialogue (2024.lrec-main)
Copied to clipboard
Ibrahim Khalil Khebour, Kenneth Lai, Mariah Bradford, Yifan Zhu, Richard A. Brutti, Christopher Tam, Jingxuan Tu, Benjamin A. Ibarra, Nathaniel Blanchard, Nikhil Krishnaswamy, James Pustejovsky
| Challenge: | In dialogue modeling, there is considerable attention on “dialogue state tracking” (DST) but “common ground tracking” identifies the shared belief space held by all participants in a task-oriented dialogue: the task-relevant propositions all participants accept as true. |
| Approach: | They propose a method for automatically identifying the current set of shared beliefs and ”questions under discussion” of a group with a shared goal. |
| Outcome: | The proposed method predicts moves toward building common ground relative to ground truth in a multimodal interaction with an AI. |
Neural Metaphor Detection with Visibility Embeddings (2021.starsem-1)
Copied to clipboard
| Challenge: | Using Visibility Embeddings, sequence metaphor labeling is improved . many metaphors involve noticeable differences between the abstractness of words constructing them . |
| Approach: | They propose to concatenate sequence metaphor labeling with BiLSTM inputs to obtain improvements . they use visibility embeddings to provide a good estimation of a word's concreteness . |
| Outcome: | The proposed method improves the problem of sequence metaphor labeling with BERT . it allows for consistent and significant improvements at almost no cost . |
GLAMR: Augmenting AMR with GL-VerbNet Event Structure (2024.lrec-main)
Copied to clipboard
Jingxuan Tu, Timothy Obiso, Bingyang Ye, Kyeongmin Rim, Keer Xu, Liulu Yue, Susan Windisch Brown, Martha Palmer, James Pustejovsky
| Challenge: | Abstract Meaning Representation (AMR) is a general-purpose semantic encoding for language. |
| Approach: | They propose an AMR interpretation of Generative Lexicon semantic components using a verb-net-encoded verb-node graph. |
| Outcome: | The proposed extension is compatible with current AMR specification and can be automated. |
Encoding Gesture in Multimodal Dialogue: Creating a Corpus of Multimodal AMR (2024.lrec-main)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) was designed to represent sentence meaning in English text, but recent research has explored its adaptation to broader domains, including documents, dialogues, spatial information, cross-lingual tasks, and gesture. |
| Approach: | They propose to annotate a multimodal (speech and gesture) AMR corpus in a task-based setting and capture coreference relationships across modalities. |
| Outcome: | The proposed corpus captures coreference relationships across modalities, enabling fine-grained analysis of how gesture and natural language interact. |
A Two-Level Interpretation of Modality in Human-Robot Dialogue (2020.coling-main)
Copied to clipboard
| Challenge: | modal expressions are used to communicate and align world knowledge, but there is no obvious manner to ground them in the shared environment. |
| Approach: | They propose a two-level annotation scheme for modality that captures both content and intent and a task-oriented, pragmatic representation that maps to our robot's capabilities. |
| Outcome: | The proposed model can be grounded and dynamically interpreted. |
The CLAMS Platform at Work: Processing Audiovisual Data from the American Archive of Public Broadcasting (2022.lrec-1)
Copied to clipboard
| Challenge: | The Computational Linguistics Applications for Multimedia Services (CLAMS) platform provides access to computational content analysis tools for multimedia material. |
| Approach: | They describe the CLAMS platform as it is and its initial prototype implementation from 2019 . they use a common multi-modal representation language called MMIF to create a workflow . |
| Outcome: | The CLAMS platform is a new version of an initial prototype from 2019 . it can be used to add metadata to mass-digitized multimedia collections . the proposed version is based on the American Archive of Public Broadcasting data . |
Grounding Meaning Representation for Situated Reasoning (2022.aacl-tutorials)
Copied to clipboard
| Challenge: | a tutorial aims to build agents that understand language using a simulated environment . situated reasoning is a critical aspect of human language understanding . |
| Approach: | This tutorial combines a synthesis of multimodal grounding and meaning representation techniques with formal and computational models of situated reasoning. |
| Outcome: | This tutorial combines multimodal grounding and meaning representation techniques with formal and computational models of embodied reasoning. |
Integrating Generative Lexicon Event Structures into VerbNet (L18-1)
Copied to clipboard
| Challenge: | Efforts to use the verb lexicon's semantic representations have revealed a need to revise the form to allow for greater flexibility in representing complex events. |
| Approach: | They propose to restrict the form to first-order representations to simplify use by planners and integrate with the Generative Lexicon's event structure. |
| Outcome: | The proposed representations simplify use by and integration with planners and allow for greater flexibility in representing complex events and for a more nuanced portrayal of the Agent's role. |
Linguistically Conditioned Semantic Textual Similarity (2024.acl-long)
Copied to clipboard
| Challenge: | Semantic textual similarity (STS) is a fundamental NLP task that measures the semantic similarity between two sentences. |
| Approach: | They propose to use a conditional STS dataset to measure sentences’ similarity conditioned on a certain aspect to reduce the inherent ambiguity posed by the sentences. |
| Outcome: | The proposed method improves the performance over baselines on the C-STS dataset with over 80% F1 score. |
The Coreference under Transformation Labeling Dataset: Entity Tracking in Procedural Texts Using Event Models (2023.findings-acl)
Copied to clipboard
| Challenge: | et al., 2023) show that entity coreference resolution is improved when events bring about changes in entities that are not reflected in text mentions. |
| Approach: | They propose to perform transformation-based entity linking prior to coreference relation identification to improve entity coreference. |
| Outcome: | The proposed model improves coreference resolution of entities mentioned under a process-oriented model of events. |
Competence-based Question Generation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing models of natural language understanding rely on question answering and logical inference benchmark challenges to evaluate performance of systems. |
| Approach: | They propose a method to generate CB questions using English cooking recipes . they argue that a broader effort needs to be put on measuring linguistic competencies . |
| Outcome: | The proposed method performs poorly on large pretrained language models until they are provided with additional contextualized semantic information. |
The VoxWorld Platform for Multimodal Embodied Agents (2022.lrec-1)
Copied to clipboard
| Challenge: | a retrospective of the VoxWorld platform is presented . it is a platform for rapidly building and deploying embodied agents with contextual and situational awareness. |
| Approach: | They present a retrospective on the development of the VoxWorld platform . they focus on three different agent implementations and the functionality needed to accommodate them . |
| Outcome: | The VoxWorld platform has evolved from a theoretical model to a platform capable of multimodal interaction and hybrid reasoning. |