Papers by Ricardo Usbeck
The Lifecycle of “Facts”: A Survey of Social Bias in Knowledge Graphs (2022.aacl-main)
Copied to clipboard
| Challenge: | Knowledge graphs are used in a variety of downstream tasks and in hybrid AI systems. |
| Approach: | They propose to examine the lifecycle of knowledge graphs with respect to bias influences. |
| Outcome: | The proposed models are based on the lifecycle of knowledge graphs and their embedded versions . they show that the KGs manifest biases and propagate harmful prejudices . |
The Role of Output Vocabulary in T2T LMs for SPARQL Semantic Parsing (2023.findings-acl)
Copied to clipboard
| Challenge: | Using a knowledge graph question answering task, we replace the entire SPARQL vocabulary with alternate vocabularies. |
| Approach: | They replace the entire SPARQL vocabulary with alternate vocabularies . they find absolute gains in the range of 17% on the GrailQA dataset . |
| Outcome: | The proposed substitutions show that the model performance improves on the GrailQA dataset. |
Proxy Indicators for the Quality of Open-domain Dialogues (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for evaluation of open-domain dialogues are expensive and require human annotators to evaluate their quality. |
| Approach: | They propose to use a deep-learning model trained on the general language understanding evaluation benchmark to serve as a quality indication of open-domain dialogues. |
| Outcome: | The proposed model can infer various quality metrics and derive a component-based overall score. |
DialoKG: Knowledge-Structure Aware Task-Oriented Dialogue Generation (2022.findings-naacl)
Copied to clipboard
| Challenge: | Recent research focused on knowledge distillation methods where the underlying relationship between the facts in a knowledge base is not effectively captured. |
| Approach: | They propose a novel task-oriented dialogue system that effectively incorporates knowledge into a language model by using structural information of a knowledge graph. |
| Outcome: | The proposed system views relational knowledge as a knowledge graph and introduces (1) a structure-aware knowledge embedding technique, and (2) a Knowledge graph-weighted attention masking strategy to facilitate the system selecting relevant information during the dialogue generation. |
RoMe: A Robust Metric for Evaluating Natural Language Generation (2022.acl-long)
Copied to clipboard
| Challenge: | Empirical results suggest that RoMe has a stronger correlation to human judgment over state-of-the-art metrics in evaluating system-generated sentences across several NLG tasks. |
| Approach: | They propose an automatic evaluation metric incorporating several core aspects of natural language understanding (language competence, syntactic and semantic variation). |
| Outcome: | The proposed evaluation metric is trained on language features such as semantic similarity combined with tree edit distance and grammatical acceptability, using a self-supervised neural network. |
Message Passing for Hyper-Relational Knowledge Graphs (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to link prediction over knowledge graphs (KGs) are designed to work over triple-based models, where facts are represented as binary relations between entities. |
| Approach: | They propose a message passing based graph encoder - StarE capable of modeling hyper-relational knowledge graphs (KGs) they propose to encode an arbitrary number of additional information along with the main triple while keeping the semantic roles of qualifiers and triples intact. |
| Outcome: | The proposed model outperforms existing models across multiple benchmarks and shows that leveraging qualifiers is vital for link prediction. |
Revisiting Supervised Contrastive Learning for Microblog Classification (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for microblog classification use pre-training language models (LMs) however, pre-trained LMs are resource-intensive and not suitable for small labs. |
| Approach: | They propose to fine-tune transformer-based language models with a SCL loss for English microblog classification by comparing two benchmarks. |
| Outcome: | The proposed method has a performance gain of up to 11.9 percentage points across all subtasks. |
Knowledge Graph Question Answering Leaderboard: A Community Resource to Prevent a Replication Crisis (2022.lrec-1)
Copied to clipboard
| Challenge: | Knowledge Graph (KG) Question Answering (QA) is a rapidly growing field in research and industry. |
| Approach: | They propose to create a new leaderboard for any KGQA benchmark dataset as a focal point for the community. |
| Outcome: | The proposed model provides a central and open leaderboard for any KGQA benchmark dataset as a focal point for the community. |
Language Model Transformers as Evaluators for Open-domain Dialogues (2020.coling-main)
Copied to clipboard
| Challenge: | Computer-based systems for communication with humans are a cornerstone of AI research since the 1950s. |
| Approach: | They propose to use transformer neural networks to predict one or more words based on an already given context to provide an efficient, automatic indication of dialogue quality. |
| Outcome: | The proposed language models show that human evaluators have a positive correlation between the output of the models and scores. |
Treating Dialogue Quality Evaluation as an Anomaly Detection Problem (2020.lrec-1)
Copied to clipboard
| Challenge: | Dialogue systems for interaction with humans are becoming more popular . the best way to estimate their success is through means of human evaluation . |
| Approach: | They investigate the effectiveness of perceiving dialogue evaluation as an anomaly detection task. |
| Outcome: | The proposed approach is based on four models and shows negative results . the proposed approach could be used in the future to improve human-led dialogue evaluations. |