Papers by Angeliki Metallinou
FANTAstic SEquences and Where to Find Them: Faithful and Efficient API Call Generation through State-tracked Constrained Decoding and Reranking (2024.findings-emnlp)
Copied to clipboard
Zhuoer Wang, Leonardo Ribeiro, Alexandros Papangelis, Rohan Mukherjee, Tzu-Yen Wang, Xinyan Zhao, Arijit Biswas, James Caverlee, Angeliki Metallinou
| Challenge: | Existing supervised and in-context learning approaches suffer from high training costs, poor data efficiency, and generated API calls that can be unfaithful to the API documentation and the user’s request. |
| Approach: | They propose an output-side optimization approach that uses state-tracked constrained decoding and a lightweight model as the discriminator to rerank the beam-searched candidate generations of the large language model. |
| Outcome: | The proposed approach outperforms existing supervised and in-context learning approaches in the DSTC8 and API Bank datasets in terms of call generation accuracy, inference efficiency, and context efficiency. |
Multi-User MultiWOZ: Task-Oriented Dialogues among Multiple Users (2023.findings-emnlp)
Copied to clipboard
Yohan Jo, Xinyan Zhao, Arijit Biswas, Nikoletta Basiou, Vincent Auvray, Nikolaos Malandrakis, Angeliki Metallinou, Alexandros Potamianos
| Challenge: | a dataset of task-oriented dialogues assume conversations between the agent and one user at a time . but multi-user task-orientated dialogues are richer, containing deliberation and deliberations . a novel task is proposed to rewrite a task-focused query that retains only task-relevant information . |
| Approach: | They propose to rewrite a task-oriented chat between two users as a concise task-orientated query that retains only task-relevant information and is directly consumable by the dialogue system. |
| Outcome: | The proposed method surpasses existing models on multi-user dialogues and generalizes to unseen domains. |
Controlled Text Generation for Data Augmentation in Intelligent Artificial Agents (D19-56)
Copied to clipboard
| Challenge: | Data availability is a bottleneck during early stages of development of new capabilities for intelligent artificial agents. |
| Approach: | They propose to use conditional variational auto-encoders to augment training data of a popular commercial artificial agent with a small set of phrase templates to generate new semantically similar phrases. |
| Outcome: | The proposed approach outperforms the previous controlled text generation techniques with limited data and significantly outperformed the previous methods. |
Toward More Accurate and Generalizable Evaluation Metrics for Task-Oriented Dialogs (2023.acl-industry)
Copied to clipboard
Abishek Komma, Nagesh Panyam Chandrasekarasastry, Timothy Leffel, Anuj Goyal, Angeliki Metallinou, Spyros Matsoukas, Aram Galstyan
| Challenge: | Existing methods to dialog quality estimation focus on evaluating individual turns or collecting dialog-level quality measurements from end users immediately following an interaction. |
| Approach: | They propose a dialog-level annotation workflow called Dialog Quality Annotation . they propose to annotate dialogs for attributes such as goal completion and user sentiment . |
| Outcome: | The proposed model outperforms existing methods for dialog quality estimation . it shows that high-quality human-annotated data is important for dialog-quality evaluation . |
Simple Question Answering with Subgraph Ranking and Joint-Scoring (N19-1)
Copied to clipboard
| Challenge: | Knowledge graph based simple question answering is a major area of research in question answering. |
| Approach: | They propose a framework to describe and analyze existing knowledge graph based simple question answering approaches. |
| Outcome: | The proposed model achieves a state-of-the-art (85.44% accuracy) on the SimpleQuestions dataset. |
Alexa Conversations: An Extensible Data-driven Approach for Building Task-oriented Dialogue Systems (2021.naacl-demos)
Copied to clipboard
Anish Acharya, Suranjit Adhikari, Sanchit Agarwal, Vincent Auvray, Nehal Belgamwar, Arijit Biswas, Shubhra Chandra, Tagyoung Chung, Maryam Fazel-Zarandi, Raefer Gabriel, Shuyang Gao, Rahul Goel, Dilek Hakkani-Tur, Jan Jezabek, Abhay Jha, Jiun-Yu Kao, Prakash Krishnan, Peter Ku, Anuj Goyal, Chien-Wei Lin, Qing Liu, Arindam Mandal, Angeliki Metallinou, Vishal Naik, Yi Pan, Shachi Paul, Vittorio Perera, Abhishek Sethi, Minmin Shen, Nikko Strom, Eddie Wang
| Challenge: | Traditional goal-oriented dialogue systems require annotations which are hard to obtain for every new domain, limiting scalability. |
| Approach: | They propose a data-driven approach to building goal-oriented dialogue systems . they use a seed dialogue simulator to generate annotated conversations instead of collecting annotations . |
| Outcome: | The proposed system improves turn-level action signature prediction accuracy by 50% . the system is scalable, extensible and data efficient . |
Fast and Scalable Expansion of Natural Language Understanding Functionality for Intelligent Agents (N18-3)
Copied to clipboard
| Challenge: | a recent paper describes efficient deep neural network architectures for expanding natural language capabilities of virtual agents. |
| Approach: | They propose deep neural network architectures that maximize re-use available resources . they use data from Amazon Alexa to accelerate expansion of new natural language domains . |
| Outcome: | The proposed methods increase accuracy in low resource settings and enable rapid development with less data. |