Papers by Vittorio Castelli
On the Importance of Diversity in Question Generation for QA (2020.acl-main)
Copied to clipboard
| Challenge: | In this paper, we show that textual diversity in automatic question generation is beneficial for downstream QA. |
| Approach: | They propose to use textual diversity to promote automatic question generation as a quality measure for QA. |
| Outcome: | The proposed measure of QG quality correlates well with evaluation on QA. |
The TechQA Dataset (2020.acl-main)
Copied to clipboard
Vittorio Castelli, Rishav Chakravarti, Saswati Dana, Anthony Ferritto, Radu Florian, Martin Franz, Dinesh Garg, Dinesh Khandelwal, Scott McCarley, Michael McCawley, Mohamed Nasr, Lin Pan, Cezar Pendus, John Pitrelli, Saurabh Pujar, Salim Roukos, Andrzej Sakrajda, Avi Sil, Rosario Uceda-Sosa, Todd Ward, Rong Zhang
| Challenge: | TECHQA is a domain-adaptation question answering dataset for the technical support domain. |
| Approach: | They propose a domain-adaptation question-answering dataset for the technical support domain that contains actual questions posed by users on a technical forum . |
| Outcome: | The TECHQA dataset highlights two real-world issues from the automated customer support domain. |
Importance of Synthesizing High-quality Data for Text-to-SQL Parsing (2023.findings-acl)
Copied to clipboard
Yiqun Hu, Yiyun Zhao, Jiarong Jiang, Wuwei Lan, Henghui Zhu, Anuj Chauhan, Alexander Hanbo Li, Lin Pan, Jun Wang, Chung-Wei Hang, Sheng Zhang, Jiang Guo, Mingwen Dong, Joseph Lilien, Patrick Ng, Zhiguo Wang, Vittorio Castelli, Bing Xiang
| Challenge: | Existing text-to-SQL parsers lack the data to perform well with augmented synthetic data. |
| Approach: | They propose a framework that imposes strong typing constraints and incorporates key relationships from schema. |
| Outcome: | The proposed framework improves on the high-quality synthesized SQL and natural language question (NLQ) models have significant accuracy boosts and achieve new state-of-the-art performance on spider. |
CFO: A Framework for Building Production NLP Systems (D19-3)
Copied to clipboard
Rishav Chakravarti, Cezar Pendus, Andrzej Sakrajda, Anthony Ferritto, Lin Pan, Michael Glass, Vittorio Castelli, J. William Murdock, Radu Florian, Salim Roukos, Avi Sil
| Challenge: | Using a new orchestration framework, we build, test, and deploy interactive NLP and IR systems to production environments. |
| Approach: | They introduce a new orchestration framework for building, experimenting with, and deploying interactive NLP and IR systems to production environments. |
| Outcome: | The proposed framework is well suited to a variety of use cases but is not suitable for academic benchmarking or industry specific use cases. |
Pre-training Intent-Aware Encoders for Zero- and Few-Shot Intent Classification (2023.emnlp-main)
Copied to clipboard
Mujeen Sung, James Gung, Elman Mansimov, Nikolaos Pappas, Raphael Shu, Salvatore Romeo, Yi Zhang, Vittorio Castelli
| Challenge: | Existing methods for IC training do not provide sufficient examples for each intent . a novel pre-training method is proposed to provide a better understanding of intents . |
| Approach: | They propose a method that uses contrastive learning with intent psuedo-labels to produce embeddings that are well-suited for IC tasks. |
| Outcome: | The proposed method achieves 5.4% and 4.0% higher accuracy than the current state-of-the-art method on four IC datasets. |
Answer Span Correction in Machine Reading Comprehension (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing MRC systems produce only partially correct answers when presented with answerable questions. |
| Approach: | They propose a method that yields statistically significant performance improvements over existing MRC systems. |
| Outcome: | The proposed method yields statistically significant performance improvements over state-of-the-art MRC systems in monolingual and multilingual evaluation. |
Benchmarking Diverse-Modal Entity Linking with Generative Models (2023.findings-acl)
Copied to clipboard
Sijia Wang, Alexander Hanbo Li, Henghui Zhu, Sheng Zhang, Pramuditha Perera, Chung-Wei Hang, Jie Ma, William Yang Wang, Zhiguo Wang, Vittorio Castelli, Bing Xiang, Patrick Ng
| Challenge: | Existing models for diverse-mode entity linking (EL) work well on per modality configurations, but it is more challenging to design a unified model for diverse modality. |
| Approach: | They propose a generative diverse-modal model that integrates text, image and table . they propose combining a multimodal encoder-decoder paradigm with a fine-tuning GDMM . |
| Outcome: | The proposed model outperforms state-of-the-art models by 8.51 F1 on average for diverse-modal EL. |
Comparing Biases and the Impact of Multilingual Training across Multiple Languages (2023.emnlp-main)
Copied to clipboard
Sharon Levy, Neha John, Ling Liu, Yogarshi Vyas, Jie Ma, Yoshinari Fujinuma, Miguel Ballesteros, Vittorio Castelli, Dan Roth
| Challenge: | Currently, studies on bias and fairness in natural language processing focus on a single language and/or across few attributes (e.g. gender, race). However, biases can manifest differently across languages for individual attributes. |
| Approach: | They adapt existing sentiment bias templates in English to Italian, Chinese, Hebrew, and Spanish for race, religion, nationality, and gender. |
| Outcome: | The proposed model favors groups that are dominant in each language's culture, indicating bias amplification, after multilingual finetuning. |
Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learning (2023.acl-long)
Copied to clipboard
Alexander Hanbo Li, Mingyue Shang, Evangelia Spiliopoulou, Jie Ma, Patrick Ng, Zhiguo Wang, Bonan Min, William Yang Wang, Kathleen McKeown, Vittorio Castelli, Dan Roth, Bing Xiang
| Challenge: | Existing methods for data-to-text generation focus on specific types of structured data. |
| Approach: | They propose a method that provides a unified representation that can handle various forms of structured data such as tables, knowledge graph triples, and meaning representations. |
| Outcome: | The proposed method improves zero-shot and few-shot scenarios and can adapt to new structured data. |
Cross-Task Knowledge Transfer for Query-Based Text Summarization (D19-58)
Copied to clipboard
| Challenge: | Existing methods for summarization data corpora are limited to extractive and abstractive summarizing. |
| Approach: | They propose to use machine reading comprehension (MRC) and query-based text summarization to produce extractive and abstractive summaries from pre-trained MRC and MT models. |
| Outcome: | The proposed model outperforms existing methods on CNN/Daily Mail and Debatepedia datasets and can be used as a baseline for future systems. |
Taxonomy Expansion for Named Entity Recognition (2023.emnlp-main)
Copied to clipboard
Karthikeyan K, Yogarshi Vyas, Jie Ma, Giovanni Paolini, Neha John, Shuai Wang, Yassine Benajiba, Vittorio Castelli, Dan Roth, Miguel Ballesteros
| Challenge: | Training a Named Entity Recognition model involves fixing a taxonomy of entity types . however, requirements evolve and a model may need to recognize additional entity types. |
| Approach: | They propose a method that uses only partially annotated datasets to train a model to recognize additional entity types. |
| Outcome: | The proposed approach performs better with partially annotated datasets than other approaches . the gap between the proposed approach and other approaches is large in additional datasets . |
Aligning to Constraints for Data-Efficient Language Model Customization (2025.findings-naacl)
Copied to clipboard
Fei Wang, Chao Shang, Shuai Wang, Sarthak Jain, Qiang Ning, Bonan Min, Vittorio Castelli, Yassine Benajiba, Dan Roth
| Challenge: | General-purpose language models (LMs) are aligned to diverse user intents, but fall short when it comes to specific applications. |
| Approach: | They propose a framework that uses constraints to automatically produce supervision signals for user alignment with constraints. |
| Outcome: | The proposed framework can produce supervision signals for user alignment with constraints. |
Scalable Cross-lingual Treebank Synthesis for Improved Production Dependency Parsers (2020.coling-industry)
Copied to clipboard
| Challenge: | scalable Universal Dependency (UD) treebank synthesis techniques are used to improve production-grade parsers. |
| Approach: | They propose a data augmentation technique that uses synthetic treebanks to improve production-grade parsers. |
| Outcome: | The proposed technique improves LAS performance on seven languages by up to two points on production models trained on original UD treebanks. |
Benchmarking Query-Conditioned Natural Language Inference (2025.findings-acl)
Copied to clipboard
| Challenge: | Query-conditioned natural language inference (QC-NLI) is a new approach to detect inconsistencies in large language models. |
| Approach: | They propose a task of Query-Conditioned Natural Language Inference to determine the semantic relationship between two documents conditioned on a query. |
| Outcome: | The proposed task is based on a query-conditioned natural language inference (QC-NLI) it is used to determine the relationship between the premise and hypothesis given a given query. |
RAG-QA Arena: Evaluating Domain Robustness for Long-form Retrieval Augmented Question Answering (2024.emnlp-main)
Copied to clipboard
Rujun Han, Yuhao Zhang, Peng Qi, Yumo Xu, Jenyuan Wang, Lan Liu, William Yang Wang, Bonan Min, Vittorio Castelli
| Challenge: | Existing datasets for question answering based on retrieval augmented generation (RAG-QA) are either constructed using a single source corpus or consist of short extractive answers, which fall short of evaluating large language model (LLM) based RAG-QA systems on cross-domain generalization. |
| Approach: | They propose a dataset that integrates short extractive answers from multiple documents into a single coherent narrative. |
| Outcome: | The proposed dataset integrates short extractive answers from multiple documents into a single coherent narrative, covering 26K queries and large corpora across seven different domains. |
Towards Robust Neural Retrieval with Source Domain Synthetic Pre-Finetuning (2022.coling-1)
Copied to clipboard
Revanth Gangi Reddy, Vikas Yadav, Md Arafat Sultan, Martin Franz, Vittorio Castelli, Heng Ji, Avirup Sil
| Challenge: | Existing neural IR systems rely on lexical matching for query-passage alignment, while masked language models use a dual encoder architecture to encode passages and questions into continuous vector representations. |
| Approach: | They propose to enhance the out-of-domain generalization of Dense Passage Retrieval (DPR) through synthetic data augmentation only in the source domain. |
| Outcome: | The proposed model outperforms existing models in in-domain and zero-shot evaluations on Wikipedia-based datasets. |
CiteEval: Principle-Driven Citation Evaluation for Source Attribution (2025.acl-long)
Copied to clipboard
Yumo Xu, Peng Qi, Jifan Chen, Kunlun Liu, Rujun Han, Lan Liu, Bonan Min, Vittorio Castelli, Arshit Gupta, Zhiguo Wang
| Challenge: | Current evaluation frameworks rely on NLI to assess binary or ternary support from cited sources, which is suboptimal for citation evaluation. |
| Approach: | They propose a citation evaluation framework based on fine-grained citation ratings within a broad context and construct a multi-domain benchmark with high-quality human annotations. |
| Outcome: | The proposed framework provides a high-quality human annotation benchmark and a suite of model-based metrics that exhibit strong correlation with human judgments. |
Generate then Select: Open-ended Visual Question Answering Guided by World Knowledge (2023.findings-acl)
Copied to clipboard
Xingyu Fu, Sheng Zhang, Gukyeong Kwon, Pramuditha Perera, Henghui Zhu, Yuhao Zhang, Alexander Hanbo Li, William Yang Wang, Zhiguo Wang, Vittorio Castelli, Patrick Ng, Dan Roth, Bing Xiang
| Challenge: | Open-ended Visual Question Answering (VQA) requires models to reason over visual and natural language inputs using world knowledge. |
| Approach: | They propose a new VQA pipeline that deploys a generate-then-select strategy guided by world knowledge for the first time. |
| Outcome: | The proposed pipeline expands the knowledge coverage from in-domain training data by 4.1% on OK-VQA, without additional computation cost. |
Multi-Stage Pre-training for Low-Resource Domain Adaptation (2020.emnlp-main)
Copied to clipboard
Rong Zhang, Revanth Gangi Reddy, Md Arafat Sultan, Vittorio Castelli, Anthony Ferritto, Radu Florian, Efsun Sarioglu Kayi, Salim Roukos, Avi Sil, Todd Ward
| Challenge: | Existing approaches to transfer learning target data to in-domain text . prior work has adapted pre-trained LMs to specific domains . |
| Approach: | They extend the vocabulary of a pretrained language model with domain-specific terms to create synthetic tasks that help it transfer to downstream tasks. |
| Outcome: | The proposed approaches show significant performance gains on extractive reading comprehension, document ranking and duplicate question detection tasks. |