Papers by Yogarshi Vyas

Characterizing and Measuring Linguistic Dataset Drift (2023.acl-long)

Copied to clipboard

Tyler Chang, Kishaloy Halder, Neha Anna John, Yogarshi Vyas, Yassine Benajiba, Miguel Ballesteros, Dan Roth

Challenge:	Existing metrics for dataset drift have not considered specific dimensions of linguistic drift that affect model performance.
Approach:	They propose three dimensions of linguistic dataset drift: vocabulary, structural, and semantic drift.
Outcome:	The proposed metrics are more effective than previous metrics at predicting out-of-domain model accuracies compared to popular fine-tuned embedding distances .

Severing the Edge Between Before and After: Neural Architectures for Temporal Ordering of Events (2020.emnlp-main)

Copied to clipboard

Miguel Ballesteros, Rishita Anubhai, Shuai Wang, Nima Pourdamghani, Yogarshi Vyas, Jie Ma, Parminder Bhatia, Kathleen McKeown, Yaser Al-Onaizan

Language Models

Challenge:	Existing models for temporal ordering of events rely on pretrained representations, transfer and multitask learning, and self-training techniques.
Approach:	They propose a neural architecture and a set of training methods for ordering events by predicting temporal relations by pre-training models.
Outcome:	The proposed models can predict temporal relations between two pairs of events within a span of text and identify temporal relationships between them.

Contrastive Training Improves Zero-Shot Classification of Semi-structured Documents (2023.findings-acl)

Copied to clipboard

Muhammad Khalifa, Yogarshi Vyas, Shuai Wang, Graham Horwood, Sunil Mallya, Miguel Ballesteros

Language Models

Challenge:	Xu et al., 2020 focus on semi-structured document classification in a zero-shot setting . positional, layout, and style information play a vital role in interpreting such documents .
Approach:	They propose a matching-based approach that relies on a pairwise contrastive objective for pretraining and fine-tuning.
Outcome:	The proposed method significantly improves Macro F1 in the zero-shot learning setting.

Linking Entities to Unseen Knowledge Bases with Arbitrary Schemas (2021.naacl-main)

Copied to clipboard

Yogarshi Vyas, Miguel Ballesteros

Knowledge Representation

Challenge:	Existing work on entity linking relies on a knowledge base that is not known at training time.
Approach:	They propose a method to flexibly convert entities with several attribute-value pairs from arbitrary KBs into flat strings and use it to generalize the model.
Outcome:	The proposed model is 12% more accurate than baseline models on English datasets.

Comparing Biases and the Impact of Multilingual Training across Multiple Languages (2023.emnlp-main)

Copied to clipboard

Sharon Levy, Neha John, Ling Liu, Yogarshi Vyas, Jie Ma, Yoshinari Fujinuma, Miguel Ballesteros, Vittorio Castelli, Dan Roth

Sentiment Analysis

Challenge:	Currently, studies on bias and fairness in natural language processing focus on a single language and/or across few attributes (e.g. gender, race). However, biases can manifest differently across languages for individual attributes.
Approach:	They adapt existing sentiment bias templates in English to Italian, Chinese, Hebrew, and Spanish for race, religion, nationality, and gender.
Outcome:	The proposed model favors groups that are dominant in each language's culture, indicating bias amplification, after multilingual finetuning.

Dynamic Benchmarking of Masked Language Models on Temporal Concept Drift with Multiple Views (2023.eacl-main)

Copied to clipboard

Katerina Margatina, Shuai Wang, Yogarshi Vyas, Neha Anna John, Yassine Benajiba, Miguel Ballesteros

Semantic Text Processing

Challenge:	Temporal concept drift is a problem of data changing over time.
Approach:	They benchmark 11 pretrained masked language models on a series of tests to evaluate temporal concept drift.
Outcome:	The proposed framework evaluates 11 pretrained masked language models on a series of tests . it aims to reveal how robust an MLM is over time and provide a signal in case it has become outdated .

Taxonomy Expansion for Named Entity Recognition (2023.emnlp-main)

Copied to clipboard

Karthikeyan K, Yogarshi Vyas, Jie Ma, Giovanni Paolini, Neha John, Shuai Wang, Yassine Benajiba, Vittorio Castelli, Dan Roth, Miguel Ballesteros

Language Models

Challenge:	Training a Named Entity Recognition model involves fixing a taxonomy of entity types . however, requirements evolve and a model may need to recognize additional entity types.
Approach:	They propose a method that uses only partially annotated datasets to train a model to recognize additional entity types.
Outcome:	The proposed approach performs better with partially annotated datasets than other approaches . the gap between the proposed approach and other approaches is large in additional datasets .

A Multi-Modal Multilingual Benchmark for Document Image Classification (2023.findings-emnlp)

Copied to clipboard

Yoshinari Fujinuma, Siddharth Varia, Nishant Sankaran, Srikar Appalaraju, Bonan Min, Yogarshi Vyas

Information Retrieval

Challenge:	Existing document image classification datasets have several limitations and we present two new datasets that overcome these limitations.
Approach:	They propose to use two newly curated multilingual datasets that overcome these limitations and propose to develop multilingual Document AI models.
Outcome:	The proposed datasets overcome limitations in document image classification and open the door for future research into improving Document AI models.

Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics (2021.tacl-1)

Copied to clipboard

Paula Czarnowska, Yogarshi Vyas, Kashif Shah

Responsible & Trustworthy NLP

Challenge:	Existing fairness metrics quantify the differences in a model’s behaviour across a range of demographic groups.
Approach:	They propose to unify existing fairness metrics and compare them to three generalized fairness measures to reveal the connections between them.
Outcome:	The proposed measures can be explained by differences in parameter choices, and the results are consistent with previous studies.

Efficient Classification of Long Documents Using Transformers (2022.acl-short)

Copied to clipboard

Hyunji Park, Yogarshi Vyas, Kashif Shah

Information Retrieval

Challenge:	Several methods have been proposed for classifying long textual documents using Transformers, but there is a lack of consensus on a benchmark to enable a fair comparison among different approaches.
Approach:	They propose to use a dataset to evaluate the relative efficacy of various models for long document classification using Transformers.
Outcome:	The proposed models outperform simple baseline models and yield inconsistent performance across datasets.

Simple Yet Effective Synthetic Dataset Construction for Unsupervised Opinion Summarization (2023.findings-eacl)

Copied to clipboard

Ming Shen, Jie Ma, Shuai Wang, Yogarshi Vyas, Kalpit Dixit, Miguel Ballesteros, Yassine Benajiba

Summarization

Challenge:	generating aspect-specific and general opinion summaries is challenging due to the lack of annotated data.
Approach:	They propose two unsupervised approaches to generate aspect-specific and general opinion summaries by training on synthetic datasets constructed with aspect-related review contents.
Outcome:	The proposed method outperforms existing methods on space and Oposum+ and on other metrics.

Identifying Semantic Divergences in Parallel Text without Annotations (N18-1)

Copied to clipboard

Yogarshi Vyas, Xing Niu, Marine Carpuat

Machine Translation

Challenge:	Parallel sentence pairs are sentences that are translations of each other and convey the same meaning in the source and target languages.
Approach:	They propose a model which detects meaning divergences in parallel sentence pairs . parallel sentence pair are translations of each other, therefore often assumed to convey the same meaning .
Outcome:	The proposed model detects divergences more accurately than models based on word alignments.

Robust Cross-Lingual Hypernymy Detection Using Dependency Context (N18-1)

Copied to clipboard

Shyam Upadhyay, Yogarshi Vyas, Marine Carpuat, Dan Roth

Cross-Lingual Transfer

Challenge:	Existing approaches to cross-lingual hypernymy detection are sparse and can be trained on related languages with negligible loss of performance.
Approach:	They propose a family of unsupervised approaches for cross-lingual hypernymy detection which learns sparse, bilingual word embeddings based on dependency contexts.
Outcome:	The proposed approach significantly improves performance on this task, compared to approaches based only on lexical context.

Weakly Supervised Cross-lingual Semantic Relation Classification via Knowledge Distillation (D19-1)

Copied to clipboard

Yogarshi Vyas, Marine Carpuat

Multilinguality

Challenge:	Existing work characterizes differences in meaning between words across languages using semantic relations . however, because of translation ambiguity, semantic relations are not always preserved by translation.
Approach:	They propose a cross-lingual relation classifier trained only with English examples and a bilingual dictionary to account for translation ambiguity when transferring knowledge from English to cross-linguistic settings.
Outcome:	The proposed model outperforms baselines that rely on bilingual embeddings or dictionaries for cross-lingual transfer and approaches fully supervised systems on English tasks.