Papers by Dheeraj Mekala
Coarse2Fine: Fine-grained Text Classification on Coarsely-grained Annotated Data (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing text classification methods focus on a fixed label set, but many real-world applications require extending to new fine-grained classes as the number of samples per label increases. |
| Approach: | They propose a problem called coarse-to-fine grained classification that leverages label surface names as the only human guidance. |
| Outcome: | The proposed method outperforms existing methods on two real-world datasets. |
Contextualized Weak Supervision for Text Classification (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for weakly supervised text classification generate pseudo-labels in a context-free manner, thus, the ambiguous, context-dependent nature of human language has been long overlooked. |
| Approach: | They propose a framework that provides contextualized weak supervision for text classification . they leverage contextualized representations of word occurrences and seed word information . |
| Outcome: | The proposed framework provides contextualized weak supervision for text classification . it leverages representations of word occurrences and seed word information to differentiate interpretations . the proposed framework also disambiguates initial seed words, making it fully contextualized . |
META: Metadata-Empowered Weak Supervision for Text Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for weakly supervised text classification use text data alone to generate pseudo-labels . strong label indicators exist in metadata and it has been long overlooked due to challenges . |
| Approach: | They propose a framework that leverages metadata as an additional source of weak supervision by combining text data and metadata into a text-rich network. |
| Outcome: | The proposed framework exploits metadata as an additional source of weak supervision. |
Smaller Language Models are capable of selecting Instruction-Tuning Training Data for Larger Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Instruction tuning language models can be expensive and expensive to train . current methods require extensive training on large datasets, resulting in high training costs. |
| Approach: | They propose a novel approach to selecting training data based on the learning percentage of the samples. |
| Outcome: | The proposed model performs better on models ranging from 1B to 13B in size compared to training on the entire dataset. |
ZEROTOP: Zero-Shot Task-Oriented Semantic Parsing using Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing LLMs cannot generalize to domain-specific parsing tasks in a zero-shot setting. |
| Approach: | They propose a task-oriented parsing method that decomposes parse problem into abstractive and extractive question-answering problems. |
| Outcome: | The proposed method decomposes a parsing problem into abstractive and extractive question-answering (QA) problems. |
BFClass: A Backdoor-free Text Classification Framework (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Various trigger design strategies have been explored to attack text classifiers, however, defending such attacks remains an open problem. |
| Approach: | They propose a backdoor-free training framework that poisons a subset of training data by injecting trigger patterns and setting their labels as the target labels. |
| Outcome: | The proposed framework can detect all the triggers, remove 95% of poisoned training samples with very limited false alarms, and achieve almost the same performance as the models trained on benign training data. |
X-Class: Text Classification with Extremely Weak Supervision (2021.naacl-main)
Copied to clipboard
| Challenge: | Weak supervision is a problem in text classification, but it requires corpusspecific knowledge. |
| Approach: | They propose a framework for extremely weak supervision that can be used to train a text classifier. |
| Outcome: | The proposed framework outperforms seed-driven weakly supervised methods on 7 benchmark datasets. |
TOOLVERIFIER: Generalization to New Tools via Self-Verification (2024.findings-emnlp)
Copied to clipboard
Dheeraj Mekala, Jason Weston, Jack Lanchantin, Roberta Raileanu, Maria Lomeli, Jingbo Shang, Jane Dwivedi-Yu
| Challenge: | Existing tools and APIs present a challenge for generalization, despite frequent parameter updates and the daily introduction of new tools. |
| Approach: | They propose a method which distinguishes between close candidates by self-asking contrastive questions during tool selection and parameter generation. |
| Outcome: | Experiments on 4 tasks from the ToolBench benchmark show an improvement of 22% over few-shot baselines. |
SELFOOD: Self-Supervised Out-Of-Distribution Detection via Learning to Rank (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for supervised OOD detection require expensive manual annotation of in-distribution and OOD samples. |
| Approach: | They propose a self-supervised OOD detection method that requires only in-distribution samples as supervision. |
| Outcome: | Experiments with multiple classifiers on coarse- and fine-grained datasets show the proposed method performs well in both coarse-and fine-grid settings. |
MORL-Prompt: An Empirical Analysis of Multi-Objective Reinforcement Learning for Discrete Prompt Optimization (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Current RL-based prompt tuning techniques focus on maximizing the average of reward functions, which does not necessarily lead to prompts that achieve balance across rewards. |
| Approach: | They compare RL-based discrete prompt optimization techniques with a new set of target language models to find prompts that maximize an average of rewards. |
| Outcome: | The proposed methods perform better on two NLP tasks, style transfer and machine translation, and achieve a better balance of all rewards. |
A Benchmark on Extremely Weakly Supervised Text Classification: Reconcile Seed Matching and Prompting Approaches (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for XWS-TC rely on minimal human guidance . X-WS-tc methods require no humanannotated datasets . |
| Approach: | They propose a benchmarking method to compare two approaches to XWS-TC . they use seed-matching and prompting a language model with instructions to decode label words . |
| Outcome: | The proposed methods are more tolerant to human guidance and more robust to model-based methods. |
Progressive Sentiment Analysis for Code-Switched Text Data (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Multilingual transformer language models are used in cross-lingual transfer learning for many NLP tasks such as text classification and named entity recognition. |
| Approach: | They propose a framework that takes the distinction between resource-rich and low-resource language into account and progressively trains from resource-dominated to low-rsource samples. |
| Outcome: | The proposed model outperforms existing methods on low-resource languages and performs well on medium-resourced to high-res languages. |
Leveraging QA Datasets to Improve Generative Data Augmentation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in NLP have substantially improved the capability of pretrained language models to generate high-quality text. |
| Approach: | They propose to reformulate data generation as context generation for a given question-answer (QA) pair and leverage QA datasets for training context generators. |
| Outcome: | The proposed approach improves performance for few-shot and zero-shot classification datasets on multiple classification dataset. |
DOCMASTER: A Unified Platform for Annotation, Training, & Inference in Document Question-Answering (2024.naacl-demo)
Copied to clipboard
| Challenge: | DOCMASTER is a platform for annotating PDF documents, model training, and inference, tailored to document question-answering. |
| Approach: | They propose to integrate layout information into a unified platform for annotating PDF documents, model training, and inference tailored to document question-answering. |
| Outcome: | The proposed platform is designed for annotating PDF documents, model training, and inference, tailored to document question-answering. |
LOPS: Learning Order Inspired Pseudo-Label Selection for Weakly Supervised Text Classification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Weakly-supervised text classification methods are noisy due to their heuristic nature . selection of correct pseudo-labels has a huge potential for performance boost . |
| Approach: | They propose a pseudo-label selection method that takes learning order into account . they propose to select samples that are learnt earlier based on their pseudo-labels . |
| Outcome: | The proposed method is ineffective and unstable due to erroneous predictions from poorly calibrated models. |