Papers by Sayan Ghosh
Pragmatic Reasoning Unlocks Quantifier Semantics for Foundation Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Generalized quantifiers are used to indicate the proportions predicates satisfy (e.g., some apples are red). |
| Approach: | They propose a framework to model quantifier semantics for textbased foundation models by combining natural language inference and the Rational Speech Acts framework. |
| Outcome: | The proposed framework shows a 20% improvement over a literal listener baseline in predicting percentage scopes for quantifier comprehension even with no training. |
Adversarial Scrubbing of Demographic Information for Text Classification (2021.emnlp-main)
Copied to clipboard
Somnath Basu Roy Chowdhury, Sayan Ghosh, Yiyuan Li, Junier Oliva, Shashank Srivastava, Snigdha Chaturvedi
| Challenge: | Existing frameworks to debias contextual representations can encode undesirable attributes, like demographic associations of the users, while being trained for an unrelated task. |
| Approach: | They propose an adversarial learning framework to debias contextual representations by encoding undesirable attributes while being trained for an unrelated task. |
| Outcome: | The proposed framework debiases representations on 8 datasets while remaining informative on the target task. |
How Helpful is Inverse Reinforcement Learning for Table-to-Text Generation? (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to Table-to-Text generation suffer from issues such as missing information, repetition and repetition. |
| Approach: | They propose to use Inverse Reinforcement Learning (IRL) to solve the Table-to-Text task . they use multiple interpretable unsupervised reward components that are combined linearly to form a composite reward function. |
| Outcome: | The proposed task outperforms strong RL baselines marginally in the Table-to-Text task. |
ePiC: Employing Proverbs in Context as a Benchmark for Abstract Language Understanding (2022.acl-long)
Copied to clipboard
| Challenge: | Large language models have shown exciting progress on several NLP benchmarks . however, evaluating their ability for complex analogical reasoning remains under-explored . |
| Approach: | They propose a dataset of narratives for employing proverbs in context as a benchmark for abstract language understanding. |
| Outcome: | The proposed dataset provides fine-grained annotation of aligned spans between proverbs and narratives and contains minimal overlaps between narratives with proverb . the results show that large language models struggle on these tasks compared to humans, and these tasks pose multiple learning challenges. |
Wetin dey with these comments? Modeling Sociolinguistic Factors Affecting Code-switching Behavior in Nigerian Online Discussions (P19-1)
Copied to clipboard
| Challenge: | Multilingual individuals code switch between languages as part of a complex communication process. |
| Approach: | They propose to model the social and contextual factors eliciting code switching in a rich contextual environment by analyzing 330K articles and 389K comments labeled for code switching behavior. |
| Outcome: | The proposed model shows that topic-driven variation, tribal affiliation, emotional valence, and audience design all play complementary roles in behavior. |
Beyond Labels: Empowering Human Annotators with Natural Language Explanations through a Novel Active-Learning Architecture (2023.findings-emnlp)
Copied to clipboard
Bingsheng Yao, Ishan Jindal, Lucian Popa, Yannis Katsis, Sayan Ghosh, Lihong He, Yuxuan Lu, Shashank Srivastava, Yunyao Li, James Hendler, Dakuo Wang
| Challenge: | Existing low-resource learning techniques focus on label annotation while neglecting the natural language explanation of a data point. |
| Approach: | They propose a novel architecture that leverages an explanation-generation model to produce explanations guided by human explanations and a prediction model that utilizes generated explanations toward prediction faithfully. |
| Outcome: | The proposed architecture produces explanations guided by human explanations, a prediction model that utilizes generated explanations toward prediction faithfully, and a data diversity-based AL sampling strategy that benefits from the explanation annotations. |
Learning to Mediate Disparities Towards Pragmatic Communication (2022.acl-long)
Copied to clipboard
| Challenge: | Recent work explores pragmatic reasoning based on Rational Speech Act (RSA) and Theory of Mind in communication (Zhu et al., 2021). |
| Approach: | They propose a framework where the speaker attempts to learn the speaker-listener disparity and adjust the speech accordingly by adding a light-weighted disparity adjustment layer into working memory on top of speaker’s long-term memory system. |
| Outcome: | The proposed framework can learn and adapt to different types of listeners by adding a light-weighted disparity adjustment layer into working memory on top of speaker’s long-term memory system. |
Leveraging Multiple Teachers for Test-Time Adaptation of Language-Guided Classifiers (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent approaches focus on language-guided classifiers that can generalize in zero-shot settings, but their performance varies significantly between different language explanations in unpredictable ways. |
| Approach: | They propose a framework that uses data programming to adapt a language-guided classifier for a new task when provided with multiple teachers and unlabeled test examples. |
| Outcome: | The proposed framework outperforms a baseline from previous work by 9.3%. |
Compare without Despair: Reliable Preference Evaluation with Generation Separability (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a meta-evaluation measure, separability, estimates how suitable a test instance is for pairwise preference evaluation. |
| Approach: | They propose a measure of separability which measures how suitable a test instance is for pairwise preference evaluation. |
| Outcome: | The proposed measure shows that instances with high separability yield more consistent preference ratings from human- and auto-raters. |
CLUES: A Benchmark for Learning Classifiers using Natural Language Explanations (2022.acl-long)
Copied to clipboard
| Challenge: | Supervised learning has traditionally focused on inductive learning by looking at labeled examples of a task. |
| Approach: | They propose a benchmark for Classifier Learning Using natural language ExplanationS that provides natural language supervision over structured data and entailment-based models that learn from explanations. |
| Outcome: | The proposed model generalizes 18% better (relative) on novel tasks than a baseline that does not use explanations. |
Mapping Language to Programs using Multiple Reward Components with Inverse Reinforcement Learning (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches focus on likelihood-based training or using reinforcement learning to fine-tune models based on a single reward. |
| Approach: | They propose an approach to fine-tune programs from natural language instruction . they propose a reward function that linearly combines them and a policy for program generation . |
| Outcome: | The proposed approach achieves better performance than competing methods using Reinforcement Learning. |
LaSQuE: Improved Zero-Shot Classification from Explanations Through Quantifier Modeling and Curriculum Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Several recent approaches have explored training machine learning models via natural language supervision, but they fail to leverage linguistic quantifiers and mimic humans in compositionally learning complex tasks. |
| Approach: | They propose a method that can learn zero-shot classifiers from language explanations by using three new strategies: (1) modeling the semantics of linguistic quantifiers in explanations; (2) aggregating information from multiple explanations using an attention-based mechanism; (3) model training via curriculum learning. |
| Outcome: | The proposed method outperforms previous work showing an absolute gain of up to 7% in generalizing to unseen real-world classification tasks. |
PRover: Proof Generation for Interpretable Reasoning over Rules (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that transformers can act as “soft theorem provers” by answering questions over explicitly provided knowledge in natural language. |
| Approach: | They propose a transformer-based model that answers binary questions over rule-bases and generates the corresponding proofs. |
| Outcome: | The proposed model generates proofs with an accuracy of 87% while maintaining or improving performance on the QA task. |