Papers by Rakesh Menon
Pragmatic Reasoning Unlocks Quantifier Semantics for Foundation Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Generalized quantifiers are used to indicate the proportions predicates satisfy (e.g., some apples are red). |
| Approach: | They propose a framework to model quantifier semantics for textbased foundation models by combining natural language inference and the Rational Speech Acts framework. |
| Outcome: | The proposed framework shows a 20% improvement over a literal listener baseline in predicting percentage scopes for quantifier comprehension even with no training. |
DelucionQA: Detecting Hallucinations in Domain-specific Question Answering (2023.findings-emnlp)
Copied to clipboard
Mobashir Sadat, Zhengyu Zhou, Lukas Lange, Jun Araki, Arsalan Gundroo, Bingqing Wang, Rakesh Menon, Md Parvez, Zhe Feng
| Challenge: | Hallucination is a well-known phenomenon in text generated by large language models . state-of-the-art LLMs still have a number of weaknesses, including the tendency to generate hallucinatory statements without considering the factuality . |
| Approach: | They propose a dataset that captures hallucinations made by retrieval-augmented LLMs . they propose to use these methods to help detect hallucinosity in QA tasks . |
| Outcome: | The proposed method captures hallucinations made by retrieval-augmented LLMs for QA tasks. |
Leveraging Multiple Teachers for Test-Time Adaptation of Language-Guided Classifiers (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent approaches focus on language-guided classifiers that can generalize in zero-shot settings, but their performance varies significantly between different language explanations in unpredictable ways. |
| Approach: | They propose a framework that uses data programming to adapt a language-guided classifier for a new task when provided with multiple teachers and unlabeled test examples. |
| Outcome: | The proposed framework outperforms a baseline from previous work by 9.3%. |
MaNtLE: Model-agnostic Natural Language Explainer (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent research suggests that practitioners prefer examining language explanations that explain sub-groups of examples. |
| Approach: | They propose a model-agnostic natural language explainer that generates faithful explanations of classifier rationale for structured classification tasks. |
| Outcome: | The proposed model-agnostic natural language explainer generates faithful explanations of classifier rationale for structured classification tasks. |