Papers by Swarnadeep Saha
Explanation Graph Generation via Pre-trained Language Models: An Empirical Study with Contrastive Learning (2022.acl-long)
Copied to clipboard
| Challenge: | Pre-trained sequence-to-sequence language models generate structured outputs such as graphs with limited supervision. |
| Approach: | They propose to use pre-trained sequence-to-sequence language models to generate graphs . they propose to learn structural constraints and semantics of graphs with limited supervision . |
| Outcome: | The proposed models can learn structural constraints and semantics of graphs with limited supervision. |
Open Information Extraction from Conjunctive Sentences (C18-1)
Copied to clipboard
| Challenge: | Recent work has highlighted the lack of proper conjunction processing as the most significant source of missed yield in Open IE. |
| Approach: | They develop a coordination analyzer that searches over hierarchical conjunct boundaries and uses a language model to score conjunctions. |
| Outcome: | The proposed system performs extraction over the simple sentences identified by CALM to obtain up to 1.8x yield with a moderate increase in precision compared to extractions from original sentences. |
ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Current commonsense-reasoning tasks are discriminative in nature, where a model answers a multiple-choice question for a certain context. |
| Approach: | They propose a generative task that generates a commonsense-augmented graph for stance prediction by using a create-verify-and-refine graph collection framework. |
| Outcome: | The proposed model is able to generate a graph that serves as non-trivial, complete, and unambiguous explanation for the predicted stance. |
ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) still struggle with natural language reasoning tasks. |
| Approach: | They propose a multi-model multi-agent framework that enhances collaborative reasoning between LLM agents via multiple rounds of discussion. |
| Outcome: | Experiments on seven benchmarks show that ReConcile significantly improves LLM reasoning on individual and team levels. |
Pre-Training BERT on Domain Resources for Short Answer Grading (D19-1)
Copied to clipboard
| Challenge: | Pre-trained contextualized representations have achieved state-of-the-art results on multiple downstream NLP tasks by fine-tuning with task-specific data. |
| Approach: | They propose to augment domain-specific data by using labeled short answering grading data for further enhancement of the pre-trained language model. |
| Outcome: | The proposed model can be enhanced by augmenting data from domain-specific resources like textbooks and labeled short answering grading data. |
ConjNLI: Natural Language Inference Over Conjunctive Sentences (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing stress tests do not consider non-boolean usages of conjunctions and use templates . large-scale pre-trained models do not understand conjunctive semantics well, we find . |
| Approach: | They propose a stress-test for natural language inference over conjunctive sentences where the premise differs from the hypothesis by conjunctions removed, added, or replaced. |
| Outcome: | The proposed stress-test for natural language inference over conjunctive sentences is challenging . it finds that pre-trained models do not understand conjunction semantics well . |
Branch-Solve-Merge Improves Large Language Model Evaluation and Generation (2024.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are often used for multi-faceted language generation and evaluation tasks that require complex user constraints or taking into account multiple aspects and criteria. |
| Approach: | They propose a Large Language Model program that consists of branch, solve, and merge modules that are parameterized with specific prompts to the base LLM. |
| Outcome: | The proposed program improves the evaluation correctness and consistency for each LLM by up to 26%, reducing length and pairwise position biases by up 50%, and allowing LLaMA-2-chat to match or outperform GPT-4 on most domains. |
MAgICoRe: Multi-Agent, Iterative, Coarse-to-Fine Refinement for Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Excessive refinement can cause over-correction and reduce performance, authors say . they say MAgICoRe is a framework for multi-agent iteration for coarse-to-fine refinement . |
| Approach: | They propose a framework for multi-agent iteration for coarse-to-fine refinement that reduces excessive refinement by categorizing problems as easy or hard. |
| Outcome: | The proposed framework beats Self-Consistency by 3.4%, Best-of-k by 3.2%, and Self-Refine by 4.0% on Llama-3-8B and GPT- 3.5. |
Are Hard Examples also Harder to Explain? A Study with Human and Model-Generated Explanations (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on explainable NLP has shown that few-shot prompting can enable large pre-trained language models (LLMs) to generate grammatical and factual explanations for data labels. |
| Approach: | They propose to use few-shot prompting to generate grammatical and factual explanations for data labels by varying the hardness of the test samples and in-context samples to investigate the link between explainability and sample hardness. |
| Outcome: | The proposed model can generate grammatical and factual explanations for data labels with few-shot prompting. |
ReCEval: Evaluating Reasoning Chains via Correctness and Informativeness (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods focus on whether the reasoning chain leads to the correct conclusion, but this view may confound reasoning quality with other spurious shortcuts to predict the answer. |
| Approach: | They propose a framework that evaluates reasoning chains via two key properties: (1) correctness, i.e., each step makes a valid inference based on information contained within the step, preceding steps, and input context, and (2) informativeness, respectively. |
| Outcome: | The proposed framework evaluates reasoning chains via two key properties: (1) correctness, i.e., each step makes a valid inference based on information contained within the step, preceding steps, and input context, and (2) informativeness, which is helpful towards deriving the generated answer. |
MURMUR: Modular Multi-Step Reasoning for Semi-Structured Data-to-Text Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | MURMUR generates highly faithful and correct reasoning paths that lead to 26% more logically consistent summaries on LogicNLG compared to direct prompting. |
| Approach: | They propose a neuro-symbolic modular approach to text generation from semi-structured data with multi-step reasoning that generates reasoning paths using neural and symbolic modules with specific linguistic and logical skills. |
| Outcome: | The proposed method improves on two data-to-text generation tasks, while achieving comparable performance to fine-tuned GPT-2 on out-of-domain data. |
multiPRover: Generating Multiple Proofs for Improved Interpretability in Rule Reasoning (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing work to generate proof graphs for formal reasoning over explicit knowledge is not unique and there may be multiple ways of reaching the correct answer. |
| Approach: | They propose to generate multiple proof graphs for reasoning over natural language rules and facts . they propose to combine all proofs and exploit correlations between them . |
| Outcome: | The proposed model outperforms PRover on multiple gold proofs on synthetic, zero-shot, and human-paraphrased datasets. |
PRover: Proof Generation for Interpretable Reasoning over Rules (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that transformers can act as “soft theorem provers” by answering questions over explicitly provided knowledge in natural language. |
| Approach: | They propose a transformer-based model that answers binary questions over rule-bases and generates the corresponding proofs. |
| Outcome: | The proposed model generates proofs with an accuracy of 87% while maintaining or improving performance on the QA task. |