Papers with Flan-T5
MoEMoE: Question Guided Dense and Scalable Sparse Mixture-of-Expert for Multi-source Multi-modal Answering (2025.naacl-industry)
Copied to clipboard
| Challenge: | Question Answering (QA) and Visual Question Answers (VQA) are well-studied problems in the language and vision domain. |
| Approach: | They propose a question-answer generation framework that learns attention across multiple sources and decodes this information for robust and unbiased answer generation. |
| Outcome: | The proposed framework can handle thousands of question types and scale to scale. |
LawInstruct: A Resource for Studying Language Model Adaptation to the Legal Domain (2025.findings-naacl)
Copied to clipboard
Joel Niklaus, Lucia Zheng, Arya D. McCarthy, Christopher Hahn, Brian M Rosen, Peter Henderson, Daniel E. Ho, Garrett Honke, Percy Liang, Christopher D Manning
| Challenge: | In general, instruction tuning is important for direct user interaction, but the legal domain is underrepresented in typical instruction datasets. |
| Approach: | They aggregate 58 annotated legal datasets and write instructions for each to create LawInstruct. |
| Outcome: | The proposed model improves on LegalBench across all model sizes, but no drop in MMLU. |
Improving the Calibration of Confidence Scores in Text Generation Using the Output Distribution’s Characteristics (2025.acl-short)
Copied to clipboard
| Challenge: | Existing methods for estimating confidence in text generation do not account for many valid answers in generation tasks. |
| Approach: | They propose task-agnostic confidence metrics that rely solely on model probabilities without the need for further fine-tuning or heuristics. |
| Outcome: | The proposed models improve the accuracy of BART and Flan-T5 on summarization, translation, and question answering datasets. |
STAR: Spectral Truncation and Rescale for Model Merging (2025.naacl-short)
Copied to clipboard
| Challenge: | Model merging is an efficient way of obtaining a multi-task model from several pretrained models without further fine-tuning. |
| Approach: | They propose a model merging technique that aims at mitigating "merging conflicts" by truncating small components in the respective spectral spaces and then an automatic parameter rescaling scheme to retain the nuclear norm of the original matrix. |
| Outcome: | The proposed model outperforms baseline models on flan-T5 by 4.2% and is robust to hyperparamater choice. |
Refining App Reviews: Dataset, Methodology, and Evaluation (2024.emnlp-industry)
Copied to clipboard
| Challenge: | App reviews are valuable for developers, but often contain typos, spelling errors, and complex sentences, hindering efficient interpretation and slowing down app improvement processes. |
| Approach: | They propose to refine 10,000 original and refined reviews from 10 mobile apps using an annotated dataset of 10,000 pairs of reviews. |
| Outcome: | The proposed dataset includes 10,000 original and refined reviews from 10 mobile applications. |
Efficient Sample-Specific Encoder Perturbations (2024.naacl-short)
Copied to clipboard
| Challenge: | Existing approaches to training encoder-decoder systems often depend on teacher-forcing with the likelihood criteria, e.g. next token prediction of the reference sequence. |
| Approach: | They propose an inference-efficient way to modify the behaviour of an encoder-decoder system according to a specific attribute of interest by using a small proxy network. |
| Outcome: | The proposed framework improves the COMET performance of Flan-T5 on Machine Translation and the WER of Whisper foundation models on Speech Recognition. |
NovaCOMET: Open Commonsense Foundation Models with Symbolic Knowledge Distillation (2023.findings-emnlp)
Copied to clipboard
Peter West, Ronan Bras, Taylor Sorensen, Bill Lin, Liwei Jiang, Ximing Lu, Khyathi Chandu, Jack Hessel, Ashutosh Baheti, Chandra Bhagavatula, Yejin Choi
| Challenge: | a new commonsense knowledge model, NovaCOMET, combines knowledge and general task models. |
| Approach: | They propose an open commonsense knowledge model that combines knowledge and general task models. |
| Outcome: | The proposed model matches or exceeds existing knowledge models on commonsense reasoning tasks. |
Entity Tracking in Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Existing studies on the ability of large language models to track discourse entities have not been conducted. |
| Approach: | They propose to investigate whether large language models can track entities . they first investigate whether Flan-T5, GPT-3 and GPT-3.5 can track the state of entities based on an English description of the initial state and a series of state-changing operations. |
| Outcome: | The proposed task investigates whether language models can track entities based on language descriptions and state-changing operations. |
Learning from Implicit User Feedback, Emotions and Demographic Information in Task-Oriented and Document-Grounded Dialogues (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Imlicit user feedback, user emotions and demographic information are promising sources for improving the accuracy and user engagement of dialogue responses, but the impact of such information on task completion and factual consistency is not known. |
| Approach: | They introduce the first English task-oriented and document-grounded dialogue dataset annotated with this information. |
| Outcome: | The proposed dataset shows that the model's responses are more informative and factual consistent. |
Factual Dialogue Summarization via Learning from Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Existing models generate fluent and coherent summaries, but inconsistencies can be found in generated summary. |
| Approach: | They propose to use symbolic knowledge distillation to improve the factual consistency of smaller pretrained models for dialogue summarization. |
| Outcome: | The proposed model outperforms baseline models in BART, PEGASUS, and Flan-T5 in factual consistency and accuracy. |
DiscoSG: Towards Discourse-Level Text Scene Graph Parsing through Iterative Graph Refinement (2025.emnlp-main)
Copied to clipboard
| Challenge: | Current approaches typically merge sentence-level parsing outputs for discourse input, resulting in fragmented graphs and degraded downstream performance. |
| Approach: | They propose a task for discourse-level text scene graph parsing that merges sentence-level outputs for discourse input and propose 'DiscoSG' a dataset of 400 expert-annotated and 8,430 synthesised multi-sentence caption-graph pairs is used to test the new task. |
| Outcome: | The proposed task improves SPICE by 30% over the baseline while achieving 86 faster inference than existing models. |
Background Summarization of Event Timelines (2023.emnlp-main)
Copied to clipboard
| Challenge: | Generating concise summaries of news events is a challenging task for newcomers to a news story. |
| Approach: | They propose a task of background news summarization that complements each timeline update with a background summary of relevant preceding events. |
| Outcome: | The proposed system performs well on a question-answering-based evaluation metric, Background Utility Score (BUS). |
MVP: Multi-task Supervised Pre-training for Natural Language Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) have achieved remarkable success in natural language generation tasks. |
| Approach: | They propose to use a large-scale natural language generation corpus to pre-train a text generation model MVP in a supervised manner. |
| Outcome: | The proposed model outperforms BART and Flan-T5 on 13 out of 17 datasets and outperformed BART by 9.3% and FlaN-T5. |
People Make Better Edits: Measuring the Efficacy of LLM-Generated Counterfactually Augmented Data for Harmful Language Detection (2023.emnlp-main)
Copied to clipboard
| Challenge: | Past work has shown that counterfactually augmented data (CADs) can improve models' performance on out-of-domain tests. |
| Approach: | They use Polyjuice, ChatGPT, and Flan-T5 to automatically generate CADs . they find that CAD generates a model that flips the original label with minimal changes . |
| Outcome: | The proposed model improves model robustness on out-of-domain test sets and individual data points. |
Reasoning or a Semblance of it? A Diagnostic Study of Transitive Reasoning in LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Evaluating Large Language Models (LLMs) on reasoning benchmarks demonstrates their ability to solve compositional questions. |
| Approach: | They investigate the transitive reasoning capabilities of two distinct LLM architectures, LLaMA 2 and Flan-T5, by manipulating facts within two compositional datasets: QASC and Bamboogle. |
| Outcome: | The proposed models leverage word/phrase overlaps across sections of test input, models’ inherent knowledge during pre-training or fine-tuning, and names of entities. |
Both Matter: Enhancing the Emotional Intelligence of Large Language Models without Compromising the General Intelligence (2024.findings-acl)
Copied to clipboard
| Challenge: | Emotional Intelligence (EI) is a key concept in the field of human intelligence. |
| Approach: | They propose a method to enhance EI of large language models by naive fine-tuning on EI-related tasks. |
| Outcome: | The proposed method improves EI of two LLM-based assistants without compromising GI. |
Mining the uncertainty patterns of humans and models in the annotation of moral foundations and human values (2025.acl-long)
Copied to clipboard
| Challenge: | disagreement in annotation (HLV) is considered a constitutive feature of subjective tasks. |
| Approach: | They investigate the relationship between disagreement in annotation and model uncertainty . they use linguistic features to calibrate models to HLV and uncertainty to analyze their impact on uncertainty. |
| Outcome: | The proposed model uncertainty is calibrated to human label variation (HLV) the proposed model is calibrate to human labels, the authors show . |
Zero- and Few-Shot Prompting with LLMs: A Comparative Study with Fine-tuned Models for Bangla Sentiment Analysis (2024.lrec-main)
Copied to clipboard
Md. Arid Hasan, Shudipta Das, Afiyat Anjum, Firoj Alam, Anika Anjum, Avijit Sarker, Sheak Rashed Haider Noori
| Challenge: | Recent performance of Large Language Models (LLMs) in low-resource languages is under-researched due to resource constraints. |
| Approach: | They present a manually annotated dataset encompassing 33,606 Bangla tweets and Facebook comments. |
| Outcome: | The proposed model outperforms other models even in zero and few-shot scenarios. |