Papers by Mahdi Namazifar
Split-Merge: Scalable and Memory-Efficient Merging of Expert LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | a zero-shot merging framework for large language models consolidates specialized domain experts into a single model without any further training. |
| Approach: | They propose a zero-shot merging framework that consolidates specialized domain experts into a single model without further training. |
| Outcome: | Experiments on code generation, mathematical reasoning, medical question answering, and instruction-following benchmarks confirm the versatility and effectiveness of the proposed framework. |
ALFRED-L: Investigating the Role of Language for Action Learning in Interactive Visual Environments (2022.emnlp-main)
Copied to clipboard
Arjun Akula, Spandana Gella, Aishwarya Padmakumar, Mahdi Namazifar, Mohit Bansal, Jesse Thomason, Dilek Hakkani-Tur
| Challenge: | Embodied Vision and Language Task Completion requires an embodied agent to interpret natural language instructions and egocentric visual observations to navigate through and interact with environments. |
| Approach: | They propose to use ALFRED to test whether models can adapt to tasks not seen during training that require the same types of language understanding as ALFred. |
| Outcome: | The proposed model can adapt to tasks that require the same types of language understanding as ALFRED-L. |
Inducer-tuning: Connecting Prefix-tuning and Adapter-tuning (2022.emnlp-main)
Copied to clipboard
| Challenge: | Prefix-tuning is an essential paradigm of parameter-efficient transfer learning . fine-tuned models require separate copies of model parameters for each task . |
| Approach: | They propose to understand and further develop prefix-tuning through the kernel lens . they propose a new variant of prefix tuning that shares the exact mechanism as prefix tun . |
| Outcome: | The proposed method improves prefix-tuning performance by training only a small portion of parameters. |
Empowering parameter-efficient transfer learning by recognizing the kernel structure in self-attention (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to fine-tune pre-trained language models are parameter efficient . fine- tuning the models requires multiple copies of the parameters, which is inefficient. |
| Approach: | They propose to use kernel-based adapters to tune only a few parameters while freezing the rest of the parameters. |
| Outcome: | The proposed methods achieve or improve strong performance over a diverse set of natural language generation and understanding tasks. |
Selective In-Context Data Augmentation for Intent Detection using Pointwise V-Information (2023.eacl-main)
Copied to clipboard
Yen-Ting Lin, Alexandros Papangelis, Seokhwan Kim, Sungjin Lee, Devamanyu Hazarika, Mahdi Namazifar, Di Jin, Yang Liu, Dilek Hakkani-Tur
| Challenge: | Intent detection is a fundamental element in task-oriented dialogue systems, usually occurring within the Natural Language Understanding component. |
| Approach: | They propose an in-context data augmentation approach that fine-tunes a pre-trained language model and synthesizes new datapoints that correspond to given intents. |
| Outcome: | The proposed method produces training data that achieves state-of-the-art on three challenging intent detection datasets and performs on par with the state- of-the art in full-shot settings. |
Enhancing Knowledge Selection for Grounded Dialogues via Document Semantic Graphs (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing conversation models treat knowledge selection as a sentence ranking problem where each sentence is handled individually, ignoring the internal semantic connection between sentences. |
| Approach: | They propose to automatically convert background knowledge documents into document semantic graphs and perform knowledge selection over such graphs. |
| Outcome: | The proposed model improves on the knowledge selection task and the response generation task on HollE and generalizes on unseen topics in WoW. |
KILM: Knowledge Injection into Encoder-Decoder Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Large pre-trained language models retain implicit knowledge within their parameters, but are susceptible to memorizing the pretraining corpora rather than capturing the knowledge within them. |
| Approach: | They propose to inject entity-related knowledge into encoder-decoder PLMs via a generative knowledge infilling objective through continued pre-training. |
| Outcome: | The proposed approach outperforms state-of-the-art models on general NLU and NLG tasks while maintaining their original performance. |
CESAR: Automatic Induction of Compositional Instructions for Multi-turn Dialogs (2023.emnlp-main)
Copied to clipboard
Taha Aksu, Devamanyu Hazarika, Shikib Mehri, Seokhwan Kim, Dilek Hakkani-Tur, Yang Liu, Mahdi Namazifar
| Challenge: | Instruction-based multitasking has played a critical role in the success of large language models (LLMs) when exposed to complex instructions with multiple constraints, they lag against state-of-the-art models like ChatGPT. |
| Approach: | They propose a framework that unifies a large number of dialog tasks in the same format and allows programmatic induction of complex instructions without manual effort. |
| Outcome: | The proposed framework unifies a large number of dialog tasks in the same format and allows programmatic induction of complex instructions without manual effort. |