Papers by Mohammad Akbari
GOLD: Generalized Knowledge Distillation via Out-of-Distribution-Guided Language Data Generation (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to generate data using LLMs are limited by sampling from the center of original content distribution. |
| Approach: | They propose a task-agnostic data generation and knowledge distillation framework for LLMs that employs an iterative out-of-distribution-guided feedback mechanism to generate data. |
| Outcome: | The proposed framework outperforms prior arts and the LLM on 10 different classification tasks and noisey generated data. |
E-LANG: Energy-Based Joint Inferencing of Super and Swift Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | E-LANG is an efficient method for building large and highly capable language models . existing methods are only applicable to encoder-only backbones and classification tasks . |
| Approach: | They propose an efficient dynamic inference approach which distributes inference between large accurate Super-models and light-weight Swift models. |
| Outcome: | The proposed method outperforms existing methods on GLUE, SuperGLUE and WMT with 3.3X computation speed and 2.9X computation cost. |