Papers by Albert Lu
Continual Dialogue State Tracking via Reason-of-Select Distillation (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing research on dialogue systems has focused on domain-specific offline systems lacking adaptation abilities. |
| Approach: | They propose a Reason-of-Select distillation method that enhances smaller models with a novel "meta-reasoning" capability. |
| Outcome: | Experiments show that the proposed method significantly improves the performance and generalization capabilities of existing models. |
Fine-tuning Pre-trained Language Models for Few-shot Intent Detection: Supervised Pre-training and Isotropization (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent studies show that fine-tuning pre-trained language models with a small set of labeled utterances in a supervised manner is helpful, but it yields an anisotropic feature space, which may suppress the expressive power of the semantic representations. |
| Approach: | They propose to regularize supervised pre-training towards isotropy by contrastive learning and correlation matrix regularizers. |
| Outcome: | The proposed methods improve supervised pre-training by regularizing the feature space towards isotropy. |
Bounding the Capabilities of Large Language Models in Open Text Generation with Prompt Constraints (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing and potential applications of open-ended text generation are farreaching, spanning domains such as QA, story generation, open-end dialogue, and ChatGPT 1 . |
| Approach: | They propose a prompt-centric approach to analyzing and bounding the abilities of open-ended generative models by a set of structural and stylistic prompts. |
| Outcome: | The proposed method can be generalized to other large models like BLOOM and OPT. |