Papers by Ting-Yun Chang
What Does This Word Mean? Explaining Contextualized Embeddings with Natural Language Definition (D19-1)
Copied to clipboard
| Challenge: | Contextualized word embeddings have boosted many NLP tasks compared with static word embeds. |
| Approach: | They propose a framework that can explain word meanings given contextualized word embeddings for better interpretation. |
| Outcome: | The proposed framework can explain word meanings given contextualized word embeddings for better interpretation. |
Data Curation Alone Can Stabilize In-context Learning (2023.acl-long)
Copied to clipboard
| Challenge: | In-context learning (ICL) is a new paradigm for few-shot learning with pretrainable large language models . however, randomly sampling examples from a training set leads to high variance in performance . |
| Approach: | They propose two methods to select training examples from a training set and then carefully curate them from corresponding subsets. |
| Outcome: | The proposed method improves accuracy over sampling from the entire training set. |
Rethinking Why Intermediate-Task Fine-Tuning Works (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained language models can be fine-tuned on intermediate labeled-data tasks before fine- tuning the models on the target task of interest. |
| Approach: | They conduct extensive experiments to study the impact of different factors on STILT . they find that the improvement from an intermediate task could be orthogonal to it containing reasoning or other complex skills. |
| Outcome: | The proposed method improves the performance of pretrained language models on various target tasks. |
When Parts Are Greater Than Sums: Individual LLM Components Can Outperform Full Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | In-context learning is a method for adapting large language models to tasks with instructions or examples. |
| Approach: | They propose a method to decompose the output of large language models into components . they propose component reweighting, which learns to linearly re-scale component activations from a few labeled examples. |
| Outcome: | The proposed method improves by 6.0% accuracy points over 24 examples given 24 examples on Llama-2-7B. |
Why Do Some Inputs Break Low-Bit LLM Quantization? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Low-bit weight-only quantization reduces memory usage but disproportionately affects certain examples. |
| Approach: | They analyze quantization errors of 50 pairs of methods on large language models and test their hypothesis . |
| Outcome: | The proposed method reduces the memory footprint of large language models while maintaining reasonable performance across benchmarks. |
Do Localization Methods Actually Localize Memorized Data in LLMs? A Tale of Two Benchmarks (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing studies on the ability of localization methods to pinpoint LLM components for memorized data are lacking. |
| Approach: | They propose to use a subset of LLM weights to evaluate localization methods . they propose to measure how much dropping out identified neurons deletes a memorized sequence. |
| Outcome: | The proposed methods show promising localization ability, despite differences in their evaluations. |
Leveraging Hierarchical Category Knowledge for Data-Imbalanced Multi-Label Diagnostic Text Understanding (D19-62)
Copied to clipboard
| Challenge: | Existing work on automatic diagnosis prediction has considered the output labels independently, so that the codes with few samples are difficult to learn. |
| Approach: | They propose to leverage domain knowledge to predict diagnostic codes given the descriptive present illness in electronic health records by leveraging hierarchical category knowledge. |
| Outcome: | The proposed model can efficiently utilize category knowledge and provide informative cues to improve the top-ranked diagnostic codes which is better than the prior state-of-the-art. |