Papers by Adrian Weller
On Evaluating LLMs’ Capabilities as Functional Approximators: A Bayesian Evaluation Framework (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have revolutionized the way we can formulate tasks in text-in-text-out format. |
| Approach: | They propose a new evaluation framework to comprehensively assess LLMs’ function modeling abilities by adopting a Bayesian perspective of function modeling. |
| Outcome: | The proposed evaluation framework enables LLMs to excel in utilizing prior knowledge to develop a strong understanding of the underlying function. |
Orthogonal Finetuning Made Scalable (2025.emnlp-main)
Copied to clipboard
| Challenge: | a recent shift in foundation models has slowed the adoption of finetuning methods . however, its high runtime and memory demands limit its scalability . |
| Approach: | They propose an input-centric reformulation that uses matrix-vector multiplications instead of cubic multiplication . they extend OFTv2 to support finetuning quantized foundation models and show it outperforms QLoRA . |
| Outcome: | The proposed model outperforms the popular QLoRA in training stability, efficiency, and memory usage. |
LLMs on interactive feature collections with implicit dynamic decision strategy (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) struggle to efficiently narrow down the search space . external engineered systems may not fully utilize the inherent problem-solving capabilities of LLMs . |
| Approach: | They propose to implicitly guide Large Language Models to enhance their interactive feature collection abilities within a single prompt. |
| Outcome: | The proposed approach improves the performance of large language models in real-world scenarios. |
ALVIN: Active Learning Via INterpolation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that Active Learning methods ignore example groups whose prevalence may vary . supervised fine-tuning remains a critical component of model development, authors say . |
| Approach: | They propose an approach that uses interpolations to create anchors between examples . they propose to use the model to identify informative examples that counteract shortcuts . |
| Outcome: | The proposed model outperforms state-of-the-art active learning methods on six datasets . it prioritizes high-certainty instances that integrate representations from different example groups . |
Mitigating Shortcut Learning with InterpoLated Learning (2025.acl-long)
Copied to clipboard
| Challenge: | Existing shortcut mitigation approaches are model-specific, difficult to tune, computationally expensive, and fail to improve learned representations. |
| Approach: | They propose to interpolate representations of majority examples to include features from intra-class minority examples with shortcut-mitigating patterns. |
| Outcome: | The proposed method improves minority generalization over ERM and state-of-the-art mitigation methods on multiple natural language understanding tasks while preserving accuracy on majority examples. |