Papers by Robby Tan
Unveiling the Achilles’ Heel of NLG Evaluators: A Unified Adversarial Framework Driven by Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have highlighted various neural metrics that align well with human evaluations. |
| Approach: | They propose a black-box adversarial framework that generates strong disagreements between human and victim evaluators. |
| Outcome: | The proposed framework can significantly improve the performance of human and victim evaluators. |
Dynamic Transformers Provide a False Sense of Efficiency (2023.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models typically lead to high computational cost during inference. |
| Approach: | They propose a slowdown attack framework that can reduce inference efficiency by 80% by leveraging existing adversarial attacks targeting model accuracy. |
| Outcome: | The proposed framework can reduce the efficiency of multi-exit models by 80% on average, validating its effectiveness and generalization ability. |
Beyond Single-Audio: Advancing Multi-Audio Processing in Audio Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evaluations of audio large language models focus on single audio inputs, but real-world applications often require processing multiple audio streams simultaneously. |
| Approach: | They propose a multi-audio evaluation benchmark that combines 20 audio inputs from 11 audio tasks to capture audio context. |
| Outcome: | The proposed model outperforms baseline models and achieves high data efficiency without human annotations. |