Papers with CAE
Chatbot Arena Estimate: towards a generalized performance benchmark for LLM capabilities (2025.naacl-industry)
Copied to clipboard
Lucas Spangher, Tianle Li, William F. Arnold, Nick Masiewicki, Xerxes Dotiwalla, Rama Kumar Pasumarthi, Peter Grabowski, Eugene Ie, Daniel Gruhl
| Challenge: | Existing benchmark aggregation methods, such as Elo-based systems, can be resource-intensive, public facing, and time-consuming. |
| Approach: | They propose a framework for aggregating performance across diverse benchmarks that generates a “Goodness” and a ‘Fastness” score. |
| Outcome: | The proposed framework achieves higher Pearson correlation with Chatbot Arena Elo scores than MMLU’s correlation with chatbot Arena scores, validating its reliability for real-world LLM evaluation. |
Improving Acoustic Word Embeddings through Correspondence Training of Self-supervised Speech Representations (2024.eacl-long)
Copied to clipboard
| Challenge: | Acoustic word embeddings are vector representations of spoken words . self-supervised learning (SSL)-based speech models are popular for speech recognition . |
| Approach: | They explore the effectiveness of the Correspondence Auto-Encoder to obtain improved AWEs . they use self-supervised learning on large amounts of unlabelled speech data . |
| Outcome: | The Correspondence Auto-Encoder (CAE) model outperforms MFCC models on language discrimination . the model achieves best results in Polish, Portuguese, Spanish, French, and English . |
Civil Rephrases Of Toxic Texts With Self-Supervised Transformers (2021.eacl-main)
Copied to clipboard
| Challenge: | et al., 2018a): a poor phrasing may make the conversation go awry. |
| Approach: | They propose a model that can help suggest rephrasings of toxic comments in a more civil manner. |
| Outcome: | The proposed model generates sentences that are more fluent and better at preserving the initial content compared to earlier systems and human evaluation. |
Cycle-Consistent Adversarial Autoencoders for Unsupervised Text Style Transfer (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for unsupervised text style transfer lack parallel data and difficulties in content preservation. |
| Approach: | They propose a neural approach to unsupervised text style transfer using non-parallel data. |
| Outcome: | The proposed approach can be trained end-to-end on two widely-used public datasets. |
Interpretable Composition Attribution Enhancement for Visio-linguistic Compositional Understanding (2024.emnlp-main)
Copied to clipboard
| Challenge: | Despite promising progress, vision-language models still exhibit significant challenges in understanding visio-linguistic concepts beyond object terms. |
| Approach: | They propose a framework that encourages the model to pay greater attention to composition words denoting relationships and attributes within the text. |
| Outcome: | The proposed framework improves the ability to discern intricate details and construct more sophisticated interpretations of combined visual and linguistic elements. |