Papers by Alon Albalak
RWKV: Reinventing RNNs for the Transformer Era (2023.findings-emnlp)
Copied to clipboard
Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Leon Derczynski, Xingjian Du, Matteo Grella, Kranthi Gv, Xuzheng He, Haowen Hou, Przemyslaw Kazienko, Jan Kocon, Jiaming Kong, Bartłomiej Koptyra, Hayden Lau, Jiaju Lin, Krishna Sri Ipsit Mantri, Ferdinand Mom, Atsushi Saito, Guangyu Song, Xiangru Tang, Johan Wind, Stanisław Woźniak, Zhenyuan Zhang, Qinghua Zhou, Jian Zhu, Rui-Jie Zhu
| Challenge: | recurrent neural networks struggle to match the performance of Transformers due to limitations in parallelization and scalability. |
| Approach: | They propose a model architecture that combines the efficient parallelizable training of transformers with the efficient inference of RNNs. |
| Outcome: | The proposed model performs on par with similarly sized RNNs, suggesting future work can leverage this architecture to create more efficient models. |
FETA: A Benchmark for Few-Sample Task Transfer in Open-Domain Dialogue (2022.emnlp-main)
Copied to clipboard
Alon Albalak, Yi-Lin Tuan, Pegah Jandaghi, Connor Pryor, Luke Yoffe, Deepak Ramachandran, Lise Getoor, Jay Pujara, William Yang Wang
| Challenge: | Prior studies of task transfer in dialogue consider only 2-4 tasks, focus on multitasks. |
| Approach: | They propose a benchmark for FEw-sample TAsk transfer in open-domain dialogue. |
| Outcome: | The proposed benchmark analyzes the transferability between 132 source-target task pairs and provides a baseline for future work. |
CausalDialogue: Modeling Utterance-level Causality in Conversations (2023.findings-acl)
Copied to clipboard
| Challenge: | Despite widespread adoption, neural conversation models have yet to exhibit natural chat capabilities with humans . despite their widespread adoption in society, chatbots have yet not shown natural chat capability . |
| Approach: | They propose a causality-enhanced method to enhance the impact of causality at the utterance level in training neural conversation models. |
| Outcome: | The proposed method improves diversity and agility of loss functions and still needs improvement . the proposed method is based on a CausalDialogue dataset . |
Addressing Issues of Cross-Linguality in Open-Retrieval Question Answering Systems For Emergent Domains (2023.eacl-demo)
Copied to clipboard
| Challenge: | a lack of cross-lingual training data in emergent domains makes it difficult to train on emerging domains. |
| Approach: | They propose a cross-lingual open-retrieval question answering system for COVID-19 . their system adopts a corpus of scientific articles to ensure that retrieved documents are reliable. |
| Outcome: | The proposed system outperforms BM25 baselines in cross-lingual settings. |
Modeling Disclosive Transparency in NLP Application Descriptions (2021.emnlp-main)
Copied to clipboard
| Challenge: | Broader disclosive transparency is difficult to define and quantify, authors say . previous work has demonstrated trade-offs and negative consequences to disclosing transparency . |
| Approach: | They propose to use neural language model-based probabilistic metrics to model disclosive transparency . they demonstrate that they correlate with user and expert opinions of system transparency a valid objective proxy . |
| Outcome: | The proposed metrics correlate with user and expert opinions of system transparency, making them a valid objective proxy. |
Do You Know About My Nation? Investigating Multilingual Language Models’ Cultural Literacy Through Factual Knowledge (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual question-answering benchmarks do not factor in regional diversity in the information they capture and tend to be Western-centric. |
| Approach: | They propose to benchmark eight standard multilingual LLMs on XNationQA and evaluate them using two novel transference metrics. |
| Outcome: | The proposed model shows greater knowledge of cultural information in English than in the dominant language of the respective culture. |
Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown human-like reasoning abilities but struggle with complex logical problems. |
| Approach: | They propose a framework which integrates large language models with symbolic solvers to improve logical problem-solving by combining them with a self-refinement module. |
| Outcome: | The proposed framework improves logical problem-solving by integrating LLMs with symbolic solvers. |