Papers by Somanshu Singla
Decentralized Arena: Towards Democratic and Scalable Automatic Evaluation of Language Models (2026.acl-long)
Copied to clipboard
Yanbin Yin, Kun Zhou, Zhen Wang, Xiangdong Zhang, Yifei Shao, Shibo Hao, Yi Gu, Jieyuan Liu, Somanshu Singla, Tianyang Liu, Eric P. Xing, Zhengzhong Liu, Haojian Jin, Zhiting Hu
| Challenge: | closed-ended question-based benchmarks struggle with saturation as newer models emerge . crowd-sourced leaderboards rely on costly and slow human judges . |
| Approach: | They propose a framework that leverages collective intelligence from all large language models to evaluate each other. |
| Outcome: | a new framework enables a democratic, pairwise evaluation of all large language models . it achieves 97% correlation with human judgements, while significantly reducing the cost. |
Dynamic Rewarding with Prompt Optimization Enables Tuning-free Self-Alignment of Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Empirical evaluations on eight recent LLMs reveal that DRPO significantly enhances alignment performance, enabling base models to outperform their SFT/RLHF-tuned counterparts. |
| Approach: | They propose a tuning-free approach to self-alignment called Dynamic Rewarding with Prompt Optimization (DRPO) it leverages a dynamic rewarding mechanism to identify and rectify alignment weaknesses . |
| Outcome: | The proposed approach outperforms existing methods and is highly adaptable to various alignment challenges. |