Papers by Justin Cheok
The Confident Liar: Diagnosing Multi-Agent Debate with Log-Probabilities and LLM-as-Judge (2026.acl-srw)
Copied to clipboard
| Challenge: | Multi-agent debate systems are typically evaluated only on whether the final answer is correct, overlooking the quality of the intermediatereasoning that debate produces. |
| Approach: | They propose a framework that pairs a two-agent debate architecture with anLLM-as-judge that scores each agent’s reasoning along instructionfollowing, justification quality, and evidence grounding, together with acritical-failure flag. |
| Outcome: | The proposed framework pairs a two-agent debate architecture with anLLM-as-judge that scores each agent’s reasoning along instructionfollowing, justification quality, and evidence grounding, together with acritical-failure flag. |