Papers by Qingshuo Guo

GenGO

Omakase Read Later

Search

Papers by Qingshuo Guo

LREC

2024.lrec-main

reinforcement learning process
proxy confidence
Pearson correlation
accuracy
downstream tasks
open-domain question answering
machine translation
code generation
general-purpose task solver
black-box calibration

2024

Natural Language Interfaces

1 papers

Does ChatGPT Know That It Does Not Know? Evaluating the Black-Box Calibration of ChatGPT (2024.lrec-main)

Copied to clipboard

Youliang Yuan, Wenxuan Wang, Qingshuo Guo, Yiming Xiong, Chihao Shen, Pinjia He

Natural Language Interfaces

Challenge:	Recent performance of ChatGPT in downstream tasks is questionable, but does it know that it does not know?
Approach:	They propose to use three types of proxy confidence to evaluate ChatGPT's black-box calibration ability.
Outcome:	The proposed model exhibits a positive correlation with accuracy in TruthfulQA and a negative correlation in the ModAr dataset.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations