Papers by Nuan Wen
COM2SENSE: A Commonsense Reasoning Benchmark with Complementary Sentences (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in pretrained language models have shown promising results on commonsense reasoning benchmark datasets. |
| Approach: | They propose a commonsense reasoning benchmark dataset with 4k sentence pairs . they propose 'gamified' model-in-the-loop setup to incentivize challenging samples . |
| Outcome: | The proposed benchmarks show that the proposed model achieves 71% standard accuracy and 51% pairwise accuracy, well below human performance. |
EventPlus: A Temporal Event Understanding Pipeline (2021.naacl-demos)
Copied to clipboard
Mingyu Derek Ma, Jiao Sun, Mu Yang, Kung-Hsiang Huang, Nuan Wen, Shikhar Singh, Rujun Han, Nanyun Peng
| Challenge: | Event information is a type of common sense knowledge that helps people understand how stories evolve and provides predictive hints for future events. |
| Approach: | They propose a temporal event understanding pipeline that integrates state-of-the-art components. |
| Outcome: | The proposed pipeline can be easily adapted to other domains, including biomedical domains. |
DEAM: Dialogue Coherence Evaluation using AMR-based Semantic Manipulations (2022.acl-long)
Copied to clipboard
| Challenge: | Existing evaluation metrics for incoherent dialogues are insufficient to accurately reflect incohérence . despite the effectiveness of large pretrained language models, not everyone is into this type of work. |
| Approach: | They propose a Dialogue coherence Evaluation metric that uses Abstract Meaning Representation to apply semantic-level Manipulations for incoherent (negative) data generation. |
| Outcome: | The proposed evaluation metric achieves higher correlations with human judgments compared to baseline methods on dialog datasets by significant margins. |
Can Language Model Moderators Improve the Health of Online Discourse? (2024.naacl-long)
Copied to clipboard
Hyundong Cho, Shuai Liu, Taiwei Shi, Darpan Jain, Basem Rizk, Yuyang Huang, Zixun Lu, Nuan Wen, Jonathan Gratch, Emilio Ferrara, Jonathan May
| Challenge: | Existing efforts to automate conversational moderation have focused on banning harmful comments or deleting them, but such efforts can inadvertently push users towards echo chambers that exacerbate polarization. |
| Approach: | They propose a framework to assess models’ moderation capabilities independently of human intervention and propose 'conversational moderation' they propose to use language models as conversational moderators to provide specific feedback on toxic behavior but struggle to influence users to increase their levels of respect and cooperation. |
| Outcome: | The proposed framework assesses models’ moderation capabilities independently of human intervention and shows that appropriately prompted models provide specific and fair feedback on toxic behavior but struggle to influence users to increase their levels of respect and cooperation. |