Papers by Priyanshul Govil
SaGE: Evaluating Moral Consistency in Large Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies on Large Language Models (LLMs) have focused on accuracy but lack universally agreed-upon answers for moral scenarios. |
| Approach: | They propose a measure called Semantic Graph Entropy to measure a model's moral consistency grounded in "Rules of Thumb" they construct a moral Consistency Corpus (MCC) with 50K moral questions and the RoTs they followed to investigate LLM consistency on two popular datasets. |
| Outcome: | The proposed measure measures moral consistency on two popular datasets . |