Papers by Praneeth Ganedi
Common Law Annotations: Investigating the Stability of Dialog System Output Annotations (2023.findings-acl)
Copied to clipboard
Seunggun Lee, Alexandra DeLucia, Nikita Nangia, Praneeth Ganedi, Ryan Guan, Rubing Li, Britney Ngaw, Aditya Singhal, Shalaka Vaidya, Zijun Yuan, Lining Zhang, João Sedoc
| Challenge: | High agreement is often used to show reliability of annotation procedures, but it is insufficient to ensure or reproducibility. |
| Approach: | They propose a protocol that increases Inter-Annotator Agreement among annotators and a standardized and codified protocol that strictly enforces transparency in the annotation process. |
| Outcome: | The proposed protocol ensures transparency in the annotation process, which ensures reproducibility of annotation guidelines. |