Papers by Arun Kirubarajan
ChatEval: A Tool for Chatbot Evaluation (N19-4)
Copied to clipboard
| Challenge: | open-domain dialog systems are difficult to evaluate due to lack of standardization and standardization in evaluation procedures. |
| Approach: | They propose a framework for human evaluation of chatbots that augments existing tools . researchers can submit their trained models to the ChatEval web interface . reproducibility and model assessment for opendomain dialog systems is challenging . |
| Outcome: | The proposed framework provides a web-based hub for researchers to compare their models with baselines and prior work. |
RoFT: A Tool for Evaluating Human Detection of Machine-Generated Text (2020.emnlp-demos)
Copied to clipboard
| Challenge: | Existing studies on how humans perceive machine-generated text are limited due to the prohibitive cost of running human evaluation studies. |
| Approach: | They propose a task to detect the boundary at which a text passage starts off human-written transitions to being machine-generated. |
| Outcome: | The proposed system evaluates machine-generated news articles on a wide range of domains. |