Papers with VTAs
A Large-Scale Real-World Evaluation of an LLM-Based Virtual Teaching Assistant (2025.acl-industry)
Copied to clipboard
| Challenge: | Empirical studies on their effectiveness and acceptance in real-world classrooms are limited, leaving their practical impact uncertain. |
| Approach: | They develop an LLM-based virtual teaching assistant and deploy it in an introductory AI programming course with 477 graduate students. |
| Outcome: | The proposed system is tested in an introductory AI programming course with 477 graduate students. |
Bringing Pedagogy into Focus: Evaluating Virtual Teaching Assistants’ Question-Answering in Asynchronous Learning Environments (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing assessments rely on surface-level metrics and lack sufficient grounding in educational theory . a new framework is proposed to evaluate VTAs in asynchronous learning environments . |
| Approach: | They propose a pedagogically-oriented evaluation framework tailored to asynchronous forum discussions . they construct classifiers using expert annotations of VTA responses on a diverse set of forum posts . |
| Outcome: | The proposed evaluation framework is rooted in learning sciences and tailored to asynchronous forum discussions. |