C. Sauerwein, T. Antensteiner, S. Oppl, I. Groher, A. Meschtscherjakov, P. Zech, R. Breu: Towards a Success Model for Automated Programming Assessment Systems, In Proceedings of the 28th Annual ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE), July 10-12, 2023, Turku, Finland. Doi: 10.1145/3587102.3588848
The evaluation of source code in university education is a central and important task for lecturers of programming courses. Thereby, lecturers are confronted with a growing number of students, increasingly heterogeneous learning types, a shortage of tutors and highly dynamic learning objectives and technologies. To support lecturers to meet these challenges, the use of automated programming assessment systems (APASs) is a promising solution. Consequently, APASs have been widely used to help lecturers supervise many students from heterogeneous backgrounds and support students in learning programming skills. Measuring the effectiveness and success of these platforms is crucial to understanding how such platforms should be designed, implemented, and used. However, research and practice lack a common understanding of aspects influencing the success of APASs. To address these issues, we have devised a success model for APASs based on established models from information systems as well as blended learning research and conducted an online survey with 414 students using the same APAS. In addition, we examined the role of mediators intervening between technology-, system- or self-related factors, respectively, and the users’ satisfaction with APASs. Ultimately, our research has yielded a model of success comprising seven constructs influencing user satisfaction with an APAS.