A Comparative Analysis of Psychometric Properties in AI-Generated and Teacher-Made MCQs Test

Abstract

This study aims to compare psychometric properties of AI-generated test and teacher made test. With the increasing use of AI in education, it has become essential to evaluate AI-generated assessments. Achievement test i.e. AI generated and teacher made test of 50 questions each, were used for this study. Validity was examined though three subject experts and three experienced teachers. Based on these experts judgement, relevance scores for items were consistently high across all experts for both tests. The reliability value of items using KR-20 for AI generated test and teacher made test were 0.9492 and 0.9271 respectively. Data was analyzed by using descriptive and inferential statistics. Descriptive statistics such as means scores, difficulty indices, discrimination indices and inferential statistics such as t-test was used for comparison of difficulty and discrimination indices of both tests. The major findings of the study were; on average, validity, reliability, item difficulty level and discrimination index of AI generated test is nearly equal to that of teacher made test. Recommendation of the study is by complimenting AI based test with teacher test, we can reduce work load of teachers while providing consistent assessment across different classes.