Validity and Reliability of Artificial Intelligence (AI) Instrument in Instructional Leadership Practice Based on Rasch Model Approach

Abstract

This pilot study was conducted to validate and examine the reliability of an instrument designed to measure Artificial Intelligence (AI) in Instructional Leadership Practice. The instrument consists of 59 items distributed among 45 principals, headmasters, and senior assistant curriculum teachers under the Ministry of Education Malaysia. The instrument was developed to measure four constructs: (i) the three pillars of AI, (ii) defining the school mission with AI, (iii) managing AI-assisted instructional programs, and (iv) creating a positive AI-assisted climate. The Rasch Model approach was used to examine the validity and reliability of the instrument in this pilot study. The Rasch approach was chosen because it allows for measuring both item and respondent reliability more rigorously than Cronbach’s Alpha. It also enables item removal based on item polarity, item fit, and standardized residual correlations. The final analysis revealed that ten items did not meet the criteria and were removed. The final instrument contained 49 items that met validity and reliability standards, making it suitable for measuring the four constructs of AI in Instructional Leadership Practice. As this was a pilot study, future large-scale implementation can be conducted to further measure these constructs among principals, headmasters, and senior assistant teachers.