Valid interpretation of educational test scores assumes that observed responses reflect the intended response process. In low-stakes settings, however, examinees may guess rapidly, omit items, or otherwise respond without sufficient engagement, so that accuracy alone may no longer represent the intended ability. This thesis developed a simulation-based machine-learning framework for detecting response-level engagement from correctness, response times, omissions, and derived behavioral features. Because true engagement is unobserved in empirical data, simulated data with known engagement states were used to train and evaluate the models. The final simulation design crossed response-time separation, omission signal strength, and disengagement prevalence, producing 27 scenarios. 

For each scenario, four independent training datasets were generated and pooled, and pretrained models were evaluated on 100 fresh target datasets per scenario. Neural network, random forest, and XGBoost models outperformed logistic regression, while response-time separation was the dominant condition affecting detection quality. Additional PCA-based embedding features and larger retained component sets did not improve performance beyond the behavioral feature set. Finally, the pretrained models were applied to the PISA PeruM1 response-time dataset used by Ulitzsch, von Davier, and Pohl (2020). 

Because this empirical dataset does not include true engagement labels, the application was interpreted descriptively When applied to the empirical PeruM1 PISA mathematics subset, the simulation-trained models produced substantively plausible prediction patterns. Answered responses generally received higher predicted engagement probabilities, omitted responses received lower probabilities, and not-reached responses were more ambiguous across models. Thus, the results support the feasibility of simulation-trained engagement detection as a model-based approach, while also highlighting that empirical applications must be interpreted cautiously when external criterion labels for engagement are unavailable.

Keywords: disengaged responding, response times, omissions, machine learning, simulation study, educational measurement