Abstract
Item preknowledge - having access to some test items prior to taking a test – is one of the way that examinees can gain an unfair advantage on a test, and determining the examinees with item preknowledge is still a challenging problem in high-stakes testing. Several different statistical methods have been developed to detect item preknowledge, most of which were utilized from item response data. Advances in testing technology and the availability of an additional source of information (e.g., response time, gaze fixation counts, click tracking) brought various dimensions into the test security field. One of the new sources of information – response times – has recently attracted significant attention in the literature as they may provide meaningful information about item preknowledge.
The Deterministic Gated Lognormal Response Time (DG-LNRT) model, which is suggested in this dissertation, uses response times to identify test takers who have item preknowledge. The model hypothesizes two latent speed parameters: one for responding to items when the examinee has item preknowledge and another for responding to items when the examinee does not have item preknowledge. The proposed model uses a statistical gating mechanism to decompose observed item response times instead of raw item responses. Based on the given model structure, real data applications and simulation studies were carried out to evaluate the performance of the DG-LNRT model.
Throughout real data applications, it was illustrated that the detection rate and accuracy of the DG-LNRT model differed considerably based on the data characteristics. (e.g., percentage of compromised items and examinees with item preknowledge, item preknowledge effect on response time). An extensive simulation study with 64 conditions was carried out to determine the most effective data characteristics on model power to detect cheating, and the results were analyzed to evaluate the significance of each factor.