Abstract
This study investigated the predictive validity and the reliability of pass/fail decisions of the College Level Academic Skills Test. Passing scores are required to receive an Associate of Arts degree and for admission to the upper division of Florida's State University System (SUS). The test consists of three multiple-choice sections (Reading, English Language Skills and Mathematics) and an Essay.The sample consisted of 7,757 students from Miami-Dade Community College who had received an Associate of Arts degree and subsequently enrolled in the upper division. Twelve percent were Non-Hispanic Black, 60% Hispanic and 25% Non-Hispanic White.Predictive validity was calculated by correlating scores with first semester GPA in the SUS. The resulting correlations were low for each section. Differences by ethnic group were found only in Mathematics where correlation with GPA was significantly higher for Hispanic students than for Black Non-Hispanic and White Non-Hispanic students.Reliability of pass/fail decisions for the multiple-choice sections was determined under three sets of passing scores using the Threshold Loss Index of Agreement utilizing a method developed by Huyhn (Berk, 1984) for a single test administration. This index assumes that pass/fail misclassifications are equally serious whether they are made by 1 point or 20.As passing scores increased, consistency of pass/fail decisions decreased. Consistency was highest for Mathematics, followed by Reading and English Language Skills. Differences in consistency across ethnic groups were found primarily under the lowest set of passing scores. Consistency was highest for Non-Hispanic White students followed in order by consistency for Hispanic and Non-Hispanic Black students. Other results indicated that as passing scores increased, about 2/3 of the students who had earned a GPA $\geq$ 2.00 in the SUS would have been denied admission on the basis of their scores.The reliability and predictive validity of the test was considered unacceptably low. Recommendations included a revision of the test to improve reliability and a large scale study using statewide data.