On the Arbitrary Nature of Cut Scores Used in Progression Testing

As I continue to be contacted weekly by students, faculty, parents, etc., about the use of progression policies in nursing programs across the country, something interesting has come to light. Schools are using varying cut scores in their progression policies. Indeed Nibert, Young and Britt (2003) noted that of the schools surveyed in thier study, scores that schools used for "benchmarking" (i.e., a "cut-score") ranged from 77 to 90 (corrected for today's scores, that would be 770 to 900).

Why should the scores schools use be different from place to place? Standard setting for educational tests is somewhat of a science (Broadfoot, 2002; Karantonis & Sireci, 2006), but it doesn't appear that most schools are using any empirical data to support the cut scores they are using. This can be inferred in that schools have varying cut scores in place for their progression policies. How are decisions on cut scores made? What evidence supports one score over another?

For example, Lewis (2006) reported on data from a HESI Exit Exam® dataset with N = 8,009, that students scoring in the 800-849 range on the Exit Exam passed the NCLEX-RN® 93.3% of the time. Students scoring in the 700-799 range pased the NCLEX-RN 85.3% of the time (see the Figure).

Why then, would schools have a cut score of 850 on the Exit Exam when data from a large N suggests that students scoring less than 850 might have more than a 9/10 chance of passing the NCLEX-RN?

Zieky and Perie (2006) suggest that when setting a cut score, the harm that will be done if students are misclassified must be considered. If students are misclassified as "likely to fail" and are therefore prevented from graduating and taking the NCLEX-RN, then there can be profound consequences for that student's life. Their job plans, financial status, personal relationships, and many other dimensions of life are instantly in jeopardy. There is a serious risk of harm, therefore, if students are misclassified as unprepared for the licensure exam when they in fact are.

On the other hand, if students are classified as likely to pass, and are therefore allowed to graduate and sit for the licensure exam, the risk of harm in that misclassification is less harmful for the student (who can take the exam again in a matter of weeks), but more important for the school, which seeks to have a high NCLEX-RN pass rate.

The only logical conclusion then is that when cut scores are chosen non-empirically (i.e., not based on available data), and possibly even arbitrarily, the risk of harm is shifted significantly to the student with schools erring on the side of not allowing qualified students to graduate and sit for the licensure exam.

The school's licensure pass rate may be protected, but the lives of many students are profoundly and negatively impacted. Many, many qualified nurses are then kept from the workforce because they are unable to test for a nursing license due to a progression policy that prevents their graduate on the basis of a score from one test.

References

Broadfoot, P. (2002). Dynamic versus arbitrary standards: Recognising the human factor in assessment. Assessment in Education: Principles, Policy & Practice, 9(2), 157-159.

Karantonis, A., & Sireci, S. G. (2006). The bookmark standard-setting method: A literature review. Educational Measurement: Issues and Practice, 25(1), 4-12.

Lewis, C. (2006). Predictive accuracy of the HESI Exit Exam on NCLEX-RN pass rates and effects of progression policies on nursing student exit exam scores. Dissertation Abstracts International, 66(11), B. (UMI No. 3195986)

Nibert, A. T., Young, A., & Britt, R. (2003). The HESI Exit Exam: Progression benchmark and remediation guide. Nurse Educator, 28(3), 141-145.

Zieky, & Perie (2006). A primer on setting cut scores on tests of educational achievement. Retrieved June 22, 2008 from http://www.ets.org/Media/Research/pdf/Cut_Scores_Primer.pdf.