Evaluation findings related to Situational Judgment Tests (SJTs) as an admission criterion for a BSCN program will be presented. Findings will discuss the impact on students from diverse learning backgrounds and the SJTs relationship to graduation GPA, then engage attendees in a sample SJT, and share strategies for optimal success.

Learning Outcomes: 1. To outline current literature related to the use of online Situational Judgement Tests (SJTs). 2. To discuss program evaluation results related to the use of SJTs and the impact on students from diverse learning backgrounds as an admission criterion to a Bachelor of Science in Nursing Program as well as the SJTs relationship to graduation GPA. 3. To engage in a sample SJT to discuss the attendees experience related to their perceived inclusivity for diverse students. 4. To share strategies to ensure that SJTs are respecting our diverse student population and working towards an inclusive and equitable admission procedure. Background Online Situational Judgment Tests (SJTs) have been used as an admission criterion for many health related program. We began using an online open video and text response SJT (CASPerTM - Computer-based Assessment Sampling Personal characteristics) as an admission criterion however for a two-year period, we did not exclude any applicants that achieved the minimum admission criteria based upon their performance on the CASper test to assess the impact of this criterion on diverse student applicants. This presentation will outline current literature related to SJTs, share our program evaluation findings related to using CASPer as an admission criterion, the relationship of SJTs to Grade point average (GPA), and engage attendees in a sample SJT to explore their perceptions related to their perception of the inclusivity for diverse student learners. Literature Review Cognitive measures such as GPA have been shown to be strong predictors of future performance in professional programs (Allensworth & Clark, 2020; Kreiter & Kreiter, 2007). However, they have little predictive value associated with professional and personal characteristics intrinsic to programs such as medicine and nursing (Siu & Reiter, 2009). Evidence suggests that traditional screening tools such as essays, interviews, letters of reference, and personal statements have significant limitations when predicting future personal and professional characteristics (Kulatunga-Moruzi & Norman, 2002; Albanese et al., 2003). A recent systematic review (Zamanzadeh, et al., 2020) reported the main non-cognitive abilities examined in nursing applicants included communication skills, teamwork, dynamism, morality, psychological strength, emotional intelligence, and warmth. Given the nature of the nursing profession, screening for personal and professional characteristics that include ethics, resiliency, empathy, communication, and collaboration should be an essential component of the admissions process. The use of a screening tool for personal and professional characteristics may identify and screen out applicants who may find nursing is not a “fit” once they are in the program thus reducing the financial burden on a student, reducing attrition rates, increasing retention and successful transition to practice post-graduation. The need for the assessment of non-cognitive abilities is further supported by reports from nurse educators of a rise in incivility and unprofessional attitudes and behaviors of nursing students (Anderson et al., 2019). A scan of the provincial nursing college website complaints and disciplinary actions between 2016 and 2023 suggest that many of the sanctions, revocation of license and/or reprimands are related to unprofessional behavior. The CASPer™ test is a 90- to 110-minute on-line SJT screening tool. There are video clips and word-based scenarios which target the applicant’s values, beliefs, and ethical principles. The questions “elicit responses from applicants on how they might behave in slightly different situations or with different information” (Dore & Reiter, 2015). Reliability values reported for CASPer in medical school applicants range from 0.70 to 0.80; predictive validity of 0.50 (Altus Assessments, 2016). CASPer offers test writing accommodations if required. The accommodation status is not shared with the academic program(s) or test evaluators. However, recent research is suggesting that diverse student learners are being placed at a disadvantage when it comes to taking an SJT. Juster and colleagues (2019) reported that Whites scored higher than Blacks on the CASPer test but indicated that this was less of a difference than the differences between the groups on the academic metrics. Similar findings were reported by Rosales et al. (2019) when examining a sample of 1245 students (n=106 Black students) and Gustafson et al. (2023) when analyzing data from sample of 1237, with 118 Black applicants and 20 Native American applicants. These recent findings cause one to pause and thoroughly examine the impact that the use of a SJT has on diverse student applicants to professional programs. There is discussion in the literature related to weighting admission scores to ensure that inclusivity is attained and resulting admissions are from diverse learners. For example, the Pareto optimization approach allows for institutional flexibility in predictive measure as well as achieving diversity goals (Gustafson et al., 2023). Livens et al. (2022) further lend support weighting admission scores to balance high predictability of success and ensuring diversity. Providing support for diverse applicants prior to the administration of a SJT has also been reported in the literature, demonstrating higher confidence and competence for under-represented groups when taking a SJT after having had a preparation program (Mahmood, 2023; Oguntala et al., 2023). What is noticeably lacking in the current literature is research related to the experiences of neurodivergent, LGBTQ2SIA+, and African descent applicants writing a SJT as an admission criterion, particularly in professional and health care programs. What research is available is limited in its' utility related to small sample sizes of diverse student learners. Therefore, it behooves every educational institution that uses a SJT as an admission criterion to analyze their internal data on an annual basis to ensure that the admission criteria are inclusive for diverse applicants and does not disadvantage them when making admission seat offers. Ongoing quality control, paired with offering preparation programs and aligning with best practices related to equitable admission policies is of paramount importance. Design As part of program evaluation, a descriptive design was used to investigate if self-identified diverse applicants (African descent, neurodivergent, and 2SLGBTQ+) were disadvantaged when calculating admission scores that included the results of the CASPer test. Secondly, we assessed the data to see if there was a statistically significant relationship between an applicant’s CASper score and their graduation cumulative grade point average (GPA) upon graduation. Data Collection: CASPer scores were provided for two admission cycles in relation to ethnicity (Black/African descent, not identified), neurodivergent, and 2SLGBTQ+, and resulting graduation GPA. During these two admission cycles, no applicants were excluded from admission if they did not reach the CASPer threshold score (3.5 using the scaled-up value; actual CASper Z-score + 5). Data Analysis: Descriptive statistics were used to assess the % of applicants that were below the CASPer threshold, and their resulting admission offer status. If a student scored below the CASPer threshold and was accepted or put on a wait list that meant that they achieved the minimum academic grade required for admission. If a student was rejected that meant either they had not achieved the minimum academic grade, or they did not submit all required documents for admission. Data was analyzed using simple linear regression to assess if there was a relationship between the independent variable (CASPer scores) and the dependent variable (GPA). Results The data set consisted of 3188 non-identified students, 261 diverse ethnicity students, 150 neurodivergent students, and 25 2SLGBTQ+. Out of the total diverse applicants, 18 applicants were below the threshold (6.9%) and 6 neurodivergent applicants were below the threshold (4.0%). There we no 2SLGBTQ+ applicants that scored below the threshold. Comparatively, the non-identified group had 214 students score below the threshold (6.7%). Our findings support the use of CASPer as an admission criterion from a quantitative point of view and found that diverse applicants were equivalent to non-identified applicants with both neurodivergent and 2SLGBTQ+ applicants having a higher percentage of students achieving or exceeding the CASPer threshold score. One limitation is the self-identified nature of ethnicity, neurodivergence, and 2SLGBTQ+ during the admission process. Out of a total of 528 students, there were 371 who had CASPer scores with a corresponding graduation GPA. CASPer scores ranged from 3.55 to 7.71 (mean 5.47, s.d, .748) and GPA scores ranged from 2.64 to 4.28 (mean 3.57, s.d, .328). The relationship between admission CASPer scores and GPA was positive (0.77) and based upon the t-value (3.533), with a p < .001, the variables have a statistically positive linear relationship. Conclusion Our data demonstrate that diverse individuals were not disadvantaged when using a SJT (i.e., CASPer) as an admission criterion over these two admission cycles in relation to admission scores. However, that does not negate the strategies that have been identified in the literature related to preparation programs to prepare students to take SJT. Nor does it mean that one can get complacent with their use as it is important to assess annually to ensure any disadvantaged students are identified immediately. The positive linear relationship between GPA and CASPer scores lend support for the use of this SJT as an admission criterion to avoid sole reliance on cognitive measures. Further research is needed to study the experiences and perceptions of our diverse student population and their lived experiences when taking a SJT to help understand how we can provide optimal support for an equitable admission process and ultimate program success. References will be available at the presentation.