Programme of Study: Assessment of the Admissions Enrollment Processes

Original Editor - Stacy Schiurring based on the course by Larisa Hoffman

Top Contributors - Stacy Schiurring

Introduction[edit | edit source]

Professional programmes are expected to perform a self-evaluation on the success of the programme.  This is often done using a quality improvement process:  Planning, Doing, Studying, and Acting.[1] Applying this process to admissions, means reflecting on the characteristics of the successful graduate, using tools that will select applicants who will most likely become the successful graduate, evaluating the effectiveness of the criteria, and modifying the criteria.  

Characteristics of the Successful Graduate[edit | edit source]

In the planning phase, where the admissions criteria are initially selected, success must be defined.  Desirable characteristics of a successful graduate of the programme include:

  • success as a practitioner
  • success in the licensure exam
  • success in the programme

Tools for Selecting the Graduate[edit | edit source]

Tools that are most often used as admissions criteria include (1) grade point average (GPA), (2) aptitude tests, (3) interviews, (4) personal statements, (5) letters of reference, and (6) personality testing.[2][3]  The predictive validity of these tools has been evaluated for the selection of students who will be successful in their future performance.[2][4]  In general, these are described as cognitive and non-cognitive assessments.

  1. Pre-professional GPA is a strong predictor of GPA in the professional programme for physical therapy.[5][6][7] [8][9](Nuciforo, 2014).  Pre-professional GPA only weakly correlates with scores on licensure exams.[6][7][10]
  2. Aptitude Tests
    • For physical therapy (PT) programmes in the United States, the verbal GRE and Quantitative GRE are predictive of the first year GPA in a PT programme[9], although this relationship is not consistently observed.[11]  Verbal and Quantitative GRE scores are the best predictor of National Physical Therapy Examination (NPTE) Scores.[12]  Scoring low on the GRE, particularly the verbal component of the GRE predicts failure on the physical therapy licensure examination.[13][8]
    • Noncognitive assessments have been proposed to identify behaviors and values consistent with healthcare professions.  The purpose of noncognitive assessments is to identify characteristics in an applicant that are challenging to obtain by other means.[13] [14] Non-cognitive assessments used in the admissions process include interviews, personal statements, and letters of reference.[2] 
  3. Interviews
    • The inter-rater reliability of interviews is quite variable [14][15][16][17], but interviews can predict performance on clinical skills.[18] 
    • To improve reliability and validity, Hollman[13] suggests creating a semi-structured interview emphasizing behaviors expected in the practicing clinician.[13]  The behaviors include (1) decision making and problem solving, (2) interpersonal skills, (3) focus on the patient or client, (4) communication, and (5) teamwork.  The interviewer rates the applicant’s description of experiences in these domains.[13]  Using this type of structured format improves inter-rater reliability[13], and improves ability of the interview to predict performance on a national licensure examination.[13]  The multi-mini interview has similar outcomes when a structured format is applied[19] comprised of multiple brief interview encounters that have a standard format and scored with a rubric.[3]
  4. Personal Statements
    • The reliability and validity of a written essay is variable[20] [21][17][22], and is likely dependent on the guidelines provided to the applicant. Concerns regarding the authors of these statements has been highlighted.  To address this concern, some authors recommend the performance of a writing task based on previously provided materials to assess communication skills (Jones, 2000).  
  5. Letters of Reference
    • The use of letters of reference as a tool for applicant selection has been scrutinized.[2]  While some authors have found that use of objective criteria to evaluate letters of reference is correlated with clinical performance [23][24], the writing ability of the author of the letter (rather than the applicant) can influence the selection process.  
  6. Personality Testing
    • Emotional intelligence is a collection of characteristics that enable a person to be aware of their own emotional state and those of others.[25][26]  Healthcare providers are expected to be self-aware, empathetic and effective at influencing others, which are essential components of emotional intelligence. 
    • Pindar[25] et al recommends the use of a tool to measure emotional intelligence (such as the Trait Meta-Mood Scale) during the admissions process.[25]  Other authors recognize the value of a measurement of emotional intelligence, but raise concerns about the lack of predictive validity.[2]

Recognizing that addition of structure to non-cognitive measures improves the validity of the measures[23][24], a holistic process applied to student admissions has been proposed.[3] 

It is based on these criteria: 

  1. Selection criteria are broad based and often linked to promoting diversity
  2. Criteria is based on experience, attributes, and metrics
  3. Individualized assessment is heavily weighted, along with race and ethnicity.[3]

The tools with the highest predictive validity for medical student’s performance include grade point average[27], aptitude tests (Donnan, 2007), and mini-interviews.[2] [27]  An undergraduate grade point average predicts graduate level grade point average.[3] Aptitude tests tend to predict outcomes on licensure exams.  Mini-interviews maybe a way to promote structure to evaluate communication and interpersonal skills.  

Pre-requisite Courses[edit | edit source]

  • Pre-requisite courses are often listed in the application criteria for healthcare programmes[28] and lack of successful completion of prerequisite courses can create a barrier to admission for pre-occupational therapy [28][29], pre-physical therapy students [28][29], and pre-speech therapy students.[30]  
  • The purpose of pre-requisite courses has been described as foundational knowledge in a subject[30] and as an indicator for success in subsequent science-based courses.[31]  [32] The impact of prerequisite courses on student learning has been questioned.[33]  Topics that are revisited at a higher level of Bloom’s taxonomy or the contextual framework applied in a new way is one way to improve student’s familiarity with a topic.[33] Without revisiting the topic, students are unlikely to be familiar with the topic.[33]     

Admissions Criteria[edit | edit source]

The admissions criteria should admit individuals who will be successful in the programme. 

  1. Student Attrition Rate
    • The simplest assessment of this process is to review the retention rate of the programme and compared to other programmes.  Student attrition in healthcare has been attributed to pressure related to workload, clinical placement, and personal circumstances.[34]  Attrition rates for healthcare students range from 6%[35] to 17%.[36]
    • Poorer student entry qualifications have been associated with student dropout or attrition rates in medical students[37] and physical therapy students.[38]  
    • Low attrition rates within a programme would suggest the admissions process is selecting students who are more likely to be successful.  
  2. Student Demographics
    • Another factor to consider in evaluating admissions criteria is demographics of the admitted group of students.   The demographics of a student group should relatively closely match the demographics of the society the students will service.   This is important to ensure that the demographics of the graduating healthcare students match the demographics of the clients they will serve.  
    • Cultural differences may be a threat to creating a therapeutic alliance between healthcare providers and patients.[39][40]  Individuals from similar cultural and ethnic backgrounds tend to remain in therapy and have less premature termination and better outcomes than when the backgrounds are not aligned. [40][41][42][43]
    • Admissions criteria can create a barrier for specific subgroups within a culture.  Intentionally evaluating the outcome of the admissions process can help identify these barriers, such that they can be revised for future cohorts.  
  3. Student Class Size
    • The number of students admitted will be constrained by two competing factors:  the workforce needs in the community and the resource needs of the programme. 
    • In the assessment of the workforce needs, an assessment of the (1) number and type of local professionals in the community, the (2) age of the professionals, and (3) distribution of professionals (urban compared rural providers and public compared to private sectors).[44]   
    • Understanding the workforce needs can highlight important growth areas for an educational programme, however, the growth must be constrained by the available resources in the programme. 
    • One of the most valuable resources in a programme are the faculty themselves.  Student to faculty ratios have increased overtime, particularly in low to middle income countries.[45]  While a large number of students to a small number of faculty does not necessarily limit student learning, it does imply limited practice opportunities.  This is particularly true in the clinical setting, where clinical supervision and mentorship are necessary for student success.[46]  
    • Limiting the number of students such as the student to faculty ratios (including clinical faculty), while simultaneously achieving workforce development goals strikes a balance in terms of numbers of students admitted.

Admissions Analysis[edit | edit source]

  • The most common way to investigate admissions criteria is through regression analysis.  Regression analysis allows programmes to understand the relationship been successful graduate outcomes and admissions criteria.[47]
  • While predictive validity of admissions criteria has been published, it is very likely that these correlations are dependent on the programme itself.  Therefore, evaluating the predictors of success within the programme is critical.  
  • Correlations between outcomes (such as student performance and graduate performance) and admissions criteria can be measured using a Pearson Correlation Coefficient.[47]  

Kingsley recommends using the following definitions:[47]  

  • -1 to -0.7 = Strong Negative Correlation
  • -0.7 to -0.3 = Negative Correlation
  • -0.3 to 0.3 = No Association
  • 0.3 to 0.7 = Positive Correlation
  • 0.7 to 1.9 = Strong Positive Correlation

It is important to look at both the strength of relationship, as well as the statistical significance of the finding.  

Resources[edit | edit source]

Clinical Resources[edit | edit source]

Recommended Reading[edit | edit source]

References[edit | edit source]

  1. Brown JF, Marshall BL. Continuous quality improvement: An effective strategy for improvement of program outcomes in a higher education setting. Nursing Education Perspectives. 2008 Jul 1;29(4):205-11.
  2. 2.0 2.1 2.2 2.3 2.4 2.5 Siu E, Reiter HI. Overview: what’s worked and what hasn’t as a guide towards predictive admissions tool development. Advances in Health Sciences Education. 2009 Dec;14:759-75.
  3. 3.0 3.1 3.2 3.3 3.4 Choi AN, Flowers SK, Heldenbrand SD. Becoming more holistic: A literature review of nonacademic factors in the admissions process of colleges and schools of pharmacy and other health professions. Currents in Pharmacy Teaching and Learning. 2018 Oct 1;10(10):1429-37.
  4. Hall JD, O’Connell AB, Cook JG. Predictors of student productivity in biomedical graduate school applications. PLoS One. 2017 Jan 11;12(1):e0169121.
  5. Payton OD. A meta-analysis of the literature on admissions criteria as predictions of academic performance in physical therapy education in the United States and Canada: 1983 through 1994. Physiotherapy Canada. 1997;49(2):97-102.
  6. 6.0 6.1 Thieman TJ, Weddle ML, Moore MA. Predicting academic, clinical, and licensure examination performance in a professional (entry-level) master's degree program in physical therapy. Journal of Physical Therapy Education. 2003 Oct 1;17(2):32-7.
  7. 7.0 7.1 Dockter M. An analysis of physical therapy preadmission factors on academic success and success on the national licensing examination. Journal of Physical Therapy Education. 2001 Apr 1;15(1):60.
  8. 8.0 8.1 Utzman RR, Riddle DL, Jewell DV. Use of demographic and quantitative admissions data to predict academic difficulty among professional physical therapist students. Physical Therapy. 2007 Sep 1;87(9):1164-80.
  9. 9.0 9.1 Shiyko MP, Pappas E. Validation of pre-admission requirements in a doctor of physical therapy program with a large representation of minority students. Journal of Physical Therapy Education. 2009 Oct 1;23(2):29-36.
  10. Zipp GP, Ruscingno G, Olson V. Admission variables and academic success in the first year of the professional phase in a doctor of physical therapy program. Journal of Allied Health. 2010 Sep 1;39(3):138-42.
  11. Kirchner GL, Holm MB, Ekes AM, Williams RW. Predictors of student success in an entry-level master in physical therapy program. Journal of Physical Therapy Education. 1994 Oct 1;8(2):76-9.
  12. Kume J, Reddin V, Horbacewicz J. Predictors of physical therapy academic and NPTE licensure performance. Health Professions Education. 2019 Sep 1;5(3):185-93.
  13. 13.0 13.1 13.2 13.3 13.4 13.5 13.6 Hollman JH, Rindflesch AB, Youdas JW, Krause DA, Hellyer NJ, Kinlaw D. Retrospective analysis of the behavioral interview and other preadmission variables to predict licensure examination outcomes in physical therapy. Journal of Allied Health. 2008 May 28;37(2):97-104.
  14. 14.0 14.1 Albanese MA, Snow MH, Skochelak SE, Huggett KN, Farrell PM. Assessing personal qualities in medical school admissions. Academic Medicine. 2003 Mar 1;78(3):313-21.
  15. Wilding J, Rheault W. Interrater reliability of an admission interview scale. Journal of Physical Therapy Education. 1998 Apr 1;12(1):26-9.
  16. Richards P, McManus IC, Maitlis SA. Reliability of interviewing in medical student selection. Br Med J (Clin Res Ed). 1988 May 28;296(6635):1520-1.
  17. 17.0 17.1 Youdas JW, Hallman HO, Carey JR, Bogard CL, Garrett TR. Reliability and validity of judgments of applicant essays as a predictor of academic success in an entry-level physical therapy education program. Journal of Physical Therapy Education. 1992 Apr 1;6(1):15-8.
  18. Balogun JA. Predictors of academic and clinical performance in a baccalaureate physical therapy program. Physical therapy. 1988 Feb 1;68(2):238-42.
  19. Cameron AJ, MacKeigan LD. Development and pilot testing of a multiple mini-interview for admission to a pharmacy degree program. American Journal of Pharmaceutical Education. 2012 Feb 10;76(1).
  20. Brown B, Carpio B, Roberts J. The use of an autobiographical letter in the nursing admissions process: Initial reliability and validity. Canadian Journal of Nursing Research Archive. 1991 Apr 13:9-20.
  21. Kirchner GL, Holm MB. Prediction of academic and clinical performance of occupational therapy students in an entry-level master’s program. The American Journal of Occupational Therapy. 1997 Oct;51(9):775-9.
  22. Hull AL, Glover PB, Acheson LS, Carter JR, Dick TE, Kirby AC, Lam M, Stevens DP. Medical school applicants' essays as predictors of primary care career choice. Academic Medicine. 1996 Jan 1;71(1):S37-9.
  23. 23.0 23.1 Brothers TE, Wetherholt S. Importance of the faculty interview during the resident application process. Journal of surgical education. 2007 Nov 1;64(6):378-85.
  24. 24.0 24.1 Peskun C, Detsky A, Shandling M. Effectiveness of medical school admissions criteria in predicting residency ranking four years later. Medical education. 2007 Jan;41(1):57-64.
  25. 25.0 25.1 25.2 Kwajaffa PS, Chidi OV, Ali MA, Mukhtar YM, Baba MU. Personality traits and emotional intelligence among health care professionals in a tertiary hospital. International Journal of Psychology and Counselling. 2020 Apr 30;12(2):31-7.
  26. Goleman D. Emotional intelligence: Why it can matter more than IQ. Bloomsbury Publishing; 1996 Sep 12.
  27. 27.0 27.1 Kreiter CD, Kreiter Y. A validity generalization perspective on the ability of undergraduate GPA and the medical college admission test to predict important outcomes. Teaching and Learning in Medicine. 2007 May 25;19(2):95-100.
  28. 28.0 28.1 28.2 Hull K, Wilson S, Hopp R, Schaefer A, Jackson J. Determinants of student success in anatomy and physiology: Do prerequisite courses matter? A task force review 2016. HAPS Educator. 2016 Apr;20(2):38-45.
  29. 29.0 29.1 Freeman S, Haak D, Wenderoth MP. Increased course structure improves performance in introductory biology. CBE—Life Sciences Education. 2011 Jun;10(2):175-86.
  30. 30.0 30.1 Sylvan L, Brock KL, Perkins A, Garrett J. Building blocks of knowledge: A close look at prerequisite coursework for graduate programs in speech-language pathology. Perspectives of the ASHA Special Interest Groups. 2020 Oct 23;5(5):1262-71.
  31. Cor MK, Brocks DR. Examining the relationship between prerequisite grades and types of academic performance in pharmacy school. Currents in Pharmacy Teaching and Learning. 2018 Jun 1;10(6):695-700.
  32. Wireman M, Russell S. Prerequisite coursework as a predictor of performance in subsequent science courses. 2019.
  33. 33.0 33.1 33.2 Shaffer JF, Dang JV, Lee AK, Dacanay SJ, Alam U, Wong HY, Richards GJ, Kadandale P, Sato BK. A familiar (ity) problem: Assessing the impact of prerequisites and content familiarity on student learning. PloS one. 2016 Jan 29;11(1):e0148051.
  34. Hamshire C, Jack K, Forsyth R, Langan AM, Harris WE. The wicked problem of healthcare student attrition. Nursing inquiry. 2019 Jul;26(3):e12294.
  35. Maher BM, Hynes H, Sweeney C, Khashan AS, O’Rourke M, Doran K, Harris A, Flynn SO. Medical school attrition-beyond the statistics a ten year retrospective study. BMC medical education. 2013 Dec;13(1):1-6.
  36. Ryan JM, Potier T, Sherwin A, Cassidy E. Identifying factors that predict attrition among first year physiotherapy students: A retrospective analysis. Physiotherapy. 2021 Mar 1;110:26-33.
  37. O’Neill LD, Wallstedt B, Eika B, Hartvigsen J. Factors associated with dropout in medical education: a literature review. Medical Education. 2011 May;45(5):440-54.
  38. Andrews WA, Johansson C, Chinworth SA, Akroyd D. Cognitive, collegiate, and demographic predictors of attrition in professional physical therapist education. Journal of Physical Therapy Education. 2006 Apr 1;20(1):14-21.
  39. Vasquez MJ. Cultural difference and the therapeutic alliance: an evidence-based analysis. American Psychologist. 2007 Nov;62(8):878.
  40. 40.0 40.1 Sue S. In search of cultural competence in psychotherapy and counseling. American psychologist. 1998 Apr;53(4):440.
  41. Schouten BC, Meeuwesen L. Cultural differences in medical communication: a review of the literature. Patient education and counseling. 2006 Dec 1;64(1-3):21-34.
  42. Anderson KN, Bautista CL, Hope DA. Therapeutic alliance, cultural competence and minority status in premature termination of psychotherapy. American Journal of Orthopsychiatry. 2019;89(1):104.
  43. Paternotte E, van Dulmen S, van der Lee N, Scherpbier AJ, Scheele F. Factors influencing intercultural doctor–patient communication: A realist review. Patient education and counseling. 2015 Apr 1;98(4):420-45.
  44. Conradie T, Berner K, Louw Q. Rehabilitation workforce descriptors: a scoping review. BMC Health Services Research. 2022 Dec;22(1):1-4.
  45. Buckner E, Zhang Y. The quantity-quality tradeoff: a cross-national, longitudinal analysis of national student-faculty ratios in higher education. Higher Education. 2021 Jul;82:39-60.
  46. Naidoo V, Stewart AV, Maleka ME. A tool to evaluate physiotherapy clinical education in South Africa. South African Journal of Physiotherapy. 2022 Aug 31;78(1):11.
  47. 47.0 47.1 47.2 Kingsley K, Sewell J, Ditmyer M, O’Malley S, Galbraith GM. Creating an evidence‐based admissions formula for a new dental school: University of Nevada, Las Vegas, School of Dental Medicine. Journal of Dental Education. 2007 Apr;71(4):492-500.