Dear Chiropractic College Leadership,
The NBCE is proud to announce the launch of a new project to evaluate advancements in Objective Structured Clinical Examinations (OSCE) in the hopes of producing a more efficient exam, while maintaining exam reliability and validity. We are calling this the Part IV Advancement and Development project. Because of your dedicated service administering the Part IV exam, we are hopeful of your support and feedback as we investigate and eventually implement improvements to the NBCE’s OSCE. We would like to take this opportunity to share the impetus behind this project, as well as our current plans and schedule.
Background
After a year of successful computer-based exam administrations, the NBCE decided to turn its focus to evaluating the current state of the Part IV exam. At the end of 2019, we performed a preliminary analysis of our current Part IV exam in comparison to Objective Structured Clinical Examinations (OSCE) given to other healthcare professionals. The NBCE’s Board of Directors saw a need to further investigate best practices in the field of OSCE testing and formed a project team with the following objectives:
1. Perform an evaluation of all OSCE exam trends by researching a benchmark group.
2. Engage stakeholders to gather feedback about the current state of the NBCE’s Part IV exam and gauge interest in improvements.
3. Perform a psychometric analysis of the current Part IV exam to identify areas of strength and areas that could be improved.
In 2020, the NBCE connected with other testing organizations including the National Board of Osteopathic Medical Examiners and the National Board of Examiners in Optometry. We extensively researched other OSCEs, and created preliminary budgets to evaluate the cost of potential changes. We sent a survey to examinees to get their opinion about the current state of the Part IV exam, and held a focus group with state board members about potential changes to the exam. Ultimately, we developed a list of “trends”, which are potential improvements to the NBCE’s Part IV exam that were discovered through our benchmark analysis. When evaluating potential improvements to the NBCE’s Part IV exam, we used the information gathered to evaluate the trends based on the following criteria:
· How well is the trend represented in the benchmark group (considered a best practice)?
· Will it impact the validity or reliability of the Part IV exam?
· Is it financially feasible to implement the trend?
· Will it impact the standardization of the exam?
· How will it impact the NBCE’s stakeholders?
· How feasible is implementation?
Through the evaluation of these criteria, we were able to identify eight potential changes that could improve the NBCE’s Part IV exam and should be evaluated for implementation.
1. Comprehensive Examinee-Guided Station Format: Combining the single-point stations (Case history, Ortho/Neuro, Physical exam, and Technique) into one comprehensive station that better mimics a real-world patient encounter.
2. Adding a Documentation Component: Examinees would be required to create a patient note (after the patient-encounter station) documenting their findings, assessment, and plan for that patient.
3. More External Contribution to Item Development: Develop a method for collecting exam content from a wider variety of people within the chiropractic community (academics, practicing DCs, and state board members).
4. Adding Video Recording to Stations: Requiring patient-encounter rooms to be set up with video recording equipment that can accurately capture the examinee's performance for remote review.
5. Moving DIM to Written Exams: Since DIM is a Computer-Based Test (CBT), we could incorporate the content into the Part III exam, eliminating the Friday testing day.
6. Eliminating Cross Country Sequestration: Removing the sequestration requirement, which currently requires us to hold examinees until the exam is completed across the country.
7. Patient Grading Component: Standardized patients rate the examinee on the “humanistic” components of the station.
8. Paperless Exam: Adding computerized testing equipment for the examinees and evaluators to eliminate scantron answer sheets.
Project Plan
The NBCE has created the following schedule to move us forward in the advancement and development of the Part IV exam.
Phase 1: Discover and Design
2021-2022
Partner with benchmark organizations to redesign the Part IV exam and plan implementation. Gather stakeholder feedback, evaluate vendors, and design a testing and rollout plan for the redesigned Part IV.
Phase 2: Testing
2023
Test the redesigned Part IV exam in the field, adjusting as necessary, communicating Part IV changes to stakeholders, and planning rollout.
Phase 3: Implementation
2024
Possible first administration of the redesigned Part IV exam.
Advisory Board
As we begin Phase 1 of this project, we will be forming an Advisory Board. We want diverse representation from the educational and regulatory communities, with oversight from the NBCE Board of Directors.
4 Directors of Clinics/ Directors of Clinical Assessment
Chosen by the Association of Chiropractic Colleges (ACC).
1 Member from the ACC
Chosen by the Association of Chiropractic Colleges (ACC).
5 State Board Members
One representative from every district chosen by the NBCE District Director.
1 Overall-In-Charge (OIC)
Chosen by the Director of Part IV.
2 NBCE Board Members
The advisory board is not directly responsible for executing project activities, but will provide valuable input on the project team’s plans by sharing insight and expertise while representing the interests and opinions of their stakeholder groups. The advisory board will also act as conduits of information to and from their stakeholder groups, sharing appropriate information about the project plan to garner support and buy-in for the project.
Stay Informed
We have created a website you can access at any point to receive the latest news about the Part IV Advancement and Development project. You can access the website here: https://www.nbce.org/pad/. On the main page, you can join our mailing list to receive project-related updates and information.
Thank you for taking the time to learn about this project. We hope we can count on your partnership and support during this exciting new endeavor. If you have any questions, or require additional information, please contact us at partivproject@nbce.org.
Sincerely,
