Accreditation is Credible Education
JUNE 2017 / IN THIS ISSUE
  • New Tools for Graduate Surveys and Employer Survey
  • Now Available: Revised Resource Assessment Tool
  • Evaluation Instruments to Support the RAM
  • Change in Personnel Fee Update
  • Annual Fees Due July 1
  • Applications Now Being Accepted for Site Visitors
  • October Accreditation Workshop
  • By the Numbers
  • Milestones
  • Upcoming Events
  • Quick Links
  • Sponsors
New Tools for Graduate and Employer Surveys

Staff Contact: Lynn Caruthers, lynn@coaemsp.org  

Effective July 1, 2017, there are new survey tools for programs to use for the Graduate Survey and Employer Survey.

The new tools are available here: http://coaemsp.org/Evaluations.htm .   Comprehensive evaluation of Paramedic program performance and outcomes is an essential component of program operation and growth. Graduate and Employer Surveys are key elements of the assessment; however, many programs have continued to struggle with obtaining adequate return rates from one or both groups. In an effort to improve the process and improve the return rate, the CoAEMSP has revised and simplified its tools for Graduate and Employer Surveys. The revised assessment tool basically asks if the graduate is a competent entry-level Paramedic provider in three areas: 1) in the cognitive domain, 2) in the psychomotor domain, and 3) in the affective domain. The rating scale mirrors the metrics used by the National Registry of EMTs (NREMT) in the clinical evaluation forms:
  • Successful/competent: no prompting
  • Marginal: inconsistent, not yet competent
  • Unsuccessful: required excessive or critical prompting
The survey tool can be administered via paper as a fillable PDF or as an electronic distribution product (e.g., SurveyMonkey ® ). The CoAEMSP electronic survey tool was developed with internal logic that opens additional questions if the selected rating was marginal or unsuccessful. If the graduate or employer selects 'competent' in all three domains the survey will consist of three questions plus an opportunity for comment. If the graduate or employer selects 'marginal' or 'unsuccessful,' additional options appear. For the paper version, obviously the skip logic is not applicable and the ‘sub-questions’ will all appear. Programs are required to begin using the revised survey tools with the next cycle of Graduate and Employer Surveys after July 1, 2017. Remember that these surveys are sent six to twelve months following graduation. Programs will continue to report the number of surveys distributed and the number/percent returned and an analysis of the results and action plan for responses that indicate marginal or unsuccessful on the Annual Report.  

For SurveyMonkey users : CoAEMSP has developed the surveys as a SurveyMonkey ® tool which will allow easy transferability of the tool. The program must have the same level of subscription to SurveyMonkey ® , which is Platinum. If your Program has access to a Platinum-level account for SurveyMonkey ® , Lynn at CoAEMSP can assist you in sharing a copy of the surveys for your Program to implement. Lynn will need your SurveyMonkey ® username to transfer a copy of the surveys to you; you may email her at lynn@coaemsp.org .
Now Available: Revised Resource Assessment Tools
Tools to Assist Programs with Meeting CAAHEP Standard III.D. Resource Assessment.

Programs are required to evaluate their services in a variety of ways, obtaining input from multiple sources. One important component is the evaluation of the appropriateness and effectiveness of the program resources. Two standard tools are provided, and required, by the CoAEMSP: 1) the Program Resource Survey completed by the Students  and 2) the Program Resource Survey completed by the Program Personnel. This latter form is used to survey the program faculty, the Medical Director, and the Advisory Committee members.  There are nine components evaluated: program faculty, Medical Director, support personnel, curriculum, financial resources, facilities, hospital and clinical resources, capstone field internship resources, learning resources, and physician interaction.  

The surveys and the Resource Assessment Matrix (RAM) were in need of a facelift and an update of the questions. This has been completed and the new surveys and an Excel® tool for data entry and RAM completion are now available and posted on the CoAEMSP website at http://coaemsp.org/Evaluations.htm.  The scoring system has been modified and has moved from the traditional five-point Likert scale to a simpler and meaningful rating system: 'yes,' 'no' or 'not applicable.' The rationale for the change in rating lies in the perceived lack of value when it comes to resources: does it affect change if the rating is ‘good’ or ‘very good’?  What information does a ‘neutral or neither agree nor disagree’ provide?  In the final analysis, a program either has the resources it needs to conduct the educational course, or it does not. As always, there is room for comment. The final overall assessment of resources by the participant is not sufficient, marginal or sufficient.  

Data from each survey is entered into the appropriate Excel® tab of the RAM and the spreadsheet automatically calculates the cumulative percentages for each question. The final percentage is also calculated for each of the nine components. If the percentage for the components is less than 80%, the program must enter an analysis and action plan in the appropriate columns. While the category percentage may be greater than 80%, do not ignore individual questions however that did not meet the threshold.  

The Word® document that was previously an option for the RAM is no longer available and the Excel version is the required tool.  

Completing the RAM is more than an exercise in data collection. Once the results have been tallied, the next important step is review and discussion of the results with the faculty, administration, Medical Director, and Advisory Committee. The RAM is one building block for program assessment and improvement.

The RAM is completed annually and programs must have a plan to survey students from all cohorts, faculty, Medical Director, and the Advisory Committee. At the time of the site visit, programs will need to present a RAM for each of the years in the accreditation cycle. The new tools are currently available for implementation. Since the rating scales are different, depending on your academic cycle, programs may wish to complete 2017 with the original forms. For programs on an academic calendar, adoption with the fall semester may be most appropriate. All programs are required to adopt the revised surveys and RAM beginning in January 2018.

Evaluation Instruments to Support the Resource Assessment Matrix (RAM)

Evaluation includes a variety of tools, administered at specific points throughout the Program.  Guidelines for administration, analysis, and implementation are identified.  See http://coaemsp.org/Evaluations.htm for the tools to assist programs with conducting the evaluations.

Instructor Evaluations

  1. New faculty members, adjunct/skill instructors, and content experts should be evaluated with sufficient frequency to determine student reception and identify areas that require counseling or positive feedback.
  2. Program Directors should observe and evaluate experienced faculty members at least annually.
  3. Evaluations should be reviewed with the individuals: document notes of the topics discussed and feedback and direction provided.  These observations may become part of a professional development plan.


Clinical and Field Internship Evaluations  

  1. All clinical sites must be evaluated by each student.
  2. Each capstone field internship preceptor must be evaluated by each student.

Course Evaluations

  1. Evaluations should be distributed and collected at the end of each term. 
  2. A final course/program evaluation should be required once the capstone field internship is complete.  
  3. The Program Director should review the evaluations and compile a list of positives/issues using a course summary report format.
  4. The course/program evaluations should be reviewed with the appropriate instructor(s).
  5. The course/program evaluations and course summary report may be reviewed with the school leaders.
  6. The course/program evaluations should be reviewed with the Program Medial Director.
  7. Discuss any changes/revisions to the Program resulting from the analysis of the evaluations at a staff meeting and document the discussion in faculty meeting minutes.
  8. Capture any other changes/revisions in writing and retain those documents.


Graduate and Employer Surveys

  1. 100% of graduates and their employers must be surveyed using the CoAEMSP tool.
  2. Record and report the return rate for both survey types.
  3. Discuss the results with the faculty, Medical Director, and Advisory Committee.

Program Evaluation

  1. Complete the CoAEMSP Resource Assessment Matrix annually and review the results with the faculty, the Medical Director, and the Advisory Committee.
  2. Consider completing a SWOT analysis with faculty and stakeholders: Strengths, Weaknesses, Opportunities, and Threats.
  3. Review the four outcome thresholds required to be reported by the CoAEMSP with the faculty, Medical Director, and the Advisory Committee and develop a plan to improve outcomes if needed.  These include: credentialing cognitive exam pass rates; credentialing psychomotor exam pass rates; retention; and positive placement (employment).
  4. Evaluate clinical and field internship resources for adequacy of clinical and field contacts: numbers and types of patients and the types of procedures, skills.
Change in Personnel Update

Staff Contacts: Lynn Caruthers, lynn@coaemsp.org or Ruth Crump, ruth@coaemsp.org 

When there is a change in key personnel (President, Dean, Program Director, Medical Director, Associate Medical Director, Assistant Medical Director, or Billing Contact), whether on a permanent, temporary, or acting basis, the CoAEMSP Executive Office must be notified no later than (15) calendar days following the effective date of vacancy. A Personnel Change form and supporting documentation must be submitted.

Effective July 1, 2017, failure to notify the CoAEMSP within the times prescribed in CoAEMSP Policies and Procedures, Policy XV, Personnel Changes, will be assessed a fee of $100.

For more about the process to submit personnel changes, visit http://coaemsp.org/Personnel_Changes.htm. For more information on the fees, visit http://coaemsp.org/Fees.htm.

Registration Now Open for October CAAHEP Accreditation Workshop for Paramedic Programs 
The workshop is focused on the new 2015 CAAHEP Standards and the new Self Study Report templates. These accreditation workshops will serve as an invaluable resource for new program directors, lead instructors, and deans as well as the seasoned program directors.
OCTOBER 12-13
CHARLOTTE, NC
Hilton Charlotte Executive Park
5624 Westpark Drive
Charlotte, North Carolina 28217

Annual Fees Due July 1
Staff Contact: Ruth Crump, ruth@coaemsp.org

Programs received an invoice for Annual Accreditation Fees in the amount of $1,700 during the week of May 1. This invoice came from Ruth Crump ( ruth@coaemsp.org).

If you did not receive an invoice, first, check your spam folder to see if the invoice ended up there; if not, notify Ruth: ruth@coaemsp.org, so that she can email it to you immediately.

The due date for the invoice is July 1. This applies to ALL programs that either hold CAAHEP accreditation or the CoAEMSP Letter of Review, or are working toward the LoR.

To avoid additional processing fees, payment is due on or before July 31.

NOTE: An Additional Processing Fee invoice of $500 will be sent the first week in August for past due Annual Accreditation Fees and Annual Satellite Fees.
Applications Now Being Accepted for Site Visitors
Staff Contact: Jennifer Anderson Warwick, jennifer@coaemsp.org

The CoAEMSP is increasing its site visitor cadre and is seeking Paramedic educators and physicians to become site visitors for CoAEMSP. The training workshop will be hosted on November 30-December 1 in Minneapolis-
St. Paul for those applicants who have been selected to be site visitors.

FAQs were compiled to help answer the questions you may have regarding the workshop and can be found here.

Application (updated) can be found here.
By the Numbers
Milestones
Claudia Clark, MA, NRP, program director of the Paramedic program sponsored by Anne Arundel Community College, Arnold, Md., earned the Master of Arts degree from the American Public University System on May 15, 2017.
Upcoming Events
CAAHEP Accreditation Workshop
OCTOBER 12-13

Charlotte, NC

Quick Links
Sponsors
Share Your Feedback and Suggestions
Is there something you would like us to cover in a future eUpdate? 
Let us know!

Sincerely,
Jennifer S. Anderson Warwick, MA
CoAEMSP
jennifer@coaemsp.org