 |
|
November 2019
· Volume 12, Issue 3
|
|
From Testing Instruments to Finance,
We're Covering Ground in This Issue
by J. Anthony Seibert, PhD, ABR Governor
Welcome to the November 2019 issue of The BEAM. In this issue, President Brent Wagner discusses the limitations of testing instruments (aka multiple-choice tests) that are used by the ABR and the realities of ensuring a balance between validity and relevancy of testing while minimizing burden and cost for candidates. Executive Director Val Jackson announces her replacement as of July 1, 2020, introduces new member volunteers on the Board of Governors and Board of Trustees, and thanks those who departed at the end of the October board meeting in Tucson. An announcement for the
Online Longitudinal Assessment (OLA) prelaunch for interventional radiology (IR), medical physics (MP), and radiation oncology (RO) diplomates is now on the
website, with a
link available to begin the process. This is a great opportunity for those enrolled in Maintenance of Certification (MOC) to become familiar with the user interface, to get a feel for how the process functions for answering and declining items, and to learn how to participate as a question rater. OLA will officially begin for those specialties on Monday, January 6, 2020.
Highlights from two of the four ABR disciplines (now a standard offering in The BEAM) cover the thoughts and perspectives of the Trustees regarding details on how to best fulfill the ABR's mission through the creation of exam content and delivery. An article by Matt Podgorsak and the MP Trustees discusses the development of new technologies and skills in contemporary clinical standard of care, and how these topics are incorporated into the examinations to ensure that board-certified medical physicists have the knowledge to provide service that benefits patients. Paul Wallner and Brian Davis describe updates of the Initial Certification Qualifying Exams in radiation oncology that are performed in concert with stakeholder organizations to improve exam development and delivery. ABR Secretary Treasurer Robert Barr gives readers an insight into the organization's finances.
Each edition of The BEAM now also features a spotlight on a section of the ABR staff by introducing the employees who are working diligently to support the mission of the ABR. In this issue, the certification services division is profiled, describing the tasks, services, and support available to candidates in their quest to become certified, and to diplomates in their quest to maintain certification.
I would like to thank Communications Coordinator Eva Wilson for her outstanding support of The BEAM by recruiting individuals to write articles, improving the quality of articles, and providing excellent editorial skills during her seven years at the ABR. Eva retired this month, in step with her intentions. Congratulations and best wishes for new opportunities that are coming her way!
Finally, links to recent ABR blogs that you might find of interest are available at the end of this issue. I sincerely hope you enjoy this edition of The BEAM and remember, now that we're producing six issues per year, the next issue will come out in January.
|
Testing Instruments Are Inherently Imperfect, but We Should Make Them
as Good as They Can Be
The inherent flaws of multiple-choice tests, both in medicine and beyond, have been debated for decades. Whether discussing the SAT, USMLE, or the ABR's certifying exams, most of us can reasonably agree with the claim that a multiple-choice test (MCT) as a predictor of performance, competence, or aptitude is far from perfect. While my medical school testing was built almost exclusively around a No. 2 pencil, today's trainees are---very appropriately---assessed using an expanded set of more sophisticated tools (e.g., the Objective Structured Clinical Examination [OSCE]). Specific to the ABR diagnostic radiology (DR) exams, we often hear criticism that the former oral exam was a better testing method because it allowed for subjectivity, discussion of nuance, and synthesis at a level beyond that reasonably tested during an MCT. Others have pointed out that the current exam structure does not meaningfully assess professionalism, critical thinking, and communication skills; all of which are requisite elements of being a highly effective radiologist.
It is important to recognize two basic characteristics of the ABR certification process that impact decisions regarding our design and use of MCTs. First, because board certification requires two distinct accomplishments (successful completion of a residency as well as passing the certifying exams), the residency program, over a four-year period, has both the opportunity and the responsibility to observe important characteristics that comprise the broad view of what being a board-certified radiologist represents to our clinical colleagues and our patients. For the ABR to assume this task of comprehensive subjective assessment during the practical constraints of a testing environment is not only unrealistic but also usurps the role of the residency program and its faculty. One of the main purposes of the independent ABR exam process, to supplement the validation by the residency program, is to represent a "standard" so that a candidate who trains in a small program in New England is certified in an approximately equivalent way as a resident from a large program in Southern California (by using a uniform test that samples a representative portion of the domain of diagnostic radiology).
Second, building on the theme of "constraints" mentioned above, ABR exams have to be reasonable in terms of the time required (we can't reasonably require a five-day exam) and cost. This is balanced against a set of rigorous processes that use volunteer subject matter experts (the majority of whom are residency program faculty) for content development; multiple levels of committee review to challenge and refine the content for relevance and reasonableness; and sound psychometric principles that seek to ensure high reliability and reproducibility. While we are mindful of these limitations and realities, the ABR is continuously looking at innovative ways, including creative use of technology, to develop exams that are valid and relevant while minimizing the burden and cost for the candidates.
|
|
|
From the Executive Director
|
New Executive Director, Board Members Named During Fall Meeting in Tucson
by Valerie P. Jackson, MD
Our twice-yearly board meetings produce meaningful results for our candidates and diplomates. The fall session in Tucson last month was brimming with key decisions that will improve our processes, products, and service for the field.
Among those actions was one that will set our organization's course for many years. The board named Brent J. Wagner, MD, as my replacement as executive director. Brent
has served as president of our Board of Governors since 2018 and has been a volunteer since 2000. He is a partner in West Reading Radiology Associates, where he has worked since 1998.
Brent has maintained a diagnostic radiology certificate since 1992 and is the author or co-author of nearly 50 publications. He regularly lectures on genitourinary topics at the American Institute of Radiologic Pathology (AIRP/ACR) course. His professional society memberships include American College of Radiology, Radiological Society of North America, American Roentgen Ray Society, American Institute of Ultrasound in Medicine (Senior Member), Society of Abdominal Radiology (Fellow), Society of Radiologists in Ultrasound, Berks County Medical Society, Pennsylvania Radiological Society, and the Pennsylvania Medical Society.
The ABR will be in great hands with Brent. He is a dedicated and knowledgeable professional who has served the field extremely well. You will be hearing and reading more about him and the organization's direction leading up to his July 1, 2020, start date.
The board also installed three new members, including the first from the public. Longtime ABR volunteers Robert Pooley, PhD, and John Suh, MD, joined Toby Gordon, ScD, as new members.
As the first public member of the ABR Board of Governors, Dr. Gordon offers an outside perspective from someone who has spent considerable time inside the healthcare industry. She joined the Johns Hopkins Carey Business School in 2010 and has expertise in the areas of healthcare policy, management of hospitals and health systems, and technology commercialization. Her professional background includes serving as vice president of the Johns Hopkins Hospital and Health System, consulting academic medical centers, teaching, and conducting research focused on the relationship between surgery costs and quality.
Dr. Pooley is chair of the division of medical physics and an assistant professor of radiology at the Mayo Clinic in Jacksonville. He is one of our outstanding oral board examiners. Dr. Suh is chairman of the department of radiation oncology at the Cleveland Clinic Taussig Cancer Institute. He has served as a written board item writer and oral board examiner. Both new trustees spent the fall meeting immersing themselves in the work ahead of them in their new roles.
We owe a tremendous debt of gratitude to Lisa Kachnic, MD, and Jerry Allison, PhD, whose terms expired in October. Both made tremendous contributions to the board, both within their respective specialties of radiation oncology and medical physics, and to the ABR as a whole. Dr. Kachnic also served as president for two years. Their efforts, ideas, and enthusiasm will be missed.
Please join me in welcoming Dr. Wagner and our new board members and thanking those whose terms have ended. We remain committed to ensuring the fulfillment of our mission: certifying that diplomates demonstrate the requisite knowledge, skill, and understanding of their disciplines to the benefit of patients.
|
|
|
New Nine-Month Financial Cycle Makes Comparisons to Previous Years Harder
by Robert M. Barr, MD, ABR Secretary Treasurer
2019;12[3]:40-41
 As a 501(c)(6) not-for-profit organization, the ABR files an IRS Form 990 yearly but is exempt from payment of taxes. Our most recent Form 990 covers financial transactions from April 1, 2018, through December 31, 2018. Since our fiscal year ending changed last year from March 31 to December 31, this most recent filing represents nine months of ABR operations, making comparison to previous years more complicated. For the future, this change should provide a more easily understood assessment of our annual finances, as it better aligns with our current program cycle.
The Form 990 is reported publicly and sheds some light on the size and scope of the ABR. Total revenue for the nine-month period was $14,097,350, which includes $571,753 from investment income.
For comparison, prior-year 12-month total revenue was $17,430,259, including $943,213 from investment income. For the calendar year, the ABR administered approximately 4,970 exams and had about 30,000 diplomates enrolled in Maintenance of Certification.
Expenses for the nine-month period totaled $12,725,890, of which the largest component, $6,844,241, was the cost of ABR salaries and benefits. The ABR maintains its headquarters in Tucson, Arizona, and leases a testing center in Rosemont, Illinois, near Chicago O'Hare International Airport. Prior-year 12-month total expenses were $16,468,080.
Although we experienced positive net revenue in fiscal year 2017 and in the shortened fiscal year 2018, we are anticipating budget shortfalls of $2.5 million in 2019 and $2.2 million in 2020, in part from nonrecurring costs related to OLA development. To avoid fee increases during periods of such programmatic enhancements, the ABR will rely on reserves that have been established to address fluctuations in capital requirements and operational expenses.
Investments at the end of fiscal year 2018 for the ABR totaled $33,669,259. This is less than twice our anticipated annual operational expenses and represents a reasonable level to minimize risk for a not-for-profit organization.
The ABR Board of Governors has fiduciary responsibility for the affairs of the ABR. We are committed to maintaining costs at the lowest possible level while meeting our mission: to certify that our diplomates demonstrate the requisite knowledge, skill, and understanding of their disciplines to the benefit of patients. We are fortunate to have a dedicated team of finance professionals, and we also engage independent auditors to review our financial statements. In addition to public reporting of our annual Form 990 filing, we maintain platinum status with
GuideStar™ as a commitment to transparency. We continuously strive to use our resources wisely to improve services for our candidates and diplomates while providing a robust and credible certification program.
|
|
|
IR, MP, RO Diplomates Can Get Started Learning OLA Before January Kickoff
|
Diplomates in interventional radiology/diagnostic radiology (IR/DR), medical physics (MP), and radiation oncology (RO) have several opportunities to get a head start on gaining familiarity with Online Longitudinal Assessment (OLA).
While OLA doesn't start for those disciplines until January 6, those participants can get a preview of the system by answering practice questions on the
OLA site
or visiting our website for information.
Our OLA web page includes an extensive
FAQs section
. There's also a
video
and
recorded webinar
on our YouTube page. Our staff will be giving demonstrations of the system at the
RSNA annual meeting from December 1 to 6 in Chicago.
OLA replaces the proctored Maintenance of Certification (MOC) exam. Most participants receive two question opportunities per week and only need to answer one of those to meet their annual progress requirement. Each question issued is available for four weeks.
OLA is a continuous assessment; participants stay involved throughout their careers unless they choose to take the traditional exam every five years. Those who choose to take the MOC exam should contact us at
information@theabr.org
.
The participation rate for diplomates in diagnostic radiology and subspecialties, who started OLA last January, was 96.5 percent as of early November, representing more than a million questions answered. In addition, almost 65 percent of those participating had completed their annual OLA progress requirement.
Our certification services team is available from 9 a.m. to 6 p.m. (Eastern time) at
information@theabr.org
or (520) 790-2900.
Anyone participating in OLA can help improve the product by offering feedback on questions. Click here to get started.
|
IC, MOC Programs Constantly Evolving
to Meet Changing Needs of Patients
by Matthew B. Podgorsak, PhD, ABR Trustee
The ABR's mission is "to certify that diplomates demonstrate the requisite knowledge, skill, and understanding of their disciplines to the benefit of patients." This is accomplished first through the Initial Certification (IC) process, in which a candidate must pass a combination of written exams and, in some disciplines, an oral exam, to become board certified. Sub
sequently, diplomates who completed the IC process in 2002 or later must continue to demonstrate that they have maintained the knowledge and skill that led to their initial certification by participating in the ABR's Maintenance of Certification (MOC) program.
For the IC and MOC programs to effectively benefit patients, the evaluation of a candidate or diplomate through participation in either program must be based on standards that include each discipline's contemporary standard of care. This means that as disciplines inevitably evolve through the discovery of new knowledge, development of new technologies, and identification of new skills, the ABR's evaluation tools must similarly evolve to remain in step with the current practice of each discipline. This is a necessary condition for board certification to be a meaningful indication of a medical physicist's ability to participate in the delivery of safe and meaningful clinical care throughout their career.
The trustee for each discipline works very closely with the chairs and members of item-writing committees to set the standards for establishing the content of the evaluation tools used in the IC and MOC programs for that discipline. For IC, these standards form the basis of the content guides published on the ABR website that define the scope of each IC exam. In the MOC program, as of January 1, 2020, the formal assessment of knowledge will be done through
Online Longitudinal Assessment (OLA), with questions similarly developed by an item-writing committee. The decision to introduce questions based on a new technology or technique into either an IC exam or the OLA program is made only after its clinical application becomes clear and robust, and deployment of the new technology or technique into clinical practice is beyond the early adopter stage. It is at this time that a thorough evaluation of a candidate's or diplomate's knowledge base must include questions about the new technology.
Recent examples of new technologies in therapy medical physics include proton therapy using pencil-beam delivery, MR-guided linac treatment delivery, and optically stimulated luminescence dosimetry. In diagnostic medical physics, dual-energy CT, breast CT, and phase-contrast imaging have recently become standard. Finally, in nuclear medical physics,
solid-state detectors, yttrium-90 radiotherapy, and PET/CT time-of-flight have become commonplace. All these clinical techniques have transitioned from the emerging technology phase to a significantly wider penetration into mainstream clinical practice. Consequently, questions related to these new technologies are now included in all discipline-respective exams and in the OLA program for that discipline.
Some changes, particularly those in major regulations, are integrated into the assessment tools more quickly. Any such changes in the IC exam material are immediately included in the discipline-specific content guides for each exam found on the ABR website.
It is certain that new technologies and techniques based on the ever-expanding scientific and medical knowledge base will continue to be introduced into clinical practice. These new developments will at first be viewed as cutting edge and invariably be supported initially by only a small number of ambitious early adopters. Since these emerging technologies are far from standard of care at this point, it is not appropriate to include them in either an IC exam or OLA. If viable, however, some of these new technologies and techniques eventually become a component of the standard of care, at which time they must be included in the IC exams and the OLA program. Similarly, questions based on techniques and technologies that have become obsolete are removed from the IC exams and OLA.
If the ABR is to remain true to its mission, the assessment tools used to ensure that board-certified clinical medical physicists have the knowledge to provide service that benefits patients must evolve to include the new technologies and techniques that define the ever-changing standard of care. The ABR trustees are keenly aware of this requirement.
|
|
|
Focus on Radiation Oncology
|
Update on Radiation Oncology Initial Certification Qualifying Exams
by Paul E. Wallner, DO, and Brian J. Davis, MD, PhD
2019;12[3]:45-46
ABR radiation oncology (RO) Trustees and staff meet and communicate regularly with leaders of discipline stakeholder organizations to improve exam development and delivery. In 2018, ABR staff and volunteers participated in several initiatives that have significantly altered some practices related to the Initial Certification (IC) Qualifying Exams (QEs).
The RO IC Advisory Committee (IC-AC) was reactivated and populated by volunteers nominated by the Society of Chairs of Academic Radiation Oncology Programs (SCAROP), Association of Directors of Radiation Oncology Programs (ADROP), and the Association of Residents in Radiation Oncology (ARRO). The committee is chaired by Andrea K. Ng, MD, MPH, of Boston, a current RO trustee. With input from committee members and their respective constituencies, several significant steps were taken: topic domains for the QEs in medical physics for radiation oncology (physics) and radiation and cancer biology (biology) have been increased in number and granularity, and quartile reporting of candidate performance will be reflected in this added granularity; reporting of performance in the clinical QE (general and RO) has been further divided from its eight clinical categories for quartile performance reporting to include separate line items for pediatric cancers and pediatric central nervous system cancers, and for nonclinical skills. Within the constraints of our contractual relationship with Pearson VUE™ test centers, dates for the 2020 qualifying exams were changed to allow for greater separation between the basic science and clinical exams; and in 2020, for the first time, relative weighting of exam content in the clinical and basic science QEs will be published on the ABR website.
Concurrent with activities of the IC-AC, ABR staff and volunteers worked with representatives of SCAROP, ADROP, and ARRO in two working groups convened by the American Society for Radiation Oncology (ASTRO) to update ABR IC study guides. One work group focused on the physics guide previously developed by ASTRO. The second work group was charged with updating the biology study guide. Acting under pressure to provide early guidance for the 2019 exam administrations, both groups delivered their work to the ABR in late 2018. The updated study guides represent improvements for candidates preparing for the exams, with greater granularity of topics and, for the first time, publication of primary and secondary references. Primary references include specification of several textbooks from which a majority of respective exam content will be sourced.
A number of critically important issues remain under discussion with stakeholder organizations. Several represent exam delivery logistics, including when during their training residents might be eligible to sit for the basic science exams, and whether the exam delivery schedule might be altered. To a significant degree, these issues are the concern of program directors and chairs, and the Board has reached out to those representative organizations for guidance.
Perhaps the most significant issue raised by discussions with stakeholder organizations has been the relative emphasis currently placed on the three elements of IC, especially the weighting of the basic sciences. Current exam structure includes one exam in physics comprising approximately 100 items, and a second biology exam of similar length. These exams are administered on the same day and residents are eligible to sit for the exams after 32 months of residency training. The clinical exam, which comprises approximately 200 items, is administered on a second day, and residents must have completed their entire 48-month training period before being eligible to sit for this exam. The relevance and weighting of the physics and biology exams has been raised as a concern, especially by younger physicians within the discipline. Consistent with input from stakeholder organizations, the ABR trustees believe that any de-emphasis of science within the RO curriculum, training programs, and assessment instruments would have potentially significant impact on the current and future science and practice of radiation oncology. They are encouraging initiation of a broad stakeholder dialogue to explore the impact of these issues and to make recommendations for the path forward.
|
|
N. Reed Dunnick, MD, Earns
SABI 2019 Gold Medal
Congratulations to N. Reed Dunnick, MD, for earning the Society for Advanced Body Imaging (SABI) 2019 Gold Medal. Dr. Dunnick, the ABR's associate executive director for diagnostic radiology, received the award during the SABI Annual Meeting in Denver. The gold medal is the highest honor given annually by SABI.
Dr. Dunnick has served 26 years as Fred Jenner Hodges Professor and Chair of the Department of Radiology at the University of Michigan. He has written 313 scientific papers, 62 book chapters, and 10 books on various aspects of radiology, especially diagnostic oncology, uroradiology, and administration.
Dr. Dunnick has served as visiting professor at 85 medical centers and as guest faculty for 460 continuing medical education courses. He has also delivered 26 named lectures. He has served on the editorial boards of 13 peer-reviewed journals.
Three Diplomates Take Home
RSNA Gold Medals for 2019
2019;12[3]:48
D. David Dershaw, MD; N. Reed Dunnick, MD; and J. Anthony Seibert, PhD, have been named winners of the Radiological Society of North America (RSNA) Gold Medal for 2019.
The gold medal is RSNA's highest honor. It is awarded every year by the organization's board of directors to people who have rendered outstanding service to the science of radiology. A unanimous vote of the board is required to win.
|
|
|
Important Deadline Coming Next Year
for Some Vascular IR Fellows
An important deadline is approaching for candidates who finished their vascular interventional radiology (IR) fellowships prior to 2010 and are not yet certified.
The application period for anyone wanting to take the 2020 IR Certifying Exam starts November 1 and runs through February 28, 2020.
The exam will be held in October 2020. This will be the last opportunity for those VIR fellowship candidates to take the Certifying Exam. Anyone who has already applied for the exam need not do so again.
|
|
|
ABR Staff Spotlight
Certification Services Team
Makes Partnerships a Priority
|
|
2019:12[3]:50-52
|
|
|
Left to right: Rachael Noble, Kristin Gudenkauf, Kyle Dignoti, and Tara Kaminsky
|
The ABR employees who spend the most time helping candidates and diplomates be successful in each phase of certification are the four members of our certification services division (CSD). Their mission is to expand their role as trusted advisors and partners with members of the diagnostic radiology, interventional radiology, medical physics, and radiation oncology communities.
Led by Associate Director Kristin Gudenkauf, CSD answers phone calls and emails from diplomates participating in Maintenance of Certification (MOC) and candidates who are eligible for board certification. Each phone call takes eight to 10 minutes. There are more than 30,000 people participating in MOC and approximately 9,000 candidates in the initial certification process.
The team is also proactive in its communications, often sensing diplomate and candidate needs before they arise regarding exams, timelines, and MOC. One of its goals is to continue expanding electronic communications to ABR constituents and personally speak with each of them at least once a year.
"We have been able to address questions quickly and have begun various outreach projects that will help us better engage with those we serve," Gudenkauf said.
Gudenkauf, who started her position not long before she had her first child this past spring, and Certification Manager Rachael Noble are the most experienced members of the team. Gudenkauf has served in several roles during her eight years with the ABR, most recently as a project manager. Noble has been with the ABR since 2012.
Certification Managers Kyle Dignoti and Tara Kaminsky joined the team this year and have quickly picked up the detailed subject matter. Dignoti worked for a nonprofit before coming to the ABR and Kaminsky was with St. Jude Children's Research Hospital in Memphis.
"Broadening my professional skills in an organization with an exceptional reputation was appealing," Dignoti said. "The focused work and commitment to excellence drew me to the ABR."
Kaminsky said, "I spent a lot of time working in Continuing Medical Education and there is a lot of similarity and overlap with the positions. There was a bit of familiarity but new challenges with this opportunity as well."
Noble played a key role in training Dignoti and Kaminsky and is impressed with what quick studies they have been.
"Kyle and Tara have both picked up the information so easily and quickly, which is impressive given how much there is to learn about the different pathways and disciplines," she said.
Gudenkauf runs a collaborative department, taking staff opinion into account when determining direction. Each of her three employees has strengths and unique traits. Gudenkauf credits Dignoti for his unwavering kindness, Kaminsky for her humility, and Noble for her determination.
"They are all unique," she said, "but they all have a passion for what we do, as highlighted by their dedication to truly becoming partners with our constituents."
With more than 18,700 diplomates already participating in Online Longitudinal Assessment (OLA) and another 10,000 starting in January, many inquiries the team receives and correspondence it sends involve OLA. Kaminsky said most of the calls she has taken about the assessment have been positive. OLA came about because diplomates sought an alternative to the 10-year MOC exam.
"They are finding the questions to be incredibly helpful in their practice and quite rewarding," Kaminsky said. "We always love receiving feedback from our diplomates."
Getting feedback is one goal of the CSD team. Another is sharing information that helps a candidate or diplomate solve an issue, which is sometimes a team effort with assistance provided by staff outside the department.
"The most fulfilling part of the job is finding resolution to a candidate's or diplomate's question or concern," Dignoti said. "Staff members support one another to better the service we provide, and that produces a feeling of great satisfaction."
Working together is key to CSD's mission. Noble enjoys being part of the division's strong camaraderie, a bond that enables the team to better partner with its constituents.
"I appreciate everyone on this team so much," Noble said. "I enjoy coming here every day to work with them."
|
|
|
List of Society Attendance
|
2019;12[3]:53
The ABR sponsors a booth at numerous society meetings throughout the year. Printed materials are available, and ABR representatives are in attendance to answer your questions. To see a list of society meetings at which the ABR plans to have a booth in 2019 - 2020, please click here.
|
|
|
Links to Recent ABR Blogs
|
Thank you . . .
for reading this issue of The BEAM. If you have comments, suggestions, or questions, please email communications@theabr.org.
|
|
|
|
|
