Which of the following is true?
The most glaring type of unfocused stem is "Which of the following is true?" Not only can the candidate not answer which statement is true without reading the options, he or she also won't even know what type of answer to look for. For example, if the stem is "Which of the following statements about metastatic lung disease is true?" the answer could be about incidence, symptoms, diagnosis, treatment, etc. In this example, the options might be as follows:
A.
Lymphangitic spread of breast cancer in a lung can be
arrested if the involved lobe is resected.
B.
Patients with metastatic lung disease from soft tissue
sarcomas have been shown to benefit from surgical
removal of the metastatic disease.
[Key]
C. A nuclear radiology lung scan is the most effective
radiographic study to detect and follow patients who are
candidates for resection of pulmonary metastases.
D.
For a survival advantage to be seen, the pulmonary
metastatic disease must be limited to only one lobe of
one lung.
If you have a "Which of the following is true?" item, the best way to rework it is to start with the answer and rewrite the question based on that answer.
How would you rewrite this item?
If you look at the answer (option B), you can see that the question is not about metastatic lung disease in general, but about the treatment of a specific type - that arising from soft tissue sarcomas. So you should make that the question:
"Metastatic lung disease from what type of primary tumor is most effectively treated by resection?"
With this new option list:
A.
Invasive breast carcinoma
B. Soft tissue sarcomas
[Key]
C.
Ovarian cancer
D.
Brain cancer
This is testing exactly the same knowledge component, but it's not making the candidate go hunting for the question. The revised question also does not test the candidate on three other knowledge components. That is, in the original item, he or she has to read and consider the validity of each of the distractors. Each item should be testing only one concept.
What vs. Which of the following
A "what" question is generally more focused than a "which of the following" question. "What" is used when there is only one possible answer to the question. For example, "What is the most common presenting symptom of
X illness?" There can be only one presenting symptom that is "most common" and the knowledgeable candidate can think of that symptom without looking at the option list.
On the other hand, if you ask the same question this way: "Which of the following is associated with
X illness?" the candidate won't know you are asking for a symptom until he or she looks at the options. The question could be about a concomitant illness, a genetic risk factor, or a variety of other things. It is important to clarify in the stem what type of answer is required. For another example, if you are testing whether the candidate knows what type of artifact is shown in an image, rather than asking "Which of the following statements about the image is true?" ask "What type of artifact is shown?"
In summary: 1) don't write "which of the following is true?" questions, 2) try to use "what" in the stem rather than "which," and 3) cover the option list to test if your item is focused.
