The first results of David Cameron's Killer Question Test were published yesterday. Sadly, everyone who knows anything about sampling and finding out what people think will tell you; in the context of the NHS the F&F Test is meaningless.
The F&F test is known in industry as the 'Net PromoterScore' and designed as a measure of brand loyalty. Would you use us Sainsbury's again rather than Tesco? As the average NHS patient has little opportunity to switch 'brands' away from the NHS, or even the local Trust, the exercise is hollow. In industry the NPS is seen as useless unless accompanied by a good 'voice of the customer' programme and the ability to dig into customer experience dynamics. Something the NHS does not have.
And, there is another little problem, compounded by the DH's F&F guidance. Sec 3.8, Survey Timeframe: "Patients should be surveyed on the day of discharge or within 48 hours of discharge. This will ensure that the answer given reflects the patient's informed opinion, based on recent experience."
Wrong! No! Couldn't be worse! It falls foul of the Halo Effect. In plain English; patients are being asked about the NHS experience by the people providing it. When the waiter asks; 'How was everything?' You say 'lovely thank you' and complain to the other half, all the way home, about the tough steak.
We know the nearer to the event a survey is carried out, the more likely it is to overstate reaction (good or bad) and if undertaken whilst the patient is still in hospital, most answers (particularly among older patients) will be positive for fear of retribution or reprisal later. On the other hand, savvy patients might use the F&F test for more manipulative reasons like campaigning to keep a service open!
Patients will not be able to make an immediate judgement on their treatment experience in its own right and usually unable to make comparisons with other brands (Trusts). The DH Guidance creates a potential 56hr window to ask questions in situ or later, perhaps on the phone or web. This distorts results because of inconsistency in how the answers are collected (framing), timing and environment. All hopelessly skewed.
So, why is the Carbuncle using a flawed system? Answer; It's cheap. Proper focus group studies cost a fortune. The F&F test is done by existing staff or volunteers, on a post-card. Second; in the context of the NHS the F&F Test can be relied upon to produce generally positive scores. Helpful for LaLite in the run-up to the 2015 elections.
Frankly, the F&F results were poor: response rate low, under 16%; combined score 63, inpatients (71) dragged down by a perfectly predictable 55 in A&E. And, as I understand it; a single response of 'extremely likely' to recommend would give a 100% score for the Trust.... scoring needs a review!
We know there are better ways of capturing 'customer opinion', nevertheless, I do believe there is a positive side to F&F. All staff will want to achieve good results and that of itself might translate into a positive impact on morale and performance.
How you conduct a survey has an important impact on the results. This survey, published yesterday, involved 3,000 ward nurses and tells us they claim to "ration" care because of time pressures. Not exactly what the DH and Carbuncle wants to hear.
Perhaps the real answer is more nurses, not more surveys.
We know the F&F test will be stolen by politicians. Ignore them. It is important the NHS focuses on the value the test will bring over time. An indicator, a marker and a reference point. I predict there will soon be pressure for better sampling and that can only be a good thing.
Most of all; this first step is an augury to the public and more important to us; the NHS is determined to get better at getting better.