You have to wonder why it is that really sensible, bright people insist on doing really daft things. Let's face it Whitehall is full of big brain-boxes and the invisibles in Leeds are no dozers either. Degrees and professional qualifications up to the eyeballs. Number 10; awash with advisors and gurus. They can't all be stupid can they? Is it them or is it us?

David Cameron is re-announcing today that the Friends and Family Test (Killer question; Would you recommend this service to friends and family?) will be in operation by April. Everyone who knows anything about sampling and finding out what people think will tell you; the F&F Test is meaningless.

Dr Rachel Reeves, principal research fellow at the, School of Health and Social Care, Uni of Greenwich, told the HSJ '...it is not the right test'; lack of rigor in the methodology, opportunities to skew results, bias and just about everything else you don't need in a sample. Even Bill Morgan, LaLa's former special advisor, wrote; '...(it) ... is too narrow and too crude to work as a measure for shared decision-making. Retrofitting a retail-centric measure onto healthcare feels awkward anyway.'

Known in industry as the 'Net Promoter Score' it is designed as a measure of brand loyalty and that's all. As the average NHS patient has little opportunity of switching brands away from the NHS, or even the local Trust, the whole exercise is hollow. In industry the NPS is seen as useless unless accompanied by a good 'voice of the customer' programme and the ability to dig into customer experience dynamics. Something the NHS does not have.

Business uses NPS to drive bigger-picture questions; Is increasing recommendation really the best way to achieve success? They may want decrease volumes and increase margins. The NHS can't do that, it is demand led, can't cherry pick and aside from fiddling the Tariff and employing serial-fraudster coders, it's a non-starter. They'll ask; if I lose 35% of my customer base per year, but most of those who stay would recommend, am I really in good shape? If an NHS service scored '35% dissatisfied' there would already be something seriously gone wrong and the Board should be in jail.

And, there is the mother-of-all problems compounded by the DH's own F&F guidance. Sec 3.8, Survey Timeframe: "Patients should be surveyed on the day of discharge or within 48 hours of discharge. This will ensure that the answer given reflects the patient's informed opinion, based on recent experience."

Wrong!  No!  Couldn't be worse! It falls foul of the Halo Effect.

In plain English; respondents are aware that the sponsors of the survey are the very people who are providing the service. Or, even simpler; patients are being asked about the NHS experience by the people providing it. Furthermore, questions can be asked whilst the patient is still in the care of the Trust. We know the nearer to the event a survey is carried out, the more likely it is to overstate reaction. Plus, if undertaken whilst the patient is still in hospital, most answers (particularly among older patients) will be positive for fear of retribution or reprisal and the wish not to be critical, face-to-face, for fear of confrontation.

Patients will not be able to judge the experience in its own right and usually unable to make comparisons with other brands (Trusts). The DH Guidance creates a potential 56hr window to ask questions in situ or later, perhaps on the phone. This distorts results because of a lack of consistency in question framing, timing and environment. All hopelessly skewed.

So, why are the big brains using a sampling system that is flawed? Two reasons. The F&F Test can be relied upon to produce generally positive scores about NHS performance. Gold dust in the run-up to the 2015 elections. Second, by publishing results, ward-by-ward, staff will start to feel proprietorial about performance. Usually that translates into a positive impact on morale and performance.

You see; they're not so stupid after all!

Have a good weekend.

---------------------------------------

 Feel like having a rant?

Please be my guest!  Use this e-address

roy.lilley@nhsmanagers.net 

Know something we don't - email me in confidence.