Official assessments of doctors’ professionalism demonstrate systematic bias, a study reveals.
The researchers, from the Peninsula College of Medicine and Dentistry, in Exeter, warn that assessment involving feedback from patients and colleagues should be considered carefully before being accepted due to the tendency for some doctors to receive lower scores than others, and the tendency of some groups of patient or colleague assessors to provide lower scores.
The research investigated whether there were any potential patient, colleague and doctor-related sources of bias evident in the assessment of doctors’ professionalism.
It is particularly relevant because the GMC is set to introduce a new system of revalidation for all doctors next year and it is likely to involve the use of multi-source feedback from patients, peers and supervisors as part of the evidence used to judge a clinician’s performance. The results will contribute to a decision on whether doctors are fit to continue practising.
The researchers used data from two questionnaires completed by patients and colleagues. A group of 1,065 doctors from 11 different settings, including mostly NHS sites and one independent sector organisation, took part in the study which has been published on bmj.com.
They were asked to nominate up to 20 medical and non-medically trained colleagues to take part in an online secure survey about their professionalism, as well as passing on a post-consultation questionnaire to 45 patients each. Collectively, the doctors returned completed questionnaires from 17,031 colleagues and 30,333 patients.
Analysis of the results that allowed for characteristics of the doctor and the patient to be taken into account, showed that doctors were less likely to receive favourable patient feedback if their primary medical degree was from any non-European country.
Several other factors also tended to mean doctors got less positive feedback from patients, such as that they practised as a psychiatrist, the responding patient was not white, and the responding patient reported that they were not seeing their “usual doctor”.
From colleagues, there was likely to be less positive feedback if the doctor in question had received their degree from any country other than the UK or South Asia. Other factors that predicted a less favourable review from colleagues included that the doctor was working in a locum capacity, the doctor was working as a GP or psychiatrist, or the colleague did not have daily or weekly professional contact with the doctor.
The researchers say they have identified “systematic bias” in the assessment of doctors’ professionalism.
They conclude: “Systematic bias may exist in the assessment of doctors’ professionalism arising from the characteristics of the assessors giving feedback, and from the personal characteristics of the doctor being assessed. In the absence of a standardised measure of professionalism, doctor’s assessment scores from multisource feedback should be interpreted carefully, and, at least initially, be used primarily to help inform doctor’s professional development.”
The GMC, which commissioned the research, said it wanted to understand more about how feedback can play a part in improving doctors’ practice.
Niall Dickson, the Chief Executive of the General Medical Council, suggested that feedback still had an important role to play in revalidation despite the findings.
He said: “This study found that feedback doctors receive may vary depending on a variety of factors, such as the specialty they work in or where they qualified. It does mean the results have to be treated with care and when we publish the final version of our questionnaires later this year, we will also produce clear guidance on how to use them.
“Being aware and taking account of how patients and colleagues view your practice is important for every doctor but it is only one part of the supporting information that doctors will bring to their appraisals. It will be considered alongside all the other information about a doctor’s practice and is not something which you can ‘pass’ or ‘fail’. It assesses an individual doctor’s strengths and areas for development to help them improve their practice – it is not a way of comparing doctors with one another.”
Read more.
Tags: GMC, Patient feedback
