Competency checks on doctors could become a costly mistake

The Medical Board of Australia could be about to make a costly mistake. Regular testing of competency being introduced is not a matter of if, but when, said the new boss of the NSW Medical Council, Dr Greg Kesby, in an interview with Australian Doctor magazine.Dr Kesby also said there needed to be a process to instil confidence within the community that all doctors’ knowledge was up-to-date.

On the face of it, competency checks of doctors sounds appealing. I’m sure some politicians will love the rhetoric. But at the same time, alarm bells were going off when I read the interview.

Unanswered questions

Does Dr Kesby imply that the Australian public has lost confidence in its health practitioners? Are there concerns about the quality of Australian healthcare? There are many other unanswered questions too.

Is there any evidence that the current quality assurance systems, such as accreditation and CPD, are insufficient? Is recertification or revalidation — such as has been implemented in the UK — an efficient way to improve the quality of healthcare? Is it possible to design a system that reliably differentiates between good and bad practitioners? And how do we define competency?

Based on numbers from Canada, AHPRA estimates that 1.5% of Australian medical practitioners are performing ‘unsatisfactorily’. I’m not sure Canadian figures can be applied to Australia, but 1.5% of unsatisfactory performers in any group is low. There are numerous models: from strengthening CPD to targeting those at high risk of complaints, to the full-bore version rolled out in the UK. But if the board tries to identify these substandard doctors, carpet-bombing the whole profession is problematic.

Expensive solution

One thing is for sure: UK-style revalidation is expensive. The process takes 1-2 working days per practitioner. Imagine thousands of Australian doctors having to take a couple of days off work to fulfil revalidation requirements.

Imagine the enormous effort it will take to manage this process. Think about the additional cost of the training, time and wages of the appraisers — who, I assume, will be doctors too — and the admin staff, extra regulation, log books, documents, IT etc.

According to the UK’s Pulse magazine, the revalidation of doctors has become a colossal enterprise, costing taxpayers £97 million (about $207 million) a year. This figure does not include revalidation of other health practitioners. Critics of the UK system have said it will not detect poor doctors, as its main purpose is to gain patients’ trust. Others feel it mainly serves to demonstrate what good practice looks like.

Policing the profession

A former UK GP-appraiser, now working in Australia, mentioned on the GPs Down Under Facebook group that appraisals used to be fun, interesting and mainly pastoral. Under revalidation, they became a form of policing the profession, he said.

Professor Kerry Breen, adjunct professor in the department of forensic medicine at Monash University, wrote in the Medical Journal of Australia last year that there was little evidence to support the idea of transposing the UK system to Australia.

He said that despite some local failures of medical regulation and hospital governance, the community had not lost faith either in its doctors or regulatory system. Indeed, it appears the public is largely content with its healthcare practitioners: Australian doctors, nurses and pharmacists have been in the top three most trusted professions for many years in the annual Roy Morgan research.

More recently, Dr Steve Wilson, an AMA WA representative, questioned in the AMA’s magazine whether revalidation would be able to address those who failed to practise to agreed levels. And if it did, he asked, would that be a sign of impairment or does it reflect personal style, or a lack of time, training, experience or adequate remuneration?

Don’t follow the UK

About 5000 doctors a year are considering leaving the UK, and many come to Australia. Bureaucracy is one reason they emigrate. Simply copying the UK’s revalidation system would be a big mistake. The last thing we need in Australia is more regulation, red tape and stressed-out doctors.

In recent years, our healthcare system has seen enough unsuccessful concepts not supported by evidence. Think about the super clinics program or the PCEHR, and the accompanying cost blowouts, delays and disappointing results.

It will be easier and cheaper to build on existing quality assurance systems.

Let’s look, for example, at refining CPD and accreditation. As they say, the main difference between a wise man and a fool is that a fool’s mistakes never teach him anything.

Developing an expensive new system with little or no supportive evidence failed in the case of super clinics and e-health, and it will fail with revalidation too.

This article was originally published in Australian Doctor Magazine.

Revalidation of doctors, or how to spot the bad apples

Wouldn’t it be great if we could spot the bad apples before we consume them? Or even better: before they become bad? In recent years medical regulators around the world have been exploring ways to identify doctors who are performing poorly.

In the UK all apples are tested once a year via a process called revalidation. But some have said it will not detect poor doctors; its main purpose is to gain patients’ trust. Others say it is meant to demonstrate what good apples look like. But one thing is for sure: Revalidation is labour-intensive and expensive.

“There is indeed an additional time cost,” said GP Dr Paresh Dawda in Australian Family Physican. “The appraisal meeting was usually 3 hours in length, and on average it took another 5 or 6 hours to collate the evidence and complete the forms, which is in keeping with an average of 9 hours found in the revalidation pilots.”

Then there are the training, time and wages of the appraisers, usually doctors too, the administrative staff, extra regulation, log books, documents, IT… Revalidation has become an enormous enterprise, costing £97M ($186M) a year, mainly because of added pressures on doctors’ time.

It seems logical that, before a country embarks on an operation like this, the problem it is trying to solve has been defined and the solution is effective.

So what’s the problem?

According to the Medical Board of Australia, evidence from Canada shows that 1.5% of doctors are not good enough. The Board has translated this figure to Australia, and thinks that over 1,350 doctors could be performing unsatisfactorily. Other research indicates that just 3% of doctors are the source of 49% of complaints.

“Where is the evidence that further regulation is needed?

Several safety mechanisms are already in place: At the moment Australian doctors must meet the Medical Board’s mandatory registration standards, including for recency of practice and continuing professional development. Doctors can be subjected to random compliance audits.

Although a majority of Australian doctors seems to support competence checks, there are serious questions about the UK-style revalidation process.

Revalidation screenshot
Screenshot: Example of questions UK doctors have to answer during the revalidation process.

AMA(WA)’s GP Dr Steve Wilson in this blog post: “Where is the evidence that further regulation is needed, which will be preventative and ultimately beneficial to the profession and the community?”

“Will it address those who fail to practise to agreed levels, and is that a sign of ‘impairment’ or more about personal style, lack of time, adequate remuneration, or lack of care, training, experience, sheer demand and workforce numbers?”

At a conference in 2013 Medical Board of Australia Chair, Dr Flynn admitted that ‘the problem that a revalidation-style system would help solve was not yet defined’.

But Dr Flynn questioned the current continuous professional education system: “Can you assure me that everyone who has done your CPD program is actually competent and practising at a reasonable standard? (…) My sense is that, for most CPD programs, they don’t do that, or at least, not to a high enough level of certainty.”

After meeting Dr Flynn in 2013, the RACGP stated in Australian Doctor magazine: “The meeting provided an opportunity for the college to discuss the strength of our current QI & CPD program, and the necessity of adding yet another mechanism to identify underperforming doctors, when processes are already in place – such as the medical boards, health quality and complaints boards and indemnity insurers.”

What’s the Medical Board up to?

“We started a conversation about revalidation in Australia in 2012,” said Dr Joanna Flynn in last week’s media release, “as part of our commitment to making sure doctors in Australia maintain the skills to provide safe and ethical care to patients throughout their working lives.”

The board has asked the University of Plymouth to answer some questions on revalidation. At first glance this seems a sensible approach.

Dr Flynn: “We have commissioned this research to find out what is working well internationally, what is in place in comparable health care systems, and what principles the Board should consider in developing revalidation in Australia. (…) this research will help make sure that the decisions the Board makes in future about revalidation are effective, evidence-based and practical.”

The aim of the project is to:

  • establish the existing evidence base for the validity of revalidation or similar in countries comparable to Australia
  • identify best practice and any gaps in knowledge for revalidation processes
  • establish the validity evidence for revalidation’s effectiveness in supporting safe practice
  • develop a range of models for the Australian context for the Board to consider.

It seems to me the research questions are broad and several steps are taken at once. For example: ‘Establishing the evidence for revalidation’ and ‘developing a range of revalidation models’ are entirely separate processes.

It appears the Medical Board has already made up its mind. The research findings will be considered by the Board in the second half of 2015. I am certainly looking forward to the results and conclusions, as well details about cost and setup of the study.

The Camera revalidation research website of the University of Plymouth doesn’t give any answers away: “The research team is currently undertaking an ambitious programme of research involving three interlinking studies to explore and understand revalidation in all its complexity.”

Putting the cart before the horse

The question is of course: Is revalidation the right solution? Are there other options? One could argue that this should have been considered before spending tax dollars on an overseas research project.

Professor Breen, from the Department of Forensic Medicine at the Monash University in Melbourne, said in the Medical Journal of Australia: “There is little to support the idea of simply transposing the UK system to Australia. Despite some local failures of medical regulation and hospital governance, there has been no widespread loss of faith of the community either in its doctors or in the regulatory system.”

“Is there a problem with medical registration in Australia that needs attention, and, if so, what should be done to fix the problem?

“The Medical Board of Australia would be wiser to start afresh by asking and answering two questions — namely, is there a problem with medical registration in Australia that needs attention, and, if so, what should be done to fix the problem?

“The medical profession in the UK appears to have accepted revalidation, albeit reluctantly, as representing the price to be paid for maintaining the existence of the GMC and for regaining public trust after a series of regulatory failures.”

“It has been claimed that revalidation will not reliably detect poorly performing doctors, and many commentators have pointed out that revalidation would not have identified Dr Harold Shipman.”

Immediate past president of the AMA, GP Dr Steve Hambleton had second thoughts too. In MJA Insight he said: “We need to make sure we maintain our currency and continue to improve health outcomes, but in terms of value for money, making everybody go through a 5-yearly process of 360-degree evaluation is not needed in the Australian health system.”

Both Professor Breen and Dr Hambleton suggested there are better ways to deal with the bad apples. Database analysis could be one solution. Other options are targeted revalidation and a revamp of the existing CPD program and accreditation. Some have argued that the focus should be on the workplace, not just on health professionals.

Journalist Paul Smith from Australian Doctor magazine was, as usual, spot on when he wrote: “(Doctors) may argue that targeted revalidation has greater merit than what they may see as carpet-bombing the entire profession.”

Red-tape stress

“Recently I cried at work,” posted Dr Adrienne Garner on the BMA blog. “Why? Because the evening before I’d been notified that my appraisal, submitted after hours of work, had been unsubmitted by my appraiser as it was ‘not sufficient for revalidation.”

“I was gutted. My mind churned with a mixture of thoughts ranging from anger to fear, through frustration and disappointment. Sleep had been impossible.”

“Under revalidation appraisals became a form of policing the profession.

Many studies show that doctors are more likely to experience psychological distress and suicidal thoughts than the general community, and there is a high rate of burnout. Pastoral care and self-reflection are important. But when they are part of a policed regulatory framework, they become a stressor in itself – which defeats the purpose.

Former Coventry GP Dr Gaurev Tewary, now working in Australia, posted on a social media platform: “I was an appraiser in the UK. My overall impression is this: Appraisals used to be fun and interesting and mainly pastoral. You did them to help people and I enjoyed supporting the profession. Under revalidation it became a form of policing the profession.”

About 5,000 doctors a year are considering to leave the UK, and many come to Australia. Bureaucracy is one of the reasons they emigrate. We must become better at dealing with bad apples, but healthcare is already a highly regulated industry and the last thing we need here in Australia is more regulation, red tape and stressed-out doctors.

I hope the Medical Board will work with the colleges and the AMA to explore better options.

Revalidation