The daunting revalidation dilemma

The Medical Board of Australia is well aware of the daunting revalidation dilemma: how to identify underperforming doctors without subjecting the rest to time-consuming and needless procedures?

The percentage of underperforming doctors is low. Nevertheless, in the UK all doctors undergo regular appraisals and are ‘revalidated’ every five years if they are deemed up to date and fit to practice.

The UK revalidation system has received its fair share of criticism. A common complaint is that the collegiate appraisal process has been ‘dumbed down’ as it changed from a formative to a summative process.

Other criticism includes the heavy time burden and paperwork, the negative impact on doctors’ wellbeing (while the profession already works in a highly stressful environment), the creation of a tick-box mentality, and a situation where some doctors are avoiding complicated situations and high-risk patients that could get them into trouble.

The good news is that the Australian Medical Board does not seem to want to copy the UK model and instead appears to be looking at countries like Canada or New Zealand, where the focus lies more on self-regulation as opposed to external regulation.

Expect the introduction of some sort of revalidation model in the next two to five years. The question is of course: are we heading in the right direction?

The purpose

Interestingly, there is still discussion about the purpose of revalidation of doctors (see picture). The overarching principle seems to be improving patient care, but whether it’s about ensuring public safety, ‘catching dodgy doctors’ or making good doctors better, is not always clear.

Some say it’s a bit of everything, which may be true but the problem is: how are we going to develop a revalidation system that does ‘a bit of everything’?

Differing message about the purpose of revalidation
Differing messages about the purpose of revalidation in the UK. Source: The Kings Fund

According to the Medical Board of Australia the fundamental purpose of revalidation is to ensure public safety. The Board has proposed a two-pronged approach to achieve this, namely improving continuous professional development (CPD) and identifying at risk doctors:

  1. To maintain and enhance the performance of doctors practising in Australia through efficient, effective, contemporary, evidence- based CPD relevant to their scope of practice.
  2. To proactively identify doctors who are either performing poorly or are at risk of performing poorly, assess their performance and when appropriate support their remediation of their practice.

To be fair, I agree our CPD model could be a lot better, focusing more on where we need to improve instead of what we want to improve.

At the same time there are concerns about the Medical Board proposal, especially with regards to the method of finding the underperformers. The Medical Board has recognised many of the issues and is currently consulting with the profession.

Strengthened CPD as proposed by the Australian Medical Board. Source: Medical Board
Strengthened CPD as proposed by the Australian Medical Board. Source: Medical Board of Australia

Two issues

The overarching problem is that there is little evidence to show that revalidation improves patient outcomes. I can see at least two other major issues:

  1. Externally enforced actions have less impact than internally-driven change in a collegiate, supportive environment. The colleges, rather than the Medical Board, AHPRA, employers or other parties, should be supported with data and resources to provide skilled remediation.
  2. The proposed profiling of doctors (e.g. over the of 35, male, trained overseas, previous complaints) appears to be a blunt approach. The tools should be sharpened, focusing more on behaviour and performance. To identify underperforming doctors we need a good screening tool. As Wilson and Jungner outlined fifty years ago, there are several criteria to be met first, to make sure we’re not doing more harm than good, especially as the percentage of underperforming doctors is low and at this stage we’re not 100 percent sure what kind of doctors we are looking for. We should also be careful not to confuse screening and assessment tools.

The way forward

The starting point should be a supportive, non punitive solution. Only when the desired outcomes through collegiate processes are not achieved, should regulators become involved. Any model must be fair, evidence-based and not create large amounts of paperwork.

Here are seven principles I believe are important when moving forward:

  1. The focus of revalidation should be heavily weighted towards self-regulation and strengthening collegiate education and remediation processes;
  2. Self-initiated gap and learning needs analysis are effective tools to direct life-long learning, supported by evidence;
  3. Peer review, performance review and outcome measurement could be used to strengthen QI&CPD but will require further evaluation;
  4. Data exchange between agencies and organisations, keeping in mind confidentiality and privacy, could identify underperformers earlier;
  5. Under performing doctors must be supported, not only via remediation but also looking after the wellbeing of the doctor involved;
  6. There needs to be clarity and transparency about potential medicolegal use of data collected during the revalidation process;
  7. The costs involved should not be carried by the profession alone.

And lastly, we really need a less punitive term instead of revalidation.

9 thoughts on “The daunting revalidation dilemma

  • Edwin, ‘revalidation’ is supposed to protect patients from underperforming doctors. Revalidation pundits argue we can identify this tiny group of doctors before they cause harm by profiling them in some way.

    We can see how banal is the evidence for profiling when the numbers of doctors who do actually ‘offend’ are the needle in the haystack of male doctors over 35 in solo practices. And that cohort naturally doesn’t include all who end up in trouble.

    But worse than banal, profiling very close to blatant discrimination. Imagine interrogating all young Aboriginal men before any crime is committed, because that cohort are over-represented in our gaols. Look at how profiling would-be terrorists on some population characteristic penalises the innocent.

    Way past blunt, it’s a failure to understand what’s going on, why those trends exist and what changes could resolve them.

    Meta-data, and the trends evident as a result, would be more helpful in predicting likely underperformance [seems it is coming anyway with the MBS review and PIP redesign].

    From that meta-data, identify the behaviour or ‘discompetence’*,work out the NNT, compared with the number harmed, as we would in any medical trial designed to increase quality and safety in healthcare before instigating a protocol.

    Prevention is better than cure. Strengthened CPD, sure – but only in the context of all the other known and delivered drivers of quality care by capable practitioners.

    These drivers include evidence based selection for all doctors seeking to work in general practice, programmatic assessment during training, completion of training that includes professional behaviour assessment as well as examination results, collegiate support systems and processes to prevent burnout, and systemic changes like a properly funded and designed medical home (noting the DOH health care home in no way meets this criteria).

    I suspect any significant, resistant and recalcitrant underperforming doctors are known in medical school if not during training. We need to have the courage and evidence to act earlier for the benefit of the patients, the profession and also the doctor in question. Don’t wait for some final tool that can’t do it with any precision anyway.

    Remediation must follow any assessment that demonstrates under performance. We know that in the training environment, but what do we know about it in the post-fellowship space? Very little and there are very few options available. Considerable research effort and educational investment needs to be made before we action assessments that lead nowhere but to worse outcomes for all or premature professional exclusion.

    Should we embrace the elements of revalidation, and in particular performance development? I think we should – not as ‘revalidation’ as if that terms means anything useful anyway – but because we learn best from constructive, even if subjective, feedback. Lots of it. I don’t know of any job where feedback isn’t a core function of improvement. This is something the colleges should explore and I think the PLAN tool is a low key way of building collegiate feedback into our profession and progression.

    Some kind of ex post facto instrument above and beyond these drivers can only assume that our current risk mitigation process as described is not for for purpose. If so we need the data on who is slipping through the Swiss cheese holes so we can quality improve the leaky layers. The obvious one might be CPD but only complaints data can tell us factually rather than intuitively.

    More likely those who end up attracting the regulators’ interest have – in the past – bypassed one or more of our layers, but we need the data to test that hypothesis. Going forward all those layers apply to all – ‘validation’ [rather than ‘revalidation’ perhaps] is a lifelong quality improvement process.

    Finally, how much safety can we provide by adding another layer? I suspect none, or very little. Medicine and health care are inherently dangerous activities and will remain so with or without further ‘validations’. It’s time we had that conversation too.

    *acknowledgement to Professor Lambert Schuwirth for this brilliant way of understanding ‘discompetence’ when and how we assess appropriate interventions

    Liked by 2 people

  • Edwin, I think the task should be – in a very general sense – to change the role of the GP. We no longer can refer to ‘general physicians’. They have disappeared over the last two decades. They are now all specialist, procedural physicians – since that is where the money is. The public hospital ‘out-patient clinics’ are still present, but grossly undersized for the real demand, with long waiting times. There is now a Chinese wall between ED and the hospital wards. I recall the response when I sent a 9 year old with a TSH of 39 to ED at Tweed Hospital. Believing that the child deserved ‘paediatric endocrinology’ to oversee the start of thyroxin supplementation/replacement. The child came back to me to following day with the curt request that I do my job. Ms Mules attended Cairns Hospital ED a week after being seen by Dr Ferguson for neck stiffness and headache, what ultimately proved to be cryptococcal meningitis. (Statistically, one case per year in Queensland at the time.) ED also missed the diagnosis. Cairns Hospital was not sued. Dr Ferguson’s MDO ultimately paid AUD $6.7 million. (Decision about a year ago.)
    We have just been told what we will be paid for a year’s care for tier 1, 2 and 3 degrees of complexity of chronic medical conditions. That is ‘total budget’. We have not yet been told whether this includes what is now paid for GP management plans, nor whether it covers the costs of running the health care homes’ clinical care consultative committees (that will include nursing and allied health professionals) touted as providing a major improvement in clinical care – and ‘saving costs’. Nor have we been told how the ‘tier assignment’ of a patient with a complex, chronic illness will be made, and how this new bureaucracy will be funded. (The criteria will either be complex, subjective and ambiguous – patients will demand they be ‘escalated’, or simplistic, arbitrary and peremptory, not uncommonly unfair. The government is simply on a savings binge.)
    Our clinical task as GPs has already become more complex, difficult and responsible. We have now been told we will be paid less money to do more work and take a longer time. All under smokescreens called ‘revalidation’ and ‘the health care home’.
    It is time that everything went into the pot again. That we characterise exactly what is now expected of us, and what we consider the reasonable payment for that is. Compared with other medical and non-medical professional incomes.

    Liked by 1 person

  • I do not know why a profession who does not expect to pay (take ownership) for their own revalidation invites at times unwarranted interference from external stakeholders. The accounting profession do not get a cent of the Government and nor does the tax office dictate to the profession for a reason. We have 5 yearly validation paid for by members. Every 5 years must undergo ethics training. It has been working for the last 10 years really well with Government.

    Liked by 1 person

    • Does your CPD coast $800 pa, not including membership fees? Does your CPD also require that, ethics aside,. you are contemporary with the latest developments? Does the CAANZ or CPA charge the tax office or other educators for them to count the points?

      Liked by 1 person

      • Benedict I have membership with both CPA and ICA and pay $1200 p.a. plus $400 to attend a 3 hour technical update. We have to do 40 points per annum. 1 point per hour. The tax laws and accounting standards are changing every month. The tax act is 10,000 pages and that excludes the rulings and case law and state law changes. Plus I get charged for being a registered tax agent and an additional fee for being a corporate agent. If I do not comply with CPD then I risk losing my registration. I also have to pay $3,000 for an externally approved accreditation surveyor to maintain a seperate practice membership which is on top of individual membership. None of this is paid for by the Government.

        Liked by 1 person

  • Dear Edwin,
    A very wise and prudent comment on the revalidation plans. You look for models only to the UK and NZ, but if I look through your 7 principles, I do see a reflection of the Dutch model. That is probably less focused on selecting the bad apples, but more on polishing. Yet one is obliged to be in a peer group/ quality circle and participate in a practice visit or an individual “visit”. It is quite difficult to become out of touch. Of course there is a lot of dissatisfaction with the extra work and the costs, but it would seem more in line with a culture of trust, rather than control.
    Problem is that the instruments are in Dutch, so less accessible, but for the framework of Quality Assurance that should not matter.
    A special greeting from rainy, windy and cold Holland
    congrats on your great blog.

    Liked by 1 person

I'd love to hear from you! Please leave a comment:

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.