TO ADDRESS UNWARRANTED CLINICAL VARIATION, MORE ANALYTICS ISN'T THE ANSWER
- Michael van Duren, MD, MBA
- 12/20/2024
Tips for new physician leaders on how to influence practice patterns
Michael van Duren, MD, MBA, has twenty years’ experience addressing the issue of variation among the care patterns of physicians treating identical conditions. This blog post distills his views on the current state of the field and makes suggestions for how we can make meaningful progress without a lot of new investment in data or analytics.
The challenge: Practice patterns vary widely
As a young physician leader, I was given the assignment to reduce the total cost of care. I had no idea where to get started. The toolset I was given was a large dataset of "average costs per episode" for several thousand physicians. It took me a few years, but I did figure out that this data could shine a light on unwarranted clinical variation and after several more years, I started making headway with reducing variation. The effort turned into a permanent team that accomplished savings of millions. Over the past twenty years, I have been refining the process of influencing clinicians to standardize care in a way that they welcome and enjoy.
In this piece, I want to pass on some of the techniques I discovered along the way so that other new leaders won’t have to spend as long experimenting on their own.
Before diving into the deep waters of using your hard earned social capital to try to change the clinical behaviors of your peers, you need to have a good reason for wanting to undertake such a treacherous challenge. I will try to convince you that unwarranted clinical variation is a problem worth diving into. There be dragons there, but also gold!
1. Variation remains unsolved
The data is clear that unwarranted clinical variation is ubiquitous.1 Almost anywhere you look, you will see three-fold variation in the rate that physicians order tests or do procedures for identical patients. Some of this is driven by differences in the patients, but that is what we call “warranted variation.” The unwarranted part is where the patients appear identical, but the differences are solely caused by which physician they happened to see.
2. Variation provides a clue for where to find savings
And much of the variation is in the area of “low value services.” This is the name for tests or procedures that are considered by scientific studies to add little value.2 For example, a Sinus CT scan for someone who presents with nasal congestion and painful sinuses. A CT might confirm the diagnosis of sinusitis, but simple treatment and waiting a few weeks would also confirm that the sinusitis resolved without the expense and harmful radiation of a CT scan. When there is unwarranted clinical variation in the areas of low value, that is the perfect place for removing unnecessary costs from the healthcare system. Despite the efforts of managed care, prior authorization systems, bundled payments, and ACOs, we have not made much of a dent in reducing the “waste” in healthcare.3
3. Physician behavior remains difficult to change
We know that there is unwarranted variation and that it causes unnecessary costs. So, why can’t we address those physicians who are ordering the unnecessary tests and procedures? The variation continues every day, and we aren’t stopping it. There are some bright spots where quality improvement projects have resulted in significant improvements in healthcare outcomes, such as reducing central line associated pneumonia, but there is not yet a robust and standardized way of addressing physician behavior change. The emerging field of behavioral economics is one area of ongoing innovation that is pointing to several promising approaches, such as using peer comparison as a tool for nudging change. (See our behavioral economics resources here.)
4. You probably already have the data
Most healthcare organizations, whether large or small, are in possession of measures of performances that show large variation. The sources of the data might be internal quality projects or external sources such as health plans, MIPS, ratings agencies, etc. And whether the metrics are aimed at quality, overuse of unnecessary services, or patient experience ratings, many team leaders already have had a sneak peak at who their “outliers” are. When you ask them whether the data has been placed in front of the physicians, more often than not, the answer is, “No.” I will address what I believe the reasons for this are later in this review. Bottom line is this — the data is already available. With few exceptions, lack of data is not the main barrier.
5. We should not blame the doctors
It would be convenient to hide behind the notion that the difficulty of changing physician behavior is because doctors don’t want to listen to advice and are unwilling to change. But that would not be true. In my experience, physicians are very interested in learning whether there is objective evidence that they are doing something that needs improvement. If the need for change is presented as simply as showing them that they are significantly different from their peers who treat similar patients, that is usually enough to get their attention and get them to consider change. If they react with the typical objections of “my patients are sicker” and “the data is no good,” then that is just evidence that they are concerned. Usually, those who complain the loudest are the ones who show the most change when you re-measure the issue. The truth is that doctors are receptive to good feedback and usually take it to heart immediately.
Imagine the conversation like this
You take the available data and ask an analyst to show the individual physician rates next to one another in a simple graph, with names attached, unblinded. Boldly, you launch the following sequence:
- You review the prepared visualization to ensure the names look right and the message is clear.
- You gather a group of physicians that know each other and have comparable practices into a group meeting and facilitate a collegial, respectful conversation.
- The data is welcomed, and rather than getting torn apart, is used as a springboard for discussing the apparent variation
- The physicians enjoy a spicy dialogue, ask each other pointed questions, and (sometimes reluctantly) land on a consensus for what would be a preferred practice going forward.
- The entire experience is appreciated, and the physicians look forward to the next meeting.
- A few months later, you come back with data that shows that upon re-measurement, there is much less variation, and quality and value have improved.
The barriers
As I see it, there are three barriers that need to be overcome to bring this ideal scenario into realization. They are very concrete and not that difficult.
Barrier 1: Deliver the data to the doctors, and in the right way
It is said that people fear the act of public speaking more than death itself. I believe that the fear of being the one to tell a doctor that they need to practice differently, ranks right up there. I have heard from administrators and from nurses, “Only another physician can tell a doctor what to do!” And from the physician leaders who are appointed to lead a group of their colleagues, “I don’t think this is the right time to upset the group with a divisive presentation like that.”
Now, there are some simple caveats to delivering the data the right way: It needs to be brought with the right tone, and the right intentions. Leaders need to be able to create a container of safety and trust, where the data will be received for the purpose of improvement, rather than punishment.
Barrier 2: Facilitate a conversation
Assuming you were able to deliver the data in a way that was respectful and helpful, the next step is to get them together to talk about it. The data alone is enough to get their attention, but usually not sufficient to show the path forward to change. The real learning comes from when they can ask each other questions, especially those in the group with the better performance, “Tell me how you do that?” The best discussions happen when the group stops fighting the accuracy of the data and starts talking about how to practice medicine in a new way.
I have found that in a group setting, where the cohort is colleagues who consider each other peers, the differences will stimulate curiosity and will serve as a source of motivation to not be on the low end of the curve. There will be subtle social norming towards the preferred behaviors.
Barrier 3: The data must be decent (not perfect)
By decent data, I mean that it ideally meets these specifications:
- Actionable — This means that the data should ideally give physicians an insight about a practice change over which they have control. They can do something to drive that metric into a better direction. Sharing data about outcomes over which they have no control has no value and will only lead to frustration.
- Attribution — This is the logic by which you apply accountability for actions, outcomes, and decisions. This can be particularly thorny in a busy hospital setting where more than one physician cares for a patient over their stay. Getting this right is critical to building trust with your physicians. Using the “attending physician” field in an EHR is not enough. You need to look into the orders and notes to see who was making the decisions and placing orders. Attribution may have to be shared across physicians in some cases. To adjust for volume differences, the best way to display comparisons is with using rates. That means that both the denominator and numerator each need thoughtful consideration regarding optimal attribution.
- Valid (statistical) — Valid data is table stakes for working with physicians. Even a hint of inaccuracy will seem justification to ignore the data. There is an art to knowing when the metric is refined enough to say “This is not perfect, but would you agree that it is sufficient to stimulate a conversation, after all, we are all measured with the same imperfect measurement.” Case examples can be helpful in supporting your metrics.
- Meaningful (impacts pts.) — While administrators have many concerns (including care quality) there is a suspicion from front line clinicians that administrators are driven by financial concerns. Where possible, try to make an empathetic connection to direct impact on patients. For example, “I know that nobody likes talking about writing discharge orders earlier in the day, but what if it were your mother, lying on a cold gurney in the hall of the ED, waiting for a bed to open up?”
Ideas to get started
If you have a “facilitator,” set them loose
You know the teammate: someone who is strong at listening, non-judgemental, and people seem to respect what they say. Why not carve out a piece of their schedule and allow them to moderate some of these discussions? Clearly they must be smart with high emotional intelligence, but need not necessarily be a doctor. Yes, you should pay and support these people in this capacity.
While some may think a department chair would be a great fit, I would recommend that you find a peer to ensure a stronger reception and less of a “top-down” approach.
No facilitator?
If you find yourself lacking an appropriate facilitator, you can still take steps to encourage physicians to interact with each other and engage with the data about their practice.
A proven method to get the data out is in a synchronized “push” fashion, for example, by using text messages. Don’t let it get lost in email where no one will see it.
Then, simply create an atmosphere or context that promotes discussion. Possibly offer a pizza lunch. Even simpler yet would be to pull up some data as a part of a department meeting and see if anyone has insights or thoughts to share on why there is so much variation between the physicians.
One group of ED physicians reported the following story. “When there are several of us working together in the ED and we all hear our phones go off at the same time, we know it is the weekly text message about our practice patterns. When we get the message all together in that setting, it always leads to some interesting discussions as we compare ourselves and share our reactions to the message.”
Start slow and easy
This is perhaps obvious, but worth mentioning for clarity’s sake. To win the team over on this approach does take a thoughtful approach. First, avoid diving into data about controversial measures. Start easy with metrics that show variation in the rates of ordering imaging or lab tests. Next, make sure discussions are focused on ideal practice rather than on individuals, and be on the lookout for any colleagues who might feel defensive. An example of a topic that might be too controversial to start with would be a process where there might be misaligned financial incentives such as fee-for-service procedures in a value-based environment. Build some regularity into your meeting schedule so you create a culture of continuous improvement and they come to expect fun discussions about practice variation. By keeping it light and somewhat frequent, you avoid too much of a spotlight on any one individual.
As you master the simpler metrics, you may start looking for more advanced metrics. That is, you may consider whether to implement analytics platforms, build your own, evaluate payer data, or consider the broader set of metrics from an accountable care organization point of view.
Summary
The challenge of leading a group of physicians towards more standard care can be met by using available metrics to explore unwarranted variation. By paying attention to proper facilitation that creates a safe and fun atmosphere, a habit of ongoing learning and sharing can be built that will strengthen group culture.
Insufficient data accuracy is too often used as an excuse to avoid starting the conversation, when all that is needed is a willingness to be curious and ask each other questions about differing practice patterns.
If you’re not sure how to get started, give me a call, as I would love to pass on the torch and share the thrill of seeing more groups of physicians enjoy the process of learning from each other.
1Song Z, Kannan S, Gambrel RJ, et al. Physician Practice Pattern Variations in Common Clinical Scenarios Within 5 US Metropolitan Areas. JAMA Health Forum. 2022;3(1):e214698. doi:10.1001/jamahealthforum.2021.4698
2Schwartz AL, Zhao X, Sileanu FE, Lovelace EZ, Rose L, Radomski TR, Thorpe CT. Variation in Low-Value Service Use Across Veterans Affairs Facilities. J Gen Intern Med. 2023 Aug;38(10):2245-2253. doi: 10.1007/s11606-023-08157-9. Epub 2023 Mar 24. PMID: 36964425; PMCID: PMC10406760.
3Phipps-Taylor, Madeline., and Stephen M. Shortell. "More than money: motivating physician behavior change in accountable care organizations." The Milbank Quarterly 94.4 (2016): 832-861.