Spot the Mistake, brief answers
Ok guys, I was hoping to have fully researched answers tonight. Sadly our library was closed and I couldn’t access the appropriate references. I’ll just try and update this with references tomorrow. Below is the brief rundown, sans the numbers.
Context is important. For the sake of argument, let’s stack the odds in our students favor and say we’re talking about the treatment of low back pain. The comparison should be on even ground.
1. Not an apples to apples comparison – single chiropractic treatment is NOT chronic therapy. If we’re going to stick with comparing to a single chiropractic treatment, than the appropriate comparison is a short course of NSAIDs – like less than one week. The mortality of short courses of NSAIDs is very small.
If he wants to stick with chronic NSAID therapy, he’s in a much harder spot. He’d need the lifetime risk of dissection for someone who uses chiropractic chronically for backpain. I promise you those statistics don’t exist for reasons I cover below.
Failing that, as a surrogate, he could use statistics on how often people use a chiropractor for chronic back pain, and how many times they go over the course of their disease. Still hard to find those stats, I haven’t found them yet.
He’d also need to know if the rate of dissection is independent for each chiro adjustment. That’s hard to grasp, so a brief analogy. Imagine I punch you in the face. There is a small, but significant chance that I will break your nose.
Now imagine I punch you in the face… every day. The risk that your nose breaks can scale in different ways.
- The risk could be independent of how many times I’ve punched you in the face. In that case, over time and multiple punches to the face, you’re more likely to get a broken nose than someone who gets punched just once. Imagine this risk to be our baseline risk.
- The risk could go down with more punches. That is, if your nose is going to be broken, it will get broken earlier on. In this case, if you’ve made it through the first few punches without a broken nose, you become less likely to break your nose over time than in the independent risk case.
- The risk could go up with more punches. Imagine your nose gets a little weaker each time I punch it. In that case, as I keep hitting you, the risk that your nose breaks is waaaay higher than our baseline case. I like to call this the Rip Hamilton case – after the Piston’s basketball player who wears a mask. He wears a mask because he’s broken his nose so many times that his doctors advised him he would likely need a significant surgery if he gets hit in the face and breaks his nose again.
Either way, no one knows which of these patterns (if any) the risk of dissection follows.
The point here, is that he used a very misleading statistic. How many people die from using Advil for a week? Very few. I’ll look for the statistic, but I suspect it’s much more comparable with the statistic given for dissection.
2. Not an apples to apples comparison – he compares the mortality with the rate of dissection. Appropriate comparison would be all causes mortality, all causes complications, with a breakdown of appropriate complications. Harder to do. Well worth the effort though, since what he gave us was worthless.
3. Where did he get these numbers? Here I’m not sure. Once I get journal access, we’ll see what I can find. I asked him after the presentation, and his answer was terrible.
He couldn’t give me his reference and said that it was an observed rate. If this is the rate observed by the chiropractors – they aren’t qualified to diagnose this. If this is a rate determined at the hospital, I know they haven’t exhaustively identified those patients who saw a chiropractor. I know this because I saw 2 patients with a vertebral artery dissection after visiting chiropractor, and in neither case did any word related to chiropractic go in the chart.
What is for sure, is that this rate was not determined prospectively, only retrospectively. If it’s self reported by chiropractors, it’s subject to a ton of biases.
Either way, I’ll be trying to find his statistic, or the real statistic as soon as I can.
The statistic for NSAIDs was likely determined prospectively by following patients on chronic NSAIDs, and determining complications related to this therapy.
4. No information about the independence of treatments – It doesn’t matter what the rate of complication of chiropractic is, if the patient going to a chiropractor is still taking the NSAIDs. In this case, any risk from chiropractic is additional risk. Lots of doctors will prescribe or recommend NSAIDs, and let patients do whatever CAM that they want, so this is an important question. Our student made no attempt to put that into the discussion.
One that wasn’t what I was going for, but also very valid is:
5. The relative efficacy of the treatments is important. A small risk from a treatment of 0 value, is an unacceptable risk.
There are more problems with this, that I’m sure I’ve missed. This is the things that jumped out at me right away though. Certainly, they’re things that a fourth year medical student should know.