Interlude 1: Evidence
leigh from the path forward made a big point on part 1,
i think one of the most important skills you can convey as a physician is how to evaluate the quality/credibility of a source. the general public is quite open to a wide variety of definitions of “expert” while we in the scientific community are far more particular
Absolutely correct, so we’ll start with a short discussion on evidence, credibility will come in another post later.
When I look at articles, or anything else cited as evidence, I’m evaluating a lot of things, including experimental design, data analysis, and statistics. But don’t fret! You don’t need to take a class in statistics or get a Doctorate degree to understand levels of evidence.
In evaluating evidence, physicians often follow something like what’s shown on this page: http://www.essentialevidenceplus.com/concept/ebm_loe.cfm?show=oxford
If that all looks like greek to you, don’t worry. It basically is. The ranking system there essentially ranks types of evidence presented in a journal article from 1 (best) to 5 (worst).It’s not actually rocket science, there are a few general guidelines. I’m not going to go number by number, but just give general guidelines based on this system that parents can follow. *HUGE DISCLAIMER: I’m not a scientist, I’m a doctor. I do not have masters, or a doctorate of any sort. When real scientists comment below that I’ve oversimplified, and have missed important things, they’re absolutely right. The guidelines actually make intuitive sense, once you get the basic question.
The basic question that underlies evidence based medicine is this: If I were going to tell you something amazing, that flies in the face of the world as you know it, what would it take to convince you?
Well, what if I called my friend, an expert on the subject and he said “Well, I haven’t really done anything to back this up, but this is what I think?” Not that convincing right? That’s the lowest level of evidence, expert opinion does not good evidence make.
What If I tracked a few patients with our disease in question over time? I cherry picked these patients based on what fits into my study. Well it’s better than just telling you right? Still not great evidence though.
How about this then: I just looked up all the cases of a particular disease in our community, and then I compared them to similar people without the condition. This is a case control study. As you can imagine, since I didn’t just cherry pick a few cases, and since I’m comparing them to people without the condition (a control group!) this is a better study type. This kind of study was among the first to show the link between tobacco and lung cancer, so we’re getting into more respectable evidence here. The big problem is to have any value, these studies need to follow groups over long periods of time, and follow a large number of people. One step up from this kind of evidence: let’s take several case-control studies on the same thing, and review them together, see if they agree. If they do, even better evidence!
A weakness of the studies mentioned above, is that they all require us to have already diagnosed a disease before we make any comparisons. So, next we have a cohort study. A cohort study takes a group of people with something in common (let’s say, people who smoke cigarettes), and compares them to a similar group, without that one thing in common (e.g. people in the same community who don’t smoke cigarettes). Then we watch these people, and eventually, we count how many had a particular outcome (say, those who get lung cancer), if those who smoked were more likely to get lung cancer, BINGO, evidence. These studies are expensive, and take foreeever.
Ok, so these are studies that let us know about things that happened already (even in the cohort study, you don’t know if smoking causes cancer until after people get cancer). Those studies aren’t as great for evaluating treatments. For evaluating treatments, the awe-inspiring gold standard is the randomized controlled trial (RCT). Here, we have different treatments and/or a placebo, and we randomize people to get one treatment or another. There are all sorts of complications with this one, what you need to know is that it’s the best kind of individual evidence. For further information, I recommend the Wikipedia article, and its references http://en.wikipedia.org/wiki/Randomized_controlled_trial. I know I know, Wikipedia is a cop out, but I want to get Part 2 out soon and go over evidence!
So what’s better than a RCT? Lots of RCTs, so we pool lots of RCT have and get a systematic review of randomized controlled trials. This is great evidence, especially when the trials all agree closely.
Now you have a basic hierarchy of evidence, so when someone tells you that they’re personal expert (Like say, the DAN! Doctors, who are taught everything they need to know to cure autism in a one day seminar) assure you that vaccines cause autism, you know what to tell them.