Can’t See the Forest Through the Trees?
Posted July 20, 2010on:
Any study that shows midwifery is more deadly, is met with a flurry of criticisms by midwives and company, supposedly based in science and statistics.
The recent meta-analysis by Wax et al in AJOG stated that homebirths are three times as deadly. As anyone who follows the debate knows, there are no randomized controls comparing homebirth to hospitals. Yet, one of the biggest complaints by the homebirth crowd was that the study failed to include a forest plot. Forest plots are for meta-analyses of randomized controls. They are merely graphics of data and therefore cannot possibly be a flaw.
The earliest accusation that this was some kind of fatal flaw that invalidated the conclusions seems to be from Gil Gyte, a childbirth educator in London. http://http://www.nctpregnancyandbabycare.com/ She feels she is qualified to comment by virtue of her association with The Cochran Reviews. The Cochran Reviews is an organization devoted to poor quality meta-analysis via its promotion of a very simplistic, cookbook method that is made worse by interpretations by its mostly amateurish researchers. Her criticisms are little more than noting or merely claiming that the reviewers did not do it according to the Cochran algorhithm. Among them, she claims that a forest plot should have been included.
Amy Romano of LaMaze’s Sense and Sensibility quickly plagiarized, er, eh , I mean, chimed in with same.
“I also take issue with the fact that the researchers did not display the standard “forest plot” that customarily accompanies a meta-analysis to illustrate how the relative magnitude of observed differences in the individual studies and the pooled analysis.”
Then Jennifer Block of PushedBirth.com excitedly passed on the misinformation as if it were a big Gotcha moment.
“For a great break down of the science, look to Amy Romano’s Science & Sensibility. She asks why the authors did not graphically display their results using the customary “forest plot,” which usually accompanies meta-analyses (perhaps because it would have shown “a confidence interval you could drive a truck through”?).”
Gold Medal for this Birth Junkies Behaving Badly Trifecta goes to Romano. It is not for the mistake itself, started by Gyte. But, rather her pervasive pattern of using statistical terminology she really doesn’t understand to dress up her beliefs, professional promotion, and self-interests in objectivity and her behavior towards her critics and her evasiveness when found out.
Amy had the clever idea of actually linking to a Wikipedia posting on forest plots from her post condemning its absence. Wiki clearly tells you in the first sentence or so it is for random studies, which homebirth studies are not.
So essentially, right under her nose was the information that, yes, many people borrow the technique for non-random analysis, but it is technically incorrect. Furthermore, it is merely a graphic representation of the data. In other words, it can’t be a methodological flaw to omit something that technically, one should not use in these circumstances. Refraining indicates better depth of understanding of complex statistics. And, it is certainly not a methodological flaw to not have drawn a picture for those unable to understand the narrative!
Ms. Romano has gotten more than one comment on her blog pointing out this and many, many other of her blatant errors in her statistical and methodological criticisms. However, none make it through moderation. None of the attempted comments were copied be re-posted here. After all, no one ever imagined pointing out a blatant misuse of statistical terms would be banned as a TOS violation, profane, spam, or unladylike.
She and Block further go on to speculate that this was not done to hide “a confidence interval you could drive a truck through”. She implies that this too is another methodological flaw. The confidence interval indicated that homebirth was anywhere from a 1/3 more deadly to 5 times more deadly. Earth to Amy. Earth to Amy. Come in Amy. Both ends of the interval are very damning to homebirth and midwifery by proxy. Homebirth is worse no matter what. No mom wants to get in this truck no matter what.
Clearly she doesn’t understand what confidence intervals mean. Their understanding of research is much like their understanding of birth – they watch a few and think they understand it enough to use it.
It is obvious to anyone who has studies advanced statistics or research methods, that these ladies decide a study is fatally flawed statistically and methodologically simply because the results don’t flatter them. Then, they proceed to pepper their condemnations with statistical terms lifted from books that have little or no relationship to the study at hand and they don’t understand. They are not trying to educate women, they are trying to deceive us.
She wrote in response to being called on the carpet about the refusal to post:
“Fwd: [Science & Sensibility] Comment: “Read this book: How to Read a Paper” Friday, July 16, 2010 12:53 PM This sender is DomainKeys verified
“Amy Romano” <firstname.lastname@example.org>
Because it’s my blog, and you’re rude.
Sent from my iPhone
Begin forwarded message:
Subject: [Science & Sensibility] Comment: “Read this book: How to Read a Paper”
Why are you erasing comments that point that you are not truly raising evidence-based statistical or methodological issues, but seem to be lifting and throwing out terminology from this book?
If you really know what these terms you are using mean, and they really apply to a situation, then you should be able to explain yourself when they are used in a completely unstandard context.
You erased one of mine and I have evidence you did same with another’s.
All you will accomplish is the genesis of the Stuff Science and Sensibility/Amy Romano Wants to Hide blog. It will probably come up higher in google than this.”
She reportedly Twittered she wasn’t going to argue with someone who insults her. So why does she insult our intelligence?
5 Responses to "Can’t See the Forest Through the Trees?"
Comments are closed.