Tuesday, May 09, 2006

A few words about scientific studies

I posted this in the comments section of "Birth of a Forum". Since it is very important to any discussion of the safety of homebirth, I am also posting it here.

I want to make two general claims, first. It is very important to understand that just because a paper appears in a peer review journal, that does not make it true. In fact, that's the entire point of peer review journals. One group presents their findings. Other groups then present findings that either corroborate or contradict the original findings. As a general matter, the truth only becomes apparent over time and multiple studies.

There are plenty of lousy studies. The chief offenders are drug companies that pay for studies to promote their drugs. Often the data in the studies is manipulated to get to the desired outcome. Others can fall into the same trap. Either deliberately, or more often out of an excess of enthusiasm, researchers subtly (or not so subtly) "massage" the data so it says what they want.

That brings me to my second general point. It is necessary to read the actual paper, not the just the abstract. The abstract contains a brief description of the study and the claims of the authors. Only by reading the paper itself can you find out whether the study was done properly, whether the results are statistically significant (exceedingly important) and whether the results actually justify the claims made by the authors.

American doctors are very suspicious, primarily because of their experience with drug companies. So when I criticize a paper, it's not because I disagree with the findings. Often what I am saying is that the results of the study either violate the rules of statistics (which render the results useless) or the authors make conclusions that are not justified by the results.

What are the red flags that should make you suspicious? In the homebirth studies there are several specific types of red flags. First, the researchers must compare apples to apples. That means that if the homebirth group contains only low risk women with babies in the vertex position, the hospital group MUST contain only low risk women with babies in the vertex position.

Second, researchers are forbidden from excluding bad data. You can't subtract the people with bad results from the homebirth group. They must stay in the group. You certainly cannot subtract people with bad results from the homebirth group and put them in the hospital group, even if they ultimately delivered in the hospital. For example, a woman in the homebirth group who is transferred to the hospital for an abruption and undergoes a C-section after which the baby dies is STILL in the homebirth group.

Keep in mind that these are not my personal rules for evaluating a study. They are the rules of statistics that exist precisely to tell you whether the results of a particular study are accurate and reliable.

3 Old Comments:

Reading this post makes me wonder how familiar you are with the homebirth safety literature. There are many methodologic flaws in some previous home birth studies, including failing to use intent-to-treat analysis (i.e., analyzing the outcomes of hospital transfers in the home birth group if they were planned home births.) There is also a huge problem with opponents of home birth citing studies that include unplanned out-of-hospital births along with planned hospital births. There is an excellent critique of the Pang et al. study (the most widely cited by home birth opponents) at http://www.lamaze.org/institute/flawed/homebirthsafety.asp.

On the other hand, there are some outstanding examples of home birth safety research which meet all of your "criteria" and then some. This includes the recent study in the BMJ on planned home births in the US and Canada. See http://www.lamaze.org/institute/advancing/rsnb/October05.asp#homebirth.

By Anonymous Anonymous, at 8:35 PM  

Amy please say if you would prefer comments in this one or the other. It is confusing to have discussions with the same topic in two different threads.

By Anonymous Anonymous, at 9:00 PM  

This post is just about scientific studies in general. Let's continue the discussion of specific papers where it originally started in "Birth of a Forum".

Amy Romano, you can find my comments about the BMJ paper under that post.

By Blogger Amy Tuteur, MD, at 10:06 PM