Evaluating cohort studies
I have repeatedly criticized the Johnson and Daviss study for using the wrong comparison group, rendering the conclusions of the study invalid.To understand this problem, you need to know about the appropriate design of a cohort study. An excellent series of articles in the British Medical Journal addresses this very question. These articles are completely independent of the issue of homebirth; they apply to the design of any cohort study. The first article in the series is Reader's guide to critical appraisal of cohort studies: 1. Role and design by Rochon et al. I encourage everyone to read the whole article, and I have quoted the relevant portions here:
In cohort studies care must be taken to minimise, assess, and deal with selection bias. A comprehensive approach is needed that includes the selection of appropriate comparison groups, the identification and assessment of the comparability of potential confounders between those comparison groups, and the use of sophisticated statistical techniques in the analysis...The authors offer three questions that must be answered when determining the validity of a cohort study:
Ideally, the comparison group in the cohort study should be identical to the intervention group, apart from the fact that they did not receive the intervention... Part of the art of designing a cohort study is choosing comparison groups that approach this ideal in order to minimise selection bias while maintaining clinical relevance...
What comparison is being made?So let's ask these three questions about the Johnson and Daviss study.
Published studies may include more than one type of comparison, but the focus of any appraisal of a cohort study is on an individual comparison between an intervention group and a comparison group in a defined population. A well written study should contain a clear definition of why the two groups were selected and how they were defined. This information is essential for assessment of clinical relevance and potential for selection bias.
Does the comparison make clinical sense?
...Cohort studies should not only describe the populations being compared but also include a discussion of the clinical context for that comparison and provide a justification for the comparison. Readers of these studies should determine if the study makes a comparison that is realistic and relevant to their decision needs.
What are the potential selection biases?
Selection bias occurs when there is something inherently different between the groups being compared that could explain differences in the observed outcomes.
It is important to keep in mind the effect the choice of comparison groups will have on potential selection bias when evaluating a cohort study... [A] form of selection bias, referred to as channelling bias or confounding by indication, occurs when patients are assigned to one intervention or another on the basis of prognostic factors and is key issue in cohort studies.
Readers should recognise the potential for selection bias in all cohort studies and carefully consider possible sources of bias...
What comparison is being made?
Johnson and Daviss have constructed a prospective cohort study to determine the effect of homebirth on mortality and intervention rates. One cohort contains all women who delivered at home with a CPM in 2000. This is compared to women who delivered in the hospital. According to Rochon et al., "Ideally, the comparison group in the cohort study should be identical to the intervention group, apart from the fact that they did not receive the intervention". So the comparison group MUST be women with the same level of risk who delivered in the hospital in the same year.
Does the comparison make sense?
Yes, it does make sense to compare women who delivered at home with a CPM to women of the same risk level as women who delivered in the hospital in the same year.
What are the potential selection biases?
According to Rochon et al., "Selection bias occurs when there is something inherently different between the groups being compared that could explain differences in the observed outcomes." The Johnson and Daviss study suffers from selection bias. The patients in the homebirth group differ from the patients in the comparison groups on the basis of prognostic factors. Pre-existing medical and obstetric complications are prognostic factors for both mortality outcomes and intervention outcomes. ONLY low risk women can be in the homebirth group. However, women in the comparison group are of all risk levels. Because of this, the comparison groups that Johnson and Daviss created (one group consisting of all women who gave birth at term from the vertex presentation in 2000, and the second group fished from out-of-date homebirth papers) suffer from selection bias. Therefore, any conclusions drawn from a direct comparison of either of those groups with the homebirth group is invalid.
Labels: Johnson and Daviss
<< Home