Print(0)

Journal Article

Critical reviews in food science and nutrition

Crit.Rev.Food Sci.Nutr.

50 Suppl 1

10

12

LR: 20150205; GR: P30 CA091842/CA/NCI NIH HHS/United States; GR: P30 CA091842-03/CA/NCI NIH HHS/United States; JID: 8914818; NIHMS248244; OID: NLM: NIHMS248244; OID: NLM: PMC3024848; ppublish

England

1549-7852; 1040-8398

PMID: 21132580

eng

Journal Article; IM

10.1080/10408398.2010.526838 [doi]

Unknown(0)

21132580

The impact of study design on the results of medical research has long been an area of both substantial debate and a smaller body of empirical research. Examples come from many disciplines within clinical and public health research. Among the early major contributions in the 1970s was work by Mosteller and colleagues (Gilbert et al., 1997), who noted that innovations in surgery and anesthesia showed greater gains than standard therapy when nonrandomized, controlled trials were evaluated compared with the gains reported in randomized, controlled trials. More recently, we and others have evaluated the impact of design in medical and surgical research, and concluded that the mean gain comparing new therapies to established therapies was biased by study design in nonrandomized trials (Colditz et al., 1989; Miller et al., 1989). Benson and Hartz (2000) conducted a study in which they focused only on studies reported after 1985. On the basis of 136 reports of 19 diverse treatments, Benson and Hartz concluded that in only 2 of the 19 analyses did the combined data from the observational studies lie outside the 95% confidence interval for the combined data from the randomized trials. A similar study drew only on data reported from 1991 to 1995, which showed remarkably similar results among observational studies and randomized, controlled trials (Concato et al., 2000). These more recent data suggest that advancing the study design and analytic methods may reduce bias in some evaluations of medical and public health interventions. Such methods apply not only to the original studies, but also to the approaches that are taken to quantitatively combine results by using meta-analytic approaches such as random effects meta-regression, Bayesian meta-analysis, and the like (Normand, 1999). By focusing attention on thorough data analysis, design issues can be understood and their impact or bias can be estimated, on average, and then ideally accounted for in the interpretation of data. Before discussing dietary data, let us first consider some of the more clearly delineated preventive exposures. Issues of study design have been addressed in terms of combining randomized trials and observational studies in evaluating preventive interventions such as Bacillus Calmette-Guerin vaccination (Colditz et al., 1994) and mammography screening (Desmissie et al., 1998). When one is interpreting the apparent heterogeneity in the results, it is important to step back and ask what is the relationship being evaluated under these different study designs? For example, a randomized, controlled trial uses the intention-to-treat analysis to preserve the merit of randomization. Such an analysis does not evaluate the exposure-disease relationship, but rather examines the impact of offering a new therapy versus an alternative therapy (regardless of adherence to the intervention, or control or placebo). On the other hand, a case-control study or a prospective cohort study will evaluate the impact of the screening test among those participants who were screened as compared with those who were never screened. In prevention studies, the design raises major issues of the timing of the exposure in the natural history of disease and also the adherence to therapy by healthy research volunteers. Case-control studies of preventive interventions such as screening mammography and prospective population-based studies of pap smears have capitalized on this variation in time since the last screen to evaluate the protective interval for a screening test (IARC Work Group, 1986). In contrast, a trial must choose a level of exposure, such as annual mammography screenings or colon screenings every 10 years with a colonoscopy, regardless of the evolving evidence on the duration of protection after a negative screening test. Continuing with the mammography example, a detailed study by Demissie and colleagues (1998) combined data from seven randomized trials and six case-control studies that

Causality, Epidemiologic Methods, Epidemiologic Studies, Humans, Mass Screening, Observation, Randomized Controlled Trials as Topic, Research Design

Colditz,G. A.

PMC3024848