The LA Times, BBC, NPR and other news outlets highlighted a recent study on breast feeding that suggests minimal behavioral or intelligence benefit for breast fed infants when assessed at age 5 years. This study stands in contrast to several others over the years that did show large differences in the IQs of breast fed children compared to their formula fed peers. The American Academy of Pediatrics (as well as Pediatric Associates of Northern Colorado) recommends breast feeding newborns for the first year of life with the introduction of other foods beginning around 6 months of life. This recommendation remains appropriate because of the other benefits of breast feeding beyond the now questionable cognitive benefit.

Drilling down on this recent study in comparison with previous studies that tried to assess the impact of breast feeding on intelligence, reveals the importance of study design when complex, multifactorial questions are being asked and serves as a good example of why these types of studies are so difficult to do well.

We know that many factors influence a child’s measurable intelligence: parental intelligence, environmental stimulation, toxic exposures (like lead), early childhood education, socioeconomic status… the list goes on. How does a researcher compare “apples to apples” when we each have a unique set of factors in our environment as well as unique genetic backgrounds?

First, the study must be designed to have a large enough sample size to capture small differences between groups. There are well-established and reliable statistical calculations that are routinely performed to allow the researcher to determine the study group size necessary to be able to detect a difference between the groups of any specific magnitude the scientist picks. That is, if the effect of the factor under study is expected to be very large, these calculations predict that a small sample size will be able to detect it unambiguously. However, if the difference between groups is small, a larger number of participants is needed to be able to detect this difference with statistical certainty. This relationship between sample size and the detectability of differences between groups applies across all scientific fields, not just in medicine.

Second, the study participants must be divided between those exposed to the test condition (in this case, breast feeding for at least 6 months) and those who are not (no breastfeeding in the first year of life). These two groups are then compared to make sure that they are otherwise entirely similar in every other pertinent way that the researchers can identify. Then if there is a difference detected between the two groups, the scientists can say unambiguously that the difference is due to the one variable under study: breast feeding.

Third, and most challenging, a study that assess an effect over several years requires that the patients enrolled in the study remain accessible to the study team for ongoing assessments. How many of us move once, twice, three times in five years? How many of us would remember to send our forwarding address to a research team we met years back? Studies that run over many years are luck if half of the patients who enrolled at the beginning can be located at the study’s end. This then impacts that first factor – study size – and therefore, the statistical power of the study to detect small differences between groups.

For this breast feeding study, the researchers made sure that the average educational attainment for the mothers was the same in each group. They matched the children for the same parts of the country (as a way to insure that the children likely were exposed to the same environmental factors). Did the children in each group have the same pre-school opportunities?

In this study of 7500 Irish children, the only statistically measurable differences in cognitive function occurred when they were assessed at age 3 years but then disappeared by the time they were 5 years old. Reanalyzing 17 other studies that tried measure a difference in intelligence in breast fed versus formula fed infants found an average difference of 1.76 IQ points; a very small and likely irrelevant difference.

Breast feeding still has many other benefits: improved immune response to infections, (for most) easier day in/day out feeding preparation, lower rates of allergies and obesity for the babies; and lower rates of breast and ovarian cancers, lower rates of post-partum depression and lower rates of type 2 diabetes in moms. It just doesn’t seem like an intelligence benefit can be supported by the data at this point in the ongoing study of this question.

It takes a lot of patience and planning to do a good scientific study. Beware reports of scientific “breakthroughs” with small sample sizes. Look for studies in well regarded scientific publications where there is review of the study’s design and findings by other peers who work in the same area of science. The field of medicine has had an incredible 50 years because the standards of evidence for scientific claims have become more uniform and academic centers incredibly rigorous in disclosing and policing conflicts of interest researchers might have. This is one of the reasons that public funding of science is so important. In a world where only the medical device makers and pharmaceutical companies fund science, how can we know whether a “study” might really be just another advertisement?