May, 2015

Study of Flawed Studies

A recent article studied flaws, errors, fraud, the inadequacies of peer-review, and the problem of non-reproducibility in research studies. The authors quoted Richard Horton, the editor of the medical journal The Lancet, as saying, “Much of the scientific literature, perhaps half, may simply be untrue.”

The authors noted that a group of researchers recently published their findings concerning their efforts to reproduce the results of 100 of the “biggest experiments” in psychology; the researchers were able to only replicate results in 39 of the experiments.  In another attempt to replicate results of cancer-research findings for potential drug trials, researchers could replicate only 11% of the findings. 

The authors stated that many scientific studies done and published annually “are poorly designed, redundant, or simply useless,” and do not properly reduce biases.  A study that analyzed 300 research papers about epilepsy was cited; the results of that study concluded that 71% of the articles were characterized as having “no enduring value,” and 55.6% of those were said to be “inherently unimportant.” One estimate concluded that about $200 billion spent on research globally “is routinely wasted.”

Exaggerated claims of research results and confusion between causation and correlation published in university press releases can contribute to exaggerated news coverage, which, in turn, can ultimately distort the available evidence bases used for later research.

Harvard professor Sheila Jasanoff was quoted as saying that science is socially constructed and “[s]cientific knowledge, in particular, is not a transcendent mirror of reality.”  Also, she said, science “is embedded in social practices, identities, norms, conventions, discourses, instruments and institutions – in short, in all the building blocks of what we term the social.”  The authors pointed-out that science is done by people, people are flawed, and “science will, inevitably, be flawed.”

Sources:  Julia Belluz and Steven Hoffman, “Science is often flawed.  It’s time we embraced that,” May 13, 2015:

DNA from Microbiome May be Used to
Identify Individuals

A recent study, published in May 2015, in the Proceedings of the National Academy of Sciences, suggests that the collective DNA of an individual’s microbiome, that is, the microbes colonized in an individual’s body -- or “gut print,” according to recent articles -- may reveal details of an anonymous participant’s health, diet, ethnicity, and, eventually, possibly identity.

The National Institutes of Health (NIH) maintains a public database of human microbiome DNA in the Human Microbiome Project (HMP).  The articles describe studies that show that, although the NIH has attempted to exclude an individual’s DNA from the HMP database, short tandem repeats – sequences used in forensic comparisons – were present in the database.  The article noted that it is still unclear whether or not the presence of human DNA in the microbiome samples could provide a precise DNA signature, but with”the rise of publicly available DNA databases,” the likelihood is increased.  For example, the articles pointed out that human-genomic researchers, using genealogy databases, were able to name five of the ‘anonymous’ participants in the international 1,000 Genomes project.  Still, “the odds of identifying someone on the basis of their microbiome is low.”

Sources: Ewen Callaway, “Microbiomes raise privacy concerns,”, May 11, 2015:  Ewen Callaway, “Microbial DNA in Human Body Can Be Used to Identify Individuals,” May 13, 2015:  Francosa, E. A. et al, “Identifying personal microbiomes using metagenomics codes,” Proc. Natl Acad. Sci., May 11, 2015:

by Neil Leithauser
Associate Editor