The Extent and Consequences of P-Hacking in Science

Citations Over TimeTop 1% of 2015 papers

Abstract

A focus on novel, confirmatory, and statistically significant results leads to substantial bias in the scientific literature. One type of bias, known as "p-hacking," occurs when researchers collect or select data or statistical analyses until nonsignificant results become significant. Here, we use text-mining to demonstrate that p-hacking is widespread throughout science. We then illustrate how one can test for p-hacking when performing a meta-analysis and show that, while p-hacking is probably common, its effect seems to be weak relative to the real effect sizes being measured. This result suggests that p-hacking probably does not drastically alter scientific consensuses drawn from meta-analyses.

Related Papers

• → EFFECT OF THE NEMATODE PHASMARHABDITIS HERMAPHRODITA ON YOUNGSTAGES OF THE PEST SLUG ARION LUSITANICUS(2002)40 cited
• → Hacker Taggers: A new type of hackers(2009)14 cited
• → Cloning and Characterization of Rat BAT3 cDNA(1999)20 cited
• → Technological Paradox of Hackers Begetting Hackers: A Case of Ethical and Unethical Hackers and their Subtle Tools(2019)8 cited
• Hacking Hollywood: discussing hackers' reactions to three popular films(2011)