Controlling the Proportion of Falsely Rejected Hypotheses when Conducting Multiple Tests with Climatological Data

Citations Over TimeTop 10% of 2004 papers

Abstract

Abstract The analysis of climatological data often involves statistical significance testing at many locations. While the field significance approach determines if a field as a whole is significant, a multiple testing procedure determines which particular tests are significant. Many such procedures are available, most of which control, for every test, the probability of detecting significance that does not really exist. The aim of this paper is to introduce the novel “false discovery rate” approach, which controls the false rejections in a more meaningful way. Specifically, it controls a priori the expected proportion of falsely rejected tests out of all rejected tests; additionally, the test results are more easily interpretable. The paper also investigates the best way to apply a false discovery rate (FDR) approach to spatially correlated data, which are common in climatology. The most straightforward method for controlling the FDR makes an assumption of independence between tests, while other FDR-controlling methods make less stringent assumptions. In a simulation study involving data with correlation structure similar to that of a real climatological dataset, the simple FDR method does control the proportion of falsely rejected hypotheses despite the violation of assumptions, while a more complicated method involves more computation with little gain in detecting alternative hypotheses. A very general method that makes no assumptions controls the proportion of falsely rejected hypotheses but at the cost of detecting few alternative hypotheses. Despite its unrealistic assumption, based on the simulation results, the authors suggest the use of the straightforward FDR-controlling method and provide a simple modification that increases the power to detect alternative hypotheses.

Related Papers

• → Thresholding of Statistical Maps in Functional Neuroimaging Using the False Discovery Rate(2002)4,997 cited
• → On the Adaptive Control of the False Discovery Rate in Multiple Testing With Independent Statistics(2000)1,637 cited
• → A step-down multiple hypotheses testing procedure that controls the false discovery rate under independence(1999)243 cited
• → Simultaneous control of all false discovery proportions in large-scale multiple hypothesis testing(2019)44 cited
• → Controlling Error Rates with Multiple Positively-Dependent Tests(2021)