Detection Higher Criticism is a statistic inspired by a multiple comparisons concept mentioned in passing by Tukey (1976). We are able to show that the resulting Higher Criticism statistic is effective at resolving a very subtle testing problem: testing whether n normal means are all zero versus the alternative that a small fraction is nonzero; the subtlety of this 'sparse normal means' testing problem can be seen from work of Ingster (1999) and Jin (2002), who studied such problems in great detail. In their studies, they identified an interesting range of cases where the small fraction of nonzero means is so small that the alternative hypothesis exhibits little noticeable effect on the distribution on the p-values either for the bulk of the tests or for the few most highly significant tests. In this range, when the amplitude of nonzero means is calibrated with the fraction of nonzero means, the likelihood ratio test for a precisely-specified alternative would still succeed in separating the two hypotheses. We show that the higher criticism is successful throughout the same region of amplitude vs. sparsity where the likelihood ratio test would succeed. Since it does not require a specification of the alternative, this shows that Higher Criticism is in a sense optimally adaptive to unknown sparsity and size of the non-null effects. While our theoretical work is largely asymptotic, we provide simulations in finite samples. We also show Higher Criticism works very well over a range of nonGaussian cases.

Estimation False Discovery Rate (FDR) controlling procedures were suggested as an estimation tool by Abramovich and Benjamini (1995), as a purely data-driven adaptive thresholding approach. A central question to statisticians is, do FDR-controlling procedures connect to any type of "optimality"? Abramovich et al (2000) studied sparse Gaussian data with the l_p-balls and l_q-loss frame and pointed out that, FDR-controlling procedures are asymptotic Minimax.

A natural question is that, is "asymptotic minimax" limited to Gaussian data, or what can we conclude for nonGaussian data? In this talk we study sparse Poisson Model and sparse Exponential Model, which are important models for nonGaussian data, and have applications in many areas such as Astronomy and Positron Emission Tomography (PET) as well. We show that, under natural and sensible frames, FDR-controlling procedures are asymptotic minimax and adaptive to unknown sparsity for Exponential and Poisson models.

The talk is based on joint works with David L. Donoho.

References

1. Abramovich, F. and Benjamini, Y. and Donoho, D. and Johnstone, I. (2000). Adapting to Unknown Sparsity by Controlling the False Discovery Rate, Technical Report, Statistics Department, Stanford University. To appear in Ann. Statist.
2. Donoho, D. and Jin, J. (2002). Higher Criticism for Detecting Sparse Heterogeneous Mixtures. Technical report, Statistics Department, Stanford University. To appear in Ann. Statist.
3. Jin, J. Detection Boundary for Sparse Mixtures, in preparation.
4. Asymptotic Minimaxity of False Discovery Thresholding for Sparse Exponential Data, in preparation.
5. Asymptotic Minimaxity of False Discovery Rate Thresholding for Sparse Poisson Data, in preparation.