Statistical Challenges in National Security

Abstract: In this talk we discuss the National Security Agency, its mission and values, and the types of problems we solve and the methods we use for solving them.

Abstract: The classification of unknown entities based on measured data is a fundamental challenge across applications as diverse as medical diagnostics, treaty monitoring, and electronic fraud detection. In many cases, the data available to support classification decisions arise from multiple sources, each with its own unique signal and noise characteristics. The method to be discussed here, known as Probabilistic Feature Fusion (PFF), provides a means to combine multi-source classification information in a manner that is statistically rigorous and accounts for the uncertainties associated with the constituent sources. PFF provides final class consistency scores that are readily interpretable within in a Frequentist framework, and allows complete traceability back to the contributing sources. It is particularly appropriate in applications related to high consequence decision support, where training data may be limited, and black box classifiers struggle to gain trust and cultural acceptance. The method will be illustrated with a practical application related to the segmentation of human skin in color imagery.

Abstract: Before the DoD acquires any major new capability, that system must undergo realistic testing in its intended environment with military users. The complex, data-limited, highly variable nature of the test environment presents many unique statistical challenges. The set of conditions in which a system will operate is typically large, and important variables are often uncontrollable during test, making rigorous experimental design a challenge. Data sets obtained from tests are almost always messy. Issues such as lurking variables, small & unbalanced sample sizes, and ordinal responses necessitate creative and sometimes sophisticated data analysis approaches. This talk will examine some of these defense testing situations in detail and discuss how statisticians in the test & evaluation community have approached associated design & analysis challenges.

The Role of Academic Research at DARPA

Abstract:  As a program manager at DARPA, I had the unique opportunity to initiate programs of relevance to the agency that nonetheless reflected my interests and expertise. As a long-time academician, an equally important goal was to make sure that these programs had long-term benefit to academia as well. Crafting these programs to meet these goals required posing questions whose solutions necessarily involved foundational advances in the state-of-the-art but whose deliverables would nonetheless contribute meaningfully to DARPA's overarching mission. In this talk, I'll provide a retrospective on several of these efforts, highlighting successes, challenges, and lessons learned.