Psychometrics: From Practice to Theory and Back Again; 20 Years of Multidimensional IRT, DIF/Test Equity, and Cognitive Diagnostic Assessment
Friday, April 17, 2009
10:30 AM in HAAS 111
Professor William Stout
University of Illinois Urbana-Champaign and Learning Sciences Research Institute, University of Illinois at Chicago
My talk presents my personal perspective about the role I’ve played in psychometrics. It includes efforts in concert with a surprisingly large group of colleagues as we have individually and collectively addressed three central issues confronting standardized testing, namely the nonparametric assessment of multidimensional latent test structure, the nonparametric assessment of test equity, and the parametric assessment of examinee skills.
The first half of my career spent as a mathematics professor (my entire career having been contentedly spent at the University of Illinois) was directed at solving problems motivated by their intrinsic intellectual appeal. In short, I was acting as artist rather than engineer. This aesthetic of pure research that works well for many in fact was increasingly sapping my research drive over time. My response was to switch fields from pure mathematics (pure probability theory actually) to psychometrics. This decision was motivated by my determination to do research where that in addition to being interesting the fruits of one’s research could be applied to important societal problems.
Because my research focus had been in probability theory, I came to psychometrics steeped in the role and importance of probability in modeling real world phenomena and in particular using complex stochastic modeling for complex real world settings, such complex settings the norm in educational measurement. In addition, I was determined to intensify the collaborative research style I had learned while doing mathematical research. For, real excitement and power can result from cooperatively doing research in carefully selected partnerships.
In the case of the measurement laboratory that I founded in the late 1980s (the Statistical Laboratory for Educational and Psychological Measurement in the Department of Statistics at the University of Illinois), the Lab focused over the next decade or so on three core issues growing out of the practice of standardized testing. These are: assessing test equity, assessing how many abilities (dimensions) the test is measuring and more generally, assessing the multidimensional structure of the latent ability space that stochastically drives test performance, and diagnosing examinee skills as a means of accomplishing formative assessment. Here, formative assessment refers to the assessment of individuals while they are still learning as a tool to facilitate both learning and the teaching process.
In each of the three research areas of latent structure, equity, and diagnosis, the challenge of appropriate probability modeling accompanied by the theoretical derivation of important implications of these models became very important. Finally (the “back again” of the title), it was always important to close the loop back to the applied setting motivating a research problem by providing a practical and easy to implement solution to the practitioner’s problem. That is, the end goal is not only the published paper we all cherish, but rather included a successful application played out in actual measurement practice. Moreover, if the work is to have an important impact on educational measurement practice, it must have easy and wide transferability to similar measurement settings. Often this last step has meant providing actual software made inexpensively available to practitioners and researchers (much of the software is distributed by Assessment Systems Corporation).
Please join us for coffee hour with Professor Stout from 3:00pm - 4:00pm in HAAS 111.