Research Statement

My research covers several methodological and substantive areas of significant interest to me, including survey methodology, applied statistics, and public health. My books and articles have been cited nearly 13,000 times by other scholars (Source: Google Scholar). The citations to my articles have increased each year during my research career, and 127 of my peer-reviewed articles have been cited over 10 times by other scholars (Source: Google Scholar). My overall Google Scholar h-index is currently 50, and this compares quite favorably to leading researchers in related disciplines who are at a similar academic rank at the University of Michigan and elsewhere. In this research statement, I provide detailed descriptions of my scientific contributions to date.

Interviewer Observations. My doctoral training in the Michigan Program in Survey Methodology, under the guidance of Drs. Frauke Kreuter, Jim Lepkowski, Rod Little, and Ed Rothman, focused on the statistical aspects of survey estimation. More specifically, I examined the role that paradata, or data describing the survey data collection process, can play in improving survey estimates. My dissertation was partly motivated by my shock in learning (from Bob Groves) that interviewers in the National Survey of Family Growth (NSFG) were being asked to record judgments of whether sampled persons were in sexually active relationships, and that these observations were being used to adjust NSFG estimates for nonresponse. How effective would nonresponse adjustments based on these observations be if they were simply judgments, prone to human error? And how much variability was there among interviewers in the quality of these observations?

The research that I performed for my dissertation resulted in three original studies, addressing gaps in the literature with regard to the quality of these "pre-survey" interviewer observations and the implications of reduced quality in these observations (and auxiliary variables more generally) for survey estimation. This research blended theories from social psychology, regarding errors in human judgments, and principles of survey estimation. If interviewer observations are recorded for all sampled units and predictive of key survey variables and the propensity to respond, then they can effectively reduce the nonresponse bias in survey estimates; however, if the observations are recorded with too much error, they can hinder the effectiveness of nonresponse adjustments. These studies were published in high-impact peer-reviewed journals (West 2013, JRSS-A; West and Kreuter 2013, POQ; West and Little 2013, JRSS-C). I have published eight additional studies to date extending this work (West and Trappmann, 2019; West and Li, 2019; West and Kreuter, 2018; West and Kreuter 2015; West et al. 2014, JSSAM; West 2013, Improving Surveys with Paradata; and two chapters in the 2018 Palgrave Handbook of Survey Research), and I recently completed an NIH R03 project (of which I was the sole PI) examining effective strategies for collecting interviewer observations in health surveys (leading to two of these recent publications).

This R03 project was motivated by my findings of unexplained variability among interviewers in the quality of these observations. I lead a team of three graduate students and two undergraduate students in coding and analyzing more than 50,000 open-ended justifications provided by NSFG interviewers for observations that they have been tasked with recording. The results of this work present strong evidence in support of the theory-driven hypotheses that 1) interviewers will vary in terms of the observational strategies that they use in the absence of standardized training on this task, and 2) strategies that rely on multiple cues will improve the quality of the observations. These findings were presented at the Annual Conference of the American Association for Public Opinion Research (AAPOR), and additional results were presented by one of my graduate students (Dan Li) at the Joint Statistical Meetings (JSM). This work has identified strategies that lead to improved and reduced accuracy in the quality of commonly collected interviewer observations (e.g., estimates of response propensity), and the two articles describing these results have been published in Sociological Methods and Research and Methodology.

In the future, I hope to experimentally evaluate the potential of standardized training based on the empirical evidence of effective observational strategies that I have discovered to reduce interviewer variability in the quality of these observations. Given my dissertation work suggesting that high-quality observations can improve the quality of nonresponse adjustments to survey estimates, I hope to demonstrate that standardized training can ultimately improve the quality of these paradata, providing survey organizations internationally with practical guidance on the collection of these observations. I also hope to perform a comprehensive international investigation of the different training strategies used by survey organizations to collect these observations, and synthesize those strategies that are found to produce observations of the highest quality. My work in this area has informed the field of survey methodology with regard to the utility of these interviewer observations for survey estimation, and also alerted survey researchers to the potential pitfalls associated with the use of reduced quality observations in estimation.

Finally, I am passionately interested in the role that post-survey interviewer observations can play in improving survey estimates and providing an indication of the quality of the survey response process. I am currently examining the ability of these observations to identify survey responses of poor quality in five major health surveys internationally. These types of observations are routinely collected in different surveys at no small cost, but the literature has not benefitted from any systematic investigations of their potential to indicate response accuracy, and the observations are almost never analyzed (regardless of the survey). Together with my co-investigators, I hope to develop a standardized methodology that will enable survey organizations to provide the public with data quality indicators for each survey respondent (based on these post-survey observations) in a public-use data file. Survey data users would then be able to examine the sensitivity of their analyses to exclusion of cases reported by the interviewers to have reduced quality. We have a chapter describing this work in the 2020 edited volume Interviewer Effects from a Total Survey Error Perspective (Chapman Hall / CRC Press).

Interviewer Effects. Human interviewers play an essential role in ensuring that the survey data collected in interviewer-administered studies are of high quality, regardless of whether the data collector uses face-to-face or telephone interviewing. My doctoral training also focused on the effects that interviewers can have on the quality of survey data using a Total Survey Error (TSE) perspective, and how interviewers can introduce errors at all different phases of the survey process if they are not carefully trained. To date, the survey methodology literature has largely focused on measuring and reducing the variance in systematic measurement errors introduced by different interviewers in interviewer-administered surveys. I have advanced our understanding of the effects that interviewers can have on multiple sources of error simultaneously (e.g., nonresponse error and measurement error; see West et al., 2020, JSSAM; West et al., 2018, JSSAM; West et al., 2018, JRSS-A; West, Kreuter and Jaenichen 2013, JOS; and West and Olson, 2010, POQ), contributed statistical methods for examining interviewer effects on multiple sources of survey error simultaneously, and developed methods for comparing independent groups of survey interviewers in terms of the effects that they are introducing (West, 2020; West and Elliott, 2014).

My initial work in this area led to a four-year NSF-funded project of which I was the PI, looking at the interviewer effects on nonresponse error and measurement error introduced by different interviewing techniques (i.e., conversational vs. standardized interviewing). I managed an original national data collection in Germany for this project that finished in late 2014, and my team of research faculty and graduate students analyzed the survey data and linked the survey responses with administrative data provided by the German government. Initial results from this work were also presented at AAPOR 2015 (where two papers were presented by my team), and these results were also presented at the 2015 conference of the European Survey Research Association and the 2015 JSM. Three papers describing results from this research were accepted for publication by JRSS-A (West et al., 2018), Field Methods (Mittereder et al., 2017), and the Journal of Survey Statistics and Methodology (West et al., 2018). This research is the first to provide the field of survey methodology with a comparison of the interviewer effects introduced by these two interviewing techniques for a variety of survey measures, and also decomposes the total interviewer variance introduced by these techniques into sampling error variance, nonresponse error variance, and measurement error variance among interviewers. With this line of research, I hope to break down barriers between different organizations committed to using only one interviewing technique, and introduce the possibility of using different techniques for different types of questions. During this ongoing research, I also first-authored an ambitious synthesis of the vast literature examining interviewer effects on various survey outcomes (with Dr. Annelies Blom), using the TSE paradigm. This synthesis was published in the Journal of Survey Statistics and Methodology.

In addition to the work described above, there are many directions in which I hope to extend the literature on interviewer effects. First, nearly all of the literature on interviewer effects has considered descriptive statistics. I have received promising feedback on two initial proposals submitted to NSF outlining a program of research examining interviewer effects on regression coefficients. Almost no work has been done in this area, despite the widespread interest of survey analysts in fitting regression models to survey data. Together with a PhD student in our program (Micha Fischer), we have had a manuscript published (in JSSAM) outlining a framework for the study of interviewer effects on regression coefficients. I also hope to research the mechanisms underlying interviewer effects on self-administered responses in surveys that involve a mix of face-to-face data collection and self-administration (for more sensitive questions). My initial work in this area (West and Peytcheva 2014) suggests that selected interviewer behaviors during self-administration can play a critical role in shifting response distributions, and that interviewers vary widely in the behaviors that they use during self-administration; I feel that methods based on virtual reality technology may prove useful here. Finally, the literature on interviewer effects is still devoid of any practical methods for estimating interviewer variance in the presence of non-interpenetrated samples (where interviewers are not assigned random samples of the full sample), and models for simultaneously estimating interviewer effects on nonresponse and measurement error. Together with my colleague Dr. Michael Elliott, I have developed and evaluated an "anchoring" methodology for this estimation problem that is currently under review at Survey Methodology.

Modular Survey Design. While I am truly passionate about the importance of advancing knowledge about interviewer observations and interviewer effects in surveys, I acknowledge that many surveys are presently taking steps to shift away from face-to-face or telephone formats for cost and efficiency reasons (and not necessarily data quality reasons). Given this slow but undeniable trend, I have recently started a line of research into modular survey designs using mobile devices (West, Ghimire and Axinn 2015), where the survey task is split into multiple modules, and respondents do not answer an entire survey in one sitting. This approach takes full advantage of new technologies used for web survey data collection, and there are countless promising directions for this research, which has the potential to make the survey response task easier for a larger number of people. I was very excited to recently have an initial paper in this area accepted for publication in a high-impact methodological journal (Survey Research Methods).

We currently have an NIH-funded R01 project in the field collecting family and fertility data, using a strictly web/mail approach applied to a national sample (the American Family Health Study). As part of this exciting new project, we are experimentally evaluating the effectiveness of a modular design approach to breaking larger surveys up into smaller modules, especially when using web and mail methods to improve cost efficiency. In addition, I'm currently working with Ipek Bilgen at NORC and Mick Couper at SRC to design an experiment that would evaluate the effectiveness of a modular web-based approach to collecting data on a variety of topics from a probability panel. Do participants find this approach easier and less burdensome? Does it produce high-quality data?

Responsive Survey Design and Paradata. Survey methodologists have developed a collection of survey management techniques known as Responsive Survey Design (RSD) in an effort to better understand uncertain sources of survey errors and survey costs, and make survey data collections more efficient both statistically and financially. These techniques rely on the use of survey paradata, and I have contributed several studies to the field of survey methodology evaluating the role of paradata (e.g., West and Sinibaldi 2013) in improving the practice of RSD and improving the quality of the survey process more generally. These contributions have focused on the use of paradata to make survey designs and survey estimation more efficient (Wagner et al. 2020, JOS; Coffey et al. 2020, MDA; Hu et al., 2020, SRM; Krueger and West 2014, POQ; Wagner et al. 2012), and provided the field with several practical techniques for interviewer monitoring (West and Groves 2013) and effective implementation of RSD ideas.

While these studies have advanced our understanding of the utility of paradata for improving RSD and survey estimation, there have been almost no advances in the statistical science of RSD since its original demonstration in 2006 by two of my colleagues. Many RSD decisions are essentially made in an ad-hoc fashion by survey organizations, and there is much work to be done in this area. Together with my colleagues Dr. James Wagner, Dr. Michael Elliott, and Stephanie Coffey (a PhD student at JPSM), I am working to develop a more systematic enhancement of RSD techniques with Bayesian methodologies, essentially learning from prior outcomes in a data collection to improve future data collection decisions. These methodologies represent a natural fit for the core ideas of RSD, and we are excited about the possibility of enhancing the science of RSD with standardized Bayesian approaches that survey organizations can use to learn from prior experiences. We have been awarded an R01 grant from NIH to conduct this research, and we have already two manuscripts accepted for publication (Wagner et al. 2020, JOS; Coffey et al. 2020, MDA) with a third evaluating the ability of Bayesian approaches to improve predictions of daily response propensity recently receiving an R&R decision from JSSAM.

I am also sincerely interested in disseminating knowledge about advances in the science and implementation of RSD to other fields outside of survey methodology, with the ultimate goal of improving the efficiency of data collections both large and small in applied fields. With this objective in mind, I collaborated with Dr. Wagner to develop an NIH R25 proposal outlining a research education program on RSD. This proposal was funded by NIH, and in July of 2017 we offered our first three courses on RSD as part of the ISR Summer Institute in Survey Research Techniques. This R25 grant will fund a five-year education program that will eventually become a self-sustaining part of the Summer Institute, and I strongly believe that this program will lead to additional advances in the science of RSD. Please visit the RSD Program web site for more information!

Multilevel Modeling and Analysis of Complex Sample Survey Data. Many of my research contributions to the statistical sciences have been in the areas of multilevel modeling and the analysis of complex sample survey data. Largely motivated by my extensive history of teaching and consulting on analytic techniques that are often required but seldom understood, I have first-authored a book that is now in its second edition on the use of alternative statistical software procedures for fitting linear mixed models (West, Welch, and Galecki 2014). The first edition of this book (West, Welch, and Galecki 2006) was nominated for the 2009 Ziegel Prize for technical writing by the journal Technometrics, and has been cited nearly 2,300 times to date (Source: Google Scholar). Working on the second edition has required me to continually understand and inform the literature with regard to state-of-the-art developments in statistical software (West et al. 2015, AJPH; Galecki and West 2013; West and Galecki 2011; West 2009), and in 2021 we began work on a third edition of this book that will provide even more state-of-the-art guidance on tools for fitting and interpreting these models.

Similar motivation led to my co-authoring a second book concerning applied approaches to design- and model-based analyses of complex sample survey data, which is also now in its second edition (Heeringa et al. 2017). I feel that this is an area where most graduate students and faculty members do not receive sufficient training, and this book reviews the relevant literature and describes best practices with regard to the analysis of survey data with a more applied audience in mind. I have written numerous collaborative articles with other colleagues applying these analytic approaches to large complex sample survey data sets, and these collaborations have led to methodological contributions as well (e.g., West and McCabe 2012). In the future, I hope to continue researching unique applications of multilevel modeling and developing advanced analytic techniques for complex sample survey data (e.g., Westgate and West, 2021; West, 2019; Heeringa et al., 2015; Raykov, West, and Traynor 2015).

I strive to ensure that researchers everywhere are using these techniques correctly (e.g., West, Sakshaug, and Aurelien, 2016, PLoS ONE; Sakshaug and West, 2014), and a research program arising from these efforts (and funded by NSF) has examined the prevalence of analytic errors in published secondary analyses of complex sample survey data. I have guided one large meta-analysis of these analytic errors to date that was presented at an international workshop on Total Survey Error and was published as a chapter in the aforementioned edited volume on TSE (West, Sakshaug, and Kim 2017). This NSF award has funded an additional meta-analysis examining the prevalence of analytic error in published analyses of data provided by the National Center for Science and Engineering Statistics (NCSES), a synthesis of the literature in survey statistics on appropriate model-based and design-based approaches to the analysis of survey data, and sensitivity analyses of NCSES data sets to examine the inferential risks of ignoring complex sampling features. We recently had much of our work in this area accepted for publication in a PLoS ONE article, the Journal of Official Statistics, and Survey Methods: Insights from the Field. With this research, I hope to add analytic error to the TSE paradigm as an important source of error affecting the quality of published survey data. I also hope to contribute much of what I have learned in these areas as a member of the Bureau of Labor Statistics Technical Advisory Committee and as an affiliated faculty member of the Michigan Institute for Data Science (MIDAS).

Substance Abuse, Mental Health, and Medical Care Research. I have been a co-Investigator on several NIH-funded research projects with prominent scholars in the epidemiology of substance abuse (e.g., McCabe), mental health (e.g., Gonzalez), and medical care (e.g., Kullgren). These substantive investigations have taken full advantage of my aforementioned methodological contributions to multilevel modeling and the analysis of complex sample survey data, and my numerous contributions in these three areas have each addressed important research gaps. Most recently, Dr. McCabe and I had an R01 proposal funded that will examine nonresponse bias in longitudinal surveys, and we have already had multiple methodological articles accepted for publication from this work (e.g., West and McCabe 2017, AJE). I look forward to continuing my collaborative work in these areas, and I am always open to ideas for collaborative proposals.

Last modified 1/26/21 by Brady T. West