Misreporting of Food Stamp Participation in a Household Survey: Results from a Single-State Pilot Study of the National Health and Nutrition Examination Survey NHANES
Data from household surveys play an essential role in monitoring and assessing the consequences of food assistance programs for the well-being of Americans. Compared to other sources—most notably administrative data—surveys like the National Health and Nutrition Examination Survey (NHANES) provide more information on individual and household characteristics such as food security, nutrition, and body weight, and these are characteristics likely to be affected by participation in food assistance programs. Thus, by using survey information on both program participation and household or individual characteristics (and statistical methods for handling non-response), surveys can provide powerful evidence on the impacts of food assistance programs.
The utility of survey data for evaluating food program outcomes is diminished by respondent misreporting of participation in these programs. At least in principle, misreporting issues can be addressed by linking administrative records to household survey data. The potential benefits of such linkage are widely recognized, especially in social policy research. In practice, however, efforts at linkage encounter many problems, but experience is growing, and with gains in experience, more of the potential benefits are being realized.
The Texas NHANES Pilot (TNP) is part of a cooperative effort by the National Center for Health Statistics (NCHS), Centers for Disease Control and Prevention of the U.S. Department of Health and Human Services, the Economic Research Service (ERS) of the U.S. Department of Agriculture, and the Ray Marshall Center for the Study of Human Resources (RMC) at the University of Texas, Austin to link administrative records from the Food Stamp Program (FSP)1 in Texas to survey records from NHANES. The Ray Marshall Center determined if NHANES participants had records in the Texas food stamp program database under contracts. We conducted this study in one state as a pilot to learn more of what would be required to develop and implement match protocols appropriate to the NHANES should more general effort ever be contemplated. The immediate goals of this venture were to: (a) assess the feasibility of matching records from these disparate sources; (b) evaluate the accuracy of the match process; (c) use the matched data to estimate the prevalence and correlates of misreporting of FSP participation for this one-state subset of NHANES; and (d) assess the likely impacts of matching error on our estimates of misreporting. Ultimately, we sought to obtain information to improve analyses of food assistance programs and policies through more informed use of valuable information gathered during household surveys.
This paper presents results to date from the TNP. Part 1 briefly describes the match process used to link data from food stamp administrative files in Texas to NHANES participants for interviews conducted in Texas for survey cycles 2005 through 2008. Part 2 summarizes the results of that data linkage in terms of how often food stamp participation status as reported in the survey did or did not correspond to food stamp participation status as indicated in the administrative records. Part 3 provides a general discussion of our results. In our larger working paper (available from the authors) we provide more specifics about the linkage algorithm, misreporting using weighted estimates and logistic regressions models, creating a face validity scoring algorithm for sensitivity analyses of the matched results, and lessons learned through this linkage project. This paper serves as a short summary of our work and findings.
It is important to remember that, while the NHANES is designed to produce nationally representative estimates of numerous measures of the health and well-being of the non-institutionalized U.S. population, the results presented herein are based on the sample from one state and cannot be generalized to the U.S. population. The results of this pilot should be interpreted with caution and in light of our objectives.
Misreporting of Food Stamp Participation in a Household Survey: Results from a Single-State Pilot Study of the National Health and Nutrition Examination Survey NHANES” (with John A. Kirlin, Lisa Mirel, and Daniel Schroeder). Unpublished paper presented at the 2012 Federal Committee on Statistical Methodology (FCSM) Research Conference, Washington, D.C., January 10-12, 2012.