By Kristen Olson (University of Nebraska– Lincoln)

Data are everywhere. Data are collected as we browse the internet, purchase items from stores, pay our taxes, go to the doctor’s office, enroll our children in school, and travel using a car or public transportation. But this outwardly observable information reflects only a small fraction of what humans do in a day, largely fails to reflect what we think and feel about our lives, surroundings, and futures, and does not capture why we think, feel, or do these activities. What is a researcher or policy maker to do when they need information that cannot be observed through records of people’s behavior? Long before digital traces of our lives were created to track everyday interactions, social, behavioral, and economic scientists used surveys to ask people about activities, opinions, and knowledge not easily found in records.

“Long before digital traces of our lives were created to track everyday interactions, social, behavioral, and economic scientists used surveys to ask people about activities, opinions, and knowledge not easily found in records.”

Survey questions can capture information about a population that cannot be known in any other way. Surveys conducted by the federal statistical system allow insights into whether someone is currently looking for work, but not employed, uses a doctor’s office versus urgent care clinic for health care, and even information about who lives in a household. Administrative data systems can be useful for understanding the quality of survey data, as some information is more reliable when found in records than survey reports. But survey data also provide insights into the quality of administrative data systems. For instance, while some might look to crime statistics based on police reports, less than half of violent crimes are reported to the police and police reporting varies dramatically across types of crimes and background characteristics of people. This information cannot be obtained from police records alone; we can only get it by asking people both about their victimization experiences and their reporting behavior. Even those who call for replacing or augmenting survey data, with administrative or commercial records or digital traces, typically require high-quality probability-based surveys for statistical adjustments. Probability-based surveys also are the benchmarks for evaluating the quality of estimates from administrative, commercial, and digital data.

Beyond the federal statistical system, surveys conducted by academic and other researchers allow insights into what people feel, think, and do on a broad array of social, economic, and political topics. Federally-funded state-coordinated surveys such as the Behavioral Risk Factor Surveillance System and Youth Risk Behavior Surveillance System have provided national policy makers and state health departments with detailed representative health data about adults and adolescents across the country for many years. Repeated cross-sectional surveys such as the General Social Survey and the American National Election Studies have provided insights into the attitudes and behaviors of adults living in the United States for more than half a century. The Panel Study of Income Dynamics, the National Longitudinal Survey of Adolescent to Adult Health, the Wisconsin Longitudinal Study, the Understanding America Study, the National Social Life, Health, and Aging Project, and the Health and Retirement Study follow the same person, and often multiple people connected to that person, over decades, permitting unparalleled insights into how early life experiences affect later life economic, health, and social outcomes. These studies form the basis for policy recommendations and decisions, are used in tens of thousands of peer-reviewed publications, have been used to train countless undergraduates and graduate students, and have contributed to the development of countless scholars across disciplines. Beyond these surveys, academic survey research centers support excellence in research and evaluation for scholars, administrators, and state and local policy makers by collecting timely local data on populations that are not covered in these larger studies.

“Supporting our knowledge about who we are and what we do as a society requires an interdisciplinary approach, working jointly across social, behavioral, and economic sciences, statistical sciences, information sciences, and computer sciences.”

All these surveys feed into a national data infrastructure system. Data infrastructure – the equivalent of roads and bridges, providing fundamental knowledge about the throughway of society – is built and maintained through efforts of the federal statistical system, academic researchers, nonprofits, and private companies. Probability-based surveys, allowing each member of a target population a known nonzero probability of being included in the study, form the backbone of this infrastructure of data about people, businesses, and farms in the United States. This infrastructure is crumbling. Through reduction of budgets and staff in federal statistical agencies, declining trust in science and other institutions, falling response rates, increasing survey costs, changes in frames and modes of data collection, and proposed elimination of the Social, Behavioral, and Economic Sciences Directorate at the National Science Foundation, what we know about people, businesses, farmers, hospitals, patients, schools and families is at extreme risk. And this is separate from the risk from bogus respondents who make up characteristics to qualify for paid nonprobability online surveys and AI-generated “silicon samples” that have little to no link to actual human respondents.

As a survey methodologist and the director of an academic survey research center, my research on the science of surveys – the original data science discipline – supports the data infrastructure of the state in which I live and the United States as a whole. Supporting our knowledge about who we are and what we do as a society requires an interdisciplinary approach, working jointly across social, behavioral, and economic sciences, statistical sciences, information sciences, and computer sciences. This approach must start by asking well-designed questions to members of the populations we want to study, understand, and support.

Kristen Olson, Ph.D. is the Leland J. and Dorothy H. Olson Professor in Sociology and Director of the Bureau of Sociological Research at the University of Nebraska-Lincoln. Dr. Olson’s research focuses on mixed-mode surveys, survey costs, questionnaire design, interviewer effects, the intersection of nonresponse and measurement errors, and within-household selection in self-administered surveys. She is a fellow of the American Statistical Association, the American Association for the Advancement of Science, and the Midwest Association for Public Opinion Research. The views expressed here are her own, and do not represent her employer, the University of Nebraska-Lincoln.

Download as PDF