In 2020, Pew Research Center launched a new project called the National Public Opinion Reference Survey (NPORS). NPORS is an annual, cross-sectional survey of U.S. adults. Respondents can answer by paper, online or over the phone, and they are selected using address-based sampling from the United States Postal Service’s Computerized Delivery Sequence File. The response rate to the latest NPORS was 32%, and previous years’ surveys were designed with a similarly rigorous approach.
NPORS estimates are separate from the American Trends Panel (ATP) – the Center’s national online survey platform. Pew Research Center launched NPORS to address a limitation that researchers observed in the ATP. While the ATP was well-suited for the vast majority of the Center’s U.S. survey work, estimates for a few outcomes were not in line with other high-quality surveys, even after weighting to demographics like age, education, race and ethnicity, and gender.
For example, in 2018, roughly one-quarter of U.S. adults were religiously unaffiliated (i.e., atheist, agnostic or “nothing in particular”), according to the General Social Survey (GSS) and the Center’s own telephone-based polling. The ATP, however, estimated the religiously unaffiliated rate at about 32%. The Center did not feel comfortable publishing that ATP estimate because there was too much evidence that the rate was too high, likely because the types of people willing to participate in an online panel skew less religious than the population as a whole. Similarly, the ATP estimate for the share of U.S. adults identifying as a Democrat or leaning to the Democratic Party was somewhat higher than the rate indicated by the GSS and our own telephone surveys.
From 2014 to late 2020, the Center approached these outcomes slightly differently. We addressed the political partisanship issue by weighting every ATP survey to an external benchmark for the share of Americans identifying as a Republican, Democrat or independent. For the benchmark, we used the average of the results from our three most recent national cellphone and landline random-digit-dial (RDD) surveys.
During this time period, ATP surveys were not weighted to an external benchmark for Americans’ religious affiliation. The ATP was used for some research on religious beliefs and behaviors, but it was not used to estimate the overall share of Americans identifying as religiously affiliated or unaffiliated, nor was it used to estimate the size of particular faith groups, such as Catholics, Protestants or the Church of Jesus Christ of Latter-day Saints. NPORS allows us to improve and harmonize our approach to both these outcomes (Americans’ political and religious affiliations).
Design and estimates
Read our fact sheet to find the latest NPORS estimates as well as methodological details. Data collection for NPORS was performed by Ipsos from 2020 through 2023 and is now performed by SSRS.
Why is the NPORS response rate higher than most opinion polls?
Several features of NPORS set it apart from a typical public opinion poll.
- People can respond offline or online. NPORS offers three different ways to respond: by paper (through the mail), online, or by telephone (by calling a provided phone number and speaking to a live interviewer). The paper and telephone options bring in more conservative, more religious adults who are less inclined to take surveys online.
- Monetary incentives. When sampled adults are first asked to respond to NPORS online, the mailing contains a $2 incentive payment (cash visible from the outside of the envelope) and offers a $10 incentive payment contingent on the participant completing the survey. When nonrespondents to that first stage are sent the paper version of the survey, the mailing contains a visible $5 bill. These incentives give people a reason to respond, even if they might not be interested in the questions or inclined to take surveys in general.
- Priority mailing. The paper version of the survey is mailed in a USPS Priority Mail envelope, which is more expensive than a normal envelope, signaling that the contents are important and that the mailing is not haphazard. It helps people distinguish the survey from junk mail, increasing the likelihood that they open and read what is inside.
- Low burden. The NPORS questionnaire is intentionally kept short. It’s about 40 questions long, including demographics such as age, gender and education. This means that NPORS takes about seven minutes to finish, while many polls take 10 minutes or longer.
- Bilingual materials. In parts of the country with sizable shares of Hispanic Americans, the materials are sent in both English and Spanish.
- No requirement to join a panel. NPORS respondents are not required to join a survey panel, which for some people would be a reason to decline the request.
These features are not possible in most public polls for a host of reasons. But NPORS is designed to produce estimates of high enough quality that they can be used as weighting benchmarks for other polls, and so these features are critical.
Why a ‘reference’ survey for public opinion?
The “R” in NPORS stands for “reference.” In this context, the term comes from studies in which researchers calibrate a small sample survey to a large, high-quality survey with greater precision and accuracy. Examples of reference surveys used by researchers include the Census Bureau’s American Community Survey (ACS) and the Current Population Survey (CPS). NPORS is not on the scale of the ACS or CPS, nor does it feature face-to-face data collection. But it does have something that those studies lack: timely estimates of key public opinion outcomes. Other studies like the American National Election Survey (ANES) and the General Social Survey collect key public opinion measures, but their data is released months, if not years, after data collection. The ANES, while invaluable to academic researchers, also excludes noncitizens who constitute about 7% of adults living in the U.S. and are included in the Center’s surveys.
NPORS is truly a reference survey for Pew Research Center because researchers weight each American Trends Panel wave to several NPORS estimates. In other words, ATP surveys refer to NPORS in order to represent groups like Republicans, Democrats, religiously affiliated adults and religiously unaffiliated adults proportional to their share of the U.S. population. The ATP weighting protocol also calibrates to other benchmarks, such as ACS demographic figures and CPS benchmarks for voter registration status and volunteerism.
Pew Research Center is weighting on political party affiliation, but isn’t that an attitude?
It’s correct that whether someone considers themselves a Republican or a Democrat is an attitude, not a fixed characteristic, such as year of birth. But there is a way to weight on political party affiliation even though it is an attitude and without forcing the poll’s partisan distribution to align with a benchmark.
Pew Research Center started implementing this approach in 2021. It begins with measuring the survey panelists’ political party affiliation at a certain point in time (typically, each summer). Ideally, the reference survey will measure the same construct at the same point in time. We launched NPORS because we control its timing as well as the American Trends Panel’s timing, allowing us to achieve this syncing.
NPORS and ATP measurements of political party are collected at approximately the same time each summer. We may then conduct roughly 25 surveys on the ATP over the next year. For each of those 25 surveys, we append the panelists’ party affiliation answers from the summer to the current survey. To illustrate, let’s say that a survey was conducted in December. When researchers weight the December ATP survey, they take the measurement of party taken in the summer and weight that to the NPORS estimates for the partisan distribution of U.S. adults during the summer time frame. If, for example, Democrats were more likely than Republicans to respond to the December survey, the weighting to the NPORS target would help reduce the differential partisan nonresponse bias.
Critically, if the hypothetical December poll featured a fresh measurement of political party affiliation (typically asked about three times a year on the ATP), the new December answers do not get forced to any target. The new partisan distribution is allowed to vary. In this way, we can both address the threat from differential partisan nonresponse and measure an attitude that changes over time (without dictating the outcome). Each summer, the process starts anew by measuring political party on the ATP at basically the same time as the NPORS data collection.
Is the NPORS design connected to the American Trends Panel?
A key feature of NPORS is that respondents are not members of a survey panel. It is a fresh, random sample of U.S. adults. This matters because some people are willing to take a onetime survey like NPORS but are not interested in taking surveys on an ongoing basis as part of a panel. That said, in certain years, NPORS serves as a recruitment survey for the ATP. After the NPORS questions, we ask respondents if they would be willing to take future surveys. People who accept and those who decline are both part of the NPORS survey. But only those who consent to future surveys are eventually invited to join the ATP.
Can other survey researchers use NPORS?
Yes. As a nonprofit organization, we seek to make our research as useful to policymakers, survey practitioners and scholars as possible. As with the Center’s other survey work, the estimates and data are freely available.
