Polls Policy And FAQs

At FiveThirtyEight, we strive to accumulate and analyze polling data in a way that is honest, informed, comprehensive and accurate. While we do occasionally commission polls, most of our understanding of American public opinion comes from aggregating polling data conducted by other firms and organizations. This data forms the foundation of our polling averages, election forecasts and much of our political coverage.

In building our polling database, we aim to be as inclusive as possible. This means we will collect any poll that has been made publicly available and meets a few basic standards:

1. The poll must include the name of the pollster, survey dates, sample sizes and details about the population sampled. If these are not included in the poll’s release, we must be able to obtain them in order to include the poll.
2. Pollsters must also be able to answer basic questions about their methodology, including but not limited to the polling medium used (e.g., landline calls, text, etc.), the source of their voter files, their weighting criteria, and the source of the poll’s funding.

However, there are some types of polls we don’t include, such as:

1. “Nonscientific” polls that don’t attempt to survey a representative sample of the population or electorate.
2. Polls that blend or smooth their data using methods like MRP (short for “multilevel regression with poststratification”). While this is a valid technique for understanding public opinion data, we exclude these polls because we consider them more like models than individual polls. (As an analogy, we think of this as using someone else’s barbecue sauce as an ingredient in your own barbecue sauce.)
3. DIY polls commissioned by nonprofessional hobbyists on online platforms like Google Surveys or SurveyMonkey. (Professional or campaign polls using these platforms are fine.)
4. Subsamples of multistate polls are not treated as individual polls of those states unless there is some method employed to verify the geographic location of the respondents and each state in the poll is weighted individually.
5. Polls that reveal leading information about the candidates before asking voters who they support. If, for instance, a poll says “Joe Biden loves puppies. Who do you plan to support: Biden or Trump?”, we won’t include it.

Polls that include hypothetical candidates — for instance, a poll testing a hypothetical three-way presidential race between Donald Trump, Joe Biden, and Liz Cheney — are included on our polls page, but are not included in our polling averages or models unless and until every politician in the question has declared their intention to seek the office in question and has had their name included on the ballot.

We do include internal and/or partisan polls that are publicly available, except in one unusual circumstance (a general election poll sponsored by a candidate’s rival in the primary¹). Polls are considered internal if they are conducted on behalf of a political party, campaign committee or other official party apparatus. Internal polls are noted with a hollow diamond next to the sponsor’s name. Polls are considered partisan if they’re conducted on behalf of any organization² that conducts a large majority of its political activity on behalf of one political party. Partisan polls are noted by a solid diamond next to the sponsor’s name.

Additionally, if we find that a sponsor organization is selectively releasing polls favorable to a certain candidate or party, we may also categorize that organization as partisan. We generally go out of our way to not characterize news organizations as partisan, even if they have a liberal or conservative view. But selectively releasing data that favors one party is a partisan action, and such polls will be treated as such. These classifications may be revisited if a sponsor ceases engaging in this behavior.

In rare cases where a pollster routinely conducts polls for partisan sponsors and we learn that it is not being transparent about sponsorship, we will count all of its polls as partisan. We also count all of a pollster’s polls as partisan if it is formally affiliated with a partisan organization; for instance, the Democratic Congressional Campaign Committee has an in-house polling operation that is always considered partisan. These cases are noted with a solid diamond next to the pollster’s name.

Polls we suspect are fake will also not be included until we conduct a thorough investigation and can confirm their veracity. We will permanently ban any pollster found to be falsifying data or engaging in betting markets that may be directly impacted by their survey work. We reserve the right to ban polls sponsored by any organization that consistently engages in dishonest or nontransparent behavior that goes beyond editorializing and political spin.

Our guidelines for inclusion are intentionally permissive. We aim to capture all publicly available polls that are being conducted in good faith. While some polls have a more established track record of accuracy than others — and we do take that into account in our models and political analysis — we exclude polls from our dataset only in exceptional circumstances.

Below are some questions we’ve been asked often over the years about the types of polls we collect. Take a look, and if you still have questions or find a poll we don’t have, please email us at polls@fivethirtyeight.com.

Frequently Asked Questions

Q: Which races do you collect polls for?

A: We collect polls for presidential, Senate, House and gubernatorial general elections in addition to presidential approval polls, vice presidential approval polls and congressional generic ballot polls at the national level. At this time, we do not collect primary polls other than for the presidency — except in cases of a “jungle primary,” as it’s possible for a candidate to win the seat outright. We also collect favorability polls for certain notable politicians.

Q: Why don’t you have any polls of the race I’m interested in?
A: The latest polls page includes all polls publicly released within two years of an election, beginning with the 2018 election cycle. If we don’t have any polls for a particular race, that means we weren’t aware of any polls for that race.

Q: Can I download this data?

A: Yes! There is a dropdown menu containing all our polling datasets at the bottom of the latest polls page, but you can also download this data and more from our data repository, which includes all our polls, forecasts and other data projects. Unfortunately, we are not able to share data on presidents’ approval ratings before Donald Trump. You can find additional information on historical presidential approval ratings, and guidelines for acquiring that dataset, on the Roper Center’s website.

Q: How do you account for a pollster that publishes multiple results for a question?

A: When a pollster publishes multiple subsamples (for example, all adults, only registered voters, and only likely voters), we include all of them in our database. Similarly, if a pollster asks a horse-race question with different sets of candidates — for example, with and without a third-party candidate — we include all versions of this question in our database. If the pollster includes multiple likely voter models, as in this Monmouth poll, we first check if the pollster indicates that one of the versions is its preferred option. If it does, we use that version; otherwise, we include them all in our database. Our election models take these multiple versions into account, preferring likely voters over other subsamples and averaging results in the case of multiple topline questions. Our approval and favorability models work similarly, though these models prefer questions among all adults over those among likely or registered voters.

Q: How do you account for a pollster that publishes numbers with and without “leaners”?

A: When a pollster publishes one version of a question with “leaners” — respondents who may be uncertain about their vote but say that they lean toward a particular candidate or party — and one without, we include only the version with leaners. If the question including leaners is a “forced-choice” question, in which respondents are not given an option of saying they are undecided when asked which way they lean, we still include only that version of the question instead of the version without leaners.

Q: What do the pollster grades mean?

A: We calculate a grade for each pollster by analyzing its historical accuracy and methodology. You can read our full ratings and methodology to better understand how we calculate the grades.

Q: What does it mean when a pollster has a rating like “A/B” and a dotted circle around the rating?

A: For pollsters with a relatively small sample of polling within three weeks of an election, we show a provisional rating (“A/B,” “B/C,” or “C/D”) rather than a precise letter grade and use a dotted circle to emphasize that the rating is provisional.

Q: Why do some pollsters not have a grade?

A: For some pollsters, we do not have any polls from the last three weeks of an election cycle, which means we can’t evaluate their historical performance. We do include polls from these pollsters in our polling averages and models, but we are unable to assign them a rating based on their historical performance; therefore, they receive less weight in our averages and models.

Q: Do you weight or adjust polls?

A: Yes. When we calculate our polling averages, some polls get more weight than others. For example, polls that survey more people or have a historical track record of accuracy get more consideration in calculating the average than polls with small sample sizes or polls that have been historically less accurate. Our polling averages also apply adjustments for things like a pollster’s house effect (a measure for how consistently a pollster leans toward one party or candidate) or trends in polls from similar states. For more information on how we weight and adjust polls for polling averages, see this detailed explanation of our forecast methodology.

Q: When do you show third-party candidates on the latest polls page or in your polling averages?
A: We include third-party candidates in every poll that asks about them. To find those polls, enter the candidate’s last name in the search box on our latest polls page or use the drop-down menus to select the corresponding race. You may have to click “More” to see all candidates for a given question. Our polling averages include every candidate who has received at least five percent support in at least five different polls from at least three different pollsters.

Q: Why do some races not have a polling average?

A: In order to ensure a polling average has enough data to accurately represent the state of the race, we do not publish an average for a race until the major party candidates have been officially selected in primaries and we have collected at least five different polls from at least three different pollsters for that race. In addition, we may not have polling averages for races that use an instant runoff, as the candidates included in the questions may differ from pollster to pollster.

Q: Why are the sample sizes sometimes missing for polls?

A: If a poll does not have a sample size listed, the pollster or sponsor did not report it and we are actively working to obtain it. These polls are still included in our averages and models with an imputed sample size (400 for district-level polls, 600 for state-level polls and 1,200 for national polls) until we obtain the actual sample size.

Q: Why do the values in some polls add up to more than 100 percent?

A: Values in some polls may add up to more than 100 percent due to rounding. For example, if a pollster published a poll that gave the president an approval rating of 46.5 percent and a disapproval rating of 53.5 percent, then those numbers would round up to 47 percent and 54 percent, respectively.

Q: Why do the margins in some polls not match what the pollster reports?

A: This often boils down to rounding. For example, if a pollster puts one candidate at 45.2 percent and another at 45.6 percent, we’ll display these two candidates at 45 percent and 46 percent, respectively. That means we’ll display their margin as a difference of 0 even though the actual margin is 0.4.

This may also occur if the pollster publicizes a version of the question that differs from the version that FiveThirtyEight includes on our polls page. For example, a pollster may publicize a horse-race question that does not include leaners, but FiveThirtyEight publishes only the version that does include leaners.

Still have questions? Send us an email and we’ll do our best to sort it out.