Can Fox News Viewers Change After Watching CNN?

Source: Neva79/Pixabay

Two researchers recently documented that some Fox News viewers can be persuaded by CNN (Broockman and Kalla, April 1, 2022). A stereotype of a Fox News viewer is that they may be closed-minded and only watch Fox News, which is probably part of why so many outlets are reporting on this research. It’s become a big deal if any partisan can accept information from outside their echo chamber. Perhaps there’s hope for our polarized politics to improve.

Initial Disclaimers or Clarifications

The researchers wrote that they didn’t mean to pick on the Fox News viewer. They explained that Donald Trump was president at the time and if a Democrat were president, they would’ve studied the CNN viewer. And it’s only a stereotype that the Fox News viewer only watches Fox News. There are of course many Fox News viewers who are already open to other news sources.

Unfortunately, the research has apparently not yet been published in an academic journal. It is a “working paper” that is “under peer review” (Aleem, 2022). The authors simply uploaded their paper to an open-source website. The academic peer-review process is not bias-free, but it is a vital part of science and can improve the reliability of research-based conclusions in the long run.

The two researchers’ universities have strong reputations which can possibly speak to the credibility of the results, but without peer review, there’s also a risk that readers and journalists can be overly accepting in a form of the halo effect or appeal-to-authority fallacy. And my continuing the hype about this unpublished research may contribute to a growing bandwagon fallacy.

Don’t get me wrong. The research seemed extremely well done in several ways, but there were also limitations and unknowns that I will mention. These caveats, barely if at all mentioned in other public reporting, suggest caution in going along with the many headlines. That said, the researchers reported several interesting results.

Results

Source: paulbr75/Pixabay

Compared to the control group, the Fox News viewers who were paid to watch CNN (at least a few hours over a few weeks) were more likely to recall negative information about Donald Trump, to doubt extreme negative statements about Joe Biden, to value the issues that CNN covered, to support protests regarding police shootings of Black men, to support voting by mail, and to acknowledge that Fox News avoids reporting “bad” behavior by Donald Trump. The paid CNN viewers also held slightly less positive views (though still positive) toward Donald Trump, Republican politicians, and Fox News itself.

There was no increase in positive views of Joe Biden or CNN. There were no effects found regarding general media attitudes, trust in CNN, trust in Fox News, or attitudes about race, climate change, or policing.

Compared to the control group, paid CNN viewers reported two months after the payments stopped that they still watched a little more CNN each week (by about 15 minutes). Many (but not all) of the other initial changes disappeared after two months.

Limitations and Unknowns

The proportion of Fox News viewers who participated, among those who were initially contacted, was less than 1 percent. Even after the researchers limited the pool to those with the relevant viewing habits, still less than 14 percent of that group became actual participants, partly because they were the ones willing to watch enough CNN. So the results may only generalize to the more open-minded Fox News viewers (a selection bias). And of course in real life, individuals won’t usually be paid to try different channels.

How the researchers randomly assigned some Fox News viewers to the paid condition (only 4 in 10) was not perfectly clear. A peer reviewer would be justified to ask for more details on the procedure as it is crucial to draw cause-and-effect conclusions.

Although some of the reported changes from watching CNN were statistically significant (meaning unlikely to have occurred by chance), the sizes of the results were “small” in statistical terms, despite multiple outlets saying that Fox News viewers “changed their minds” or were “transformed.” What “small” even means in practical item-by-item terms was very difficult to discern because the analyzed outcome measures usually comprised combinations of numerous items (and because changes were usually reported in standard deviations and not original units).

President Donald Trump Essential Reads

Golf, Integrity, and Presidents Past

Does Trump Believe His Own Lies?

[Fair warning: The next paragraph is a bit mathematical.]

Some journalists have identified single items’ raw data from the paper’s appendices, as when Yglesias (2022), and multiple others who cited Yglesias, stated that paid CNN viewers scored 13 percentage points lower than the control group in agreement with an extreme negative statement about Biden. Such reporting is interesting but not very meaningful in itself because only the combined outcome measures (and not single items) showed statistically significant changes. Also, the percentages were provided by the researchers only after dividing participants into two groups (agree or neutral/disagree) from a scale that seemed to have five response options. Thus, even a 13-point swing could technically arise from a mean difference between, say, 2.39 and 2.52 on a 1-to-5-point scale where 1 = “strongly agree” and 5 = “strongly disagree.” Such a difference represents about 3 percentage points (not 13) on the original 5-point scale.

In Sum

Overall, Broockman and Kalla did an extraordinary job compiling data from hundreds of Fox News viewers in methodologically and statistically sound approaches. It was quite an undertaking that addressed issues beyond those reported here. Nonetheless, the caveats make it hard to draw firm conclusions at this time, and many public reports overstate the outcomes. Among the significant findings, the attitudinal shifts from watching CNN were most likely very small. But very small is better than nothing in today’s divisive politics—let’s work with that and continue to try to broaden voters’ and our own sources of information.