The Center for Media Engagement partnered with ProPublica to determine if news story headlines and images can be crafted in ways that appeal to both liberals and conservatives.
We found that Facebook ads using straightforward photographs, instead of illustrated graphics, performed better among both liberals and conservatives. However, changing the headlines in an attempt to appeal to different moral values decreased clicks among both groups.
The study also revealed that liberals shared the articles more often than conservatives, but share rates were very low overall.
The Problem
Many newsrooms aim to serve audiences with diverse political viewpoints. To this end, ProPublica, a nonprofit news organization focused on investigative journalism, wanted to understand how liberals and conservatives react to their articles. In a collaborative effort, the Center for Media Engagement and ProPublica looked at whether there are ways to write headlines and use images that attract people from across the aisle.
In this study, supported by the News Integrity Initiative at the Newmark J School, we tested articles to see which would appeal more to liberals than conservatives and vice versa. Using an online survey, we identified four articles on which liberals said that they were more likely to click. None of the articles we tested appealed significantly more to conservatives than liberals. Using these four articles, we developed new headlines based on research that provided general themes and specific language on what would appeal to different political audiences. We also tested whether using photographs, instead of illustrated graphics with symbolic meaning, could influence conservative and liberal interest. We conducted these tests using Facebook Sponsored Posts.
We then conducted another test to see if we would get similar results by targeting people based on their geographic location instead of by using Facebook’s targeting tool. This allowed us to evaluate whether Facebook’s targeting yields similar conclusions to a different targeting technique.
Key Findings
- Facebook advertisements for news articles featuring straightforward photographs, as opposed to illustrated graphics with symbolic meaning, performed better among both conservatives and liberals.
- Changing the headlines to try to appeal to people across the political spectrum decreased clicks among both conservatives and liberals.
- Liberals shared the articles more often than conservatives, but the share rates were very low overall.
- Using different strategies to target political audiences – whether Facebook’s categorization of those “likely to engage with (conservative/liberal) content” or geographic areas with lopsided voting records – yield similar results in terms of which articles garner the most clicks. There are differences, however, in how much better the various articles perform.
Implications for Newsrooms
Images have a big impact when it comes to Facebook clicks. When posting articles on Facebook, newsrooms should use straightforward photographs instead of illustrated graphics with symbolic meaning. Also, if newsrooms want to investigate how their content is performing among conservatives and liberals on Facebook, Facebook’s targeting yields similar conclusions compared to a county-based targeting strategy.
Reframing headlines to try to appeal to different moral values was not an effective way to affect interest among conservatives and liberals. Instead, it appeared to make the article less interesting to users of both political viewpoints. This isn’t to say that there may not be some headlines that generate more cross-partisan appeal and additional efforts to uncover these are warranted.
The Study
Testing Headlines and Images on Facebook
In partnership with ProPublica, we put together a list of articles we thought might be more interesting to liberals or conservatives. We tested the headlines on Amazon’s Mechanical Turk and found that liberals reported they were more likely than conservatives to click on four of the articles: an article about policing in Jacksonville, an article about hate crimes, an article about civil rights cases in education, and an article about the mortality rate of pregnant black women. None of the articles appealed significantly more to conservatives than liberals.
To explore ways of narrowing the interest gap between liberals and conservatives, we examined the headlines in context of moral foundations theory, which contends that conservatives and liberals prioritize different moral values. This research provided both general themes and specific language to help us develop new headlines that could affect conservative interest.
We also tested whether using more literal images could influence conservative and liberal interest. Three out of the four original articles featured a graphic with symbolic meaning in the header of the article. For the changed images, we selected straightforward photographs (For example: a photo of Betsy DeVos for the article about civil rights complaints submitted to the Department of Education).
We ran 16 ad campaigns on Facebook using their split test feature. The campaigns tested four versions of each article: an ad with the original headline and original image, an ad with the original headline and changed image, an ad with the changed headline and original image, and an ad with the changed headline and changed image. Half of the campaigns targeted conservatives and the other half targeted liberals using a Facebook ad feature that allowed us to select for users’ likelihood to engage with liberal or conservative content.
With few exceptions, conservatives and liberals clicked on the articles at similar rates. The gap that we had expected to see based on our test on MTurk was not reflected in the click rates on Facebook.
The most popular ads in terms of clicks were the versions with the changed image and original headline. These ads outperformed the ads with the original image and original headline in every case, across all four articles and both ideologies. This indicates that the new images consistently boosted clicks. The results suggest that while graphics may be visually appealing in other contexts, newsrooms should use straightforward photographs when running advertisements or posting articles on Facebook.
In contrast, rewriting the headlines with moral foundations theory in mind had a negative impact on click rates for both conservatives and liberals.
We also examined how often the articles were shared. Overall, the share rates were low for users of both ideologies. Liberals, however, shared all of the articles more often than conservatives, but changing the headline decreased their share rate. Conservatives were slightly more likely to share the article when it featured the changed image or both the changed image and the changed headline.
It isn’t clear from this study why conservatives and liberals click on the articles at similar rates, but liberals share the content more often. Future research could explore how conservative and liberal user engagement with Facebook ads differs and investigate users’ motivations for clicking on and sharing news articles on Facebook. It could be that in general, liberals are more likely to share content. Or it could be that there is something specific about this content that prompts higher levels of liberal sharing.
Testing Facebook’s Political Targeting Tool
We conducted a second test to evaluate whether we would arrive at similar results if we targeted people based on their geographic location instead of using Facebook’s targeting tool. For this test, we only used two out of the four articles (the article about civil rights cases and the article about policing in Jacksonville).
We ran four ad campaigns using Facebook’s “likely to engage with (conservative/liberal) content” feature, just as we did for the campaigns in the first part of this study (2 ads x 2 political views). At the same time, we ran four campaigns that targeted users living in an area where people voted overwhelmingly for either Donald Trump or Hillary Clinton in the 2016 presidential race. Based on the 2016 voting data, we pulled the 100 most politically homogenous counties from each party and targeted the campaigns to a random set of the zip codes from those counties.
Generally, the original targeting method and the zip code targeting method yielded similar results. As before, the article that featured the changed image and ProPublica’s original headline garnered the highest click rate among users of both ideologies.
Users who were targeted using Facebook’s political targeting tool clicked on all of the articles more often than those who were targeted by zip code. This is likely due to the fact that Facebook’s tool targets users by their likelihood to engage with content.
Methodology
For the initial test, 232 participants were recruited using Amazon Mechanical Turk during September of 2018. We tested 15 ProPublica articles in a Qualtrics-based online survey, all of which were formatted to appear as Facebook sponsored posts. Participants were asked to evaluate how likely they would be to click on the article on a scale of 1 (extremely likely) to 5 (extremely unlikely). Out of the 15 articles, four garnered significantly more liberal interest than conservative interest. None of the others produced differences by ideology. These are the four articles we used in the Facebook ad campaigns.
In collaboration with ProPublica, we rewrote the headlines and selected new images for the four articles. These revised versions were contrasted with the original version using the split test function on Facebook. The first eight ad campaigns ran from May 22, 2019 to May 24, 2019. Facebook users saw one of four versions of an ad: an ad with the original image and original headline, an ad with the changed image and original headline, an ad with the original image and changed headline, or an ad with the changed image and changed headline. All of the ads linked to the original article on ProPublica’s website.
We ran a second round of eight campaigns from July 13, 2019 to July 15, 2019. These campaigns were identical to the May tests. Before their completion, two of the campaigns were halted by Facebook due to a new requirement for ads related to politics to disclose who paid for them. We did not use the data from the interrupted campaigns in this report. We ran the two campaigns that had been interrupted again from July 20, 2019 to July 22, 2019. To comply with Facebook’s disclosure requirement for political ads, the ads in this test displayed the tag “Paid for by Center for Media Engagement.” The most popular ads in terms of clicks for both conservatives and liberals were consistent across the campaigns, whether the “paid for” label was present or not.
We targeted a random sample of U.S. zip codes for each ad campaign, so no Facebook user could see more than one article. We also excluded users who liked the ProPublica Facebook page, in case they were already familiar with the articles we were testing.
The eight campaigns created to test Facebook’s political targeting method ran from August 26, 2019 to August 28, 2019.
For the zip code targeting method, the campaigns targeting conservatives were restricted to a random sample of zip codes from the top 100 counties that had the highest percentage of the population vote for Donald Trump in 2016. The campaigns targeting liberals were restricted to a random sample of zip codes from the top 100 counties that had the highest percentage of the population vote for Hillary Clinton in 2016. These political zip codes were excluded from the zip code samples used in the other campaigns.
We calculated the click-through rates and share rates by dividing the number of clicks or shares by reach, or the number of unique Facebook users who had the opportunity to interact with the ad.
SUGGESTED CITATION:
Murray, Caroline and Stroud, Natalie Jomini. (2019, December). Images that generate clicks among both conservatives and liberals. Center for Media Engagement. https://mediaengagement.org/research/images-that-generate-clicks