Solutions journalism is reporting that focuses on responses to social problems, instead of just the social ills themselves. It investigates how people, institutions, and communities work towards solutions, and whether and why solutions work.
Newsrooms across the country have implemented reporting with a solutions journalism focus, but little is known about how the public and communities respond. In this report, we ask an overarching question: Can a solutions journalism campaign change public discourse? The research described in this report was jointly funded by the Solutions Journalism Network and the Center for Media Engagement (with grants from the Democracy Fund, Hewlett Foundation, and Rita Allen Foundation).
On March 6, 2016, the Chattanooga Times Free Press published a series focused on poverty called “The Poverty Puzzle.” In this report, we track whether the series changed public discourse by looking at Twitter and public government documents.
Key Findings
Findings suggest that:
- The public tweeted more about poverty in the days following the publication of the series.
- Community churches and opinion leaders did not use more poverty-related terms in their tweets following the series.
- In the Chattanooga City Council meeting minutes, there were 4 mentions of poverty-related terms in the 12 meetings prior to the series, compared to 26 mentions in the 12 meetings following the series.
Implications for Newsrooms
This study provides some support for the ability of long-form solutions journalism series, like the Chattanooga Times Free Press’ “The Poverty Puzzle,” to affect public discussion, as measured through social media.
There are some limitations to this content analysis – for example, as most Facebook accounts are private, we were only able to examine public Twitter accounts for social media discussion about poverty. Further, we note that this study cannot prove causal relationships – it is possible that something else happened at the same time as the Times Free Press series that is responsible for the patterns we document. Inspection of the tweets, however, makes this possibility unlikely. Finally, it isn’t possible to know whether a certain component of the series, such as its length or its use of solutions journalism, might be responsible for the observed patterns. It also could be something specific to the issue of poverty. Indeed, research shows that tweets follow a different pattern than online news in response to major news events like elections.1
The share of poverty-related tweets in Chattanooga rose significantly following the series publication. The spike, however, lasted only for a few days. This suggests that while impactful, the attention span of social media to the issue was limited. Yet social media in general, and Twitter specifically, may display short-lived attention to issues as compared to other media or measures of public opinion. We also uncovered that the public led the news-driven conversation about poverty without the influence of opinion leaders. Finally, the spike in poverty-related word usage in the City Council meeting minutes suggests that the series may have influenced government dialogue.
We followed up with Joan McClane, author of “The Poverty Puzzle,” about the impact of the series beyond Twitter. “The community response to The Poverty Puzzle has been overwhelming,” she told us. “Many organizations and citizens have reached out to us to get involved in ways ranging from mentoring to using the series in their classrooms to holding forums to funding innovation regarding poverty.” She said she also found the response to be compassionate and respectful across political divides. Her experience emphasizes that future research should look at how solutions journalism sparks community dialogue and reactions beyond social media.
The Study
For this research, we chose to use Twitter as our method of monitoring online discourse as many user accounts are accessible to the public and not set to private, as can be the case on Facebook. Pew Research reports that only 16% of U.S. adults use Twitter and even fewer – 9% – get news from Twitter. Thus our analysis captures only a small fraction of the possible ways in which people could respond to the series. Nonetheless, the ability to examine unprompted responses from Twitter users within a geographic area is a considerable advantage.
To do this research, we identified the most common words in the Times Free Press series related to poverty. We then analyzed how frequently these terms appeared in tweets from Chattanooga and in Chattanooga City Council meeting minutes. We collected 5,393,325 tweets from Chattanooga, TN from December 1, 2015 to June 1, 2016 using the program Crimson Hexagon. Of these, 4,768 contained poverty-related terms derived from the text in the series. We looked at public tweets that were geotagged in Chattanooga, TN and at tweets from community opinion leaders (including community churches) identified by the Times Free Press. We also gathered the minutes from City Council meetings during the same period. More details about our method can be found at the close of this report.
Public Tweets About Poverty Increased in Days After Series Published
Our analysis shows that tweets about poverty in the community increased following the publication of “The Poverty Puzzle” series by the Chattanooga Times Free Press. Comparing the week before the series to the week afterward, tweets on the topic rose by 27.9 percent. However, the uptick did not extend beyond several days.
Below is a graph of poverty-related tweets from the public by day. Looking at the 7-day moving average (the darker line in the graph), there is a notable increase in poverty-related tweets following the launch of the series on March 6, and another uptick in poverty-related tweets on March 31.
Note: The 7-day moving average takes an average of the percentage of poverty tweets from the three days before and the three days after a particular day. This technique helps to smooth out day-to-day variation.
We statistically assessed the change in Twitter-based poverty discourse following the publication of the Times Free Press’ series. The three days directly following the series’ publication show a significant increase in poverty-related tweets, but this trend does not continue beyond the first few days.
We looked at the tweets to get a better sense of the conversation. Some tweets from linked directly to the story in the Times Free Press. Others discussed poverty in Chattanooga. For example, this tweet “I’ve been homeless 3 times. The problem isn’t drugs or mental illness — it’s poverty” linked to another story about poverty. Another subset of tweets in the week following the series’ publication shared opinions about revelations from the series, such as: “Paying school cafeteria employees more than fast food restaurants would doesn’t mean we pay them enough. $8.43 isn’t a living wage.”
We also observed another increase in poverty-related tweets from citizens in Chattanooga four weeks post-publication. The uptick in the fourth week, most noticeable on March 31, seems to be tied to widely tweeted news that a Chattanooga waitress filed a minimum-wage class action lawsuit against Ruby Tuesday.
Discussion of Poverty in City Council Meetings Increased After Series
There was an uptick in poverty-related words used in Chattanooga City Council meetings after the series was published.
We gathered minutes from city council meetings before and after the publication of the series – 24 meetings took place in this period. In the 12 meetings before the series was published, the 11 poverty- related words appeared only four times. In contrast, in the 12 weeks following the publication of the series, those same words appeared 26 times. The graph below displays the sharp upticks following the March 6 publication of “The Poverty Puzzle” in the Times Free Press.
We examined the meeting minutes for several of the peaks to understand the qualitative nature of the discussion. The March 22nd and May 17th meetings included a report from the Youth and Family Development divisions, which discussed services for those with low incomes. On May 5th, a local resident addressed the council about the issue of poverty. There also was a special presentation that referenced unemployment. The series was not directly mentioned, and these data cannot tell us whether the series is responsible for the spikes.
Opinion Leaders Did Not Drive Poverty Discussions on Twitter
Our analysis of tweets from opinion leaders, identified in conjunction with local reporters, did not show the same responsiveness to the Times Free Press poverty series as did the public tweets.
A graph of poverty-related tweets from the opinion leaders across time can be found on the next page. There was no notable increase in poverty-related tweets following the launch of the series on March 6. Of the 63 accounts we identified as opinion leaders, only 23 tweeted in the six months during which we collected data. Those 23 accounts averaged slightly more than 11 tweets per account over the time period. At least in this instance, Twitter may not be the place where opinion leaders engage.
The only spike we observe comes some nine and a half weeks after the series was published. We examined the tweets in this spike, which lasts from May 12 to May 19, and found a flurry of tweets about an upcoming church conference about poverty called “Re-Think Poverty.” It appears that the publicity about this upcoming event drove an increase in poverty-related tweets from opinion leaders. As will be discussed shortly, opinion leaders affiliated with churches were responsible for this uptick.
Tweets related to the Re-Think Poverty conference included: “Good intentions aren’t enough–gather your ministry networks at Rethink Poverty & learn how to help without hurting” and “How is your church connecting people to institutions and services that can help them escape poverty?”
As we did with the public, we assessed whether there was any statistically meaningful increase in poverty-related Tweets by opinion leaders following the Times Free Press’ poverty series. This analysis, described in detail in the method section at the close of this report, reveals a significant mid-May uptick, but little else.
Note: The 7-day moving average takes an average of the tweets from the three days before and the three days after a particular day. This technique helps to smooth out day-to-day variation.
Churches Drove Modest Uptick Due to Event, Not Coverage
In addition to the 16 churches listed as opinion leaders by the Times Free Press, we looked at an additional 24 churches that call Chattanooga home. We found no tweets matching our poverty- dictionary terms from these additional churches. However, we should note that the May uptick in poverty tweets among the opinion leaders (detailed in the section above) does stem from churches. This increase can be attributed to three accounts tweeting about the upcoming Re-Think Poverty conference.
Methodology
This study involved comparing the language in the Times Free Press poverty series with the language in tweets from various groups, as well as city council minutes, across time. First, we identified the most common words in the Times Free Press poverty series. From the top 5,000 unique words,2 two researchers identified 78 words that related to poverty. We collected all tweets geotagged in Chattanooga, TN from December 1, 2015 to June 1, 2016 (the series was published March 6) that contained these 78 words. For each of the 78 words, a random 25 tweets were coded as either relating to poverty or not. From the initial list of 78 words, we identified 11 words for which two coders agreed that 80% or more of the tweets related to poverty. There were no disagreements between the two coders in terms of which words were, and which words were not, consistently related to poverty. Using this subset of 11 words, we again queried the Twitter API for poverty-related tweets.3 We used these poverty-related tweets for the analysis included in this report.
Next, we looked at three groups: opinion leaders, churches, and the public. The list of community leaders (including some churches) came from one of the series’ reporters at the Times Free Press. To this list, we added other members of the local government, ultimately ending up with 64 Twitter users. We also found Twitter accounts for another two dozen churches and church leaders not on the original opinion leader list. We constructed a second group consisting of these additional churches, which does not overlap with the opinion leader list. To create a dataset of tweets from the public in Chattanooga, we looked at all tweets geotagged in Chattanooga and then excluded accounts identified as a) opinion leaders, b) reporters for the Times Free Press, including the main newspaper account, and c) churches.
Once the user groups were established, we collected all tweets, regardless of their topic, from each group of users in the given time period (this served as a denominator when calculating the percentage of poverty-related tweets). After that, we again queried Twitter’s API for tweets from those users, but only those that contained the poverty-related words (this served as the numerator when calculating the percentage of poverty-related tweets). We then analyzed the percentage of poverty-related tweets over time. In order to understand the strength of the post-series spikes, we used regression analysis to test the association of given time periods to the prevalence of poverty-related tweets in both groups.
The city council meeting minutes were downloaded from the Chattanooga city government website. There were 24 meetings, representing 12 weeks before and after the launch of the Times Free Press poverty series. These documents were queried for the poverty dictionary words, and counts for each week were recorded.
Statistical Analysis
Aside from graphing the time series, we used OLS regression models to assess Twitter discourse about the topic after the Times Free Press’ poverty series. We first tested for time lags – that is, the extent to which the previous day’s poverty-related tweets predict that day’s tweets – and identified a one-day lag, which we control for in the analysis. We also control for the base number of geotagged tweets from Chattanooga by day.
As the regression results indicate, the first week the series was published is a significant predictor for poverty-related tweets. Additionally, the fourth week post-publishing indicates a significant increase in poverty tweets.4
After finding a week one uptick, we then completed a day-by-day analysis to understand how long the uptick lasted. The results, shown in the table below, suggest that the uptick lasted for 3 days.
We follow the same model for the analysis of opinion leaders’ tweets. The OLS regression analysis controls for a previously identified one-day lag in tweets, but the only significant predictor of poverty-related tweets is the above-mentioned May spike (at 9.5 weeks), driven by tweets about the Re-Think Poverty conference. We also control for the base number of tweets from opinion leaders by day (Opinion Leader denominator).
SUGGESTED CITATION:
McGregor, Shannon and Stroud, Natalie Jomini. (2016, September). How one community responded to solutions journalism. Center for Media Engagement. https://mediaengagement.org/research/how-one-community-responded-to-solutions-journalism
- See S. Mo Jang and Josh Pasek, (2015). “Assessing the carrying capacity of Twitter and online news.” Mass Communication & Society, 18(5), 577-598. [↩]
- Stop words (common words like: and, or, the, so, etc.) were removed from the corpus so that the most frequent words had meaning. [↩]
- (employment OR good-paying OR income OR inequality OR low-income OR low-wage OR poverty OR recession OR unemployment OR upward-mobility OR wage) [↩]
- Since the dependent variable — the number of poverty-related tweets — is a count variable, we also conducted a negative binomial regression. The results are the same. [↩]