Traditionally, identifying truth in politics comes from mainstream media and its fact checking. A recent Gallup poll, however, showed that only 29 percent of Americans trust fact checking.
Research in behavioral science suggests that we can address the spread of misinformation through a number of other effective strategies, which are brought together in the Pro-Truth Pledge (PTP) project. Several months ago, I wrote a post explaining the Pro-Truth Pledge and its mission. Since that time, two peer-reviewed studies have provided evidence of its effectiveness in changing the behavior of pledge-takers — private citizens and public figures both— to be more truthful, for more than a month after they have taken the pledge. Both studies were published in prestigious psychology journals, Behavior and Social Issues and the Journal of Social and Political Psychology.
Quantitative evidence shows the Pledge is effective
The study in the peer-reviewed Journal of Social and Political Psychology suggests that taking the pledge results in a statistically significant increase in alignment with the behaviors of the pledge. The survey involved 24 participants filling out Likert scale (1–5) surveys self-reporting their Facebook engagement with news-relevant content on their own profiles and also with other people’s posts and in groups before and after they took the pledge, with 1 at lowest level of alignment to the pledge behaviors and 5 being full alignment. To avoid the Hawthorne effect of study participants being impacted by observation, the study did not evaluate current behavior, but past behavior.
We only recruited participants who took the pledge 4 or more weeks ago to fill out the survey, and asked them about their behavior after taking the pledge. Giving them this period also gave people an opportunity to have the immediate impact of taking the pledge fade from their mind, thus enabling an evaluation of the medium-term impact of the PTP on sharing news-relevant content.
This study method was informed by the approaches used by studies of whether honor codes address cheating, which is the most comparable form of intervention to the PTP. Such studies similarly rely on self-reporting by students on whether they have cheated or not cheated.
The study found that on one’s own Facebook profile, the median alignment with the PTP score before taking the PTP is 4 (SD=1.14), and the median alignment score after taking the PTP is 4.5 (SD=0.51). For engaging with newsworthy content on other people’s profiles and in groups, the median PTP alignment score before taking the Truth Pledge is 3.5 (SD=1.06).
The median PTP alignment score after taking the Truth Pledge is 4.5 (SD=0.65). For sharing content, 70.83% of participants (17 of 24 respondents) reported an increase of their PTP alignment after taking the PTP. The figure below provides a visual summary of the preliminary survey data.
Figure 1, Visual summary of preliminary survey data with PTP alignment in Facebook engagement
We conducted a second study, the one published in Behavior and Social Issues, to address the weakness of the first study’s reliance on self-reporting. The second study sampled 21 people, and involved researchers observing and evaluating the quality of Facebook engagement by study participants on their own Facebook profile.
Similarly to the first study, the second study avoided the Hawthorne effect of study participants being impacted by observation by evaluating past behavior. Researchers looked at the first ten Facebook posts with news-relevant content made four weeks after the pledge. Then, the researchers compared these ten posts to the first ten posts for the same period the year before the study participant took the pledge. Each post was coded according to quality, from 1 of lowest level of alignment with the PTP, to 5 of highest alignment.
The second study showed that the average PTP alignment before taking the pledge was 2.49, and after taking the pledge was 3.65, and conducted a paired t-test to examine whether Pro-Truth Pledge Alignment is significantly different after taking the PTP. The null hypothesis H0 for the paired t-test states that there is no significant alignment difference before and after taking the pledge and the alternative hypothesis H1 proposes a significant difference. There was a significant difference in the scores for Pledge Alignment before(M=2.49, SD=0.60) and after (M=3.65, SD=0.41) taking the PTP; t(20) = -8.86, p < 0.001. An estimation of the effect size indicates that the found difference can be considered to be large (Cohen’s d=-1.93). Thus hypothesis H0 can be rejected in favor of hypothesis H1. These results suggest that taking the PTP really does have an effect on inducing truthful sharing behavior on Facebook.
Figure 2, Showing the results of PTP alignment before and after taking the PTP.
The figure above provides a visualization of the results. The thick black line shows the median. The small colored lines represent change among individuals. Note that every individual who took the PTP has improved their sharing on Facebook to be more aligned with the PTP, some drastically.
The first study showed that individuals self-report behaving more truthfully both on their own profiles and in other contexts on Facebook — such as on the profiles of friends and in groups — after taking the PTP. The improvement was large, with a clear statistical significance, of about 1 unit on a 1–5 scale. The second study focused on observing people’s behavior on their own profiles, and confirmed that pledge-takers behaved more truthfully four weeks after taking the pledge. Again, the improvement had clear statistical significance, and was large, also about 1 unit on a 1–5 scale.
In other words, people’s self-reports about their improvement of behavior on their own profiles — and the extent of their improvement — were corroborated by external observers. We can thus assume that people also behaved more truthfully on other people’s profiles and in groups, even though we have no realistic way of observing that.
Overall, these two studies provide compelling evidence that people improved the honesty of their behavior on Facebook because they have heard about and signed the pledge, and there is no reason to believe they would have improved if they did not hear about and sign the pledge. The combination of these two studies provides solid evidence that taking the PTP decreases the spread of misinformation on social media.
The Pledge approach to misinformation
The Pro-Truth Pledge team, made up of behavioral science experts and concerned citizens, determined that tilting the scale requires a two-pronged approach, targeting both private citizens and public figures. Research shows that without any intervention, people tend to ignore information that goes against their beliefs, and are more likely to deceive others both when they see others do so and when it benefits their in-group.
However, increased risk of suffering negative consequences, being reminded of ethics, publicity and committing in advance to honesty all decrease the incentive to lie for ordinary citizens. For public figures, research suggests that transparent, clear information about who is truthful, coupled with reputational rewards for socially beneficial behavior, such as honesty, and enforced with penalties for dishonesty, are the most vital interventions.
The Pro-Truth Pledge incorporates all of these elements by asking signees to commit to 12 truth-oriented behaviors, such as fact-checking, retracting incorrect information, and celebrating others for changing their minds toward the facts, that are meant to counter our brain’s tendencies toward deceiving themselves and others. Private citizens who sign the pledge get the benefit of contributing to a more truth-oriented society. Public figures and organizations get more substantive rewards for signing the pledge, in the form of positive media and public recognition.
The pledge crowdsources the truth by holding them accountable through a reporting and evaluation mechanism. Thus, the PTP offers the same kind of credibility mechanic offered by the Better Business Bureau: just like the BBB holds businesses accountable to ethical business practices, the PTP holds public figures and organizations accountable for truthfulness in their public statements.
Similarly, just like anyone can lodge a complaint to the BBB and a business will have to respond to a legitimate complaint, anyone can lodge a complaint to the PTP. If after initial investigation the PTP evaluators consider the complaint legitimate, the public figure or organization will be asked to respond.
Since its launch in December 2016, the PTP has over 8,000 signers so far across the globe, including over 1200 public figures and organizations. The PTP has already had positive media coverage in prominent US and international media venues on the right and left. They include Newsweek , The Guardian , The Daily Caller , Raw Story , Scientific American , The Hindu , NPR , CBS News , and others. Unsurprisingly, signers include prominent leaders in the misinformation-fighting movement, and one of the most prominent scholars of misinformation, Stephan Lewandowsky, is on the Advisory Board of the organization running the Pro-Truth Pledge project, Intentional Insights.
Perhaps more surprising is that over 600 Republican, Democratic, and minor party politicians across the political spectrum took the pledge, including US state legislators Eric Nelson (PA) , James White (TX) , and Ogden Driskell (WY) and national congress members Beto O’Rourke (TX) , Matt Cartwright (PA) , and Marcia Fudge (OH).
Over 800 other public figures have taken the pledge, such as globally-known public intellectuals Peter Singer , Steven Pinker , Michael Shermer , and Jonathan Haidt , religious leaders such as Pierre Whalon , Jim Burklo , and Lorenzo Neal , media personalities such as Ethan Bearman , Andrew Keen , and Bob Fitrakis , and civic activists such as Bill Shireman , Andrew Hanauer , and Ricken Patel. Over 80 organizations , whether news-oriented ones such as Media Bias/Fact Check , Media in Action , Columbus Free Press , and Fugitive Watch , or civic-oriented ones such as Stand Up Republic , Faith and Freedom Coalition Delaware , George Mason University Center for Climate Change , and BridgeUSA also took the pledge. While they may have many disagreements, they are united around a commitment to truthful behavior.
The pledge organizers make a strong effort to reach across the political divide. Included on the Advisory Board of the organization running the pledge project is the founder of the Houston Tea Party Society, Felicia Winfree Cravens (who described why she took the pledge here), and also a conservative Christian pastor, Lorenzo Neal (who described why he took the pledge here). The very popular far-right website The Daily Caller published a positive story about the pledge. A number of Republican legislators, such as members of state legislatures Eric Nelson (PA) , James White (TX) , and Ogden Driskell (WY).
Many organizations and public figures added the PTP badge to their website and social media, in similarity to how businesses add the BBB badge to their website, which is one of many ways that these organizations and public figures can get a positive reputation boost while also promoting truthful behavior.
Will the PTP work to tilt the scale toward truth? In order to tell, we’ll need to evaluate whether and why people are taking the pledge and also whether the pledge changes their behavior. Michael Smith, a candidate for Congress in Idaho, took the Pledge, and later posted on his Facebook wall a screenshot of a tweet by Donald Trump criticizing minority and disabled children. After being called out on it, he went and searched Trump’s feed. He could not find the original tweet, and although Trump may have deleted it, the candidate edited his own Facebook post to say, “Due to a Truth Pledge I have taken, I have to say I have not been able to verify this post.”
Further research is needed to determine whether that is indeed the case. We also do not know whether presenting the PTP in a semi-voluntary context, such as when students are presented with an honor code with an implicit expectation that they sign it in order to attend the college of their choice, will maintain the impact of the PTP: further research is needed as well.
The evidence so far shows that the Pro-Truth Pledge has the potential to protect our democracy from the tide of lies. Whether it will succeed depends on how many people go to ProTruthPledge.org to sign the pledge, spread the word, lobby public figures to sign it and monitor those who do. The early results look promising.
