Richard John (University of Southern California) will deliver a talk titled, "On human detection of deception on social media." See abstract below.
Abstract: False posts to social media following extreme events such as natural disasters and terror attacks have become a pervasive concern for both crisis responders and the public. The current study assesses how well individuals can identify false social media posts, extending previous research on lie detection of oral communication. We report 3 experiments in which over 1000 US participants were presented with a series of actual Tweets posted within 48 hours following soft target terrorist attacks in the US or Europe. Experiment 1 contains three conditions where base rates of false information were 25%, 50%, and 75%, respectively. In Experiment 2 and 3, respondents were incentivized using one of three payoffs varying in the relative cost of false positives and false negatives. In each experiment, respondents were randomly assigned to one of three conditions and provided a binary judgment of the authenticity of information for 20 separate Tweets. ROC analysis showed that respondents performed only slightly better than chance (AUC statistic ranges from 0.49 to 0.56 across all three experiments), consistent with previous meta-analysis from lie detection research. Furthermore, sensitivity and specificity shifted across three conditions in accordance to the manipulations, yet not as much as predicted from an optimal threshold analysis. Participants were responsive to the manipulations, but overall poor at detecting false information. Participants who self-identified politically as conservatives performed worse than liberals and moderates across all three experiments.
