Title: Using Forecasting Tournaments to Improve Intuitive Predictions And Increase Open-Mindedness
Abstract: I will report the results of four forecasting tournaments that occurred between 2011 and 2015 that were designed to discover the best ways of crowd sourcing numerical forecasts and aggregating them as accurately as possible. Over 20,000 participants from all over the world estimated the probabilities of geopolitical events, from pandemics and military conflicts to international agreements and refugee flows. Participants were randomly assigned to experimental conditions that varied in terms of elicitation methods (surveys versus prediction markets), social dynamics (independent forecasters versus forecasters who worked collaboratively in teams) and training in probabilistic reasoning. Accuracy was defined by the Brier scoring rule and its variance decomposition. By measuring the cognitive, psychological, and political traits of forecasters, we learned what factors were most highly correlated with top performers. Finally, by studying the strategies of “superforecasters” – those who scored among the top 2%, we learned what contributed to their success and how they made forecasts that were more accurate than U.S. Intelligence Analysts predicting the same events but with classified information. Accuracy depends on selecting the right people, creating the best forecasting environments, and taking advantage of simple statistical aggregation methods.
Dr. Meller's bio information is available here.
