Topological Data Analysis (TDA) to Solve Big Data Government Problems

Last week on I discussed how the public sector is under an increasing pressure to turn big data into actionable insights. This data includes traffic, planning and development, census and population, crime and fraud intelligence, energy consumption, smart grid optimization, and more. The big issue, however, is that scientists, researchers and domain experts inside government agencies are under increasing pressure to find insights from complex data with less resources and limited technology. Moreover, a crucial issue is that the methods existing today are hypothesis-based and require persons to ask the right questions in order to find answers to our nation’s most pressing problems. 

During my tenures at NSF and DARPA, I experienced this first hand. But in the late 1990’s, it occurred to me that a branch of mathematics now called Topological Data Analysis (TDA) might be useful to look at data in a completely new way. Topology is the study of shape, and its properties allow us to extract meaning from shape to better understand what’s hidden in data. At Ayasdi, TDA is a core component of our machine-learning platform allowing us to automatically discover patterns and anomalies in complex data without asking questions. More importantly, today, TDA is proving to be a useful method transforming the way that that government agencies find insights in complex data leading to outcomes that have measurable impact. 

To learn more about TDA check out the video below of Ayasdi Co-Founder and Stanford Professor of Mathematics Gunnar Carlson