Predicting the future in the financial markets is notoriously hard and – due to a number of well understood tenets, even more difficult to do for a sustained period of time.
In the case of publicly traded securities, price series (and their relationship to macroeconomic indicators) often demonstrate stability over selected time periods. These trends and relationships – referred to as regimes – can also shift quickly to form new patterns as the market enters a new phase.
We see this frequently, and even the casual market observer will be able to construct a handful of these events quickly. Think of the collapse of oil prices this year, the Euro crisis from a year ago, the events that precipitated the Great Recession at the end of 2008.
Those investors with a deep understanding of the characteristics of each regime, as well as the ability to recognize early indicators of the onset of new ones can take advantage of that information to create winning trading strategies, create better asset allocation strategies and develop more accurate liquidity forecasts.
Easier said than done.
The ability to analyze highly complex market and economic data to uncover and capture the key characteristics of each regime is exceptionally difficult. Conventional statistical tools and machine learning techniques generally limit analyses to small sets of explanatory variables and demand that analysts hypothesize relevant relationships and analytical forms prior to analysis. As a result, uncovering regimes, their explanatory variables, and the implications for the future is difficult and time-consuming.
Ayasdi’s application of machine intelligence enables portfolio managers to rapidly uncover subtle, valid combinations of features that characterize different market regimes. It then pinpoints similarities to past regimes to help them more accurately assess the performance of various asset classes. Further, our approach surfaces the complex relationships between market regimes and liquidity proxies to aid the creation of more precise liquidity forecasting models.
Figure 1. A similarity map of 30 years of market data. Note the cycles, those represent business cycles and how you transition through those cycles represents the trading opportunity.
Underpinning this is Topological Data Analysis (TDA). As opposed to making global assumptions regarding all the underlying data, TDA effectively constructs an ensemble of models, each representing different market regimes and responsible for a different segment of the data. An ensemble of asset allocation or liquidity forecasting models can be much more accurate as they are each optimized for different segments of the data, thus reducing the possibility of systematic errors in the model output.
One of our data scientists, Carl Dietz spoke about a real world example at the RiskUSA event last year. At 20 minutes it might be the most efficient way to come up to speed on these new techniques. It is a great watch – explained from a practitioners perspective to other practitioners.