Conventional Analytics Have Hit the Wall

Market regimes refer to the price trends for publicly traded securities and their relationship to macroeconomic indicators and other variables. While they can remain stable over certain periods of time, they tend to shift very quickly. The key lies in being able precisely characterize each regime, quickly recognize the onset of new ones, and immediately understand their impact on asset returns so as to capitalize on the opportunity to drive more alpha and mitigate risk. However, gaining a precise understanding of market regimes requires the ability to analyze highly complex market and economic data, over a broad time horizon, to uncover their key characteristics, and their impact on returns and risk forecasts.

Traditionally, analysts use three primary approaches to understanding the impact of market and macroeconomic information on asset returns.

Fundamental approaches are useful for identifying long-term business cycles through an analysis of global economic information and market indices. However, these approaches tend to be people-intensive. They require teams of analysts to comb through primary and secondary sources to find patterns and opportunities in limited amounts of time. They fall short when it comes to systematically capturing the nuanced, local variations in market conditions and their potential impact on asset returns.

The primary goal of technical analysis is to analyze historical time periods to identify the factors that drive similar price movements (e.g., interest rates and volatility levels). It often draws on other indicators such as moving averages and the Relative Strength Indicator (RSI) to gauge the performance of an asset or market. However, this analysis is typically conducted manually and it does not capture the impact of market movements in other asset classes. For instance, this approach would struggle to capture the impact of movements in Crude Oil or the Euro on the movement of the S&P 500. In addition, strategies based on technical indicators tend to get over-utilized and lose their predictive power in certain market conditions, and as such might not perform as well with new out-of-sample data points.

More rigorous quantitative approaches focus on trying to identify when market regimes transition from one state to another using techniques such as regime-switching models. However, these approaches require analysts to hypothesize the number of such regimes and the parameters that describe them, prior to conducting an analysis. In addition, as market regimes tend to evolve and shift fairly rapidly, these models struggle to keep pace with changing patterns. As a result, uncovering regimes, their explanatory variables from high-dimensional data, and the implications on asset returns can be difficult and time-consuming. There is a need for a new approach.

Ayasdi Machine Intelligence for Accurate Returns and Risk Forecasting

Ayasdi for Market Regime Forecasting draws on the power of the Ayasdi Machine Intelligence Platform to uncover the precise drivers of returns, risk, and liquidity from complex data and help create more accurate predictive models. The key benefits are as follows:

Comprehensively Analyzes all Data

A precise understanding of market regimes aids the creation of better asset allocation strategies and more accurate liquidity forecasts. However, this requires the ability to analyze highly complex market and economic data over extended periods of time, all at once. Most modeling techniques compel analysts to look at small sets of variables that increases the likelihood of missing important relationships that have predictive power.

The Ayasdi Machine Intelligence platform draws on innovations in topological data analysis (TDA) and machine learning to analyze hundreds of market and macroeconomic variables simultaneously. The software automatically groups these variables using a notion of similarity to create topological models that reveal subtle, valid combinations of variables that characterize the current regime. The topological models or similarity maps that TDA generates uniquely preserves the multivariate relationships between hundreds of market and macroeconomic variables, across a broad time horizon. This map also captures the local relationships and stabilities, across time-periods and variables. It helps with identifying the key drivers of analogous time periods, that drive the creation of more accurate forecasts of risk and return.

Finds Precise Drivers of Returns and Risk

During periods of stability the factors that determine asset prices include variables related to supply and demand, cash flows, as well as risk and return. However, during periods of market instability, prices can be affected by an entirely different set of variables, such as flight to safety, funding risk, and collateral calls. As a result, price movements tend to have no relationship whatsoever to the variables that are typically used to compute prices.

Ayasdi Machine Intelligence helps portfolio managers uncover subtle, valid combinations of features that characterize different market regimes, hard to uncover using conventional machine learning techniques. It then rapidly pinpoints similarities to past regimes to help them more accurately forecast the performance of various asset classes. The software can also surface the complex relationships between market regimes and liquidity proxies to aid the creation of more precise liquidity forecasting models. The software helps strike a fine balance between nuanced and systematic examination. It helps detect subtle relationships while providing a systematic way of excluding spurious correlations.

Drives Alpha with More Accurate Forecasting Models

Conventional statistical tools and machine learning techniques limit analyses to small sets of explanatory variables, and require analysts to hypothesize relevant partitions and analytical forms prior to analysis. As a result, uncovering regimes, their explanatory variables, and the implications for the future can be difficult and time-consuming. Ayasdi Machine Intelligence provides a way of rapidly examining vast amounts of data to be able to rapidly respond to evolving market structures and the associated opportunities and risks.
As opposed to making global assumptions regarding all the underlying data, Ayasdi Machine Intelligence helps construct an ensemble of models, each representing different market regimes and responsible for a different segment of the data. An ensemble of returns, risk, or liquidity forecasting models can be much more accurate as they are each optimized for different segments of the data, thus reducing the possibility of systematic errors in the model output.

Provides a Foundation for Developing Predictive Models

Using Ayasdi Machine Intelligence, hedge funds and asset management firms can create an effective framework for executing a regime-based asset allocation strategy. It provides them with a better understanding of the current market regime and the implications for the future. An understanding of market regimes coupled with knowledge of the outcomes of specific risk premia in similar conditions can inform better asset allocation strategies and more accurately forecast liquidity in the current market regime resulting in significant transaction cost savings.

Ayasdi Machine Intelligence for Accurate Returns and Risk Forecasting

How Ayasdi Helps Create More Accurate Returns and Risk Forecasting Models

Portfolio Managers

The intelligence derived from Ayasdi’s software can supplement portfolio managers’ professional experiences, helping them create effective regime-based asset allocation strategies and more precise liquidity forecasting models.

Quantitative Analysts

Ayasdi Machine Intelligence equips quantitative analysts with a systematic way of exploring all the data to produce more accurate returns and risk forecasting models.

IT Teams

IT infrastructure teams benefit from the option of using Ayasdi Machine Intelligence as the foundation for developing and deploying a wide range of returns and risk forecasting models across the investment firm.

Learn More


Ayasdi machine intelligence for financial services

Download Now


Regime forecasting using machine intelligence

Download Now


TDA and machine learning: better together

Download Now


Forecasting Returns Using Machine Intelligence

Watch Now


Forecasting Market Regimes with Machine Intelligence

Watch Now

Get Started

Ayasdi's regime forecasting application is designed for sophisticated market participants in the hedge fund, asset management and capital markets industries. The product is highly configurable and Ayasdi generally partners with firms that have a clear idea of the problem they are trying to solve with this uncorrelated approach.

Schedule a Live Demo