Modeling the Future of Regulatory Risk

Today, alongside our Envision release, we added a new application – this one targeted at the challenges of rapidly distilling complexity down to something that can be understood by regulators, internal model review boards or internal business audiences. This application adds to our ever growing family of intelligent apps.

The application descended from our pioneering work with a major bank where we were tasked with helping them to model their capital markets business for the critical Federal Reserve Stress Test (CCAR). We quickly realized that the challenges that the bank was struggling with were everywhere in the post-meltdown, regulation-heavy financial services world.  

This new normal demands a modeling solution that prioritizes scope, collaboration, speed, accuracy, impartiality and transparency. That is a lot to ask of a single application, much less a single algorithm.

The result of these challenges is a workflow that doesn’t make any sense – a workflow that removes those best able to validate the model until after all the work is done. Yet that is what we do.

The reason has to do with how the modeling process works in most enterprises. Presently, 80% of the modeling timeline is dedicated to extracting and preparing data (ETL). What this means is that 20% of the modeling timeline goes into actually building and validating the model.

Given the scheduling constraints associated with iteration – this leaves little time to explore the problem from a principled perspective. With no time and minimal business input, enterprises fall back on building an algorithmic black box because it is faster and easier than the alternative. This may deliver predictive power, but it comes at the cost of explainability – and that cost can be exceptionally steep when there is a regulator involved.

Ayasdi’s Model Accelerator, built on Ayasdi’s award-winning machine intelligence platform, is particularly well suited for complex, high-dimensional, modeling problems where the goal is to principally reduce the number of features/variables being used for modeling purposes – thereby facilitating the process of building, documenting and deploying those models into production while creating exceptional transparency.

The application excels at a variety of modelling challenges but is particularly effective at risk modeling problems such as loss-given default (LGD), probability of default (PD) and other regulatory modeling problems. These models typically make use of simple regression techniques in order to support the simplicity and explainability requirements associated for regulatory oversight, however, Model Accelerator supports a range of approaches. 

Here is how it works.

Model Accelerator starts by including pre-packaged transformations that derive from base data and are specific to these modeling problems, such as the calculation of log, square, delta lags and so forth. These transformations create additional features that are detected and explained by Ayasdi’s machine intelligence approach. This is important as the model form (e.g. linear) is constrained in many regulatory settings. Without variable enrichment, it is exceptionally difficult to build accurate models.  

Since the Model Accelerator draws upon Ayasdi’s particular capabilities for distilling complexity the addition of this additional resolution creates better outcomes – where other modeling solutions struggle with high dimensional data.

Model Accelerator then segments all variables, both base and transformed ones, into self-similar groups using semi-supervised learning. The subject matter expert can then let the application auto-select the best combination of features, or interact with an easy-to-use interface to select the variable(s) that are meaningful to the business in their estimation.  This is as simple as checking a box.  Every variable will have statistical significance and the list will be sorted by absolute correlation but not every variable will be noteworthy to the business leader.

Of particular note is the ability of the application to maintain a full audit trail of all the selections made by the software and the human operators.

Finally, using the selections made by the business leader, the model accelerator application constructs all possible models from among the the down-selected variables – presenting a candidate model and a full suite of challenger models.

 

Variable Group Discovery  

At the heart of Ayasdi’s Model Accelerator is a powerful approach to data science that encompasses both unsupervised learning capabilities as well as supervised learning capabilities. By leveraging the learners, Model Accelerator is able to identify groups of variables and explain why they were selected.

While the selection of an outcome variable presages an element of supervision, the Model Accelerator uses topological data analysis to look at all of the possible relationships encoded within the enriched base-level data; finding hidden patterns that hold predictive value. These patterns are frequently obscured and cannot be detected by other approaches.  By taking an unsupervised approach and looking at all of the data enterprises can often validate or disprove organizational assumptions while becoming more sensitive to environmental changes impacting the business.  

This approach also avoids modeling pitfalls such as multicollinearity and the tendency of forward or backward selection methods resulting in local minima.   

While Ayasdi’s unique capabilities in unsupervised learning are a key feature model developers and line of business leaders often want explicit control over the model through the selection of key variables.  This approach is fully supported and adds the benefit of documenting any bias associated with the selection of certain variables and the elimination of others. This leads to better models and greater transparency.  

Collaborative Workflow

For the vast majority of modeling solutions on the market today, the only interface that a business user or regulator has to the model is PowerPoint or Word.  This is particularly problematic for the business users. Those with the domain knowledge to think through the model, to guide its development and ensure its applicability to the business rarely ever interact with the model until the results are presented.  Often it takes weeks or months to gain consensus using this approach of build, explain, evaluate as stakeholders have to be scheduled to interact with the candidate model.  This is sub-optimal on multiple levels and introduces risk for the organization that can be dramatically reduced or eliminated altogether.

In the AMA, carefully designed application workflow and collaboration features allow the model designer to anchor discussion while facilitating reviews with line of business owners and internal review managers on statistically defensible decisions. Once a base model has been created, a full suite of supporting or alternative models are be generated in parallel, enabling “real time” and interactive iteration while all stakeholders are in the same meeting.

Accelerating Model Design and Development

The AMA can be used both as a model generation engine and as a model execution engine.  This accelerates development cycle time considerably. In most scenarios, the model is created by a modeling or data science team, shared via PowerPoint or Word with the business owner and, if validated is sent to IT to be hand-coded, requiring a full software development lifecycle, which can be extremely expensive.  With AMA’s workflow and collaboration features models can go from development directly into production.  

Creating Operational Flexibility for Model Deployment

The AMA can be deployed in number of configurations including high frequency configurations.  Furthermore, AMA can output to other modeling execution engines that the enterprise has in production through industry-standard formats (PMML).

Global and Local Models

Frequently, the modeling requirements demand both a global model and local models.  This frequently occurs geographically (e.g. Asia and the component countries) but can be found in any number of situations.  The local models have the same sets of variables as the global models, however the interaction among those variables can be fundamentally different. The AMA can successfully construct models at both granularities rapidly and with full documentation – enabling a more detailed understanding of both the entire region and the local markets.

Summary

We are really excited about Model Accelerator. The world is a different place and the time has come for a modeling application that can take advantage the challenges it brings. Feel free to reach out to us here to get a demo of how it works.