Nassim Taleb has released a paper with the IMF: A New Heuristic Measure of Fragility and Tail Risks: Application to Stress Testing

From Business Insider: NASSIM TALEB: The Fed Is Looking At The Banking System All Wrong

Nassim Taleb has long been a critic of traditional forecasting methods like the ones underlying these stress tests. He even coined a now oft-repeated term to capture his criticism – “black swan” – which became a huge New York Times bestselling book.

Now, he warns that “fragility is especially high for the banks with the worst outcomes” according to a new metric he’s developed to better analyze the risks facing the banks.

In a new white paper with researchers at the IMF, Taleb explains the reason why all of the stress tests conducted by central banks and international financial institutions like the Federal Reserve, the ECB, and the IMF come up short:

First, many stress tests focus on the point estimates of very few scenarios, and often pay little attention to how the impact would change in case of different scenarios, e.g., a slightly more severe one. Second, if stress tests do not take into account the possibility of model and parameter error, it can be misleading to rely only on the point estimates of even well-designed stress tests. Without considering the potential for these errors, one could miss the convexities/non-linearities that can lead to serious financial fragilities.

A better approach, according to Taleb and his IMF co-authors Elie Canetti, Tidiane Kinda, Elena Loukoianova, and Christian Schmeider, is to measure the difference between outcomes arising from different scenarios instead of focusing on the estimates of potential losses themselves.

According to Taleb, this is the real way to measure the “fragility” of a bank or a country in the event of a negative economic shock. Because point estimates are so prone to errors from faulty model assumptions, measuring the distance between them to detect how quickly losses pile up as the economic shock gets larger becomes a vastly more reliable measure of risk.

In other words, it’s not the size of the losses themselves that is important. Instead, it’s the rate of change of potential losses as the economic situation deteriorates that determines how fragile a bank is, by Taleb’s standards.