Written by David Fagleman.
AI poses a risk to financial stability. The financial regulators are aware of this and have started exploring the impact, but the government’s focus on growth and international competitiveness puts this planning in jeopardy. Extreme care should be taken. We know the impact of the last financial crisis. We can only imagine the impact of one driven by AI.
This time last year it seemed as though everyone was talking about Artificial Intelligence (AI). Public discourse was filled with potentially disastrous outcomes, ranging from the existential threat to humanity’s existence, to international security and job replacement. Despite these concerns, which may have been intentionally blown out of proportion by AI developers to increase their market value, governments around the world have made commitments to encourage the development of AI.
The new UK government is continuing with the light-touch regulatory approach of its predecessor and is placing AI at the heart of its agenda to “deliver change, sustained growth and improved public services”. Civil society organisations have voiced concerns over these plans, highlighting the risks and calling for a more transparent approach to its use in public service deliver, which despite government commitment continues to be lacking. There is however one area of AI adoption that we should be looking at more closely as its failure could have cataphoric consequences on people’s lives, and that’s the use of AI in financial services.
The financial services industry affects every citizen’s life and is a major contributor to the UK’s total economic output. But it also poses a major risk, as demonstrated by the catastrophic consequences for the wider economy in the aftermath of the global financial crisis (GFC) of 2007–09, which resulted in the deepest recession in terms of lost output since quarterly data were first published.
The use of AI in financial services is not a hypothesis, it’s a reality. Last month, the Bank of England and the FCA published its third survey of AI and Machine Learning (ML) in UK financial services. It found that 75% of the firms surveyed are already using some form of AI in their operations. That’s up from 53% in 2022. 17% of all cases are using foundation models, which apply advanced ML to very large quantities of data for a wide range of uses. Its use is mainly in low-risk situations, such as automating customer support and mitigating external risks from cyber-attacks, fraud and money laundering. However, some firms are using AI for more high-risk situations, especially from a financial stability perspective: 16% are using AI for credit risk assessment; 11% are using AI for algorithmic trading; and 4% of firms are using AI for capital management. In all three areas there are plans by firms who aren’t currently using AI to do so in the future.
Last year, the Lab published a report AI in Financial Services: How to avoid the big risks, in collaboration with The Financial Inclusion Centre. In it, we argued that while there are benefits to integrating AI into financial services it also presents significant systemic risks, some known and others unknown, to financial stability.
For example, AI algorithms used in trading are susceptible to ‘herd mentality’ bias and mispricing risk if it captures data at times of market euphoria, or based on misinformation, with human investors following suit. These ‘hallucinations’, where generative AI produces inaccurate information, have been recognised by financial firms and the Bank of England as a potential risk to firms and consumers relying on AI as a source of financial advice or information.
There is also a risk of an accountability gap caused by a lack of understanding of how the AI works and what it’s capable of. This leads to two undesired outcomes: 1) a failure of governance leads to firms selling products, or introducing processes, that haven’t been fully assessed for risk; and 2) a lack of knowledge at the regulator results in an inability to understand market activity. Both have unfortunate similarities to the build-up to the global financial crisis.
The world’s financial markets are fragile at the best of times and can respond negatively to unexpected events. The International Monetary Fund has said that the use of foundation models presents significant new challenges to cybersecurity, and its adoption in sensitive and heavily regulated sectors such as finance warrants “careful contemplation”. Gary Gensler, the outgoing chair of the US Securities and Exchange Commission, has said that the immediate risk from AI is not a robot takeover but a new financial crash, and that the likelihood of an AI-driven financial crisis within a decade would be “nearly unavoidable” if there were no regulatory intervention.
History tells us that financial crises occur following periods of deregulation and sector growth, combined with risky behaviour. The new government has asked the FCA to encourage more risk taking to accelerate efforts to support the growth and competitiveness of financial firms. This has echoes of 1970s deregulation and pre-2007 behaviour with the added element of a powerful and complex technology that we don’t fully understand. As the Lab argued last year, unless the regulatory regime acts in a more precautionary manner, we may very well see another financial crisis. The difference will be that this time it will be driven by AI, and the extent of the impact could go well beyond anything we’ve seen before.