How to build enterprise resilience in the face of growing AI risk Matt Tippets | usagoldmines.com

New research from StackGen analyzing nearly 178,000 public technology incidents found AI-related incidents now account for more than one in 10 reported outages, roughly six times the rate in 2023.

AI agents have deleted data, databases, or live systems autonomously. Those agents acted with valid credentials, meaning traditional monitoring did not identify anything unusual until the damage was done.

With AI becoming embedded in business processes across claims processing, coding, customer support, decision support, fraud detection, HR, risk analysis, and supply chain planning, outages and unintended outcomes are a growing risk throughout the enterprise. And while the risk may feel unprecedented or novel, established resilience practice provides the path forward.

AI is creating often-unseen dependencies and risk

AI-embedded business processes are just one aspect of enterprise AI risk.

AI is also accelerating cyber risk for enterprises. Attackers can now employ AI to scale deepfakes, phishing, social engineering, reconnaissance, and develop exploits.

Additionally, AI systems may change behavior over time, creating drift and explainability gaps. This is most likely when AI data sources, integrations, models, and prompts change.

Enterprise employees and teams may be using unapproved AI tools with sensitive information or in business-critical workflows. This shadow AI also creates significant enterprise risk.

Meanwhile, AI adoption is creating operating dependencies faster than governance is maturing.

Recovery complexity adds another layer of risk.

If an AI-enabled workflow fails, produces incorrect decisions, or becomes unavailable, organizations may not know the business impact or have a manual fallback. This can leave enterprises without valid recovery strategies while trying to determine the scope and consequences of a problem while the disruption is already unfolding.

AI failures may be new, but the resilience requirement is familiar: Organizations need to understand what depends on AI, what happens when those dependencies fail, what the business stands to lose, and where action matters most.

Organizations also need to determine whether backup models can be used during disruptions in addition to putting deterministic or even manual solutions in place as workarounds. If AI is the only option, you may have a single point of failure.

Many enterprises are exposed without realizing it.

Understanding probabilistic and deterministic processes

As AI becomes embedded in business operations, every business needs to stop and ask which processes can tolerate answers that are “probably right” and which cannot. An AI-generated recommendation used to inform a decision will likely tolerate “probably right” answers; a process that executes a financial transaction, determines a regulatory obligation, or controls a critical operation cannot. Those require a predictable, repeatable result.

The questions become where can we tolerate the uncertainty AI introduces, and what happens to the business when the answer is wrong?

To answer those questions, you need to understand the business. What are the critical services it provides and what are the processes, technology, people and third parties they depend on? With that context, organizations can evaluate AI risk through four business questions:

What is impacted? If an AI-enabled process fails or produces an incorrect result, which business services, customers, operations, and dependencies are affected?

What happens next? How could that failure propagate or create downstream consequences?

What is the financial exposure? What could the resulting disruption, error, or delay cost the organization?

What should we prioritize? Where are additional controls, human oversight, fallback processes, or other resilience measures most important?

AI risk becomes a business decision about consequence and tolerance. They help organizations determine where probabilistic outcomes are acceptable, where additional safeguards are required, and where the potential impact is too significant to tolerate uncertainty.

Making explainability a buying criterion

If a vendor can’t clearly explain how and why its model reaches an output, that is more than a feature gap. It can become an unquantified source of business risk.

As Harvard Business Review (HBR) explains, even if an enterprise outsources AI technology, it owns the risk. HBR points to recent lawsuits against Cigna, iTutorGroup, Peloton, and Workday as examples, noting “courts and regulators are holding [the enterprises that use AI systems they did not build] responsible when those tools discriminate, mishandle data, or harm customers.”

Compliance and legal problems can quickly lead to financial and reputational damage through competitive disadvantage, lost customers or customer trust, investor sell-offs, higher capital costs, and stock price drops.

When selecting suppliers, businesses should therefore evaluate whether AI-assisted outputs can be explained, audited, and defended to regulators. Explainability should be considered alongside cost, performance, security, and reliability, rather than addressed after deployment.

Map dependencies to prevent single points of failure

Whether it’s one model an entire workflow depends on or a supplier your vendor depends on, reliance on AI and frontier models has created new concentration risk. Dependency mapping matters as much when managing AI risk as it does in other critical supplier relationships and, since we’re still in the early years, its importance will grow as adoption grows.

Understanding model and data dependencies is key because AI outputs are only as reliable as the data, context, and controls behind them. And because enterprises are adopting AI through cloud platforms, data services, model providers, and third-party applications they don’t fully control, enterprises also need to understand the downstream consequences if one of those dependencies becomes unavailable or unreliable.

Recent events demonstrate how quickly disruption can spread beyond its apparent point of origin. The Persian Gulf conflict disrupted global oil flow and created pressure in supply chains across agriculture, manufacturing, semiconductor, and transportation.

The ransomware attack on Change Healthcare similarly demonstrated how disruption at one highly connected organization can create operational consequences across an entire ecosystem, forcing healthcare organizations to use manual processes and other workarounds.

AI creates the same dependency challenge, as the recent OpenAI outage shows. When it, as a foundational AI provider, became unavailable, the disruption extended to the services and applications, like ChatGPT and Codex, built on top of it.

A model provider, data source, cloud service, or AI agent may appear to support one application while actually sitting upstream of dozens of business processes. Without mapping those relationships, enterprises cannot reliably determine the broader impact when something fails.

Cataloging AI agents as assets

AI agents are proliferating across enterprises at a rapid rate. Failing to keep track of these agents can create a form of shadow AI risk.

For example, if an employee or team responsible for an AI agent leaves or changes focus, that orphaned AI agent may continue running without the necessary ownership or oversight. An AI agent may retain permissions to access data it no longer needs. AI agents that fall outside of enterprise awareness and management can also lead to compliance and audit failures, excessive autonomy, incident response blind spots, uncontrolled costs, and other problems.

Catalog agents the same way you catalog other critical assets. Enterprises should know what each agent does, who owns it, which systems and data it can access, which business processes depend on it, and what happens if it fails or behaves unexpectedly.

That visibility gives organizations a clearer understanding of their AI environment, their exposure, and the controls needed to manage it.

Enterprise resilience helps organizations stay ahead of disruption

AI is just one area of enterprise risk, but its rapid adoption is creating new dependencies across critical operations.

That makes AI risk an executive- and board-level concern. Organizations that build resilience into how AI is adopted, governed, and managed across the enterprise will be better positioned to absorb disruption without losing control of the business.

We’ve featured the best antivirus software.

This article was produced as part of TechRadar Pro Perspectives, our channel to feature the best and brightest minds in the technology industry today.

The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/pro/perspectives-how-to-submit