An AI Black Swan is a plausible, high-impact event involving artificial intelligence that sits outside an organisation’s current operational risk taxonomy. It is not a forecast; it describes a class of uncertainty that ordinary risk lists may not capture, so the response to unknown unknowns is resilience rather than prediction.

The term draws on the broader idea of rare events with consequences that are hard to anticipate from past experience. In AI, a familiar risk can become a Black Swan concern when scale, speed or interaction changes its effect, or when a new capability creates a failure mode that existing controls do not address. The boundary is not fixed: as evidence accumulates, a previously unfamiliar scenario may become an operational risk that can be assessed and managed more directly.

Eight categories for resilience planning

These categories are prompts for scenario planning. They are not predictions, an exhaustive catalogue or proof that any particular event will happen.

Emergent Capability: A system develops or reveals a capability that was not anticipated from testing, changing the consequences of its use.
Systemic Concentration: Dependence on a small number of models, providers or infrastructure layers creates a shared point of failure.
Alignment Failure: A system persistently pursues an objective in ways that conflict with human intent or organisational constraints.
Multi-Modal Integration: Combining text, image, audio, video or sensor inputs creates interactions and vulnerabilities that separate testing missed.
Adversarial Breakthrough: A new attack or manipulation method bypasses controls that had appeared robust.
Supply Chain Vulnerability: A failure or compromise in a model, dataset, component or supplier affects downstream AI services.
Multi-Agent Emergence: Interacting agents produce outcomes that are not readily explained by the behaviour of any single agent.
Quantum Cryptographic Transition: Advances in quantum computing undermine cryptography that protects AI data, models or infrastructure before replacements are fully deployed.

Prepare without pretending to predict

Resilience planning asks what the organisation would need to notice, decide and do if a scenario began to unfold. It includes monitoring for weak signals, clear escalation routes, tested authority to pause a system, fallback processes and recovery plans. Exercises can test whether staff know who may make an urgent decision and whether critical services can continue without the affected AI capability.

These preparations do not eliminate uncertainty. They reduce dependence on one forecast being correct and improve the organisation’s ability to adapt when evidence changes. Black Swan thinking is most useful when it broadens readiness while keeping everyday controls focused on risks that can already be observed and assessed.