On unpredictable events in risk analysis
Glette-Iversen, I. & Flage, R. (2024). On unpredictable events in risk analysis. Safety Science, 180.
Our opinion
A new work by Norwegian colleagues written by a team specialized in industrial risk analysis at the University of Stavanger. The theme of accounting for unpredictable events in risk analysis is gaining traction as the world becomes more complex. The content is well-supported and relevant to all high-risk industries. A must-read!
Our summary
The uncertainty of the future is a challenge for all sciences, and particularly for predictive risk analysis. This is not new, but recent years have shown the increasing inability of traditional approaches and methods to account for the most extreme and severe events, often referred to as “black swans”, “tail events” (in statistics), or in English as “outliers” and “wildcards”.
These events share the characteristic of being unpredictable. The authors of the article aim to better understand what is meant by the word “unpredictable”.
Five major causes and two missing dimensions
The attempts to explain these events mention five major causes:
Two challenge the current risk prediction model:
- The fact that these events may simply be unimaginable, inconceivable for our societies in their current state;
- Their explanation by complexity theory, which suggests that they involve irreducible uncertainty.
And three remain compatible with the current model:
- The fact that the available models are still imprecise regarding the area of activity and the cause-effect relationships;
- The absence of known precursors or warning signals, again due to a lack of knowledge;
- And/or the fact that the available models indicate the absence of assignable probability to events, which in turn makes them theoretically impossible for specialists, regardless of their potential severity.
In the face of predicting these marginal risks, risk analysis—while maintaining its logic and foundations in frequency, severity, precursors, and cause-effect relationships—gradually acknowledges two missing dimensions in the current framework:
- The importance of the time dimension in prediction, both in the initial calculation of occurrence during the system's life and in the time allowed for the assessment and nature of consequences after the accident. There is a recurring difficulty in evaluating long-term consequences compared to the relative simplicity of assessing short-term effects.
- The importance of identifying the gaps in our knowledge models: what do we know about what we don’t know (“known-unknowns” as opposed to “unknown-unknowns”)?
The main identified causes
The rest of the article revisits the main causes identified above and expands on them.
The impossibility to assign a probability to these events:
This impossibility to assign a probability can be interpreted in different ways.
Objectively, through a calculation that proves impossible, even when a very large number of iterations (almost infinite) are considered. It may also be due to a lack of knowledge and data to establish a reliable calculation. It is worth noting Cavalcante’s (2013) alternative interpretation, which suggests that unpredictable events follow the same mechanisms as other more conventional events. They can therefore be estimated in terms of frequency, but it is the “severity” component that cannot be estimated, leading to a misjudgment of recorded events that are poorly assessed.
Subjectively, through a belief that the event would be impossible. This latter case is always possible, but there is no guarantee that the person making this assumption has sufficient knowledge to declare the event impossible.
The absence of a precise model of behavior and cause-effect relationships:
Again, several interpretations are possible. It could be the model itself or the unpredictable events that cannot be modeled because they are still unknown. In this case, it would be enough for these events to occur once to become concrete and part of the model's inputs. Others refer to complexity and attribute the lack of predictability to the absence of cause-and-effect relationships and the non-linearity of certain events.
It should be noted that this path of continuous knowledge acquisition seems infinite and only postpones the problem. It continuously closes gaps in ignorance, allowing predictability to be restored, but it also constantly opens new gaps with the discovery of missing knowledge and new unpredictable events “just one step further”.
The absence of precursors or early warnings refers to the idea of “surprise”:
In this case, the model may be accurate, the conditions ideal with no limitations on access to lessons from the past, yet the surprise still often occurs as an exceeding of usual conditions with catastrophic intensity and scale beyond all expectations (Cox, 2012).
The existence of irreducible uncertainty:
We speak of ontological uncertainty, a fundamental lack of knowledge and information. This aligns with the idea of unimaginable events, beyond the realm of lived experience (“black swans”, “unknown-unknowns”).
Russo (2017), on the other hand, refers to unpredictable uncertainty, adopting an epistemic rather than ontological perspective. Unpredictability would be reduced through the acquisition of knowledge. However, some authors point out that the acquisition of knowledge before the event is impossible, meaning there is no solution to improve prediction beforehand, as epistemic solutions only apply after the event.
What framework should be used to understand and concretely evaluate unpredictability in traditional risk analysis?
In the context of risk analysis, risk is viewed as a combination of events (A) and their consequences (C) with an associated uncertainty regarding their occurrence (U).
We accept the existence of A’ and C’ as specific conditions in the context, with the idea that certain events and effects can be added to the basic model in a particular context. The uncertainties inherent to A’ and C’ allow for the measurement of an unpredictability Q within a knowledge context K (which necessarily does not cover all the knowledge needed to model all risks). A probability P of the occurrence of the unpredictable can be established based on the strength/belief attributed to this existing knowledge SoK (Strength of Knowledge): Q = (P, SoK).
This belief in the value of our knowledge—and its limits—depends on our understanding of the world, the consensus of expert opinions, the insights and confidence built through the general behavioral model linking A to C, and the historical perspective gained from past occurrences of surprises. Our perception of time also plays a role, both in the occurrence of events (centennial, decennial, annual, etc.) and in the timeframe within which risk observations are made (10 years for a structure… or its entire lifespan?), as well as in the timeframe for measuring the consequences of losses.
It should be noted that depending on the hypotheses formed about unpredictability (referencing the points mentioned previously), the components involved in measuring this unpredictability may vary (these could include A’, C’, Q, K, and/or SoK).
Unpredictability within the framework of modern risk analysis approaches
The lack of knowledge refers to three recurring situations/conditions:
- The impossibility of describing the spectrum/variety of contexts that feed into A’ and C’, making certain contexts and their descriptors A’ and C’ completely unexpected.
- The impossibility of describing the probability of certain events recognized as possible (an expression of uncertainty Q).
- The absence of reference for certain precursors that have never been observed before.
The degree of contextual unpredictability will depend on these three non-exclusive conditions, not to mention the reference time considered for measurement (short-term and/or long-term), which becomes an essential element in modulating the values of these conditions.
Overall, the measurement of unpredictability appears more as a variable, being the result of fluctuations within a spectrum relatively constrained by these three conditions.
It should be noted that measuring short-term effects is always more predictable than measuring long-term effects. Moreover, when long-term effects can be measured, they can challenge what was previously believed to be predictable (or not) in the short term by changing the scope of observation of the consequences and the criteria to be adopted, transforming short-term uncertainty into established facts. This temporal dimension thus contrasts:
- A prospective value of unpredictability, where the event remains feared with a variable strength of knowledge ranging from conviction beyond scientifically demonstrable evidence (justified true beliefs) to something more pragmatic limited to verified knowledge that evolves over time (justified beliefs), with variations in calculations regarding its probability, cause-effect behavior, etc.
- A retrospective value of unpredictability, where knowledge of the event is definitively established, making it possible to validate or invalidate certain beliefs and convictions.
More generally, the analysis reminds us that all the elements described previously are situated within a flow of time that continuously changes the measurement of unpredictability, necessitating a continual revision of risk analysis to avoid rapid obsolescence.