In most incident investigations, including those involving cyberattacks, industrial accidents, and plane crashes, the findings often point to the same cause: human error. Upon closer inspection, however, it often turns out that the real cause was something else. Our tendency to pin everything on human error is related to a common cognitive distortion known as hindsight bias.
There is an interesting research study that explains how this works.
Essentially, the people investigating an incident already have a certain bias because they know the end result. This means that they automatically look at what happened with a certain level of prejudice. They have the impression that the professionals involved, such as the industrial engineers or pilots, perceived certain events leading up to the incident as being more significant than the actual events at the fateful moment of crisis. They often feel that these professionals “should have known” that these factors would “inevitably” lead to an accident.
In reality, these professionals are constantly working under conditions that require them to consider a multitude of factors and make quick decisions. In the heat of the moment, they simply cannot always know that a specific factor will lead to an accident. This is because accidents often occur as a result of an unfortunate cascade of factors that would not be critical on their own, but become critical when stacked or combined. Interestingly, this fact is confirmed by the real-life incident commonly known as the Miracle on the Hudson. You can read about it here.
As the research study suggests, the real causes of similar incidents are problems with the design and architecture of systems that do not account for such issues.
For example, when investigating a plane crash, it is easy to blame the pilot for not noticing a specific factor, such as a tumbler switch being in the wrong position. In the context of the here and now, we already know the outcome and therefore emphasize this factor as the primary one. However, at the moment of crisis, the pilot did not know this and may have been distracted by another factor. The position of the tumbler switch may have been fine under normal conditions, but an unfortunate set of circumstances may have combined to cause an accident.
Hindsight bias in real-life scenarios
A similar effect often occurs in real life. Have you ever understood or done something that seemed obvious in retrospect? You get all self-critical: why didn’t I understand or do that sooner? Personally, this happens to me fairly often. However, things were never quite so obvious. The final decision or action was just within the range of possibilities. This perception is caused by the cognitive distortion known as hindsight bias.
This same cognitive distortion is why people may start whining that it was obvious all along that the dollar exchange rate would rise, or that some other thing was going to happen that had merely been within the range of possibilities.
Likewise, when some new project or idea blows up, there are often people who claim that they had been working on the same thing for the past 30 years, and they should get credit for it. What’s more, they often seem genuinely sincere when they say it.
But what does cybersecurity have to do with all this?
Well, a similar effect happens in the sphere of cybersecurity. Today’s software/hardware systems conceal an enormous amount of complexity and connectivity, especially if you look at in-vehicle or industrial systems. This leads to a sort of “combinatorial explosion” of a multitude of interactions, not only between a given system and external systems, but also between components within a single system.
But how are most of these systems usually created? First, an initially insecure system is developed, and then external security tools are applied to this otherwise insecure system. However, with so many interactions occurring in the original system, it becomes increasingly difficult to maintain this type of external security fence around all of these interactions. In this case, hacking a single component can lead to an ever-increasing range of methods that cybercriminals can use to expand an attack on the system. A successful hack of these types of systems is only a matter of time. Sooner or later, a cyberattack and/or human error, as we often see with plane crashes, will serve as the last missing domino that leads to system failure.
Therefore, alternative approaches that create secure-by-design systems are not just futuristic dreams that we should start thinking about and gradually working toward. Instead, we need to aggressively implement them now to keep pace with the changes that have already occurred.
In most incident investigations, including those involving cyberattacks, industrial accidents, and plane crashes, the findings often point to the same cause: human error. Upon closer inspection, however, it often turns out that the real cause was something else. Our tendency to pin everything on human error is related to a common cognitive distortion known as hindsight bias.
There is an interesting research study that explains how this works.
Essentially, the people investigating an incident already have a certain bias because they know the end result. This means that they automatically look at what happened with a certain level of prejudice. They have the impression that the professionals involved, such as the industrial engineers or pilots, perceived certain events leading up to the incident as being more significant than the actual events at the fateful moment of crisis. They often feel that these professionals “should have known” that these factors would “inevitably” lead to an accident.
In reality, these professionals are constantly working under conditions that require them to consider a multitude of factors and make quick decisions. In the heat of the moment, they simply cannot always know that a specific factor will lead to an accident. This is because accidents often occur as a result of an unfortunate cascade of factors that would not be critical on their own, but become critical when stacked or combined. Interestingly, this fact is confirmed by the real-life incident commonly known as the Miracle on the Hudson. You can read about it here.
As the research study suggests, the real causes of similar incidents are problems with the design and architecture of systems that do not account for such issues.
For example, when investigating a plane crash, it is easy to blame the pilot for not noticing a specific factor, such as a tumbler switch being in the wrong position. In the context of the here and now, we already know the outcome and therefore emphasize this factor as the primary one. However, at the moment of crisis, the pilot did not know this and may have been distracted by another factor. The position of the tumbler switch may have been fine under normal conditions, but an unfortunate set of circumstances may have combined to cause an accident.
Hindsight bias in real-life scenarios
A similar effect often occurs in real life. Have you ever understood or done something that seemed obvious in retrospect? You get all self-critical: why didn’t I understand or do that sooner? Personally, this happens to me fairly often. However, things were never quite so obvious. The final decision or action was just within the range of possibilities. This perception is caused by the cognitive distortion known as hindsight bias.
This same cognitive distortion is why people may start whining that it was obvious all along that the dollar exchange rate would rise, or that some other thing was going to happen that had merely been within the range of possibilities.
Likewise, when some new project or idea blows up, there are often people who claim that they had been working on the same thing for the past 30 years, and they should get credit for it. What’s more, they often seem genuinely sincere when they say it.
But what does cybersecurity have to do with all this?
Well, a similar effect happens in the sphere of cybersecurity. Today’s software/hardware systems conceal an enormous amount of complexity and connectivity, especially if you look at in-vehicle or industrial systems. This leads to a sort of “combinatorial explosion” of a multitude of interactions, not only between a given system and external systems, but also between components within a single system.
But how are most of these systems usually created? First, an initially insecure system is developed, and then external security tools are applied to this otherwise insecure system. However, with so many interactions occurring in the original system, it becomes increasingly difficult to maintain this type of external security fence around all of these interactions. In this case, hacking a single component can lead to an ever-increasing range of methods that cybercriminals can use to expand an attack on the system. A successful hack of these types of systems is only a matter of time. Sooner or later, a cyberattack and/or human error, as we often see with plane crashes, will serve as the last missing domino that leads to system failure.
Therefore, alternative approaches that create secure-by-design systems are not just futuristic dreams that we should start thinking about and gradually working toward. Instead, we need to aggressively implement them now to keep pace with the changes that have already occurred.
