The cybersecurity industry focuses on risk. As the risk landscape evolves, we aim to identify early and adapt before our own organizations are affected. Incidents at other organizations are a canary for ourselves, and we can't afford to ignore them.

Tech is in a rough spot right now. Tech layoffs have been rampant for months, and organizations are reevaluating investments. While not totally immune, cybersecurity has been impacted less than other areas of the tech sector. I believe there are three primary reasons driving this:

  1. Cybersecurity control failures are one of the top concerns for executives.
  2. Many teams face severe skills gaps, making layoffs more difficult.
  3. Recent high profile breaches reinforce the costs associated with cybersecurity failures.

This leads me to wonder: what would happen if we did not have these tailwinds? What happens if defenses become more effective, and there is a material reduction in incident frequency and severity? Could meeting our goals expose organizations to greater risks?

It is impossible to predict the future, but I could see successful defensive programs leading to chronic underfunding and ultimately major catastrophes. Yet, I also believe that security teams can align themselves with shareholder and executive interests, leading to a sustainable long-term culture of security. In this post, I will explore what could lead to both, and my thoughts on how this next era should be navigated.

The Collapse of Credit Suisse

Cybersecurity has a lot of similarities to financial risk management. This makes financial risk management a useful parallel to predict future trends for the security industry. Most relevant for this discussion is that the better job you do, the less noticeable you become. This means that if an organization has an effective cybersecurity strategy, it risks facing underinvestments or cuts that would lead to it no longer being effective.

The 2008 collapse of Lehman Brothers directed the attention of regulators, executives, and customers to financial risk management. The subsequent bull run, lasting well over a decade (with a minor exception for early pandemic), made risk management seem less necessary. However, in the early 2020s, a number of high profile bank collapses resumed. One of the most prominent was Credit Suisse.

Credit Suisse (CS) was one of the two major banks in Switzerland. CS experienced a number of scandals, linked to risk management failures (most prominently Greensill and Archegos). The Archegos report emphasizes that the failures were not just technical, but also cultural. They suffered severe underinvestment in risk management personnel and technology:

Both the first and second lines of defense were generally under-resourced both in terms of absolute headcount and expertise  (146).

Swiss banks, including Credit Suisse, had long been associated with safety. The post-2008 period should have been the perfect opportunity for them. The report does not go into the cultural reasons why CS deprioritized risk management. However, it was a long, gradual transition including:

  • Prime Services Risk (PSR) highly skilled employees were replaced with junior ones as they left (28)
  • The replacement for the head of PSR was a marketing executive with no risk management experience (28)
  • PSR failed to invest in risk technology (28)
  • PSR employees were severely overburdened (52)

While no reason was attributed in the report, I suspect a major reason is that between 2010 and 2020, the banks that collapsed were small and immaterial to the global system. In good times, risk management is just an expensive cost center and time suck. Risks at the large banks were being managed well, so the cost and time dedicated to it seems less important. Only once the controls had a catastrophic failure did risk management become relevant again.

The Risk of Complacency

I hear a lot of people in the cybersecurity industry talk about the benefits of defense as if it is self-evident. Every high profile breach reinforces the benefit of investments in defense. However, if organizations do meaningfully reduce their exposure, and breaches become less common, it could become less self-evident.

When a major hack is on the front page of the Wall Street Journal or the top of the 6 o'clock news, it reinforces to people outside of security why investments are necessary. To go back to our Credit Suisse example, it would be as if there were a bank run on a Global Systemically Important Bank ever year. In that environment, every bank would be investing in risk management.

I am not a fan of the fear mongering after a major attack, and most people I know aren't either. A lot of the marketing after a breach is tiring, and things can get blown out of proportion. That being said, I think we benefit a lot from it, even if we don't explicitly exploit said fears.

Let's say that all of the investment pays off, and there is a material decrease in frequency and severity of an attack. What happens then? I think there would be a real risk of complacency and undermining of what made such achievements possible. Security teams would remain expensive. Users could become frustrated at how the controls disrupt their productivity. Since the threat would feel more distant, it would be easy to become complacent and regress.

We are many years out from reaching this reality. However, cybersecurity teams should be prepared to justify their importance outside of this period of heightened concern. That would bolster their support today and reduce the risk of gradual declines in the future. Thankfully, small changes in message and mission could help foster greater alignment and reduce the risk of complacency.

Velocity not Maturity

What defines a successful security program? Is an organization ever secure? There are two ways that I have seen to evaluate security programs: velocity and maturity. Velocity looks at continual improvement over time while maturity takes a snapshot of time and looks at if an organization is sufficiently secure.

I think that most people within the industry look at security through the lens of velocity. However, I think a big reason for this is because of how young defensive cybersecurity is and how common attacks still are. However, I also think that we could get ourselves into the trap of talking about security at major American/European companies in maturity terms rather than velocity.

In the absence of frequent examples to the contrary, maturity is a lot easier to talk about. Are planes safe? For the most part, at least on major airlines, yes. Are cars safe? Well, sort of, but we are continuing to make them safer.

People think of aircraft as safe because of the rarity of events in the west. I personally have tremendous faith in aviation, even despite the turbulence over the past couple of years. In two weeks, I am flying transcontinental on a 737 MAX 9, despite a mid-air door blowout recently, and I have no concerns. At the same time, if I were in Indonesia, I would be more careful with the airlines I choose.

Preliminary NTSB report on Boeing 737 Max 9 Alaska Airlines flight finds missing bolts led to mid-air door blowout
Bolts holding a door plug in place on an Alaska Airlines flight were missing, the National Transportation Safety Board found in a preliminary report released on Tuesday.

For airlines and manufacturers, it is also convenient to say aircraft are safe. It instills greater customer trust, and can counteract often dramatized fears of flying. At the same time, this perspective that aircraft are safe can lead to underinvestment and loss of confidence in the event of major issues.

I suspect that the industry, especially third party vendors, could start to walk into the trap of saying that "yes, your organization is secure." This is what our stakeholders want to hear. Yet, the industry must resist. If we say this, it becomes easier to relax and divest, opening the door to future attacks. After all, our planes were safe and our deposits secure until 737 MAX started diving into the ground and Credit Suisse disappeared.

Adversarial Dynamics

From planes to banks and beyond, there is one thing that I think will make it easier to keep the velocity perspective on cybersecurity: the fact that we have active adversaries. The way to reach a state of "maturity" is to engage in a race between defensive and the most advanced offensive forces such that it crowds out other less-wealthy offensive forces.

I think that major companies could, with enough investment, reach a point where their risk profiles are significantly reduced and attacks generally become less frequent and less severe. This would be good for organizations and a sign that security teams were having an impact. In this situation, it would still be vital to invest and phrase communications as continuing to become more secure rather than being secure.

Value-Added Security

My article thus far has looked at security as primarily a cost center. However, I don't think that security has to only be a cost center, especially for large companies. I have talked and written about this opportunity before, but will reiterate it here as well.

BSides Toronto - Quality Engineering
Download the Slides Quality Engineering AppSec.pdf 15 MB download-circle Sign Up for my Newsletter What does it take to have a successful SDLC program? In this talk, I step beyond the policy, and share my experiences turning a failing software development team into one that was both fast and
Services: The AppSec Inflection Point
AppSec teams should adopt a service-oriented model and API proxies to improve security across stacks.

Large organizations often have many internally developed applications. Often, these can be siloed, especially if the company is not explicitly a tech company. This decentralized development can both severely inhibit productivity and make remediation more complicated.

Netflix is one of the best examples of this approach. They realized that they did not have the resources to do individual application tests across their organization. So, they instead developed "well-integrated, secure by default central platforms." The Zuul gateway is a particularly interesting example. It centralizes many risky operations, but also brings a host of other benefits:

  • Authentication
  • Insights
  • Stress Testing
  • Canary Testing
  • Dynamic Routing
  • Load Shedding
  • Security
  • Static Response handling
  • Multi-Region Resiliency

I think that many other organizations could adopt a similar approach. If secure by default solutions make engineers more efficient, they would be more likely to adopt said solutions. That would also reduce security assessment costs (since high risk services would be shared), further decreasing the burden on security teams.

Crucially, it shifts security from a cost center to a value add. Investments in secure platforms reduce costs elsewhere while also boosting engineering productivity. As investor and executive attention shifts away from security towards other priorities, the value proposition remains.

Lean Operations

The third component to remember is to maintain lean operations. As mentioned in the previous section, security teams have a unique advantage because they are focused on cross-enterprise operations, allowing them to scale. The problem is that since scale dilutes costs, it can become easy to become bloated. Overhiring was a major driver for the recent rounds of tech layoffs.

There is absolutely a cybersecurity skills gap. Yet, I think that the real gap is less than the numbers passed around online. Extensive investments in automation, centralized platforms, and AI will reduce the amount of manual work. These could lead to losses for hypothetical jobs in the future, but I think that strong employees would adapt rather than be laid off.

The tech hiring race led to underutilized "A players," which also led to the hiring of "B players." Certainly, not everyone who experienced layoffs were "B players," and it isn't entirely the fault of underutilized "A players" that they are underutilized. However, when layoffs happen, there may be less control than what could happen if performance is continually evaluated to keep teams efficient and lean. It would be better for everyone to avoid the bloat before it happens.

Conclusion

The world is in a turbulent period, and the impact on software engineering is unfortunate. However, market cycles will naturally take place. Security may be primed for both underinvestment and bloat, which could hurt organizational security in the future.

I hope that attacks become less common and less severe, and I believe that could happen in the future. If that happens, it is important to maintain velocity and continue the steps that got us to that goal. If security is seen as a continually evolving, lean, value-add to organizations, it could protect investments in security and reduce the risk of security facing the same issues that financial risk management faced. Maintaining stability and velocity is essential for organizational security, and the current period of investment should be leveraged to maintain velocity in the future.