In this month’s newsletter, we gathered some valuable advice from different sources on ways to continuously improve your threat modeling skills. Threat modeling through step-by-step approaches, updates, and add-ons, embedding it in the DevOps lifecycle… There are so many ways to find out why and how to threat model!  

Enjoy the read, and spread the threat modeling message! 

The TMI line-up for this month:

• Developer-driven threat modeling at OutSystems, written by Rui Covelo
• Book bites: “Threats: What Every Engineer Should Learn From Star Wars” by Adam Shostack
• Threat Modeling talk by Sarah-Jane Madden: Introducing Threat Modeling to Established Teams
• Threat Modeling Connect: A browse through the archives
• Tips & tricks: Threat Modeling as Code with PyTM Written by Georges Bolssens and a new podcast on threat modeling!
• An update on our upcoming training

We first started threat modeling at OutSystems when the whole R&D Team could fit in a large room. Now, the team is 10 times larger and spread around the world. Still, threat modeling remains part of our secure development lifecycle. Here is why we started threat modeling, how we still do it, and the bumpy but invaluable ride it has been.

Let’s start with the basics. Threat modeling is a structured, iterative approach to identifying, qualifying, and addressing security problems in a system or application based on a four-question framework (Shostack, 2014):

• What are we building?
• What can go wrong?
• What can we do about it?
• Have we done a good job?

These four questions allow you to gather context about the Development team’s work to really understand what the team is building and think about possible problems and potential solutions. You can then re-evaluate your solutions until you feel comfortable that you have addressed all relevant issues. In more elaborate projects, you can go into more detail and use more structure, tools, and documentation.

The more complex your application is, the more complex your threat model can be. Regardless of that, if you are answering these questions, you are threat modeling, even if you’re in a team of one like I was.

The obvious choice for a growing problem

I joined OutSystems eight years ago. We had just launched our cloud offering in record time, and I was eager to bring in my systems engineering skills seasoned with a security mindset. The team was small, and we didn’t even have a security team, so I started to list what I would like to change on the security front. It was a big list.

I showed it to my manager and asked for a budget to “fix all the things.” As a small team doing a lot of work for our cloud offering, my manager had no choice but to ask me to prioritize and plan to implement the most critical security fixes first.

For me, someone who had barely scratched the surface of a career in security, this sounded outrageous. It’s security! Everything is important, right? Not really. This simple act of leadership from my manager turned out to be one of the most crucial turning points in my career. How do I prioritize security issues? What are the most critical issues?

Being new to the company, my first step was to get more information about everything to better understand the risks involved. Before I proposed solutions, I broke down each problem as simply as I could so that anyone could understand them. Then I offered solutions based on what I discovered — and that’s when it hit me. I was answering some very familiar questions:

• What are we building?
• What can go wrong?
• What can we do about it?

We prioritized based on risk, selected the top ones, and planned the fixes. We could then answer the last remaining question:

Yes, I was threat modeling. The methodology caught the attention of a few leaders in the company, tired of endless debates around security, often leading to over-engineered solutions to fix minor problems. Threat modeling allowed Engineering teams to have structured conversations around security, identify relevant issues, and then focus on the most important ones first.

Now, all we had to do was have every team do it early in the development process to maximize the value of the methodology by hopefully finding problems before development starts. And that’s when the bumpy ride began.

Getting Buy-In From Developers

OutSystems was a very small company at the time. We didn’t have a security team and we had a single threat modeling champion (me). There was no way this would work without developers’ buy-in and help. The first bump you may encounter when implementing threat modeling is getting that buy-in from developers. To them, threat modeling will sound like yet another process to follow on the path to delivering software. A delay. An annoyance. And it will continue this way until developers see the methodology’s value.

You can help them do this by surprising the team with newfound problems they didn’t know they had or showing how threat modeling can help fix the problems they know they have. The absence of an established security team at the time worked to my advantage. The methodology sounded like an idea coming from one developer to another.

We started small, focusing on a few teams with clearly defined security issues, exposed assets, or other identified security concerns that needed prioritization. Then we rolled out the process to more teams. And the following feedback was consistent:

1. “We don’t know anything about security or how to do threat modeling.”

Not knowing about security or threat modeling is easy to tackle with documentation, guidelines, examples, and an introductory session. In no time, I saw developers chiming in with security problems that security-savvy engineers weren’t aware of. How do you create this mindset? I advise focusing on whatever can go wrong rather than trying to think like a hacker. Hackers invest in tools that we don’t have while using their creativity. So, instead, look at what impacts users, the company, and the business, and start from there. As soon as someone starts identifying one of these, the whole team usually chimes in, offering solutions and more things that can go wrong.

2. “We don’t have time to do threat modeling.”

The feeling of being pressured to deliver soon is a common reason for not making the time for threat modeling. This argument can be addressed by having the initiative’s sponsor or team lead state it as imperative right from the start. These are also the same people who can help reprioritize items to ensure this event takes place. Threat modeling should be part of the design process and if you are estimating effort for designing, include threat modeling.

Having overcome these two hurdles, the biggest challenge for me was defining when threat modeling is relevant. But more on that later.

Teaching How to Threat Model

As I pointed out before, we didn’t have a security team in the early days, and the R&D Team was growing quickly. There’s only so much one security-savvy developer can do, so it was vital to get help from other security-savvy developers and turn them into security champions – these champions led by example, teaching and asking questions across all teams. Sometimes asking, “Have you threat modeled?” goes a long way.

Our first approach to teaching teams was to create high-level guidelines, point out some training sessions, and let the teams autonomously decide what worked for them in terms of process, format, and risk scoring for prioritization.

Guess what? It didn’t work.

The feedback from developers was that high-level documentation and freedom of choice gave them too much to decide and waste time on. What helped was step-by-step documentation, which we posted on our internal wiki, with real-life examples and a few one-hour recorded training sessions. We also shared some training courses anyone could take to learn more about the methodology.

Examples include complete and real threat models from other teams and requirements, assumptions, and threat lists that developers can check and use if relevant to their project. STRIDE can be helpful, but it still confuses some developers that prefer actual threat examples.

However, some teams feel the need for an introductory face-to-face session so that they can ask questions. The help from our security champions and, later on, from security engineers was invaluable in getting the word across. We handled these sessions on an ad hoc basis, proactively scheduling meetings with teams who had many questions or promptly accepting meeting requests from those looking for answers.

Prioritizing Threats

A critical output of threat modeling is a prioritized list of threats so that you can focus on the most critical first. There are plenty of scoring frameworks that can help create some objectivity and common language around the risks involved, prioritizing the threat list in a way everyone can understand. You’ve probably heard about DREAD, CVSS, or OWASP’s risk rating methodology.

On the other hand, risk frameworks have some caveats. There is no one-size-fits-all risk framework. Some are overly complicated, and others are too simple. If you take them too seriously, you might still end up prioritizing risks wrongly, dismissing important ones, or focusing on those that are unlikely to occur.

I’ve seen developers spend copious amounts of time trying to get the score right and still get it wrong. When you have a threat shortlist you will address it anyway, it does not make sense to waste time trying to obtain the perfect score.

Still, they are a valuable tool if you need to establish priorities. One team created its own risk framework as the existing ones did not cater to their concerns. Since they used a customized framework that wasn’t familiar to other teams and lacked documentation and examples, understanding it could be somewhat challenging. Still, for that group, it helped them achieve their particular purpose.

When is Threat Modeling Relevant?

This is a relatively common question. You will find many tips and rules of thumb online from looking at changes in your attack surface (the entry points of your organization or system that are susceptible to hacking), dataflow, new APIs, new assets, or, sometimes, features. But they basically usually tell you that you only need to do threat modeling when you have “relevant changes” to your application or system.

Unless you use these tips to focus on the particular changes that are more likely to create threats, such advice can be either too generic or too specific and often hard to follow or even lead to not looking at problems. In my experience, there are plenty of things that can go wrong with very simple changes to your system.

On the other hand, more than following a formal process, threat modeling creates the right security-by-design mindset, which pertains to delivering products and capabilities that are foundationally secure, with security strategies imbued right from the software design stage and a thought-structuring methodology.

The trick relies on starting simple and getting more complex as needed. Making the threat modeling process lean and fast so that simple projects get a simple threat model process. You should always think about what you are building and ask yourself what can go wrong. And that is already part of threat modeling. The more detailed threat modeling happens when you have a complex system where more attention is needed.

Here are some lessons learned over the years about threat modeling in a fast-growing and fast-changing environment.

How We Do Threat Modeling at OutSystems

I have to confess: our original dream of having all developers on board doing threat modeling regularly and autonomously didn’t come to fruition. Not entirely, at least. The Development Team keeps growing, and we can’t afford to release anything without ensuring all security practices are in place.

A security review is a mandatory requirement for everything reaching the production stage. Threat modeling is one of the practices within the security review process.

Fortunately, we now have a skilled Security Team that can help. But some things have remained the same: developers are at the heart of the threat modeling and security review processes. What started as a workaround is now an essential part of the process.

The reason for this is simple. Security is everyone’s job and developers know what they are building better than anyone. At the same time, the most significant value of threat modeling is not in the resulting formal documents or sign-offs but in learning about the security implications of what you are building.

At OutSystems, we do threat modeling at the design phase and in every development initiative. We do it even if the output simply describes what you are doing and there are no particular security risks involved.

If we have a long list of threats to address, we use simple risk frameworks to keep us on the right path. We use them as a tool but not as a formal score. There is no need for scoring if all threats are to be addressed or if none of the threats is relevant.

Development teams drive at least the first iteration of the threat model, which will get reviews and contributions from the Security Team. This can happen async or during one-hour meetings, depending on the complexity of the solution.

From the outputs of threat modeling, Development teams, and Security teams redesign, build mitigations to the threats, and create tests. Mitigation strategies can include design changes, new features, and detection and recovery capabilities. The threat model is also used to develop and fine-tune automated testing to ensure threats are adequately covered. Everyone’s input is relevant.

We focus on changes but keep track of previous models. Why? It allows us to keep improving our overall security while continuing to learn.

The bumpy ride has proved invaluable. I am sure the way I describe how we do threat modeling today will change in the future. As systems change, teams change, and need change, we adapt and change processes. But the crux of it remains unchanged: create a secure-by-design approach where you focus on the problems you need to solve before you create them.

References

Shostack, A. (2014). Threat Modeling: Designing for Security. Wiley.

Adam Shostack: “Threats: What Every Engineer Should Learn From Star Wars” (Wiley, 2023) 

Adam Shostack writes: Yoda teaches us that “when 900 years old you are, forgotten what it was like to be a beginner, you will.” I may have the quote slightly wrong, but as experts in cybersecurity, it can be easy to forget that you had to learn about the relationship between authentication and identifiers, or the difference between authentication and authorization. When we combine this with the many paths people take into cybersecurity, there was a need for a standard text that people can read to learn about the threats we use in threat modeling. 

Threats: What Every Engineer Should Learn from Star Wars is my solution for this problem. Mixing clear technical descriptions with fun Star Wars stories, this book delivers on the promise of ‘what every engineer should learn from Star Wars. Available now as paperback, ebook, or audiobook, wherever fine books are sold.” 

Global AppSec Dublin: Introducing Threat Modeling To Established Teams – Sarah-Jane Madden

Global AppSec Dublin has provided us with a ton of great material such as this great video by Sarah-Jane Madden on the introduction of threat modeling in established teams. This is a topic we encounter frequently in our threat model coaching, so make sure to give it a watch!

Read more