Welcome to the first edition of Threat Modeling Insider in 2022.
With this newsletter, we deliver guest articles, white papers, curated articles and tips on threat modeling that help you bootstrap or elevate your threat modeling knowledge and skills.
Our “TMI” line-up:
- A guest article by Jeevan Singh covering “Threat Modeling Redefined: The Self-Serve Threat Model” ;
- Curated resources covering new standards from NIST and OWASP including threat modeling, and an ATT&CK-like Common Threat Matrix for CI/CD Pipelines;
- A Toreon blog post covering 9 benefits of Threat Modeling;
- Tip of the month: the OWASP Threat Dragon egg is hatching;
- Our new Threat Modeling Practitioner hybrid learning journey.
Our New Year’s resolution is to send out more newsletters in 2022 🙂
Introduction by Seba and Jeevan
My last international flight before COVID was to RSA in San Francisco in February 2020.
There I met Jeevan Singh again. Jeevan attended one of our first Black Hat trainings and has introduced and improved the threat modeling practice within Segment a lot.
Jeevan wrote one of the best blog posts on practical threat modeling in 2021 on how to have the engineers take the lead in threat modeling.
I am very happy that Jeevan agreed to include it in our newsletter.
Seba, 18 January 2022
So here it is:
I am Jeevan Singh, I run the Application Security program at Segment.
I am very honored to be featured in this iterations of Toreon’s Threat Modeling Insider newsletter.
A year ago we at Segment decided to do something radically different, we wanted the Engineering organization to take over threat modeling responsibilities. We wanted to go down this route for many reasons:
- Engineers will find more vulnerabilities than Security Engineers because they know their systems better
- Security won’t be a bottleneck for the Engineering team if the Engineering team does the threat model themselves
- Since every engineer knows how to threat model, that means every feature we build will have security built into it Before going down this route, I reached out to security experts to see if they agreed with the potential outcomes.
Before COVID hit, I even had an a coffee with Sebastien, to talk more about my plans.
We both agreed that this would be the next evolution of threat modeling.
Segment is very big on giving back to the community, if you have any questions about the article or the open sourced training, please feel free to reach out to me at email@example.com.
I would be happy to answer any of your questions.
Jeevan Saini on March 29th 2021
It was a cold and dark mid-December day in Vancouver; another Staff Security Engineer and I were a part of a developer-led threat modeling session. We couldn’t find a single additional security concern for the developer-led team. The developer leading the project had considered all the options!
When the session was over, the Staff Security Engineer asked: “What have we done? What purpose do we even have any more?” We joked about losing our jobs, but we knew that this is the best kind of problem to have!
Record scratch, freeze frame: You may be wondering, “How did we get here?”
Why is threat modeling important?
If you’ve never heard of Threat Modeling before, you’re not alone. Threat Modeling can mean very different things to different people. At Segment, threat modeling is the process of systematically identifying assets within a system or feature, brainstorming ways to compromise those assets, and collaboratively designing ways to defend them.
Threat modeling is a critical step in Segment’s Security Development Lifecycle. Before we make a feature from Segment available to the public, the Security Engineering (SecEng) team reviews it. Our job is to reduce the risk of a bad actor exploiting a vulnerability in that feature, which ultimately ensures the safety of our clients’ data.
What are the benefits of threat modeling?
We love to do threat modeling because we get to fix vulnerabilities before they are coded into the system. It is much cheaper to fix vulnerabilities like this during the design phase. Fixing vulnerabilities in production might mean that a developer has to learn about the vulnerable system from scratch, on-the-fly, and spend valuable time context switching. We also save money because we reduce the number of vulnerabilities discovered in our bug bounty program. And since developers spend less time fixing vulnerabilities, it means they get to focus on building features, not fighting fires.
From a security perspective, it also means fewer security incidents where we have to activate our SIRT and possibly tell our customers we’ve messed up. Everyone wins, as we’ve essentially created a Utopia for developers and Security engineers!
What are the challenges of threat modeling?
Threat modeling sounds great, but it isn’t without its challenges. As much fun as threat modeling is, there are usually only a handful of folks at an organization that are good at threat modeling. Security teams can often become bottlenecks on threat modeling and reviews due to the ratio between Security Engineers and Engineers in an organization. Some people think threat modeling is too expensive when counted in engineering-hours, given the number of people involved. But given the engineering-hours you save in responding to incidents, this isn’t true.
With Segment growing at a very rapid pace and our feature set growing even quicker, it was time to try something different. Self-serve threat modeling checks all of the right boxes: it removes bottlenecks, and helps deal with an unbalanced ratio of security engineers to developers. If we could meet those requirements, we’d have built ourselves a Segment Threat Modeling Utopia.
Why a self-serve threat model?
In our security Utopia, the Engineering teams perform the threat modeling exercises, and the Security team are optional attendees. Security can then pick and choose which sessions they attend based on interests, priorities, and size of attack surfaces.
The security “goal state” is important to model, because it solves many problems. In our current world, the Security team’s availability becomes a bottleneck, but in our goal state, no engineering team is blocked by a lack of Security personnel, because the engineering teams can run their own threat modeling sessions and the Security team are optional attendees. In our current world, threat modeling can take up additional engineering hours to review, where in the goal state, threat modeling activities are built into the design review, which is already a standard meeting in the development cycle. And while threat modeling is hard to scale when only a handful of people are responsible for it, it scales beautifully when everyone can do it.
Developers know their features better than anyone: they understand what works well, where the gaps are, and where they might run into trouble. In contrast, Security Engineers can only spend a few hours on a specific feature when they assess it, but are expected to provide in-depth security concerns about that feature. It makes a lot more sense for the feature experts to discuss potential security concerns, rather than the folks that are looking at the problem from the outside.
Size matters. The Engineering teams are considerably larger than the Security Engineering team. As a Security Engineer, I know when more folks look at a problem, we discover more potential issues. We believe that with training and practice, coupled with their intimate knowledge of their features’ architectures and codebases, our Engineering teams will be better at threat modeling than our Security team. With this outcome on the horizon, it just makes sense to invest in self-service threat modeling training.
How we built Utopia
Self-serve threat modeling is an Engineering-wide initiative and it has taken considerable developer time. We have a comprehensive curriculum that goes through several stages to make sure this project is a success.
There are five distinct phases to have Engineering self-serve threat model:
- Traditional Phase
- Training Phase
- Observation Phase
- Review Phase
- Security Optional Phase
1. Traditional Phase
This is less a stage, and more “where we were before we started the self-serve threat modeling project.”
All features need a security review during the design phase. Security is responsible for the review and will build the threat model for the feature when appropriate.
2. Training Phase
In order to shift to a self-service model, the first and most important step is to train developers how to perform a threat-modeling exercise. During this stage, you teach developers the threat modeling workflow, strategies for how to find threats, how to prioritize them, and how to decide which threats should be addressed.
(We’ll cover the actual training in more detail below.)
3. Observation Phase
This is where Segment is currently in our journey.
The vast majority of developers have been fully trained, and now teams lead their own threat modeling sessions. The Security team is still (virtually) in the room, and are there to help guide engineers to a successful session.
The people who are leading the session create all of the necessary pre-read material before the threat modeling session begins. We use an established threat-modeling section as part of our standard design document template. The design document provides the appropriate context, but developers are asked to create architectural diagrams, explain what is in-scope and out-of-scope for the meeting, and catalog the assets involved in the project. The assets could be infrastructure, the application system itself, data in the system; anything that’s worth protecting gets listed out as an “asset.”
Security’s role during this phase is to attend the meeting and help guide the conversation whenever required. Security does NOT tell the developers what the vulnerabilities are, but rather guides the team to discover the vulnerabilities themselves. Although a threat model is occurring, this is still a great time to teach developers how to discover vulnerabilities themselves.
There are still situations where developers do not discover important vulnerabilities, even with hints, and that is fine. When they observe this, Security gets involved and helps guide the thought process so the development team learns to discover those vulnerabilities. Again, this is a teaching moment, and everyone involved goes away feeling better about the threat model.
We’ve had many developer-led self-service threat modeling sessions so far, and we know that things are going well when a Staff Security Engineer says “Well, I feel completely useless in this meeting.”
Threat modeling is not an easy skill to learn, and it takes practice to become good at it. It is our responsibility as Security Engineers to create a culture of learning. As our teams start to discover critical- and high-severity vulnerabilities more consistently, they graduate on to the next phase.
4. Review Phase
In this phase, the Engineering teams are so good at threat modeling that the Security team no longer needs to be in the meeting. Our job is to review the artifacts from the threat modeling session, and validate that we don’t anticipate additional critical- or high-severity vulnerabilities.
You might keep finding medium- and low-priority vulnerabilities. That’s great! You’re encouraged to share them with the team, and explain how you discovered those vulnerabilities.
To go on to the next phase, the Engineering teams must discover critical-, high- and medium-severity vulnerabilities consistently. As an aside, after a few cycles of this, the Security team should start seeing a noticeable decline in bug-bounty and penetration test findings for those teams’ features.
5. Security-Optional Phase
This is the ideal world that we Security Engineers dream about. Once teams are at this stage, it is optional for Security to even review threat models.
We believe it may take us a year or two to get to this phase, but we know that in the end, it will have been worth the effort.
Training deep dive
The team at Segment reviewed many options for training. We ultimately decided that the best way to proceed would be to build the training from scratch. The training itself had to be Segment-relevant and specific to our cultural norms.
The training was delivered to the entire Engineering organization, and anyone else who wanted to sign up: this included product managers, tech writers, and members of the general IT staff. This meant that the training had to be engaging for all levels of skill and experience. Classes were made up of a mix of folks that were junior, mid-career, and very senior. The training had to also be relevant to people who have never done threat modeling before, as well as to others who are security champions.
We broke the training down into three parts, which we delivered in three-week intervals:
- Part 1 – Introduction to Threat Modeling
- Part 2 – Foundations of Threat Modeling
- Part 3 – Live Threat Modeling
Introduction to Threat Modeling
The goal of the first training session is to introduce the concept of threat modeling, teach the workflow, and show developers the tools Security uses to find vulnerabilities.
We introduced the concept of threat modeling to the developers by making parallels with how you might think about personal safety. This is much more relatable than diving right into software vulnerability concepts.
Because we started this process in April of 2020, in the very first exercise we talked about what our grocery shopping habits were like before, and then after the pandemic. This exercise got us talking about risks, and how they evaluate and decide their tolerance for risks. We work through several non-software risk exercises to help make threat modeling relatable.
Later in the session we introduce STRIDE to the class, and do a deep-dive and talk about each vulnerability class. We then talk about real-life vulnerabilities and how they impact applications. We do some light exercises to help the class get used to asking the right security questions.
Foundations of Threat Modeling
The goal of the second training is to get the developer’s hands dirty with exercises to solidify their knowledge.
We introduce the concepts of “assets,” and the value of a diversity of viewpoints in the threat modeling sessions. It’s important that as we go through a threat modeling exercise, everyone in the class understands the assets in the system we’re assessing. It’s also important that all members of the Engineering teams get involved in the threat modeling exercises. Each member of the team has a slightly different background and experience, and so they each see the system in a different way. When finding potential vulnerabilities, everyone’s voice is important.
As a part of this training, we have a lengthy discussion about a fake Segment feature: a credit-card billing system. The class is broken up into small teams, and each team reads a design document and talks through it to discover potential vulnerabilities in the system. At the end of the evaluation, everyone comes back together, and we discuss and compare the top three threats from each group. The class learns how to read a design document with a focus on security, how to identify security vulnerabilities, and how to prioritize risks.
Fake Segment application we learned upon
Live Threat Modeling
Performing an end-to-end threat modeling session with the team cements the concepts that we discussed.
In the final training, we do an actual threat modeling exercise. We discuss the feature we are threat modeling, and review the architectural diagrams. We then list the assets in the system, and discuss what is in, and out of scope for the exercise. Next, we spend some time finding potential vulnerabilities, and discuss how to prioritize them. Finally, we discuss when the vulnerabilities we discovered should be remediated, based on severity and priority.
We’ve learned a LOT while rolling out our self-serve threat modeling program. Here’s some advice if you’re considering doing something similar.
One of our initial learnings was to fail fast and iterate on the training contents. There were two teams willing to sit through the very first training iterations.
Immediately after the initial sessions, I held focus groups to get attendees’ feedback and better understand how to improve the training. It had an immediate impact – we adjusted the length of our sessions, and changed what concepts we introduced and when. With feedback from the first team, we modified our approach, and ran the training for the second team. With the feedback and suggestions from those two initial teams, we solidified the training content and approach. We were then ready to roll it out for the rest of the organization.
Trust your counterparts
I was initially hesitant, but I have been totally impressed by our developers. They learn quickly and are interested and engaged in the training. When you create a safe space for them to learn and make mistakes, you’ll get to your self-service goals much faster. Trust that they can figure out how to uncover vulnerabilities, and coach the team on how to find them.
Security culture improvements
As I rolled out the threat modeling training, I noticed that people in the classes really enjoyed the training. This led to more people reaching out to me about the concerns that they have about vulnerabilities within their system, allowing me to give them proactive advice and support. In the past, developers only contacted me sometimes, but now they reach out far more often, and in greater numbers.
The simplicity in the training made the scary topic of “security” accessible to everyone, which makes them less afraid to ask us security-related questions. Many folks have even asked about more security learning resources, and how to join application security related meetups.
With this project, we identified and created security champions at Segment. There are folks that are interested in learning about, security, and who often impress us with their security knowledge. We have only seen benefits from this program.
How do we get to Utopia?
Every company is unique! I recommend that you start somewhere, and adjust the training to fit your culture.
We at Segment don’t want you to start from scratch! Today, we’re open-sourcing Segment’s Threat Modeling training content for you to adapt and modify. You can find the links below.
We hope that you can use these training modules, give feedback on how we can make it better, and share what you find with the community.
The slides are great, but how can I learn more?
We have a strong sense of community here at Segment and in addition to sharing our training slides and this blog post, we have an email address so you can submit your questions and thoughts about the program. I am always interested in hearing feedback, good or bad!
Email address: firstname.lastname@example.org
Future improvements, even in Utopia
So, we’re on our way to Utopia, and things couldn’t look brighter. Time to rest on our laurels, right? Wrong. We’re constantly thinking about how we could improve our threat modeling training, and build sustainable systems. We’ve trained all of our current developers, but what about new Segmenters in the future? We’re thinking about how best to incorporate this training into onboarding for new hires.
We’re also thinking about how to keep this knowledge fresh. Maybe you took the training months ago, but haven’t had to use it since, and now you have this huge project – yipee! Threat modeling refresher courses to the rescue! Finally, for the people who really enjoyed the training and want to learn more, or who really want to deep dive on advanced strategies for large and complex systems, we’re working on a Threat Modeling “201” class.
Curated threat modeling content
Will new standards from NIST and OWASP push threat modeling mainstream?
Expect a growing demand for threat modeling expertise and tooling in the coming years. Both NIST and OWASP doubled down on threat modeling in 2021.
The National Institute of Standards and Technology (NIST) included Threat Modeling as the first software verification technique in their Recommended Minimum Standard for Vendor or Developer Verification of Code.
Open Web Application Security Project (OWASP) has published a new version of the OWASP Top 10. OWASP has added Insecure Design to the Top 10. This item focuses on design flaws and advocates for the use of Threat Modeling.
Threat modeling to solve our supply chain attacks?
Supply-chain attacks are one of the most serious risks for a lot or organizations. Threat modeling is not only useful for our applications, we should also consider the integration and deployment pipelines of our software. With CI/CD on the rise we should have a better understanding of the attack surface and the attacks against it. The Mercari security team created an ATT&CK-like Common Threat Matrix for CI/CD Pipelines and shared it with the Cybersecurity community.
Toreon blog post: 9 benefits of Threat Modeling
Next time you need to pitch threat modeling, here is a list of 9 benefits you get from Threat Modeling your application or other system, during the design phase.
This is the second post in our series about Threat Modeling. The series is meant to provide basic information about the practice of Threat Modeling, why we love it and how it can make your organization more secure. Check out our previous post here: What is Threat Modeling?
Tip: OWASP Threat Dragon egg hatching
The OWASP Threat Dragon tool has been promoted to OWASP Lab project status in November. The latest release before 2.0 was 1.6.0, including Automated threat and context threat generation, mainly based on OWASP Automated Threats.
Threat Dragon is a free, open-source, cross-platform threat modelling application including system diagramming and a threat rule engine to auto-generate threats/mitigations.
Extra: New threat modeling practitioner training
As announced in our previous newsletter, we have transformed our threat modeling training towards a new hybrid learning journey to become a Threat Modeling Practitioner. We have integrated all the expertise and experience of Toreon on threat modeling in a new hybrid learning journey.
The course is a blend of self-paced learning and hands-on live labs with exercises based on real-world projects. Since 2021 we deliver our threat modeling training through our subsidiary Data Protection Institute (DPI).
Discover and register for our new Threat Modeling Practitioner training on the DPI website.
We aim to make this a community-driven newsletter and welcome your input or feedback. If you have content or pointers for the next edition, please share them with us.
Book your seat in our Threat Modeling Practitioner course
If creating or updating a threat model is essential to your line of work, then this course is for you. Our threat modeling practitioner training targets software developers, architects, product managers, incident responders, and security professionals.