Soatok's Informal Guide to Threat Models

After a long day of exhausting conversations about Hybrid Post-Quantum Cryptography, random jackasses trying to play gotcha with endpoint attacks against end-to-end encrypted messaging apps, and message board discussions in the wake of dumb politicians pushing more “age verification” bullshit on us all, it’s become abundantly clear to me that the phrase “threat model” is a foreign concept to most people.

Except, y’know, as a buzzword.

To be up front: If you’re here looking for an academic resource with over 100 citations on how to write a formal threat model document for your new startup which involves multiple blockchains, this probably isn’t the gay furry blog for you. Maybe start with STRIDE and system theory.

But if you’re looking to build an intuition for what questions a good threat model should answer, and you’re starting from zero, you’re probably in the right place.

So let’s talk about threat modeling.

Threat Modeling For Neophytes

At a high level, don’t overthink this too much.

While a threat model is a formal cybersecurity process that some infosec folks actually specialize in, you can run informal threat models in the design and architecture phases of developing a new product or service and no one can stop you. You might just end up with a better result.

A threat model should, at minimum, answer these basic questions:

What are we even protecting to begin with?- If you can’t answer this, you have a lot of ground work to do.

Who/what wants to harm what we’re protecting?- Hackers, activists, cyber-stalkers, social media harassment networks

• Natural disasters / bad karma
• Underpaid and overworked employees who get fed up
• Idiotic legislatures paid by large corporate lobbyists to pass stupid laws that hurt everyone
• Nation State Adversaries!!!!1oneon

How might (2) attack (1)?- Attack scenarios go here

• Murphy’s Law goes here

What will we do to prevent (3) from happening?- Murphy’s Law also goes here!

And, like, okay. If you can check those off, you can call your document a threat model in some sense.

However, this is often useless in practice because some crucial details are omitted.

How are the assets (1) related / connected?- Think in graphs, not lists.

• Not all targets are equal value.

What assumptions are we making, especially with (4) and (5)?- I’ll say more about this below.

What threats are we deliberatelynotaddressing?- You literally cannot address every possible attack that any person will ever imagine in the unforeseeable future, so don’t pretend to.

Too many people take assumptions (6) for granted, ironically, but it’s incredibly important to be as clear about what your assumptions are.

If one of your assumptions is wrong, then your model is incomplete (at best), or your list of accepted risks (7) needs to be reconsidered.

For example: The Invisible Salamanders attack breaks abuse reporting in some end-to-end encrypted messaging designs, but only if you introduce abuse reporting.

The attack is possible because one of the assumptions that went into the AEAD schemes in question (AES-GCM, ChaCha20-Poly1305) is that there is only one valid key for a given message. The second you introduce multiple valid keys for a given message (or confused deputies for that matter), you’ve gone outside the security guarantees of your algorithm–which, as an attacker, makes for a fun trick.

Being clear about your assumptions allows you to identify your own unknown unknowns. You don’t have to be perfect.

In fact: Threat models are supposed to be living documents, not point-in-time snapshots. Update them whenever you deem appropriate.

How to Get Started

If you’re looking to do threat modeling professionally, you probably want to read the Threat Modeling Manifesto. Here’s how I approach it.

First, write down the 7 items above in a format that you can vaguely copy and paste rapidly. You’re going to need it.

Next, map out (on a large piece of graph paper, preferably–or the digital equivalent) the components of the system you’re designing or analyzing. If any widget directly talks to, depends on, or interacts with another widget, you need that relationship drawn in whatever convention is most useful to you.

Once this is setup, you want to draw a box around the entire graph, and then pretend you’re playing Fortnite: Every so often, the box gets smaller and focused more on each individual component. Each iteration, note all the inputs and outputs to each component, and try to answer as many of the 7 items as you can.

Repeat until you’ve drilled down as far as your abstraction allows you, then brainstorm what assumptions you have about the layers you aren’t drilling deeper into.

What you’re doing is starting at the highest level and working your way down into more specific pieces.

Your database probably doesn’t depend on the security of X25519 the same way that your load balancer does. But your database also probably shouldn’t have an RSS feed built into it either. Take note of inappropriate relationships and aim to sever them if you can.

Example: My Own Work

You may or may not already be aware that I’m working on delivering key transparency to the Fediverse. The work is being tracked on publickey.directory if you’re curious about the state of it after this blog post goes live.

This work began with a specification, which includes a prominently featured threat model.

The threat model is organized into the following sections:

• Assumptions (stated up front)
• Assets
• Actors (both attackers and people we want to protect), given role names
• The risks, which have one of four statuses attached Prevented by design: Attack simply won’t work lol 😀Mitigated: Attacks shouldn’t succeed, unless an assumption is wrong. Most interesting for researchers to focus on.Addressable: There’s a way to mitigate the risk, but it requires effort or care. Operators should be aware of this.Open: This is a risk we cannot or will not mitigate. These are the attacks that will succeed.

This threat model isn’t perfect, of course. I didn’t perfectly relate the assets and actors to each other in a human-readable graph. There might be blind spots in the risks section I never considered. I might have forgotten to write down some assumption that matters for the security of the system.

If you can look my project’s threat model and see its shortcomings, you probably understand the assignment well enough to write your own.

But enough thinly-veiled shameless self-promotion. You won’t learn much only looking at my example of fur-in-the game. We also need an example of a bad threat model doc, and boy howdy do I have one ready.

Bad Example: Matrix’s Threat Model

I’ve picked on Matrix before (twice), so if you’ve read those blogs, this won’t be news to you.

** This** is Matrix’s threat model (latest, v1.18):

The attacker could attempt to prevent delivery of messages to or from the victim in order to:

• Disrupt service or marketing campaign of a commercial competitor.
• Censor a discussion or censor a participant in a discussion.
• Perform general vandalism. An attacker could cause the victim’s server to exhaust a particular resource (e.g. open TCP connections, CPU, memory, disk storage)

An attacker could send messages which created an unrecoverable “split-brain” state in the cluster such that the victim’s servers could no longer derive a consistent view of the chatroom state.

An attacker could convince the victim to accept invalid messages which the victim would then include in their view of the chatroom history. Other servers in the chatroom would reject the invalid messages and potentially reject the victims messages as well since they depended on the invalid messages.

An attacker could try to firewall traffic between the victim’s server and some or all of the other servers in the chatroom.

An attacker could send large volumes of messages to a chatroom with the victim making