Investigated
How do threat models change conclusions?
Short answer
Image: matinmavaddat.substack.com
Mainstream security practice says threat models change conclusions by specifying the adversary, access, harms, assumptions, and defenses, making risks and proportionate actions clearer. Critics argue that unknown threats, unrealistic assumptions, specialist knowledge, disclosure risks, and outdated models can distort that clarity. The main disagreement is whether modeling primarily improves decisions or can create false confidence.
The views on this question
Two lenses on the same evidence. Source weight and the primary source ratio show what each rests on.
Alternative
Source weight 50%A critical view argues that threat models can create an appearance of rigor without reliable conclusions. Unknown surveillance, incomplete technical or legal knowledge, impractical attacker assumptions, manual documentation, disclosure risks, and model obsolescence can distort priorities. On this reading, changing the model may expose uncertainty or organizational incentives rather than reveal a single objectively correct answer.
- Threat models can underestimate risks when threats or vulnerabilities are hidden, unknown, or difficult to assess accurately.
- Publishing a threat model may create a trade-off: useful detail can be sensitive, while safe disclosure may be too vague to inform readers.
- Some AI-security research models assume attacker access or capabilities that do not match common real-world settings, potentially changing conclusions about practical risk.
- Critics argue that specialist knowledge, hand-annotated diagrams, friction, and stale documentation can make threat modeling less accessible or actionable.
Follow the white rabbit
Deeper threads worth pulling on next.
- When do models create false confidence?A different rabbit holeInvestigate cases where narrow or overfitted threat models led organizations to underestimate real risks and suffer unexpected failures.
- How to model unknown adversaries?A different rabbit holeExplore techniques like red teaming, adversary emulation, and worst-case scenario planning for threats with limited intelligence.
- What are threat model assumptions?Different angle, same storyBreak down common explicit and implicit assumptions in threat models and how they influence outcomes and blind spots.