Rethinking Threat Modeling for AI: Quick Sessions Offer New Insights

A recent challenge faced by cybersecurity teams has underscored the urgency for an agile approach to threat modeling, especially as AI technology begins to embed itself further into business processes. Adam Shostack, a recognized expert in threat modeling, received a request from a client whose app, optimized for customer data through AI, raised significant risk concerns. Responding to the client's need for rapid insights, Shostack structured his analysis within a tight 15-minute window.

In this brief session, he successfully pinpointed numerous potential threats that traditional modeling frameworks like STRIDE might overlook, including hallucinations and biases associated with language model interactions. His framework, PHANTOM-B, specifically tailors threat analysis to the unique aspects of applications using AI components.

PHANTOM-B stands for Prompt Injection, Hallucination, Anthropomorphization, Non-explainability, Training Issues, Overreliance, Missing Security Engineering, and Bias. This model is not intended to replace STRIDE; rather, it complements it by providing targeted insights for those elements of a system that engage with AI-driven architecture. The goal is straightforward: streamline threat modeling to make it more accessible and less burdensome, thus increasing its prevalence in regular practices.

"Let's make it inexpensive to do this work," Shostack emphasizes, presenting his framework at the Black Hat USA 2026 conference. As organizations race to secure AI deployments quickly, many are struggling to adapt existing security methodologies to these new technologies.

The Limitations of Traditional Threat Modeling

Threat modeling has long been a complex task, often reserved for only the most critical applications due to time constraints and the constant evolution of software. Many organizations falter in maintaining consistent practices, which leads to gaps in security coverage. “AI didn’t break threat modeling; it exposed underlying weaknesses,” says Jeff Williams, founder of OWASP.

The challenge isn't merely identifying potential threats but involves understanding how different applications operate. Historically, this has relied on outdated practices, such as cumbersome surveys or static documentation that can quickly become outdated as applications evolve.

Generative AI further complicates traditional modeling efforts, as the emergent behaviors of these systems can be unpredictable. Unlike conventional software that operates on structured rules, AI systems generate probabilistic responses, leaving teams grappling with uncertain outcomes and increased risks of exploitability.

According to Brian Glas of CODIFIC, the shift to non-deterministic AI necessitates a reevaluation of risk assessment strategies. He notes that as AI blurs the line between commands and data, it creates vulnerabilities that conventional security practices are ill-equipped to address.

Adapting Threat Modeling to AI

The concept of conducting a quick, focused threat modeling session may seem oversimplified, yet Shostack argues that such exercises are not intended to cover every aspect exhaustively. Instead, they aim to identify key risks that inform immediate strategic decisions: should the system undergo modifications, or is further analysis warranted?

This method takes cues from agile development principles, promoting faster iterations that allow cybersecurity teams to efficiently revise their assessments without destabilizing ongoing projects. Each session can surface actionable narratives regarding potential system failures, enabling better understanding of accepted risks for CISOs and technical staff alike.

The nuances captured by PHANTOM-B, particularly concerning anthropomorphization and non-explainability, underline common pitfalls in AI applications. Users might mistakenly anthropomorphize models, attributing understanding or reliability that they do not possess. The charting of control measures, like narrowing data access or introducing approval points for AI-driven decisions, is vital to curtail improper trust in AI capabilities.

Avoiding Common Threat Modeling Pitfalls

While it's tempting to focus exclusively on emerging AI threats, this strategy can lead organizations to neglect foundational security principles, exposing them to risks that are easily preventable. Williams cautions against prioritizing vulnerabilities like prompt injection while failing to adhere to established application security standards.

Security professionals should not lose sight of context either; understanding the full scope of how AI components fit within broader application architectures is essential. Missing this step invites vulnerabilities that could compromise the system.

Continuous threat modeling is key in the current digital climate. Teams need to avoid attempting to catalog every possible risk and instead focus first on protecting critical assets and understanding the preventive measures already in place. Once those foundations are established, modeling various attack vectors helps identify weaknesses and reinforce defenses effectively.

Incorporating insights from frameworks such as PHANTOM-B, OWASP guidelines, and NIST recommendations provides a pathway forward as organizations navigate the complex landscape of AI security. Adopting a creative yet structured approach to threat modeling will enable teams to stay responsive in a rapidly evolving domain.