Security conversations about AI tend to get lost in exotic scenarios: prompt injection, model poisoning, jailbreaks. Those are real and worth attention. But for most organisations, the largest risk is duller and more immediate. It is that sensitive data walks out the door inside ordinary AI usage, and nobody logged it leaving. This paper builds the threat model from that starting point.
The threat that is already happening
Before anyone attacks your model, your own people are quietly moving data into tools you do not control. It is called shadow AI, and it is not hypothetical. IBM's 2025 Cost of a Data Breach research found that one in five breached organisations were breached in connection with shadow AI, that these incidents carried a cost premium of hundreds of thousands of dollars, and that the great majority of organisations that suffered an AI-related breach lacked proper access controls on those AI tools. The global average breach already sits at 4.44 million dollars. Shadow AI is not a compliance footnote. It is a live breach vector with a price tag.
The mechanism is mundane. An employee pastes a customer list, a contract, a patient summary or source code into a public assistant to get help. The data is now on someone else's servers, possibly retained, possibly used to improve a model, and entirely outside your logs and your control. No firewall stopped it because it left through a browser tab.
The supply-chain dimension
Every external AI service you adopt is another supplier in a chain that regulators increasingly expect you to secure. NIS2 made supply-chain security a statutory obligation for in-scope entities. DORA created direct oversight of critical cloud providers precisely because so much of European finance leans on the same few of them. The EU cybersecurity agency ENISA has documented how supply-chain attacks reach their targets not head-on but through a trusted supplier. Each AI dependency you add is one more link that can be attacked, and one more relationship you have to assess, contract for and monitor.
A useful reframing for a security review. Do not ask only "is this AI vendor secure?" Ask "what new attack surface and what new supplier does this AI create, and could we avoid creating it at all?" Sometimes the most secure integration is the one you did not build, because the capability lived inside your own boundary instead.
What on-premise inference removes
Keeping the model inside your network does not make you invulnerable. You still have to secure the box, patch it, control access to it and watch it. What it does is remove entire categories of risk rather than adding controls to manage them.
When inference happens in your building, the exfiltration path through a third-party API is closed, because there is no third-party API in the sensitive path. The supplier you would have to secure and monitor is not in the chain. The international transfer you would have to justify never occurs. The provider that could log your prompts, change its retention policy, or be compelled to hand data to a foreign authority is not holding your data, because your data never reached it. A sealed system with an immutable audit log lets you prove, rather than hope, that sensitive information stayed where it belonged.
Building the model
Put the pieces together and the threat model for enterprise AI has a clear centre of gravity. The headline risk is not a clever adversary defeating your model. It is routine data leakage through unsanctioned or externally hosted tools, compounded by a growing web of AI suppliers you now have to secure. The most effective single control is not another monitoring product. It is architecture: give people a capable assistant inside the boundary so they stop reaching for the ones outside it, and keep the sensitive inference on hardware you own so the data has nowhere external to go. Everything else, the prompt-injection defences and the rest, is worth doing on top of that foundation. It is not a substitute for it.
This paper is general guidance, not a security assessment of your environment. Your own threat model depends on your data, your sector and your obligations, and is a conversation for your security team. The direction it points, though, is consistent with where European regulation has been heading for years. Shorten the chain. Keep the sensitive data home.