Microsoft and Nvidia Set Open-Weight AI Security Checks for Enterprise Deployments

Open-weight AI can keep producing strong results after its safeguards have weakened. That gap is the security problem Microsoft and Nvidia want enterprise teams to catch: task performance alone doesn’t show whether a modified model still responds safely.

A joint paper published last week sets out 30 risk types across five categories and a process to assess, reduce and monitor them. Its central point is that openness of the weights doesn’t determine risk on its own. As the paper puts it, risk depends on the model, its use case and deployment, and the safeguards around it.

The framework covers cybersecurity, data protection, content, human-AI interaction and bias. Cybersecurity risks include malicious code generation and tool calls, attempts to extract safety policies, excessive token generation, and behavior that strays from the system’s intended goal. The companies say the guidance draws on NIST and ISO risk-management standards and is the first in a planned series, according to a Microsoft blog post.

Test the artifact, then test it again

Fine-tuning, model merging, quantization and configuration changes can erode safety protections while leaving routine task performance intact. That makes separate assessments of capability and safety essential: a model that still answers ordinary requests accurately may have changed in how it handles unsafe ones.

Microsoft and Nvidia recommend evaluating the precise configuration an organization plans to deploy, then repeating those checks after significant changes. Behavioral tests can show how a model responds, but they can’t establish whether the tested file is the publisher’s original artifact. The paper recommends cryptographic signatures for provenance checks; when signatures aren’t available, it says organizations should record integrity as an unresolved risk instead of treating test results as proof of origin.

Gartner’s proposed vetting steps add independent adversarial testing, static scanning of model files, source and integrity checks, and registration of approved models in an enterprise registry. It also recommends checking provenance both before and after deployment.

Application controls matter as much as model checks

A model’s evaluation doesn’t establish that the surrounding agent or application is safe. Enterprises also need to assess permissions, data access, connected tools, runtime behavior and operational controls, the paper says. Its guidance spans the model, runtime, harness and application layers, with monitoring of inputs, outputs and tool use after launch.

Gartner recommends routing open-weight model traffic through governed AI gateways that can enforce authentication, defend against prompt injection, protect sensitive data and create audit logs. For self-hosted deployments, it advises isolating runtimes. Cloud API users should consider controls such as private network paths and customer-managed encryption keys.

Demand is rising: Gartner said client inquiries about open-weight models through its research inquiry service increased by more than 30% in the second quarter of 2026 compared with the first. The firm projects that open-weight models will make up 50% of enterprise AI use cases by 2028, up from 20% at the time of its report.

Ownership has to include people and process

Self-hosting changes staffing needs as well as infrastructure, said Sanchit Vir Gogia, chief analyst at Greyhound Research. “Self-hosting is a staffing decision as much as an infrastructure one, and the people to carry it must move with the model,” he said.

Gogia cautioned that self-hosting doesn’t automatically mean losing every safeguard associated with managed services. One possible gap is a provider’s misuse-investigation function; the customer’s responsibilities vary with the service and its configuration.

He sees the Microsoft-Nvidia guidance as useful groundwork, with further detail still needed for runtime, harness and application risks. One unresolved operational question, he said, is who must retest and patch a model when a new technique for stripping safeguards appears. His advice for CISOs is to assess the model, deployment and intended use before approving production workloads, and to keep ownership of the organization’s risk register rather than relying only on a vendor’s taxonomy.