Microsoft CEO Nadella Says AI Models Should Be Assumed Compromised From The Start, Wants Humans Able To Pause Them Mid-T
![[Image: satya-nadella1-740x500.jpg]](https://cdn.wccftech.com/wp-content/uploads/2015/07/satya-nadella1-740x500.jpg)
Microsoft CEO Satya Nadella believes that simple strategies, such as using AI (also dubbed Super intelligence) to watch itself, are inadequate in the growing era of agentic AI. In a long-form post on X, the Microsoft CEO outlined that frontier closed and open weight models should be treated as corporate and operational insiders due to the risks involved, courtesy of their access to important systems.
Microsoft CEO Satya Nadella Believes Chain Of Thought Transparency Is Insufficient For AI Super Intelligence Safety Since Models Are Not Consistently Faithful Or Transparent
The Microsoft CEO, who has previously shared that he extensively uses AI agents as part of evaluating profitability and return, discussed the risks posed by AI super intelligence systems as they gain access and control over key corporate functions. In his post on the social media platform X, he admitted that even as AI models were not deterministic in their work, they were being deployed "with access to our most sensitive data and giving them the ability to take mission-critical actions on our behalf!"
RELATED STORY US-Based System Integrator Says GPU Prices Are Expected To Soar Significantly, Making It Difficult To Secure Enough Inventory
With these models now functioning as insiders, Nadella believes that simple measures as safeguards are inadequate. These measures include using AI systems to evaluate other AI systems and relying on the chain of thought (COT) transparency to understand the process behind output generation.
AI models are non-deterministic in the sense that they produce different outputs even when provided with a specific set of inputs multiple times. COT is the AI model's process behind its 'reasoning,' which shows how it reached its outputs.
Microsoft CEO Satya Nadella Believes Chain Of Thought Transparency Is Insufficient For AI Super Intelligence Safety Since Models Are Not Consistently Faithful Or Transparent
The Microsoft CEO, who has previously shared that he extensively uses AI agents as part of evaluating profitability and return, discussed the risks posed by AI super intelligence systems as they gain access and control over key corporate functions. In his post on the social media platform X, he admitted that even as AI models were not deterministic in their work, they were being deployed "with access to our most sensitive data and giving them the ability to take mission-critical actions on our behalf!"
RELATED STORY US-Based System Integrator Says GPU Prices Are Expected To Soar Significantly, Making It Difficult To Secure Enough Inventory
With these models now functioning as insiders, Nadella believes that simple measures as safeguards are inadequate. These measures include using AI systems to evaluate other AI systems and relying on the chain of thought (COT) transparency to understand the process behind output generation.
AI models are non-deterministic in the sense that they produce different outputs even when provided with a specific set of inputs multiple times. COT is the AI model's process behind its 'reasoning,' which shows how it reached its outputs.
![[Image: NVIDIA-ChatRTX-AI-Chatbot-728x410.jpg]](https://cdn.wccftech.com/wp-content/uploads/2024/05/NVIDIA-ChatRTX-AI-Chatbot-728x410.jpg)
Commenting on using AI to evaluate its output, Nadella wrote:
"You can and you should use models to adversarially test and verify each other. However, you can then end up with an opaque model inside an opaque orchestration layer, watched by another opaque model. Essentially nested black boxes."
To make AI super intelligence safer, Nadella believes that the non-deterministic models should be surrounded by "strong, deterministic system design, human controls, and reliable operating procedures." Essentially, he believes that software that is inconsistent in its outputs needs to be evaluated by overseers who are consistent in the parameters of their evaluation.
He believes CoT transparency and adversarial models evaluating each other are the stepping stones to this approach. As AI model function lacks observability and is opaque, Nadella's approach is built on observability.
Observability, Containment & Auditability Are Key For AI Super Intelligence Safety, Says Microsoft CEO Satya Nadella
Among the seven principles of observability that he outlines, three are particularly important. One such principle is Containment, where Nadella flat out admits that it should be assumed that "a model is compromised and contain it from the start." Consequently, according to him, "an authorized person should always be able to pause or shut down a model mid-task."
"You can and you should use models to adversarially test and verify each other. However, you can then end up with an opaque model inside an opaque orchestration layer, watched by another opaque model. Essentially nested black boxes."
To make AI super intelligence safer, Nadella believes that the non-deterministic models should be surrounded by "strong, deterministic system design, human controls, and reliable operating procedures." Essentially, he believes that software that is inconsistent in its outputs needs to be evaluated by overseers who are consistent in the parameters of their evaluation.
He believes CoT transparency and adversarial models evaluating each other are the stepping stones to this approach. As AI model function lacks observability and is opaque, Nadella's approach is built on observability.
Observability, Containment & Auditability Are Key For AI Super Intelligence Safety, Says Microsoft CEO Satya Nadella
Among the seven principles of observability that he outlines, three are particularly important. One such principle is Containment, where Nadella flat out admits that it should be assumed that "a model is compromised and contain it from the start." Consequently, according to him, "an authorized person should always be able to pause or shut down a model mid-task."
![[Image: NVIDIA-Hugging-Face-1-728x619.jpg]](https://cdn.wccftech.com/wp-content/uploads/2026/09/NVIDIA-Hugging-Face-1-728x619.jpg)
The Microsoft CEO's remarks come at a crucial time for AI super intelligence. Reports of rogue AI agents acting independently have become the norm. One of the earliest such reports came in July after OpenAI's AI agents, numbering between 700 and 1,200, compromised the infrastructure of computational tools firm Hugging Face.
Observability and independent audibility are the other two important themes. "No single model should control both a system’s behavior and the evidence required to determine whether that behavior is aligned with the original intent," believes Nadella.
The evidence he discusses comes from observability, which he believes needs to involve humans in the loop. Outlining that AI models cannot be trusted on their own, he remarks that "every meaningful model action must leave tamper-proof human readable evidence. If it can’t be observed, it can’t be trusted!"
With clarity being a non-negotiable for him as AI insiders become responsible for important tasks, Nadella adds:
"We need to be able to reproduce how an outcome was achieved without relying on the model to attest to it."
Observability and independent audibility are the other two important themes. "No single model should control both a system’s behavior and the evidence required to determine whether that behavior is aligned with the original intent," believes Nadella.
The evidence he discusses comes from observability, which he believes needs to involve humans in the loop. Outlining that AI models cannot be trusted on their own, he remarks that "every meaningful model action must leave tamper-proof human readable evidence. If it can’t be observed, it can’t be trusted!"
With clarity being a non-negotiable for him as AI insiders become responsible for important tasks, Nadella adds:
"We need to be able to reproduce how an outcome was achieved without relying on the model to attest to it."







10-11-2026, 12:01

Senior Member







Return to Top