October 12th, October 10thMicrosoft CEO SATYA NADRA PUBLISHED A LONG ARTICLE IN X CALLING FOR A STRONG BUSINESS AI The system creates "emergency brakes": an authorized person should always be able to suspend or close the model while it is in operation. Businesses cannot rely on security assurances from model providers alone, but must assume from the outset that the model may be invaded and isolated and controlled。
Nadra noted that traditional software behaviour can be traced back to a specific code path, but that current model behaviour is difficult to match to a given training data or group weight configuration. Businesses, however, have given these systems access to sensitive data to perform key tasks on their behalf, and trust structures must be revisited. He stressed that models as internal risks were not assumed to be malicious, but that any powerful subject with access to critical systems could be mistaken or compromised。
He suggested that the model, the framework for the implementation of its work and the scope for allowing action should be separated from each other, and that competencies and protection mechanisms must be outside the model. Even using another model for counter-testing, it may simply allow a non-transparent model to oversee another non-transparent model and cannot independently prove that the system is reliable。
Specific measures include:
Not to allow a single model to be the only source of important results, nor to allow it to validate its work on its own。
For each of these important operations, there is a tamper-proof, readable record and continuous testing of failures, attacks and borders。
It is up to the enterprise to decide independently on the terms of access and scope of action, to maintain an independent audit, a manual stoppage and a mechanism for timely disclosure of security incidents。