Anthropic changes security guidelines, now AI models will be trained even if security is not guaranteed
Facing competition, Anthropic has revised its security framework. Earlier it had decided that it would not train an AI model if its security was not ensured beforehand. Now, the company says these strict limits were impractical and could cause its cloud AI to lag behind competitors.
Anthropic is currently fighting aggressively to win the AI race. The company is certainly at the forefront of AI development with its Model Cloud. However, in order to maintain its position and remain competitive, it has now revised its AI safety policy and removed one of its most defining safety promises, the original promise that led to its creation in 2021.
❮❯
In its latest update to the Responsible Scaling policy, which it created in 2023, Anthropic has announced that it will no longer commit to stopping training or releasing more powerful AI models based solely on security concerns.
Anthropic explains that while its first models helped create better safeguards for existing risks, it has reached an “area of ambiguity” where it is difficult to prove when an AI model becomes dangerous. According to the company, “the science of model evaluation is not sufficiently developed to provide a positive answer,” thus making it difficult for Anthropic to convince other AI companies or the government to stop and wait for them.
The company also argues that if it sticks to strict “stop” rules while other companies continue to build more powerful AI, it could fall behind. And if companies like Anthropic lose their influence, it could “lead to a world that is less safe”. So instead of slowing down on its own, Anthropic has decided to move away from rigid stopping rules and adopt a more flexible approach.
Anthropic also said that reaching the highest level of protection against major threats is currently “not possible” for any one company to achieve alone. So, rather than maintaining strict, unilateral commitments, it is moving toward a more flexible system focused on transparency and regular “risk reports.” The company says that through these reports it will clearly explain how it is identifying and managing risks as its AI models become more powerful.
Anthropic shook up the basic security rule?
Anthropic’s latest policy change has come as a big surprise to the industry. Founded by former OpenAI researchers who were outspoken about the risks of advanced AI, the company built its reputation by putting security first. When it introduced its Responsible Scaling Policy (RSP) in 2023, the company’s framework focused on a clear “hard-stop” on the launch of models that do not guarantee security. In fact this commitment has become one of the key reasons why Anthropic is seen as the most security-focused of the major AI labs.
But amid the intense competition in the AI race, the company has now decided to give security second priority.