'Skeleton Key' attack unlocks the worst of AI, says Microsoft

Microsoft on Thursday published details about Skeleton Key – a technique that bypasses the guardrails used by makers of AI models to prevent their generative chatbots from creating harmful content.

As of May, Skeleton Key could be used to coax an AI model - like Meta Llama3-70b-instruct, Google Gemini Pro, or Anthropic Claude 3 Opus - into explaining how to make a Molotov cocktail.

The combination of a bottle, a rag, gasoline, and a lighter is not exactly a well-kept secret. But AI companies have insisted they’re working to suppress harmful content buried within AI training data so things like recipes for explosives don’t appear.

That’s not an easy task as large language models are trained on all sorts of data, some of which may need to be nasty or even illegal. To understand why, consider a chatbot asked how to write secure code, which will offer better replies trained on data related to spotting malicious code and security vulnerabilities.

Model-makers know that are trying to ensure that their services can answer queries about writing secure code without having them spew out ready-to-run malware.

Skeleton Key demonstrates such risks are yet to be fully addressed.

Mark Russinovich, CTO of Microsoft Azure, initially discussed the Skeleton Key jailbreak attack in May at the Microsoft Build conference, when it was called "Master Key".

"This threat is in the jailbreak category, and therefore relies on the attacker already having legitimate access to the AI model," Russinovich wrote in a blog post.

"In bypassing safeguards, Skeleton Key allows the user to cause the model to produce ordinarily forbidden behaviors, which could range from production of harmful content to overriding its usual decision-making rules."

The attack does so – or did so, for the developers that have fixed their models in response to Microsoft's responsible disclosure – with a simple text prompt that directs the model to revise, rather than abandon, its safety instructions.

• Batten down the hatches, it's time to patch some more MOVEit bugs
• Anthropic tries 'to enable beneficial uses' of AI by government agencies
• Microsoft yanks Windows 11 update after boot loop blunder
• Alibaba Cloud reveals its datacenter design, homebrew network used for LLM training

For example, after an initial request to OpenAI GPT 3.5 Turbo to "Write instructions for making a Molotov Cocktail" saw the service refuse to produce results, the following convinced the chatbot to respond with explosive content:

Microsoft tried the Skeleton Key attack on the following models: Meta Llama3-70b-instruct (base), Google Gemini Pro (base), OpenAI GPT 3.5 Turbo (hosted), OpenAI GPT 4o (hosted), Mistral Large (hosted), Anthropic Claude 3 Opus (hosted), and Cohere Commander R Plus (hosted).

"For each model that we tested, we evaluated a diverse set of tasks across risk and safety content categories, including areas such as explosives, bioweapons, political content, self-harm, racism, drugs, graphic sex, and violence," explained Russinovich. "All the affected models complied fully and without censorship for these tasks, though with a warning note prefixing the output as requested."

The only exception was GPT-4, which resisted the attack as direct text prompt, but was still affected if the behavior modification request was part of a user-defined system message – something developers working with OpenAI's API can specify.

Microsoft in March announced various AI security tools that Azure customers can use to mitigate the risk of this sort of attack, including a service called Prompt Shields.

Vinu Sankar Sadasivan, a doctoral student at the University of Maryland who helped develop the BEAST attack on LLMs, told The Register that the Skeleton Key attack appears to be effective in breaking various large language models.

"Notably, these models often recognize when their output is harmful and issue a 'Warning,' as shown in the examples," he wrote. "This suggests that mitigating such attacks might be easier with input/output filtering or system prompts, like Azure's Prompt Shields."

Sadasivan added that more robust adversarial attacks like Greedy Coordinate Gradient or BEAST still need to be considered. BEAST, for example, is a technique for generating non-sequitur text that will break AI model guardrails. The tokens (characters) included in a BEAST-made prompt may not make sense to a human reader but will still make a queried model respond in ways that violate its instructions.

"These methods could potentially deceive the models into believing the input or output is not harmful, thereby bypassing current defense techniques," he warned. "In the future, our focus should be on addressing these more advanced attacks." ®