Mystery solved? Microsoft thinks it knows how Chinese hackers stole its signing key

When Microsoft revealed earlier this year that hackers based in China snooped on the email accounts belonging to senior U.S. officials, the news was accompanied by a mystery: How did the hackers obtain the signing key — the closely guarded, critical piece of Microsoft’s security infrastructure — that they used to bypass security protections?

On Wednesday, Microsoft revealed the results of its internal investigation regarding how that key was stolen, the findings of which describe a series of cascading security failures that resulted in a signing key ending up in the hands of apparent Chinese hackers.

The theft of that key allowed the hackers to access the email accounts belonging to U.S. Secretary of Commerce Gina Raimondo and the U.S. ambassador to China, Nicholas Burns, at a crucial moment of diplomatic engagement between Washington and Beijing. The incident has raised major questions about Microsoft’s security and business practices and whether the firm understood the full scope of the attack.

According to Microsoft, the series of events that culminated in an espionage campaign targeting U.S. officials began with a “crash dump” — the set of information that describes the state of a computer or program when it fails. When a Microsoft consumer-signing system crashed in April 2021, the resulting crash dump included a signing key. That key should have been redacted but due to an error was not.

In the belief that the crash dump did not contain any sensitive information, the file was moved from Microsoft’s highly secure production system into its less secure corporate network. At some point after April 2021, a Chinese hacker compromised an account belonging to a Microsoft engineer with access to the debugging file containing the signing key.

If a crash dump is the garbage generated by a failing computer system, stealing a signing key via a crash dump is like rifling through a garbage can and discovering the key to the family safe.

Microsoft’s investigators cannot be sure that this is how the key was stolen but believe the crash dump file the most likely method.

Trey Herr, the director of the Atlantic Council’s Cyber Statecraft Initiative, described the crash dump as a “brutal vector of compromise” and commended Microsoft on tracing the breach to its source but added that questions remain about the company’s design and security choices. Microsoft’s Wednesday blog post, still has “no good answer on why this key was allowed to sign so far across different services,” Herr said.

Sen. Ron Wyden, D-Ore., said in a statement to CyberScoop that while “Microsoft deserves credit for providing additional details about the hack” the firm “has an obligation to explain why it deviated from best practices and its own advice when it came to protecting highly sensitive encryption keys.”

Wednesday’s post is unlikely to be the final word on the breach. The U.S. Cyber Safety Review Board is currently investigating the breach as part of a broader examination of cloud security.

In a measure of irony, Microsoft no longer has access to the security logs that would contain the definitive evidence that the key was exfiltrated via the crash dump. When the attack was first revealed earlier this year, Microsoft came under intense criticism for its customer logging policies, which required clients to upgrade to a more expensive service to have access to the more comprehensive logs containing evidence of the attack. Due to its log retention policies, Microsoft doesn’t have access to the logs that would detail how the signing key was stolen.

Update, Sept. 6, 2023: This article has been updated with a statement from Sen. Ron Wyden.