SolarWinds: the supply-chain attack that changed everything

The SolarWinds compromise isn't really a vulnerability in the usual sense. Nobody found a bug in Orion and exploited it. The attackers got into the company's build system and made it compile their code into Orion itself. The result shipped as a normal, digitally signed update.

The build system was the target

SolarWinds' own investigation gives this timeline:

1. September 2019. Earliest suspicious activity on SolarWinds' internal systems.
2. October 2019. An Orion release contains test modifications, apparently the attackers checking that they could insert code into builds.
3. 20 February 2020. An updated injection tool starts inserting the backdoor into Orion builds. CrowdStrike, working with SolarWinds, named that tool SUNSPOT.
4. March–June 2020. Orion updates released in this window carry the backdoor. The malicious DLL in one analysed update was signed on 24 March 2020.
5. June 2020. The attackers remove the malicious code from SolarWinds' environment.
6. December 2020. The compromise is discovered and disclosed.

SUNSPOT watched the build server for an MsBuild.exe process compiling Orion. When it saw one, it replaced one source file, InventoryManager.cs, with a backdoored copy for the length of the build, then put the original back. The source repository never held the backdoor, so reviewing source wouldn't have shown it.

What SUNBURST did

The backdoor, which Mandiant named SUNBURST, lived in SolarWinds.Orion.Core.BusinessLayer.dll. Mandiant's analysis, published on 13 December 2020, describes it this way:

• It waited. After installation it stayed dormant for up to two weeks.
• It checked its surroundings. Obfuscated blocklists of forensic and anti-virus processes, services and drivers decided whether it ran at all.
• It blended in. Its command-and-control traffic imitated Orion's own Improvement Program protocol.
• It used generated subdomains of avsvmcloud[.]com to find its command server, encoding victim details into DNS names.

Scale

In its SEC filing of 14 December 2020, SolarWinds said fewer than 18,000 customers may have installed an affected version. In a February 2021 White House briefing, the deputy national security adviser for cyber said about 18,000 entities had downloaded it. She said nine federal agencies and about 100 private-sector companies had been compromised in follow-on activity. Most installations were never used. The attackers chose their targets.

On 13 December 2020, CISA issued Emergency Directive 21-01. It told federal agencies to disconnect or power down Orion versions 2019.4 through 2020.2.1 HF1. On 15 April 2021, the US government attributed the operation to Russia's Foreign Intelligence Service (SVR).

The CVE with the same name

Searching "SolarWinds CVE" usually turns up CVE-2020-10148. That's a separate authentication-bypass flaw in the Orion API, rated CVSS 9.8. CISA documented attackers using it to run commands on an Orion server and plant a different piece of malware, the SUPERNOVA web shell. It isn't the SUNBURST supply-chain compromise. The two are often conflated.

What changed afterwards

• Executive Order 14028 (12 May 2021) directed US federal guidance on software supply-chain security, including giving purchasers a software bill of materials.
• NIST SP 800-218, the Secure Software Development Framework, was published in February 2022 as part of the work under that order. It became the common reference for secure development practices in US federal software buying.
• Build systems became production. The industry started treating CI servers and build agents as systems that need the same hardening, monitoring and access control as what they deploy.

What it means for a pipeline

• A signature proves who built it, not what went in. SUNBURST was correctly signed. Signing is necessary but proves nothing about a compromised build.
• Make builds reproducible where you can. If two independent builds of the same commit must produce the same output, a build-time injection shows up as a mismatch.
• Keep build agents short-lived and isolated. SUNSPOT sat on a long-lived build server. Ephemeral, single-use build environments leave less room for persistence.
• Watch what monitoring tools talk to. Orion needed wide network access to do its job. Tools with that kind of reach deserve tight egress rules and their own alerting.

Sources

• CISA — Emergency Directive 21-01: Mitigate SolarWinds Orion code compromise
• CISA — AA20-352A: Advanced persistent threat compromise of government agencies, critical infrastructure and private sector organisations
• SolarWinds — Form 8-K filed with the SEC, 14 December 2020
• SolarWinds — New findings from our investigation of SUNBURST
• Mandiant — Highly evasive attacker leverages SolarWinds supply chain to compromise multiple global victims with SUNBURST backdoor
• CrowdStrike — SUNSPOT: an implant in the build process
• The White House — press briefing with Anne Neuberger, 17 February 2021
• NVD — CVE-2020-10148
• CISA — AR21-112A: CISA identifies SUPERNOVA malware during incident response
• NIST SP 800-218 — Secure Software Development Framework