By now you have probably heard about the SolarWinds supply-chain compromise that has impacted government and businesses all over the world. This story is still unfolding so I won’t try to explain everything in detail, rather, I’ll attempt to explain the situation for the less-technical reader and link to some resources so that you can follow the story.

Overview

The story broke last week when cybersecurity company, FireEye, disclosed in a blog post that "A highly sophisticated state-sponsored adversary stole FireEye Red Team tools.” Five days later, in a subsequent blog post, FireEye wrote, “We have discovered a global intrusion campaign” and, “discovered a supply chain attack trojanizing SolarWinds Orion business software updates in order to distribute malware we call SUNBURST.”

In addition to FIreEye being compromised in this attack, Graham Cluley reports, “The United States Department of Commerce, Treasury, State Department, National Institutes of Health, Homeland Security, and Pentagon have had their networks compromised in what appears to have been a massive supply-chain attack on American government systems.” According to an SEC filing, SolarWinds believes that up to 18,000 of their customers had a vulnerable installation of their Orion tools.

What is SolarWinds?

SolarWinds is a 21 year-old technology company based in Austin, TX that makes network management and monitoring tools that companies and organizations use, to keep track of the computers on their network and manage the health and status of those computers.

If that explanation didn’t help you understand, imagine if you had a computer on your home network that identified each computer, printer, mobile device and smart/IoT device (TVs, thermostats, doorbells, security cameras, light bulbs, etc.) and alerted you if one of those devices went offline or needed a patch, or was running out of storage space. Well that is what the SolarWinds Orion product does at a very high level. Companies, governments and organizations all over the world use SolarWinds to manage their IT assets.

SolarWinds Customers

SolarWinds has more than 300,000 customers worldwide. Many of them are high-profile customers which include 425 of the U.S. Fortune 500, all ten of the top ten U.S. telecommunications companies, all five branches of the U.S. military, the U.S. Pentagon, State Department, NASA, NSA, Postal Service, NOAA, Department of Justice, the Office of the President of the United States, all five of the top five U.S. accounting firms and hundreds of universities and colleges worldwide, according to a screenshot of the SolarWinds website, posted in a blog by KrebsOnSecurity.

This impressive customer list was on the SolarWinds website until earlier this week when The Verge reported that, “SolarWinds has removed a list of high-profile clients from its website in the wake of a massive breach.”

What Happened?

SolarWinds creates updates and patches for their products regularly. Some of those updates are to add features and some of them fix problems or vulnerabilities in the software.

On March 24, 2020, a software update was posted to the SolarWinds website for customers to download and install on their Orion system. This update included malware that gave threat-actors remote access to any Orion system on which it was installed.

What seems to have happened is that a nation-state threat-actor who has been identified as a Russian hacking team, (code named Cozy Bear or APT 29) was able to infect this patch update with malware that created a backdoor for Cozy Bear to access Orion systems on SolarWinds’ customer networks. This is not an easy process since Orion systems will not allow patches or updates to be installed unless the patch is signed with a digital certificate from SolarWinds.

Software developers treat their code-signing certificates as one of their most valuable assets so obtaining one is extremely difficult but quite powerful if achieved. In this case, the patch update was signed with an official SolarWinds certificate. There are a few ways that Cozy Bear could have pulled this off but no matter how it was done, it is an impressive number of compromises to achieve without being caught for 9 months.

The Malware

When any of the 18,000 SolarWinds Orion customers installed this malicious update, the malware on the Orion system would wait for a period of time, up to 2 weeks, then run some jobs and connect to a command and control server at the domain, avsvmcloud[.]com to get updates and provide a back door into the networks where Orion was installed. This domain was a key factor in retaining control of the infected systems. Once the security community discovered this, Microsoft worked with U.S. government agencies and GoDaddy, to take control of that domain and thereby remove CozyBear’s access to the infected Orion systems. On December 15th, Brian Krebs tweeted, “Looks like the domain used to control the malware infrastructure in the SolarWinds compromise is now controlled by Microsoft.” This means that Microsoft was able to take control over the domain name so the infected Orion devices would not be able to reach the command and control server.

Supply-Chain Attacks

The IT world has been talking about supply-chain attacks for years and this attack underscores the continued need to focus on defending against them. Cyberscoop wrote, “It’s not the first time a supply-chain attack has caught the cybersecurity community off guard. In 2017, hackers compromised HandBrake, a video conversion tool, to distribute a remote access toolkit. The same year, hackers with suspected links to China laced malicious software in the file cleaning program CCleaner to ultimately target more than two million users. In perhaps the most globally infamous supply-chain security nightmare to date, Russian hackers exploited a software vulnerability in Ukrainian tax software to lock up computers around the world. The U.S. has charged hackers linked with the Russian Main Intelligence Directorate, or the GRU, for the NotPetya attack. The European Union has also sanctioned Russian hackers for the attack.”

Updates

Legal Fallout

On Wednesday December 16, Brian Krebs wrote, “the potential legal fallout for SolarWinds in the wake of this breach continues to worsen. The Washington Post reported Tuesday that top investors in SolarWinds sold millions of dollars in stock in the days before the intrusion was revealed. SolarWinds’s stock price has fallen more than 20 percent in the past few days. The Post cited former enforcement officials at the U.S. Securities and Exchange Commission (SEC) saying the sales were likely to prompt an insider trading investigation.”

Additional Victims

[Dec 17, 2020] ZDNet reports, “The state-sponsored hackers who breached US software provider SolarWinds earlier this year pivoted to Microsoft's internal network, and then used one of Microsoft's own products to launch attacks against other companies, Reuters reported today citing sources familiar with the investigation.

Microsoft now joins a list of high-profile entities that have been hacked via a backdoored update for the SolarWinds Orion network monitoring application.

The vast majority of these victims are US government agencies, such as:

• The US Treasury Department

• The US Department of Commerce's National Telecommunications and Information Administration (NTIA)

• The Department of Health's National Institutes of Health (NIH)

• The Cybersecurity and Infrastructure Agency (CISA)

• The Department of Homeland Security (DHS)

• The US Department of State

• The National Nuclear Security Administration (NNSA) (also disclosed today)

• The US Department of Energy (DOE) (also disclosed today)

• Three US states (also disclosed today)

• City of Austin (also disclosed today)”

Additional References

Krebs On Security articles

• Dec 14, 2020 U.S. Treasury, Commerce Depts. Hacked Through SolarWinds Compromise

• Dec 15, 2020 SolarWinds Hack Could Affect 18K Customers

• Dec 16, 2020 Malicious Domain in SolarWinds Hack Turned into ‘Killswitch’

• Dec 18, 2020 VMware Flaw a Vector in SolarWinds Breach?

FireEye Discovered SolarWinds Breach While Probing Own Hack [Dec 14, 2020]

Russian government hackers are behind a broad espionage campaign that has compromised U.S. agencies, including Treasury and Commerce [Dec 14, 2020]

Infographic by Security Boulevard [Dec 17, 2020]

A moment of reckoning: the need for a strong and global cybersecurity response [Dec 17, 2020]

Hackers last year conducted a 'dry run' of SolarWinds breach [Dec 18, 2020]