SolarWinds Hack (The Patch is the Attack)

A current assessment of the SolarWinds hack

The attack was initially detected by the affected IT security company, FireEye [2], around December 8, 2020. FireEye warned against the use of its security products, but denied that stored and unpublished vulnerabilities (zero-day vulnerabilities) had been read. The perpetrators manipulated an update of the network monitoring platform Orion of SolarWinds Inc. to install a backdoor (two have already been published to date) in approximately 18,000 of the estimated 300,000 customer systems (i.e. supply chain attack). Their clientele comprises of the public sector in the USA, Great Britain, and the world’s largest companies in various sectors (defense companies, technology companies, banks, consulting, pharmaceutical/chemical, telecommunications and raw materials companies) in North America, the Middle East, Asia, Europe, Germany[3], and all the states in the European Union.

Given that the attack has an immense impact (i.e. copying of data and programs, manipulation of programs, etc.), it is likely to continue being studied in detail[4], and also imitated. Attack documentation despite the expected remarkably high cost price, is sold like hot cakes to criminals and interested security agencies. The probability of occurrence for these attacks is internationally rated to be very high. Therefore, organizations and authorities alike should prepare themselves by taking preventive measures.

The following scenario illustrates the severity of the SolarWinds cyberattack. U.S. federal agency systems were compromised in this attack and the U.S. Cybersecurity and Infrastructure Security Agency (CISA) had to issue an emergency directive, which instructed all federal agencies to immediately shut down all the affected Orion products.

Alarmingly, the SolarWinds attack is not an isolated incident. Cyberattacks such as these have been on the rise in recent times. Microsoft[5] alone has sent out more than 13,000 warnings to its customers in the past two years. The primary aim of the backdoor installation to remotely control global systems through targeting the manufacturer’s customers. Perpetrators at present seem to be only partially concerned with the financial success (i.e. extortion) from an attack. This also applies to attacks within the healthcare sector, which are currently not (yet?) specifically targeted at individual patients.

The methods used by attackers are consistently at a particularly high technical level and demonstrate years of experience. Such specialists can be found not only in all industrialized countries but also in so-called developing countries. However, such attack techniques have not been heavily researched and taught at public universities. The first criminal attempts date back to the beginning of the 1970s in Germany.

Summary and ideas

The attacks on IT systems are being increasingly carried out by companies that specialize in them.

The expenditure for attack preparation typically amounts up to about $500 K from escalating to the many victims of an attack (approx. 18,000 in this case). However, each case of an attack is expected to bring in a revenue of $500 K – $10,000 K per victim. Pre-financing of these attacks are made possible through organized crime or intelligence agencies, which supplements the expectation that these attacks are technically well-crafted.

The SolarWinds attack was planned and implemented over the span of about 3 years. It has been pointed out in international studies (in German-speaking countries) that roughly 6 – 18 months had passed between the spying of data and programs alone, and the first unauthorized access.

A dangerous illusion is the assumption that once the IT production team is up and running again, the crisis of the attack has been averted. This misconception, while a popular opinion, is untrue in most cases where restarting is not a sign of averted attacks. Where exploitable attack points such as undetected security vulnerabilities (zero-day vulnerabilities), backdoors, covert channels and the like have not been eliminated, renewed attacks must be expected. The likelihood of this issue is high, given the market power (i.e. technical capabilities, core personnel) of commercial hacking companies. The powerlessness in the face of these hacking companies also reveals the helplessness of the affected U.S. government agencies.

Perpetrators usually attack when the situation seems advantageous. Theoretically, only certain organizations whose financial creditworthiness are considered to be sufficiently good are targeted. These perpetrators may attack repeatedly when the opportunity arises.

1. Current situation on the Internet
There is a current disparity between general awareness and these cyberattacks. Politicians and decision-makers broadly lack an understanding of the risks of these attacks on their own IT systems. The classic response to an attack scenario is to accordingly ask the IT manager whether everything is safe, without seeking a second opinion. Independent advice from the ‘outside’ is commonly unsought. The misjudgement of these cyber risks presents a real danger. Especially since these attacks are usually proceeded with caution and perpetrators will intentionally conceal the attack from the victim for up to several years.

2. Perpetrators
Some people will believe that it was the Russians (Pompeo knows), and some will believe that it is Chinese (as former president Trump puts it). A few even speculate about Korea – but only because a Korean word was ‘found’ in the source code (North Korea rather?). If nothing else comes to mind, the hackers are automatically assumed to be at least ‘close to the state’. However, all these perceptions stem from nothing more than the usual political propaganda of politicians (cf. the ‘rogue states’). Furthermore, these convictions can only be clarified in an extremely technically complex way.

The typification of perpetrators according to script kiddies, freaks, hackers, crackers, etc. is seemingly outdated. Present-day attacks have diverse and complex possibilities, which require competencies and personnel in all areas of cybersecurity. These requirements cannot be provided by individual companies, municipal administrations, or private individuals[6]. In the past 5 years, there has been a surge of cyberattack companies that are established internationally. These companies carry out new attack procedures that are developed worldwide, according to the motto ‘Crime as a Service (CaaS)’[7] against payment for clients.

Perpetrator groups[8] such as script kiddies, insiders, hackers, hacktivists, cybercriminals, state-sponsored groups, ‘intelligence agencies’ (e.g. government institutions such as security agencies) are a thing of the past. A clear distinction between these former perpetrators and current hacking groups is the commercialization of attacks. Modern-day cyberattacks are carried out by specialized companies under contract for a fixed fee or a revenue share of, for example, 30% in the case of ransomware. These companies have a corporate structure with departments such as personal, marketing, accounting, and production, etc., minimally in place. Thus, they have adequate resources to carefully analyze whether and how the organization intended as a victim is actually liquid to their desired extent (i.e. profit orientation). The personnel strength of these attack companies can reach 20 employees, with up to 15 IT specialists and additional free-lancers brought in for special tasks.

3. Affected parties
The affected parties include several U.S. federal ministries and companies which have come forward or have been published. The German Federal Office for Information Security (Bundesamt für Sicherheit in der Informationstechnik BSI) has informed the affected German companies. While 18,000 – 35,000 customers were affected in the SolarWinds case, this statistic reaches a total of more than 300,000 parties worldwide.

4. Attack targets
It is difficult to attain credible information as the available reports on reached attack targets are diffused, coupled with the consideration of marketing statements that were made. It is safe to assume that valuable company data were targeted (i.e. security tools, exploits, medical devices). Another likely attack target is the data stored in private and public clouds (e.g. Microsoft Office 365 accounts). Additionally, it is postulated that manipulations were carried out on the control data of production processes (e.g. IoT[10]) for vaccine production[11], or sabotage in the cases of production for chemicals and medicines). While the uses of this data for terrorist purposes should also not be ruled out, this has yet to be proven.

5. Attack sequence
Overall, the SolarWinds hack has a technical significance which is comparable to the ongoing (!) hack on the German Bundestag[12], Stuxnet[13], or NSA[14]. These attacks, when combined, present the state of the world attack technology, which can be seen in the techniques that are used.

This report focuses on the SolarWinds hack and delves into the progression of this attack. The attack sequence is detailed below:

  1. The first evidence[15] of the unauthorized manipulation of Orion updates surfaced in October 2019 – about 14 months before the attack was detected.
  2. The exploited attack points of the SolarWinds systems are currently undisclosed, possibly even unidentified. All the potential attack points emerge from vulnerabilities that are unpatched, unpublished, or even undiscovered. The initialization of the attack took place from March to June 2020. Past experience has shown that unpublished (i.e. zero-day vulnerabilities), or at least the vulnerabilities that are not known to SolarWinds or have not been patched, are exploited due to this reason. The following steps may be repeated at will by the attackers, as long as the entry points have not been identified and patched.
  3. Two (or more) groups of attackers have made themselves independent of this vulnerability by installing two backdoors, at a minimum, in the SolarWinds system. These backdoors were not published or identified by SolarWinds.
  4. The tampered update was made to appear authentic through a correct digital signature [16]. Code signing is one of the most important security measures of global software companies. If this signature is forged, it opens the door to any abuses of authentication and integrity checking.
  5. The malicious code is obfuscated (i.e. steganography) in the source code of the update. The runtime environment, in regard to operations, is checked to see if it is managed from a corporate network or, say, an analyst’s workstation.
  6. An update for the SolarWinds Orion Business Software was manipulated with almost 4,000 lines of code[17], which led to a backdoor being installed in the customers’ system (Orion Monitoring Software) for the very first time[18]. As long as the backdoor has not been identified and closed, the following attack steps may be repeated at will[19]. This applies analogously to the second backdoor that has been published[20], as well as to any other backdoors.
  7. The expectation of further backdoors are realistic. Similar attacks in the future should be expected on the grounds that not all the backdoors have been identified and patched.It is through the backdoor that further (and possibly updated) code from a command&control server is infiltrated, or a permanent connection between the attackers and target system is generally established. The transport protocol is similar to that of the SolarWinds protocol. Files are transferred or executed; the system is parameterized with system services being activated and deactivated; computers are rebooted.
  8. The backdoor is conveniently embedded in one of the SolarWinds modules that is installed in the target system. If the attackers obtain information on the other software technologies (e.g. standard software from vendors like Microsoft) used in the target system, the backdoor can be installed there as well. An attack lasts as long as the backdoor remains exploitable. In other words, the built-in backdoor is the linchpin. Attackers build in several backdoors for resilience reasons. However, the victim often believes that the danger has been repelled after identifying a (first) backdoor, and that the attack is therefore over. In some cases, victims even ask for ‘proof’ as to why they are still looking for further backdoors. It should not be forgotten that attackers may take further steps where possible, such as the copying and deleting of (all) data from the attack victim and the encryption of this data (i.e. ransomware). The copying of security information is especially relevant when related to the collection of unpublished security holes – e.g. for law enforcement purposes. It is noted that FireEye has denied such a theft, even before the investigation was completed.
  9. After the discovery of this attack, the backdoor was identified by the manufacturer and closed with a signed patch. Hence, it can be assumed that the attackers do not use this (closed) back-door anymore. The deployment of other backdoors remains a speculation at this juncture.
  10. There is often more than half a year gap between the installation of a backdoor and its exploitation, and this period may last up to 18 months. The decisive factor behind this duration is when the attackers are decidedly sure that the victim will not notice their attack.There is no way to prove that a system is backdoor-free. While this means that proof cannot be provided (e.g. in the aforementioned hacking cases such as NSA and Bundestag), it does not signify that the case is actually closed. It should be acknowledged that attackers will move cautiously to not give any hint of their activities.

6. Damage and amount of damage
Serious damage assessments cannot be made due to the person-year effort which is required. Official estimates are likely to remain secret.

The attackers used novel malicious code that was not (yet) stored in the Department of Homeland Security’s (DHS) multi-billion dollar intrusion detection system – ‘Einstein’.

A cleanup of the known manipulations is expected to take a duration of longer than 6 months.

However, the USA also attacks other states in this form[21].

7. Protective measures after attack detection
The manufacturer has recommended an update to the latest Orion Platform version 2020.2.1 HF 1 as soon as possible, to ensure the security of the environment. However, it is doubtful whether a simple update of the Orion Platform is sufficient to eliminate the infection, given the complexities involved. Anyone who has utilized the compromised software builds has no choice but to independently check and forensically analyze the affected systems. The signatures of the two published backdoors are available for this purpose.

It is easy to identify backdoors that are at least partially known, as in the SolarWinds case. The identification of more backdoors forms the difficulty, especially those that have not been detected as yet, or are not published. The latter requires a sophisticated methodology. It is easier to identify backdoors that misuse documented input or output interfaces.

The scope of recovery measures is dependent on the value of the processed data and controlled processes, which can be determined from a risk analysis. It ranges from a simple update of the Orion software to an immediate disconnection from the Internet, installation of new devices and software, and a check of all the stored data. After all, attack software can be stored anywhere – in (standard) software, in firmware and microcode of devices and controls, and also in data. A current and extremely comprehensive check is required before the system resumes its operations.

The attempt to simply restart without taking any further remedial action is a deed of negligence. Anti-virus programs and installing the latest updates (etc.) formulates the next level of defense against this particular attack[22]. However, these measures are unlikely to detect any modifications to the attack. Affected parties should carefully consider whether the successful attack should be made public.

8. Preventive measures
Commercial and government intrusion detection systems are of little use if they fail to detect documented attacks. Legal measures[23], such as the requirement to report attacks within 60 calendar days, fall completely flat in the face of detecting these attacks. Attack detection typically happens at the 6-18 month mark, and took 13 months in this case. The U.S authorities have created an impression that they develop excellent attacks, but are not in an adequate position to defend themselves against such attacks by third parties, as exemplified in the instance of the SolarWinds attack.

In Germany, a great emphasis is placed on the surveillance (i.e. decryption of all communications) of all citizens. The monitoring of Internet traffic and protection against criminals is thus seemingly neglected. The recurring crypto debate can be described to serve as a distraction for citizens from the real risks of the Internet.

Politicians should ask themselves how they intend to guarantee the fundamental right to physical integrity[24] – for example, in the cases of hospitals and the supply of vaccines[25]. Fundamentally, attack capabilities are increasing, as demonstrated in attacks such as the SolarWinds case. This shift renders previous methods of prevention less effective, as such attacks can no longer be detected, investigated or even repelled, even by well-funded companies. Policies should be aimed at identifying attacks and warning companies and authorities in good time by pointing out previously unpublished security loopholes, backdoors and covert channels. Such an initiative belongs under the IT security law.

Two basic techniques that help to identify backdoors and covert channels[26] are (1) the analysis of a system’s resources and (2) a thorough static source code analysis. However, past experience has shown that only 30% of covert channels are detected through the usage of tools.

Microsoft’s suggestion[27] is to create a signature about the attack practiced in SolarWinds and compare it with current data streams, which is comparable to anti-virus programs. This measure may detect the SolarWinds hack, but hardly comes close to helpful in detecting any other attacks.

A constructive approach toward this issue of prevention is presented in the ‘Internet Governance Forum’ (IGF)[28] of the United Nations and the ‘Council to Secure the Digital Economy’ (CSDE) of the IT and telecom industry.

9. Final assessment
The total damage accrued from this attack can only be estimated by the affected companies and authorities. It will not only take great effort, but is also only possible if their logs have been created automatically at various levels.

Attack vectors, beyond the 2 published backdoors, are likely yet to be identified (possibly without even using the Orion software). Statements which were made such as “was not spied on, not sabotaged” have not been technically justified. Additionally, the ‘classic’ security errors were evident in this case, such as the publication of passwords, and too long a reaction time after malware detection.

The impression given here might be that this case is one of the few exceptional ones. However, it is not. Comparable attacks (perhaps not within this scope) are commonplace. Accordingly, 5 days after this case was published, the U.S. Cybersecurity and Infrastructure Security Agency (CISA) issued an emergency directive asking U.S. agencies using SolarWinds products to forensically analyze the case and block network traffic to addresses outside the organization. Agencies without the appropriate expertise were asked to immediately shut down the products due to possible compromise.

Written by: Prof. Dr. Hartmut Pohl
https;//www.softScheck.com
Hartmut.Pohl@softScheck.com

Original article (German) can be found here.

Disclaimer: The English version is a translation of the original in German for information purposes only. In case of a discrepancy, the German original will prevail.

[2] https://bit.ly/35gbyb5

[3] For example, the source code base of Windows (Microsoft) was successfully accessed (https://bit.ly/2JA91AC); so far unconfirmed (but probable) are accesses to the supply chain, which – as with the access to the SolarWinds supply chain – enabled backdoors in over 85% of all computers in the world. The political and economic consequences were studied decades ago (https://bit.ly/3rK8ZHN), but were not understood: Worldwide, almost all computers and thus the Internet can be shut down by attackers within a few days or even abruptly. Terrorist interests (sabotage) cannot be ruled out.

[4] https://bit.ly/38NCIH1

[5] https://bit.ly/34YHuQP

[6] https://bit.ly/3o2ZO2Y

[7] https://bit.ly/2WW1jE2

[8] https://bit.ly/353iecp

[9] A rough (unconfirmed) overview of CISA can be found on the Internet (https://adobe.ly/386Cvj1): Belkin, Cisco, CrowdStrike, Deloitte (since June 20019), FireEye (with CIA involvement), Intel, Nvidia, Siemens, VMware. A number of US government agencies were also compromised by the malicious software. For example, the hackers reportedly managed to penetrate the Department of Homeland Security, the Department of Treasury, the Department of Commerce and the Department of Energy, and the systems of the U.S. Atomic Weapons Agency, airport networks such as Austin, the NSA, … Thus, the sectors affected are telecommunications, aerospace, and defense and health care. Furthermore, companies in Great Britain and Turkey are mentioned, as well as cloud/hosting providers in particular, such as Amazon, DigitalOcean, Microsoft Azure. Also, the UK National Health Service, the European Parliament and NATO.

Classic ransomware attacks, on the other hand, seem to be those on Aida, Funke, Hetzner, Symrise, etc. The German government stated that there were no accesses to its systems.

Simultaneously, Microsoft has also admitted to a successful attack – although it has not published how long the attackers have been active in Microsoft networks. (https://reut.rs/352s1PQ)!

Since the attack took place months ago, some companies no longer have the forensic data that is essential for a full investigation.

[10] https://bit.ly/382txUb

[11] https://bit.ly/382Sq1Y

[12] https://bit.ly/3pHJl4n

[13] https://bit.ly/2L7igZy

[14] https://bit.ly/38QBUB4

[15] https://bit.ly/38Prwd3, https://on.wsj.com/3hIujZG

[16] For reasons of practicality, the message (in this case the update) is first hashed and this hash value is encrypted into a check digit using a (strictly secret) private key from SolarWinds. Only with the corresponding public key the check digit can be decrypted again, so that the update appears authentic from SolarWinds and unchanged. The unauthorized use of the signature method therefore requires that the attackers could read and use the private key without authorization!

[17] https://bit.ly/38Prwd3

[18] Backdoor or trapdoor. Concealed (undocumented) sequence of instructions (programs, program parts in hardware, firmware, microcode and/or software) that enables access to an IT system by bypassing the security system (access control system).

[19] Therefore, a kill switch was installed on the associated command & control server, which automatically deletes the back-door when called by the manipulated software. https://bit.ly/350NqZQ

[20] Web shell ‘Supernova’ embedded in Orion code by another attacker.

[21] In June 2019, The New York Times reported that U.S. Cyber Command had penetrated Russian electric utilities deeper than ever before and deployed malware. https://bit.ly/38MwOG3

[22] https://bit.ly/34ZsUZh

[23] https://bit.ly/2MoRjBl

[24] https://bit.ly/2MoRjBl

[25] https://bit.ly/3aVA84z

[26] Covert channel. Logical channel that is not intended for information transmission – nevertheless enables unauthorized and covert (non-documented) transmission, i.e. exchange of information and thus violates the security policy of the IT system. Two classes of covert channels are distinguished covert storage channels and covert timing channels. A covert channel is a channel that allows information to flow between at least two cooperating entities in a manner that is contrary to the security objectives – without being controllable by access control, i.e. it violates the security policy.

[27] https://bit.ly/3834v76

[28] https://bit.ly/3o3kPKK