Episode 99: SolarWinds Supply Chain Attack Affects Government and Fortune 500 Businesses

Earlier this week, we learned that SolarWinds, the largest provider of network management tools for government and enterprise organizations fell victim to a supply chain attack. This attack affected their Orion network management system. Reportedly, 18,000 enterprise and government customers installed malware that was digitally signed by a valid certificate as part of an update from SolarWinds’ servers. Microsoft took control of one of the primary command-and-control domains, and a security researcher stated that he alerted the company in 2019 that anyone could access SolarWinds’ update server by using the password “solarwinds123.”

We also talk about a vulnerability in the PageLayer plugin and a wormable zero-click XSS bug found in the Jabber client.

Here are timestamps and links in case you’d like to jump around, and a transcript is below.
0:31 Reflected XSS in PageLayer Plugin Affects Over 200,000 WordPress Sites
3:06 SolarWinds supply chain attack confirmed, Microsoft takes control of C2 domain. Hundreds of organizations affected.
12:32 Wormable code-execution flaw in Cisco Jabber has a severity rating of 9.9 out of 10

Find us on your favorite app or platform including iTunes, Google Podcasts, Spotify, YouTube, SoundCloud and Overcast.

Click here to download an MP3 version of this podcast. Subscribe to our RSS feed.

Episode 99 Transcript

Ram Gall:
Hi, and welcome to Think Like a Hacker, the podcast about WordPress, security, and innovation. I am Wordfence QA Engineer and Threat Analyst, Ramuel Gall, and with me is Director of Marketing Kathy Zant.

Kathy:
Hey, Ram. It sounds like we’ve got a very busy week in the world of security. What’s going on?

Ram:
Well, first of all, there’s the SolarWinds thing, which is a big, and just keeps on getting bigger and bigger and bigger and expanding like a supernova of, well, insecurity.

Kathy:
Yeah. Before we get into that, though, I wanted to ask you about reflected cross-site scripting vulnerability that you found in the PageLayer plugin, because that’s affecting over 200,000 WordPress sites. What did you find?

Ram:
The PageLayer plugin, it’s a page builder for WordPress. And in this case, it had two vulnerabilities actually, where you could go to the settings page and send in a request to update the font size or the background color. And it did have appropriate checks, it had a permission check, and a nonce check, if you tried to save those settings. But if you just sent those without adding a parameter that says, “Save this,” it would reflect those back to you. And this meant that an attacker could run malicious JavaScript in that.

Ram:
The thing about reflected cross-site scripting is it does take social engineering. You’d have to basically trick an administrator into clicking a link that sent those requests. But once you have that, that script would run in that administrator’s browser. And once it’s doing that, it can do all sorts of things that the administrator could do, including for instance, adding a malicious administrator account or adding a backdoor to a theme header. Having a nonce on the page to prevent cross-site request forgery doesn’t do much because the script that’s running in the administrator’s browser is running on the site and it can read those values and use them when making those requests.

Ram:
And unlike other social engineering things, it’s not like a phishing page where the administrator has to enter their credentials. If the administrator is currently logged into their site and they click on that link, everything that happens from there is automatic.

Kathy:
Interesting. So, this could definitely be used to take over a site.

Ram:
Correct. Pretty much any cross-site scripting that can happen in the administrative panel, can be used to take over a site.

Kathy:
Okay. So if you’re using PageLayer, definitely get your site updated. But WordPress users that are using Wordfence free or the premium version, they were protected?

Ram:
Correct. Our built-in cross-site scripting protection covered this.

Kathy:
Okay. So it’s always good to have just by the nature of having Wordfence firewall on your website, even if there’s a cross-site scripting vulnerability that we haven’t discovered, that we haven’t worked with the developer to fix, you’re going to be protected from any attacks that could leverage that vulnerability, correct?

Ram:
Correct. And every now and then we get a new vulnerability that manages to pull off cross-site scripting in a way that the built-in protection doesn’t cover. And then we jumped right on it and we push out that protection to our users.

Kathy:
Wordfence, good to have. So we talked last week about a pretty shocking problem with FireEye. And this week, it just got a whole lot worse. I mean, crazy worse. What’s going on?

Ram:
So it sounds like it was a supply chain attack in a product that FireEye used. And I think we’ve covered supply chain attacks before in the WordPress ecosystem.

Kathy:
Yeah. But for people who have never heard of a supply chain attack, let’s talk about what that means.

Ram:
So a supply chain attack is effectively where an attacker manages to insert some sort of malicious payload into trusted software. Something that a bunch of people are going to be downloading, installing and using, and they trust it because it’s from a reputable source.

Kathy:
Right. So this happened with WordPress, with some plugins that were purchased from the original developer and a malicious actor came in and purchased those plugins, put malware in them, so when people went to update their plugins, they got the malware, right?

Ram:
Correct. That was the Mason Soiza thing, wasn’t it?

Kathy:
Yeah. Mason Soiza, our old friend. We have some interesting blog posts on the Wordfence blog about all of our research that went into that. So the same type of thing is happening here with the SolarWinds supply chain attack. What exactly happened?

Ram:
I guess the issue was in the Orion platform. So I guess I should give some background on what SolarWinds does. They offer network monitoring and IT management solutions, which is basically a fancy way of saying they have software that they will sell you and you can use it to keep an eye on what’s going on in your network and to make configuration changes if you want to.

Ram:
So the problem was with their Orion platform, which is a one thing to rule them all deal. It runs on a server and you feed it different accounts that you want to give permissions to run as in order to do that monitoring, in order to make those configuration changes. I guess the issue was in the update versions 2019.4 through 2020.2.1. According to the news release article, we’ve seen those were released between March 2020 and June 2020, but I went and looked at the changelog and the 2019.4 version was actually released in November of last year.

Kathy:
So we would suspect then that that 2019.4, what did you find? It was November 5th that it was actually released? But there’s a discrepancy in what’s being reported?

Ram:
Yeah, it’s possible that the attackers uploaded an altered version of that release that only got altered in March 2020, depending on where in the supply chain it was compromised. If they just replaced the existing binary with their own signed binary, then could have just happened since March. But the version that was initially compromised was from the end of last year.

Kathy:
Gotcha. And do we know how SolarWinds was actually compromised?

Ram:
So, we don’t actually have a lot of information on that. A security researcher, Vinoth Kumar, last year told the Reuters that he alerted the company that anyone could access their update server by using the password “solarwinds123.”

Kathy:
Oh, those are my favorite kind of passwords.

Ram:
I know, right? And I guess he found this in a public GitHub repository that was not supposed to be public. And here’s the thing, different analysts have mentioned that this was likely not how they got in. But if they had a public GitHub repository, even if the SolarWinds changed the password that might still reveal information on their internal tooling and it might’ve had other credentials that he didn’t find. So it’s just indicative of a generally poor security posture. Don’t get me wrong. If you’re using open source code then it’s good to have eyes on it. But if you’re actually relying on security through obscurity, which you should not entirely rely on that at all, but if that’s what you’re relying on, then something like this can be devastating.

Kathy:
Definitely. So, I mean, what can we take away from that? So if you’re seeing a developer that has a poor security posture, they’re using “solarwinds123” as a password for important systems, and you see that even mentioned somewhere, it gives you a clue that you should possibly dig further. Because isn’t security all about trust and whether it’s your plugin on your WordPress site or something that’s protecting your giant enterprise network, trust is key, right?

Ram:
It really is. And I mean, SolarWinds was very much a trusted provider. They have/had 300,000 customers. I don’t know how many they still have. I’m sure they still have plenty. But the good news is that only 33,000 of those were using Orion and fewer than 18,000 are believed to have installed the malware-released update. But that’s still 18,000 organizations, including some really high profile ones. Wasn’t it the Treasury Department?

Kathy:
Yeah. We saw news this weekend that the Treasury Department, and I think the Commerce Department as well were compromised. That information actually came out over the weekend before we knew what was going on with SolarWinds.

Ram:
Speaking of stuff that happened before we knew this was going on, didn’t Silver Lake and Thoma Bravo, didn’t a couple of private equity firms also sell more than a hundred million dollars in SolarWinds shares each?

Kathy:
Yeah. I was looking, poking around at some of the financial data of what … Because SolarWinds is a publicly traded organization, and the Washington Post reported that investment firm Silver Lake and Thoma Bravo, a private equity firm, had both sold shares on December 7th before all of this. Now something also interesting that I found as well as poking around, on December 12th, their CEO was no longer employed, Kevin Thompson. And he had sold stock in mid November, as did a number of other high profile executives at SolarWind.

Ram:
This does seem eerily similar to the sequence of events in the Equifax breach. I am sure it will all come out.

Kathy:
Yeah. So what’s going on with Microsoft, because Microsoft was affected by this as well?

Ram:
It looks like Microsoft managed to seize control of their C2 domain, or at least one of their C2 domains. We’re not sure how many C2 domains they had, but this means that Microsoft, theoretically, would be able to shut this down to some extent, or at least prevent it from being exploited further. And that is assuming two things. That’s assuming that the malware doesn’t have any alternate C2 domains that Microsoft doesn’t control and that the attackers weren’t able to send a signal to the malware to update to a different C2 domain in the meanwhile. Which, I mean, that might not be as much of a possibility. I haven’t actually examined the malware myself, but we are aware of C2 controlled malware that does have those capabilities where it will have a list of C2 domains, it will update that list based on input it gets from the C2 servers, and from alternate login paths as well, just in case they lose control of those.

Kathy:
And so it looks like Microsoft put out an advisory for their customers talking about this malicious DLL that was calling out to the remote network infrastructure using AVSVMcloud.com as this domain that the DLL could get second stage payloads from and compromise or exfiltrate data. So Microsoft now has control of that particular domain. So if there are still compromised servers out there that might be phoning home, they might be able to detect that and determine what was happening with this malware.

Ram:
That does sound like it’s likely to be the case. That best case scenario, they’re able to help notify which companies were actually exposed.

Kathy:
SolarWinds did have on their site a list of some of the companies that were using their products as marketing tool, but that was taken down. We do have obviously some screenshots of that in some archives, but what types of companies were using SolarWinds?

Ram:
Financial providers, backbone internet service providers, large organizations, the top five accounting firms. Now these are just customers of SolarWinds. These are not necessarily customers that used Orion. But it is a bit odd that they took down their list of customers.

Kathy:
It is odd. I don’t know if they’re trying to do that in order to protect those customers, but once it’s on the internet, it’s on the internet. Someone will find it and someone will figure out exactly who the customers are.

Ram:
Unless you want to preserve it, in which case you will never be able to find it again. Like GeoCities. I mean, someone is preserving some of the GeoCities sites, but they couldn’t get all of them.

Kathy:
Those were good old days. So we’ll keep an eye on SolarWinds. And as this, obviously still a developing story, still trying to figure out what is going on, who’s affected, how they’re affected. And there are rumors about that this is a state-sanctioned attack. They’re pointing fingers at the Russians, but the Russians are denying that it’s them. So it’s still up in the air exactly what is going on. So obviously we will keep you informed and let you know what is happening.

Kathy:
So now we have another story about Jabber. What exactly is Jabber?

Ram:
So Jabber is a communication client. I don’t know if you’ve used Link or Microsoft Office communicator, or more recently something like Teams or Slack, but it’s basically a messaging client. And it’s an instant messaging client frequently used by internal office communication, that kind of thing. Or used for internal office communication, similar to Slack though. It’s been around a lot longer. It uses the XMPP protocol.

Ram:
However, it does have something in common with Slack and Teams. We discussed this in our last episode. Now it’s not built like Electron, but it does use an embedded Chromium browser. This vulnerability uses the onanimationstart attribute based cross-site scripting to bypass its XSS filters. And by the way, the Wordfence built in firewall and the built-in cross-site scripting protection does block that. So just in case you were wondering. Not that it would have helped in this case because you can’t install Wordfence on the Jabber client.

Ram:
But this was fairly similar to the Teams and the Slack vulnerability we’ve mentioned in previous episodes in that once you have cross site scripting in what is basically a portable web application, if it’s a messaging application, you effectively have a wormable zero-click vulnerability. Because you can send a message to someone with a malicious JavaScript, and that malicious JavaScript will take control of the browser that is the messaging client can say, “Hey, I’m going to find all the other people in this person’s contact list and send them a copy of the script.”

Kathy:
So it could just replicate on its own and take over everybody using Jabber eventually, if it wasn’t stopped and patched.

Ram:
Yeah. I mean, it would only be able to take over the Jabber client and stuff that had been shared through the Jabber client. But then as with Teams, people share sensitive information via internal messaging services all the time. So if your messaging client is a web app that uses an internal browser, then a cross-site scripting vulnerability immediately becomes very severe.

Kathy:
Okay. But this is patched. They had an initial patch that came out in September, and then they just patched it again. Is that true?

Ram:
Yeah, they had to repatch it.

Kathy:
Okay. Gotcha. All right. So if you’re using Jabber, make sure that you update, update every time.

Ram:
They’re not automatic for Jabber, unlike Teams. At least they weren’t the last time I used it. That may have changed since then, since it’s been a couple of years.

Kathy:
Well, we will keep you all posted and all of the security news. We have another treat coming next week. We have episode what, 100?

Ram:
Episode 100, and it’s going to be a very interesting episode, and I look forward to it.

Kathy:
Yeah. We have something planned, so make sure you are following us on Twitter for an update when that comes out. Make sure that you have us in your favorite podcasting app on your device and subscribe to us. You can even subscribe to a specific newsletter just for this podcast. Go to the wordfence.com/podcast page, and there is a form there where you can subscribe if you just want a very short email when a new episode is available. Until then, thank you for listening to Think Like a Hacker. And if you want to find Ram, you are where, @RamuelGall?

Ram:
Yep. Ramuel Gall on Twitter. I don’t have a Facebook anymore.

Kathy:
I don’t blame you. I can’t get anything done if Facebook’s around. I am @KathyZant on Twitter. So follow us there for all the latest news in security, WordPress, and innovation. We’ll talk to you next week.

Ram:
Bye.

Kathy:
Bye.

You can find Wordfence on Twitter, Facebook, Instagram. You can also find us on YouTube, where we have our weekly Wordfence Live on Tuesdays at noon Eastern, 9:00 AM Pacific.