On Dec. 10 a crisis-level cybersecurity vulnerability was discovered and the likelihood that chemical facilities even know there’s danger is pretty slim, according to Eric Byres, P.Eng, ISA Fellow, founder and CTO, aDolus Technology Inc., Victoria, British Columbia, Canada. It all lies in Log4j logging software, which is pretty prevalent in the chemical industry.
“Log4j basically allows systems to record events, particularly events that are transmitted over the network: my hard drive is full, somebody's trying to log into my computer, somebody's trying to log into this PLC,” explains Byres. Log4j is free open-source Java software that people have been using in their code to manage the collection and management of those logs.
Complicating detection is the fact that people don't buy Log4j and install it; they get it from suppliers -- anybody who is making software or hardware for the plant floor.
“One of the things we're finding right now is that even the suppliers don't know that they've got Log4j,” notes Byres. “I won't name names or embarrass anybody here, but there is a large industrial-control-systems vendor that we were looking at and we see third-party software that they've embedded and it's got Log4j in it. I'm willing to bet they don't know they've got it right now.”
Byres says there are several reasons these vulnerabilities are considered serious: The first reason is they're trivial to exploit. Two, you get phenomenal control.
“Basically, what happens is that the Log4j designers put in a bunch of defaults that allow you to send a log that would actually execute commands, not just log itself, but actually do things on the computer that was receiving the log.”
Add to those the fact that this software has been used for several years and you have a perfect storm brewing.
“Man, is it widely used because it was good, and it shows up in everything,” says Byres. “You wouldn't find it in a PLC, but you'd likely find it your engineering workstations, your management consoles, your HMIs. If you have anything that collects logs over the network, there's a reasonable chance that the developer said, ‘Hey, look at this free software, we'll use it.’”
Eric Byres, CTO of aDolus, discusses cybersecurity with Trish & Traci.
A money-grab hack prompted executive orders to get cybersecurity under control at firms supplying the government. Private companies will have to comply even if they aren’t the primary contractor.
Oh, and guess what? “There's one more thing that makes it nasty -- the type of equipment that run Log4j are the keys to the kingdom. They are the management servers, they're the administration tools,” says Byres. “If you're an attacker and you can grab control of something like a log-management server, you own the joint. That’s way better than just grabbing a random laptop.”
Marcello Delcaro, software analyst at aDolus, explains that it's so embedded that it's really hard to detect. “If you're not looking for it, then you're not going to know that you don't know. So unless there was some indicator of compromise, that's not something that would be easily recognizable.”
There is hope in terms of a directive from the Cybersecurity & Infrastructure Agency (CISA) regarding workarounds, according to Byres. “Putting in firewall rules that will detect and block anybody trying to send logs that are formatted in the way that will allow you to take over a locked server or learning how to reconfigure your log servers so that those default configurations aren't there.”
Byres points to a blog on the aDolus website that offers four or five workarounds you can implement to mitigate exposure.
But before you can put in a workaround you first have to determine if you have Log4j code. That’s where software bill-of-materials tools can come in handy. Similar to checking out the ingredients on a can of soup, if you go buy software you should get an ingredients list, explains Byres. “Why? Because of days like today.”
One of the main objectives at aDolus is to find the ingredients lists in software. Noting the potential crisis at hand, Byres says his company will analyze software at no charge.
“Right now we're a little bit buried, but probably within a day or two we could give you a very quick analysis of whether you've got Log4j,” says Byres.
If this is so dangerous, why aren't we seeing Colonial pipeline-type outages? Why aren't we seeing the ransomware guys launching lots of ransomware attacks? According to Byres, it's because these guys are so overwhelmed with the opportunities. All they're doing is collecting access points.
“They'll use them later for all sorts of things whether it's intelligence gathering, whether it's for ransomware, whether it's for other exploitation. That's the risky part.”
