24 November 2019

The motto of the IT meets Industry 2019 (IMI) conference in Mannheim was What happens if shit happened. During the World Cafe session, the participants dealt with the following scenario:

1. The cyber-criminal overcame all hurdles you put in place to protect your production systems from attacks.
2. The anomaly detection capabilities in place recognized the attack late.
3. The engineering station (ES) is compromised.
4. You isolated the engineering station from the network for further analysis.
5. The good news is that the process control system (PCS) is still operable.
6. The bad news is that it’s not clear whether the control program in the PCS is also compromised.

You decide to download the control program from the backup into the PCS. This is no uncommon scenario. The Rogue7 (1) attack described at the Black Hat 2019 and Triton (2) work this way. One of the participants put it this way: No Backup, No Mercy! Unfortunately, it’s not that simple.

Where is the current backup stored?

Under normal conditions, the current control program is stored on the engineering station. But this version is not usable because the engineering station is compromised.  If the backup is well organized, a copy of the control program is available from a NAS or a dedicated backup system

Is it really the current version?

This is very important if you want to recover the PCS to the state before the attack happened. Unfortunately, the Recovery Point Objective (RPO) in production is zero. That means, that the latest version of the control program is required for recovery. Older versions require, in the best case, manual reworking, thus a longer downtime and higher financial loss.

Is the PCS restorable from this version and fully operable afterwards?

Have you ever tried a restore test during scheduled maintenance to make sure that the PCS is fully operable after the restore of the control program? Is it clear what is meant by fully operable? Do you have a procedure and check list in place to verify this?

But the worst is yet to come. If you do daily backups there is a small chance that all backup versions are compromised.  In the above scenario, the anomaly detection system detected the attack late. If you keep for instance the latest 10 versions online and the attacker was active for 14 days, then all backups are potentially compromised. So, you must retrieve a backup from a tape library, if any.

Summary

Backup in the age of cyber attacks and ransomware is a hard job, especially in production. Without a strategy and preparation for the worst case a cyber attack may become a financial disaster. The 7 Ps Rule shows the direction in incident response:

Prior Preparation and Planning Prevents Piss Poor Performance!

Want to participate in real peer to peer knowledge exchange and a World Cafe on hot topics? Join the IMI 2020 in Mannheim.

Have a great week.

---

References

1. Biham E, Bitan S, Carmel A, Dankner A, Malin U, Wool A. PPT: Rogue7: Rogue Engineering-Station attacks on S7 Simatic PLCs [Internet]. Powerpoint Presentation presented at: Black Hat USA 2019; 2019 Aug 8 [cited 2019 Aug 16]; Mandalay Bay / Las Vegas. Available from: https://i.blackhat.com/USA-19/Thursday/us-19-Bitan-Rogue7-Rogue-Engineering-Station-Attacks-On-S7-Simatic-PLCs.pdf
2. Sobczak B. SECURITY: The inside story of the world’s most dangerous malware [Internet]. 2019 [cited 2019 May 11]. Available from: https://www.eenews.net/stories/1060123327