The Inherent Vulnerability of HCI

But patching makes it better.

vSphere admins of the world sure had fun last week! VMW dropped VMSA-2021-0010 on us Tuesday, and you can imagine faces falling as virtualization administrators read the details: unauthenticated RCE on 443, CVSSv3 Range:6.5-9.8, versions 6.5, 6.7, and 7.0 all affected. Any optimism we had going into a long weekend was summarily tempered with the cold reality of unscheduled and / or emergency changes. 

To VMware's credit, the company published a blog post with a stunning amount of information regarding the severity of the issue and clearly expressed the urgency of the situation, to wit: When do I need to do something about this? Right now. The post is as good as the flaw is bad.

For administrators of vCenter Servers that supervise converged or traditional infrastructures, patching was a relatively easy task. Of course you read the release notes first (cue laugh track). In most cases, suffering the indignities of officious change management processes eclipsed the update task itself. If you've been keeping up with VCSA updates, you know that you always grab a snapshot and run an ad hoc backup through the VAMI before you do anything. Once you've taken steps to protect the data, you run the update and within minutes you're back in business. Delete the snapshot and move on.

But for admins of hyperconverged solutions (e.g., Dell's VxRail system), the process is not so simple. HCI vendors maintain a tight grip on the versions of each component in the bundled system, and mince no words in warning users to not upgrade software outside of the bundled updater. There's an expectation that when VMware publishes a software upgrade for a critical security issue, the HCI vendor will quickly incorporate that update into their bundle and make it available for download. In this particular case, VMware published a workaround along with the security advisory that allowed users of HCI to defend against exploits.

However, the gap between VMware and HCI-vendor patch releases is unacceptable when the issue is remote code execution. The finnicky requirements of the infrastructure solution force its users to remain vulnerable beyond the immediacy of moment.

Last Words

HCI is still a worthwhile solution for most vSphere environments. The onus is on the customers and users to decide how they'll account for and supplement for the unacceptable delay in incorporating critical security updates into their solutions.