The Virtual Machine Tsunami

Over the past 2 weeks, I have had a number of very interesting conversations with partners, prospects, customers, and analysts that lead me to believe that a virtual machine tsunami is building which might soon swamp the legacy, horizontal system management approaches. Here is what I have heard:

Two separate prospects told me that they have quickly consumed every available bit of capacity on their VMware server farms. As soon as they add more capacity, it disappears under the weight of an ever pressing demand of new VMs. They are scrambling to figure out how they manage the pending VM sprawl. They are also scrambling to understand how they are going to lower their VMware bill via an Amazon EC2 capability for some portion of the runtime instances.

Two prominent analysts proclaimed to me that the percentage of new servers running a hypervisor as the primary boot option will quickly approach 90% by 2012. With all of these systems sporting a hypervisor as the onramp for applications built as virtual machines, the number of virtual machines is going to explode. The hypervisor takes the friction out of the deployment process, which in turn escalates the number of VMs to be managed.

Amazon EC2 demand continues to skyrocket. It seems that business units are quickly sidestepping those IT departments that have not yet found a way to say “yes” to requests for new capacity due to capital spending constraints and high friction processes for getting applications into production (i.e. the legacy approach of provisioning servers with a general purpose OS and then attempting to install/configure the app to work on the production implementation which is no doubt different than the development environment). I heard a rumor that a new datacenter in Oregon was underway to support this burgeoning EC2 demand. I also saw our most recent EC2 bill, and I nearly hit the roof. Turns out when you provide frictionless capacity via the hypervisor, virtual machine deployment, and variable cost payment, demand explodes. Trust me.

Bottom line, we are all facing an impending tsunami of VMs unleashed by an unprecedented liquidity in system capacity which is enabled by hypervisor based cloud computing. When the virtual machine becomes the unit of application management, extending the legacy, horizontal approaches for management built upon the concept of a physical host with a general purpose OS simply will not scale. The costs will skyrocket.

The new approach will have vertical management capability based upon the concept of an application as a coordinated set of version managed VMs. This approach is much more scalable for 2 reasons. First, the operating system required to support an application inside a VM is one-tenth the size of an operating system as a general purpose host atop a server. One tenth the footprint means one tenth the management burden – along with some related significant decrease in the system resources required to host the OS itself (memory, CPU, etc.). Second, strong version management across the combined elements of the application and the system software that supports it within the VM eliminates the unintended consequences associated with change. These unintended consequences yield massive expenses for testing and certification when new code is promoted from development to production across each horizontal layer (OS, middleware, application). Strong version management across these layers within an isolated VM eliminates these massive expenses.

The VM tsunami is coming. I'm just happy to have my trusty rPath surfboard tethered to my ankle. It's going to be the ride of a lifetime.