Microsoft Embraces Linux Virtual Appliances

In a very savvy move aimed at gaining competitive advantage in the cloud computing space, Microsoft yesterday announced that they were contributing source code to the Linux kernel in order to optimize the performance and management of virtual appliances on Microsoft's hypervisor, Hyper-V. One of the goals of cloud computing is elasticity – applications scale up and down based upon the hour by hour demand for the application. Well, you cannot have hour by hour elasticity for an application when it takes days/weeks/months to install, provision, and instrument the application onto the infrastructure. Virtual appliances eliminate this challenge by allowing the application owner to pre-configure the application as a set of virtual machines that are ready to respond to demand. The “set-up” is done “off-line.” Microsoft, realizing that Linux is the de facto underpinning for virtual appliances that run on Amazon's EC2, is now contributing code to Linux that will optimize the performance and management characteristics of Linux-based virtual appliances on Hyper-V – the virtualization technology that underpins Microsoft's Azure cloud.

Microsoft's new status as a Linux kernel contributor sheds light on the amazing shift that is occurring in the battle-lines for IT infrastructure. The operating system is going to split, with one half becoming the control software for the hardware (the hypervisor) and the other becoming the control software for the application (virtual appliances). Given their huge advantage with developers based upon the installed base of applications that run on Microsoft-only application frameworks (.Net, etc.), Microsoft has determined that they need to pull out all of the stops in order to be certain they do not get ripped off the hardware in favor of VMware (the dominant hypervisor) or Xen (the hypervisor that supports Amazon's market leading cloud service). Linux is no longer the biggest threat to Microsoft in the datacenter when datacenters begin embracing a cloud architecture such as Amazon's in order to enable IT-as-a-Service.

If indeed this move enables higher elasticity and simpler management of Linux based virtual appliances that run atop Hyper-V, the competitive pressure might force VMware to follow suite and make their drivers and tools available as source code that is included in the Linux kernel. To be clear, Microsoft does not currently plan to support Linux virtual appliances on Azure, but that position may be shifting with changes of this type. With Amazon currently holding the dominant position in cloud and with VMware holding the dominant position in the datacenter for virtualization, Microsoft might have lots of crafty tricks up its sleeve to re-assert themselves in this new theater of datacenter war where hypervisors and virtual appliances rule the day.

McKinsey Recommends Virtualization as first step to Cloud

In a study released last week, the storied consulting company, McKinsey & Company, suggested that moving datacenter applications wholesale to the cloud probably doesn't make sense – it's too expensive to re-configure and the cloud is no bargain if simply substituted for equipment procurement and maintenance costs. I think this conclusion is obvious. They go on to suggest that companies adopt virtualization technology in order to improve the utilization of datacenter servers from the current miserable average of ten percent (10%). I think this is obvious too. The leap that they hesitated to make explicitly, but which was called out tacitly in the slides, was that perhaps virtualization offers the first step to cloud computing, and a blend of internal plus external resources probably offers the best value to the enterprise. In other words cloud should not be viewed as an IT alternative, but instead it should be considered as an emerging IT architecture.

With virtualization as an underpinning, not only do enterprises get the benefit of increased asset utilization on their captive equipment, they also take the first step toward cloud by defining their applications independent from their physical infrastructure (virtual appliances for lack of a better term). The applications are then portable to cloud offerings such as Amazon's EC2, which is based on virtual infrastructure (the Xen hypervisor). In this scenario, cloud is not an alternative to IT. Instead, cloud is an architecture that should be embraced by IT to maximize financial and functional capability while simultaneously preserving corporate policies for managing information technology risk.

Virtualization as a step to cloud computing should also be viewed in the context of data, not simply application and server host resources. Not only do applications need compute capacity, they also need access to the data that defines the relationship of the application to the user. In addition to technology such as VMware and Citrix's Xen technology, enterprises also need to consider how they are going to abstract their data from the native protocols of their preferred storage and networking equipment.

For static data, I think this abstraction will take the form of storage and related services with RESTful interfaces that enable web-scale availability to the data objects instead of local network availability associated with file system interfaces like NFS. With RESTful interfaces, objects become abstracted from any particular network resource, making them available to the network where they are needed. Structured data (frequently updated information typically managed by a database server technology) is a bit trickier, and I believe solving the problem of web-scale availability of structured data will represent the “last mile” of cloud evolution. It will often be the case that the requirement for structured data sharing among applications will be the ultimate arbiter of whether an application moves to the cloud or remains on an internal network.

The company that I founded, rPath, has been talking about the virtualization path to cloud computing for the past three years. Cloud is an architecture for more flexible consumption of computing resources – independent of whether they are captive equipment or offered by a service provider for a variable consumption charge. About nine months ago, rPath published the Cloud Computing Adoption Model that defined this approach in detail with a corresponding webinar to offer color commentary. In the late fall of last year, rPath published a humorous video cartoon that likewise offered some color on this approach to cloud computing. With McKinsey chiming in with a similar message, albeit incomplete, I am hopeful that the market is maturing to the point where cloud becomes more than a controversial sound-byte for replacing the IT function and instead evolves into an architecture that provides everyone more value from IT.

Killing the OS Octopus

The inspiration for this blog post title comes from a Paul Maritz (CEO of VMware) quote, during his presentation to financial analysts last week. Paul used the phrase "severing the tentacles of complexity" multiple times when referring to the new level of business flexibility that is possible when applications are liberated from their physical host by encapsulating them inside of a virtual machine with “just enough operating system (or JeOS).” They can be provisioned much more quickly because there is no need to provision physical assets. They can be moved from datacenter to datacenter more quickly because there is no onerous installation and validation process required. Indeed, virtualization enables cloud computing because the applications are no longer defined by the physical computers upon which they run. But, until VMware truly embraces a JeOS approach with their operating system support matrix, they are simply recommending “isolating” the tentacles of complexity. And the result will be a perilous and expensive condition often referred to as VM sprawl.

So what is the difference between “isolating” the tentacles of complexity and “severing” them? Isolating the tentacles means shoving the previous definition of your application running on a physical server into a virtual machine box. When you put a virtual machine box around the octopus, it can no longer create mischief with other application octopi running on the same physical host. Its tentacles are “isolated,” and utilization on the physical host can be much higher. This approach is valuable, and it has catapulted VMware into the spotlight as one of the hottest technology companies on the planet.

However, the octopus is still alive and well inside the box, and system administrators must continue to feed that hungry animal in exactly the same way they did when it was living on a physical server host. The level of maintenance has not been reduced. The level of security vulnerability has not been reduced. Although isolated, it is still a resource hog because those crazy tentacles demand CPU, and memory, and disk to flail and flap as they do. This condition of ever expanding system administration grief associated with the frictionless deployment of virtualized applications whose tentacles of complexity have simply been isolated and not severed is known as VM sprawl. And it will be a nightmare of system administration expense for those that embrace it.

In order to avoid the nightmare of VM sprawl, the tentacles of the complexity octopus must actually be severed, not simply isolated. Application developers and system administrators alike must re-think the category of the operating system in the context of a virtualized datacenter. Since the operating system is no longer the conduit for managing the hardware, it should become a simple shared library for system services required by the application. Two great example of this approach are rPath and BEA's (now Oracle) liquid VM technology. With both of these platforms, the operating system is specified in a manner that explicitly supports the needs of the application – without any extra bloat associated with the typical general purpose OS approach. As a result, the OS in both of these cases is 10X or more smaller than the smallest installation option offered by a general purpose OS. In theory, this should lead to a 10X reduction in the scope and scale of administration activities. Severing the tentacles of complexity by re-thinking the OS eliminates the perils of VM sprawl.

But VMware does not currently support this approach. They only support the legacy vendors of general purpose OS technology. Sure, these new approaches have terrific performance and value, and VMware is happy to have them contribute to the value of their virtual appliance market, but their support statement pretty clearly favors “isolation” of complexity over true elimination of complexity. But the winds of change are steadily and surely blowing in favor of this new approach in the market. Red Hat, for example, just announced that they are going to market with a bare metal hypervisor that is directly competitive with the VMware approach in lieu of their historical product architecture - where virtualization was simply a feature of the general purpose operating system. And Paul Maritz was pretty clear in his presentation that “severing the tentacles of complexity” and a “just enough operating system” approach are important to VMware. Perhaps we are drifting toward the precipice of an all out war for the definition of the future datacenter operating system. I said it back in 2006, and I'll say it again today -- let's fry up that OS octopus polvo frito and serve it with some spicy mango chutney and cold beer.

Red Hat Goes Streaking - Dumps Xen

Yesterday Red Hat announced their revised virtualization strategy. Most interesting to me was Red Hat's declaration that the hypervisor should indeed lie "naked" on the metal. Red Hat also announced end-of-life for Xen support and some stuff regarding desktop virtualization and protocols that I don't pretend to understand. While it was expected that Red Hat would end-of-life support for Xen after their acquisition of the KVM technology from Qumranet, the fact that Red Hat went streaking into the market with a bare metal approach to the hypervisor was a pretty significant strategy reversal.

Until yesterday, Red Hat had always adopted the Microsoft line on the hypervisor - it is simply a feature of a general purpose operating system. Red Hat historically claimed that the hypervisor should not lie naked on the bare metal, but instead it should be wrapped up inside the general purpose OS - a little extra bloating that never hurt anybody. It looks like some combination of market forces - likely VMware financial success and Amazon's adoption of Xen for their Elastic Compute Cloud - have forced Red Hat to consider a new approach. Now Microsoft stands alone in their contention that a hypervisor is just a new general purpose OS feature, and the rest of the market can move on to the market share land grab for the large-scale Linux datacenters. It should be an interesting race, because all three players - Xen (Citrix), KVM (Red Hat), and VI (VMware) are all coming from different positions of strength - and weakness.

In almost every case I have observed, Xen is the incumbent virtualization technology among the Linux datacenter consumers that have virtualized any production workloads. However, not many have actually virtualized because the historical compelling case for virtualization - server consolidation - falls on deaf ears among the Linux crowd. With Linux, as opposed to Windows, it is possible to run multiple application workloads on a single server without significant instability. Server utilization can be quite high without virtualization - hence no requirement for Linux server consolidation. But, the new benefits of virtualization - flexibility, security, and elasticity - apply equally, if not especially, well to the Linux workloads. If application workloads are separated into unique, small footprint, virtual machines (virtual appliances, if you like) that run atop a bare naked hypervisor, then:

- flexibility is better because one application can be managed/administered without interfering with another workload running on the same host

- security is better because vulnerable services like DNS are isolated and easier to maintain/secure due to a smaller footprint

- elasticity is better because application workloads can be scaled quickly when demand increases and also retired quickly when demand recedes (cloud, anyone?)

Because of these new business benefits, Xen is beginning to take hold in the Linux datacenters. Citrix, however, has not historically been a strong brand among the Linux savvy crowd, so it is unclear if they have the stomach for pursuing a new market segment. VMware is the 800 pound gorilla in hypervisors generally, but they too have shown very little Linux savvy (their management console only runs on Windows) - probably because 90% of their revenues come from virtualizing Windows workloads. Then there is Red Hat, the 800 pound Linux gorilla who has been in denial about the importance of a bare metal approach for the hypervisor - until now.

Based on this set of circumstances, I would say that the virtualization opportunity for the Linux datacenter market is a wide open race. Now that Red Hat has decided to run the race naked, it should be more fun to watch.

Is JeOS a Tonic for VM Sprawl?

It seems that everyone is worried about VM sprawl these days. When system capacity is easy to consume because the application is prepackaged as a virtual machine (a virtual appliance in the case of an ISV), the virtual infrastructure capacity is quickly gobbled up by those that have been waiting for IT to get around to provisioning systems for them. No more friction due to virtualization means no more waiting and wanting. It also leads to VM sprawl. But why is VM sprawl bad?

VM sprawl is bad because the scale of the management problem used to be throttled by the capital spending associated with the size of the infrastructure. Now, the scale of the management problem is equal to the true demand for application capacity as represented by the number of application images, or virtual machines, that get deployed. This scale factor associated with application images throws the old yardstick of X system admins per Y server machines out the window. What are we going to do to lower the work profile associated with so many new systems that now need to be managed?

I think at least part of the tonic for VM sprawl is the new acronym coined by Srinivas Krishnamurthi of VMware – JeOS. JeOS stands for Just enough OS and it is pronounced “juice.” In my mind, this liquid pronunciation is appropos given my view of its potential potency as a tonic for VM sprawl. A huge part of the burden of system management is the patching and associated regression testing for maintaining the security and functionality of the general purpose OS. If you can shrink the size of the OS by 90% (which is where we have measured the typical size for most applications built with our rBuilder technology) by eliminating any elements not required by the application, you can eliminate about 90% of the patching burden. More importantly, you eliminate the bigger burden of regression and stability testing that is coupled to the patching process.

With a JeOS approach, the number of virtual machines can theoretically grow by 10X the legacy approach without any impact to the cost structure associated with patching and testing the OS changes. I suspect a 10X reduction represents a real win for most shops as the OS patching and testing associated with security and infrastructure performance is a very sizable portion of their management spending. So if you are feeling the pangs associated with VM sprawl, I strongly suggest a healthy slug of JeOS each morning and once again in the afternoon to clear your system of the painful bloating that is brought on by virtualizing the general purpose OS.

Can You See the Clouds from Windows?

During the course of our webinar entitled "The Pragmatist's Guide to Cloud Computing: 5 Steps to Real ROI," several of the attendees submitted questions regarding the status of Windows as an environment for cloud applications. In a partial answer to the question, Jeff Barr, a speaker during the webinar and a member of the Amazon Web Services team, responded that a beta implementation of Windows for EC2 was now available. The problem with the notion of “Windows for EC2” is that it perpetuates the broken, legacy model of tying your application to the infrastructure upon which it runs.

In the legacy model, applications became artificially tied to the physical server upon which they ran, and server utilization was low because it is very difficult to run multiple applications on a single instance of a general purpose operating system. The reason it is difficult to run multiple applications on a single instance of a general purpose operating system is because each application has unique needs which conflict or compete with the unique needs of other applications. Virtualization technology, such as that provided by VMware or Citrix with XenServer, breaks the bond of the application to a physical server by placing a layer of software, called a hypervisor, on the physical hardware beneath the operating system instances that support each application. The applications are “isolated” from one another inside virtual machines, and this isolation eliminates the conflicts.

Amazon embraces this virtualization model by using Xen to enable their Elastic Compute Cloud (EC2) service. So what's the problem? If the OS instances are not tied to the physical servers any longer (indeed you do not even know which physical system is running your application on EC2, nor do you need to know), why am I raising a hullabaloo over a “broken model?” The reason this new model of Windows for EC2 is broken is because your application is now artificially coupled to EC2. When you begin with a Windows Amazon Machine Image (AMI), install your application on top, configure-test, configure-test, configure-test, configure-test, configure-test to get it right, and then save the tested configuration as a new AMI, the only place you can run this tested configuration of your application is on Amazon's EC2. If you want to run the application on another virtualized cloud, say maybe one provided by RackSpace, or Terremark, or GoGrid, or even your own internal virtualized cloud of systems, you have to install the application yet again, configure-test, configure-test, configure-test, configure-test, configure-test to get it right again, and then save the tested configuration on the other cloud service. Why don't we just stop the madness and admit that binding the OS to the physical infrastructure upon which it runs is a flawed approach when applications run as virtual machine images (or virtual appliances) atop a hypervisor or virtualized cloud of systems like EC2?

The reason that we are continuing the madness is because madness is all we have ever known. Everyone knows that you bind an operating system to a physical host. Operating systems are useless unless they bind to something, and until the emergence of the hypervisor as the layer that binds to the physical host, the only sensible approach for operating system distribution was to bind it to the physical host. When you buy hardware, you make it useful by installing an operating system as step one. But if the operating system that you install as step one in the new virtualized world is a hypervisor in lieu of a general purpose operating system, how do we get applications to be supported on this new type of host? Here's your answer -- what we previously knew as the general purpose operating system now needs to be transformed to just enough operating system (JeOS or “juice”) to support the application, and it should bind to the application NOT THE INFRASTRUCTURE.

Virtualization enables the separation of the application from the infrastructure upon which it runs – making possible a level of business agility and dynamicism previously unthinkable. Imagine being able to run your applications on-demand in any data-center around the world that exposes the hypervisor (any hypervisor) as the runtime environment. Privacy laws prevent an application supporting medical records in Switzerland from running in an Amazon datacenter in Belgium? No problem, run the application in Switzerland. Need to run the same application in Belgium in support of a new service being offered there next month? No problem, run it on Amazon's infrastructure in Belgium. The application has to support the covert operations associated with homeland security and it cannot be accessed via any Internet connection? No problem, provide it as a virtual appliance for the NSA to run on their private network. Just signed a strategic deal with RackSpace that provides an extraordinary level of service that Amazon is not willing to embrace at this time? No problem, shut down the instances running on EC2 and spin them up at RackSpace. All of this dynamic capability is possible without the tedious cycle of configure-test -- if we will simply bind the operating system to the application in order to free it from the infrastructure and let it fly into the clouds.

So why doesn't Microsoft simply allow Windows to become an application support infrastructure, aka JeOS, instead of a general purpose operating system that is bound to the infrastructure? Because JeOS disrupts their licensing and distribution model. Turning a ship as big as the Microsoft Windows licensing vessel might require a figurative body of water bigger than the Atlantic, Pacific, and Indian oceans combined. But if they don't find a way to turn the ship, they may find that their intransigence becomes the catalyst for ever increasing deployments of Linux and related open source technology that is unfettered by the momentum of a mighty business model. Folks with valuable .Net application assets might begin to consider technology such as Novell's mono project as a bridge to span their applications into the clouds via Linux.

I can tell you that there are lots of folks asking lots of questions about how to enable Windows applications in the “cloud.” I do not believe the answer is “Windows for EC2” plus “Windows for GoGrid” plus “Windows for RackSpace” plus “Windows for [insert your data-center cloud name here].” If Microsoft does not find a way to turn the licensing ship and embrace JeOS, the market will eventually embrace alternatives that provide the business agility that virtualization and cloud computing promises.

Will the Credit Crunch Accelerate the Cloud Punch?

It's no secret that the days of cheap capital might be over. While it is obvious that startups with lean capital structures are already embracing cloud offerings such as Amazon EC2 for computing and S3 for storage, it seems to me that this trend might accelerate further for both startups and even enterprise customers.

Cloud consumption in the startup segment is poised to accelerate as investors like Sequoia Capital warn their portfolio companies to “tighten up” in the face of this credit crunch. Even the well capitalized SaaS software providers might begin re-considering the “ridiculous” expense of building out their offerings based upon the classic salesforce.com model of large scale, proprietary datacenters with complex and expensive approaches to multi-tenancy. They might be better served by a KnowledgeTree model where on-demand application value is delivered via virtual appliances. In this model, the customer can deploy the software on existing gear (no dedicated server required) because the virtualization model makes for a seamless, easy path to value without setup hassles. Or they can receive the value of the application as a SaaS offering when KnowledgeTree spins up their instance of the technology on Amazon's elastic compute cloud. In both cases, the customer and KnowledgeTree both avoid the capital cost of acquiring dedicated gear to run the application.

Large enterprises as well will be re-considering large scale datacenter projects. When credit is tight, everyone from municipal governments to the best capitalized financial institutions must find ways to avoid outlays of precious capital ahead of the reality of customer collections. More and more of these customers will be sifting through their application portfolio in search of workloads that can be offloaded to the cloud in order to free up existing resources and avoid outlays for new capacity to support high priority projects. Just as the 9/11 meltdown was a catalyst for the adoption of Linux (I witnessed this phenomenon as the head of enterprise sales at Red Hat), a similar phenomenon might emerge for incremental adoption of cloud associated with the credit crunch of 2008. All new projects will be further scrutinized to determine “Is there a better way forward than the status quo?”

As enterprises of all sizes evaluate new approaches to minimize capital outlays while accelerating competitive advantage via new applications, rPath is offering a novel adoption model for cloud computing that might serve as a convenient bridge to close the credit crunch capital gap. For those that are interested in exploring this new model, pleas join us in a webinar along with the good folks at Forrester, Amazon, and Momentum SI on October 23rd. If necessity is the mother of invention, we might be poised for some truly terrific innovations in the cloud space . . . . and we will owe a debt of gratitude to the credit crunch for driving the new architecture forward.

VMware Strikes Back

The tech industry has been all abuzz lately about the competitive hullabaloo surrounding the new hypervisor technologies that are emerging to take on VMware's dominant hypervisor product. Microsoft launched Hyper-V with a party last week to upstage VMware's VMworld event this week. Red Hat purchased Qumranet to solidify its control of the KVM hypervisor technology. Now, VMware is striking back at the legacy OS vendors by labeling their new product category the Virtual Datacenter Operating System – a direct attack on the entrenched category of the general purpose operating system. The cold war of spies and covert operations to grab mindshare while outwardly promoting a message of peaceful co-existence has officially escalated to a hot war for the future architecture of the datacenter.

I, for one, am happy to see this rise in hostilities because I believe it will carry the industry to a much better place – and customers will be the primary beneficiary of the new approach. In the legacy datacenter, a general purpose operating system attempts to serve both the hardware infrastructure with device drivers while also serving the applications with system services. This approach has the disadvantage of artificially coupling applications to physical servers. Any attempt to move the application to another physical server typically requires that the configuration and validation process begin anew because it is extremely unlikely that the new server is absolutely identical to the previous one. This lack of flexibility leads to extreme overspending on capital equipment because an application with a period of low demand cannot relinquish its hardware resources to an application experiencing high demand. The hardware resources become application specific, and each application owner must size hardware capacity to meet peak demand. Server utilization in the datacenter averages 15 – 20%, and the general purpose OS is the culprit.

VMware has now declared that they offer an alternative approach to the general purpose operating system. The technology is not new, but marketing it under the category of an operating system is a very different tactic in this war for the datacenter. The conflict is now overt instead of covert, and this change was inevitable as VMware attempts to expand its footprint beyond its bread and butter business of Windows server consolidation and test lab operations. The new objective is the elastic datacenter and ultimately cloud computing.

The datacenter becomes elastic when applications are released by developers as coordinated sets of virtual machines (or virtual appliances in the case of a vendor release), each with Just Enough Operating System (JeOS or “juice”) attached to provide the system services required by the application. These applications can expand or contract on-demand because there is no onerous configuration process to ready the general purpose OS for a specific application. Instead, the hypervisor accepts the virtual machine and allocates it resources as specified by the OVF meta-data that is included with the image. Applications are up and running in a matter of seconds, and the process is totally repeatable to assure stability, security, and compliance as workloads scale, de-scale, and re-scale to meet the ever changing demand profiles of the enterprise application portfolio. Infrastructure can become a variable cost via this architecture because the scaling cycle can include hardware resources provided by third parties via a hypervisor layer – aka cloud computing as popularized by Amazon's Elastic Compute Cloud (EC2) via the Xen hypervisor.

The reason I label this new competitive tact by VMware as “warfare” is because the concept of a hypervisor as the infrastructure management layer with JeOS as the system services layer for the applications delivered as virtual machines destroys the value of the general purpose OS. If a hypervisor provides access to the infrastructure via device drivers, and applications receive system services from JeOS, and the flexibility of the datacenter improves, and the management of applications is simplified, and I can embrace cloud computing for variable cost infrastructure, why would I ever again buy a general purpose operating system? I won't. And customers won't either.

Unleash the dogs of war. Let's get to it so that we can all live happily ever after on the other side of history.

Hypervisor Certification Crossroads

My last blog post, Certification “aka Some Assembly Required,” stirred up an interesting brew of fan mail. It seems that I wrote what lots of people have been thinking when I proclaimed that certification is a weak promise at best -- a hoax at worst -- with most customers spending 6X their license costs on installation, maintenance, and administration of “certified” software. The pity of this whole situation is that software vendors are also paying a high price for the myriad of customer preferences regarding middleware and operating systems. And the price is about to get a lot steeper.

Applications vendors will soon find themselves at a crossroads. They must either embrace virtual appliances (with certification to the hypervisors as the critical element), embrace SaaS ala Salesforce.com as their only distribution approach (where customers get no choices regarding infrastructure or even features for that matter), or brace for an enormous expense uptick in certification costs when they must begin including all of the hypervisor products as additional customer preference items required for testing.

The certification gauntlet that application vendors have historically run in order to deliver their application to the widest possible market is often referred to as “the matrix of pain.” For every element in the stack that might be a customer preference – OS, application server, web server, database – and for every release of these elements, the application vendor creates a row and a column. Each intersecting cell above the diagonal represents a configuration to be tested as part of the R&D expense associated with release engineering. Of course the testing never occurs in the exact configuration of elements for a given customer situation, but some testing is better than none.

And, theoretically, each maintenance update of each element requires re-testing. Of course, no application vendor actually does this testing because they simply cannot afford to do it. As it is, certification gobbles up 40 – 50% of R&D expenditures (not to mention the customer service burden associated with customer variability). Don't believe me? Why do you suppose salesforce.com spends 8 – 9% of revenue on R&D for a pretty rich CRM application when most software companies spend 15 – 18%? Salesforce.com only certifies to ONE infrastructure – their own datacenter infrastructure. And they bring every customer along with every release of software so they do not have the testing expense associated with maintaining legacy code on new platforms or vice versa.

Now, with hypervisors providing customers with so much value and flexibility as the layer that abstracts the hardware from the application, vendors are going to be pushed by their customers to certify their applications against all of the popular hypervisors. And, with such a hot category, there will be lots of technology development and lots of releases. That means lots of NEW columns and rows for the “matrix of pain.” If you thought 40 – 50% of your R&D budget was painful because it sucked the wind out of new feature development, wait until that number climbs to 60 – 70% when you throw in another layer for the hypervisor. Add some more to those costs when customers start demanding that you also support Amazon's EC2 and other similar services like IBM's Blue Cloud.

Had enough? Ready to cry “UNCLE?” Better get busy figuring out how to deliver your application as a virtual appliance. For those virtual appliances, you will STILL need to release and test against the hypervisor products -- VMware, Citrix' XenSource, Microsoft's pending Hyper-V -- along with all of the “cloud” computing services that are going to emerge to compete with Amazon's EC2. Better choose your virtual appliance infrastructure provider wisely to accommodate these customer choices in the market (shameless plug for rPath).

No doubt customers will initially object to some limitation of choice regarding the elements that support the application when you wrap it all up and deliver it with a bow on top as a virtual appliance, but these are the same customers that are going to drive you to embrace hypervisors in your testing matrix. Give them a tradeoff – virtual appliances that are cheap and easy to install, maintain, and administer, or the old fashioned way with a $250,000 per year support uplift to cover your expense for navigating their “matrix of pain.” Standing at this expense crossroads, I bet they make the same decision you should make as you stand at the hypervisor certification crossroads today -- “Let's give that virtual appliance a try.”

Intel Invests Heavily in the Future of Operating Systems - VMware

Intel and VMware announced today that Intel Capital is making a $218M pre-IPO investment in VMware. This investment would give Intel a 2.5% ownership stake in VMware, with a board seat, but no real voting rights whatsoever (less than 1%) relative to VMware's parent company, EMC. So why might Intel be making such a significant investment in VMware? I believe it is because Intel plans to influence the outcome of the investment through its X86 distribution chain.

Sales of computers, especially server computers, are currently constrained by the complexity of software. Larry Ellison estimates that for every $1 in license spending for applications customers spend $6 in integration and maintenance expense. Not only is this complexity a drag on new license sales of software, it is also a drag on new sales of servers to run that software. What if a technology existed that could eliminate the drag of integration and maintenance such that customers could spend more money on new applications and servers? Maybe Intel would be interested in seeing such a technology proliferate?

Well, VMware has that technology, and I believe Intel plans to integrate VMware's technology with it's chipsets such that applications can arrive as virtual appliances, attach to the system in a matter of seconds, and then be managed by the application provider as a complete system image instead of being managed by the customer as a set of “lincoln logs” to be assembled in a unique manner each time. With VMware hypervisor technology as the replacement for the bloated, “one size fits all” general purpose operating system, customers can actually take advantage of Moore's law instead of being held back by the lack of installability, maintainability, and portability of software applications.

How many times has the hardware sales rep been stymied in the sale of a hot new machine because the customer absolutely refused to touch the machine that was currently running their ERP/CRM/SCM/(pick your alphabet soup) application? They refused to touch it because once it was installed, configured, and stabilized no one wanted to repeat the hellish process that led to that stability.

Imagine the new world order with the hypervisor embedded at the chipset level as the replacement for the bloated general purpose operating system. You simply hook up power and networking to the new box, and then “vmotion” the system image on the old box over to the new box. And magically you now have X% more processing power, I/O, memory, or whatever you need on that fancy new box. Life doesn't get much better than that for a hardware sales guy or gal.

My bet is that you see Intel pushing their downstream server partners to adopt Intel chipsets that enable VMware hypervisor functionality at the level of the BIOS. Intel and its downstream partners such as IBM, Dell, and HP will develop system management technology along with vendors like Cisco, Symantec, and McAfee that enables “value added” services to also be delivered to the machine without the general purpose operating system getting in their way. You will see these supply chain partners develop virtual appliance catalogs that contain applications that can be added to the system in the same manner that you add songs to your iPod through iTunes. When software ceases to be a drag due to operating system complexity, hardware sales accelerate tremendously. All this for a measly investment of $218M. I think Intel got a great deal.

Microsoft Attempts Software Appliances

Microsoft has been slowly revealing the roles that will be available as Server Core installation options for the upcoming Windows Server 2008. These Server Core installations are effectively Microsoft's attempt to provide a software appliance packaging option for customers in order to address the nightmare of administration and stability issues created by the general purpose OS approach (i.e. “one size fits all”). So how are they doing?

In a recent blog post, Scott Fulton replays some of the messaging that Microsoft product manager Andrew Mason delivered at TechEd 2007. According to Scott's post, Microsoft has achieved the following:

- a reduction in the attack surface of the OS from 5GB to about 1.5GB

- a corresponding reduction in the patching burden of 60%

- command line administration (although without “Powershell” - whatever that is)

- 9 server roles available sometime in 2008 or 2009, including a hypervisor role

Well, I suppose this is progress. However, if the goal is simplicity and a reduction in the burden of administration for security and patching, Microsoft still has a very long road to travel. Let's do a quick comparison with rPath's rBuilder capability:

- attack surface for “group core” is about 50MB (no, that is not a typo)

- by proxy, a reduction in the patching burden of 85% vs. the classic Windows approach (this figure does not consider at all the lesser patch requirement for Linux overall, just a straight ratio)

- command line plus graphical, Internet enabled rPath Appliance Agent administration

- infinite roles available based upon the applications generally available for Linux

As the world adopts server virtualization for X86 en masse, these critical packaging differences are going to become a huge challenge for all of the general purpose OS vendors. When hypervisor virtualization such as that offered by VMware and XenSource replaces the general purpose OS as the mechanism for managing infrastructure and attaching applications to the infrastructure (via virtual appliances), the critical packaging requirements for the OS that hosts applications will be:

- tight dependency management for the smallest possible attack surface

- configuration flexibility to optimize the system software to the application workload

- flexible kernel tooling to optimize performance across the various hypervisor management systems

- user friendly interfaces for creating and maintaining the software appliance definition

Given the technology, business, and cultural hurdles that the incumbent general purpose operating system vendors face in implementing these capabilities, it is unclear to me that they will commit to this new approach in a timely fashion. Although Microsoft acknowledges the need for these “slimmer” server core roles, they only plan to achieve a surface area reduction to 1.5GB, with limited roles, and virtually no user enabled packaging options for further optimization for the variety of applications and hypervisors. If that is all that Microsoft can achieve in the face of this obvious demand in the market for hypervisors with virtual appliances, the relevance of the incumbent players in the OS space may be diminished more quickly than anyone currently imagines.