Billy on Open Source

VMware Spits Into the Wind - Buys Zimbra

2010-01-15T16:09:00.002-05:00

What's next? Tugging on Superman's Cape? Pulling the mask off the ol' Lone Ranger? In my opinion, the future of email and collaboration belongs to Google, with Microsoft playing very strong defense shifting folks directly to Azure (which doesn't include Exchange today, but I will bet a nickel it is in the works). If the acquisition of Zimbra is an attempt by VMware to arm the service providers with a similar capability, I sincerely hope VMware is not expecting to make any money along the way.

Don't get me wrong, as I have long been a huge fan of VMware and Zimbra. The former President and CTO of Zimbra, Scott Dietzen, is a friend and sits on the board of rPath, the company that I founded. Zimbra was an rPath customer. rPath was a Zimbra customer. But I think the notion of folks running email and basic collaboration functions at small scale (i.e. any scale that doesn't match Google and Microsoft hosted solutions in the future) is a lost cause – or at least a VERY low margin cause. And I don't think VMware should get into that business (i.e. the hosted solutions business) as it is a completely different beast than selling infrastructure software licenses (ask Microsoft, they haven't got it right but they are willing to spend BILLIONS and still be wrong).

It just makes no sense for anyone in the future to take on the burden of running an enterprise collaboration service (i.e. Exchange, Zimbra) with the software model. I even believe that RIM is going to have problems sustaining the Blackberry business as I witness the integration of Gmail with my iPhone. As I survey this market, I find more and more companies quietly moving their email/calendar/messaging services to Google. If VMware thinks this move is about Microsoft, I think they are wrong. I think it is about Google, and I would not be standing in line to pull the cape or remove the mask after spitting into the wind. VMware is no south Alabama boy named Slim.

Windows 7 Diagnosed with Cancer

2009-11-17T14:08:00.004-05:00

As I read this news article about a downloadable tool for Windows 7 being "infected" with open source code, I could not help but recall the boisterous statements by Steve Ballmer during the uprising of Linux popularity in the early part of this decade. Ballmer proclaimed Linux and open source to be a "cancer" that would eat away at all that is good in the software industry. Not only the industry, but capitalism itself was at risk due to the contagious and destructive nature of this new approach to software distribution.

I suppose it is now time to begin the radiation and chemo treatments on the Redmond patient. The first tumor has been discovered, and no doubt there are others awaiting detection. As they are discovered, my guess is that the treatment will be far more damaging to the patient than the disease itself. Once you are infected with the potential of open source as a short cut to a better customer experience, paying full price on every line of code becomes challenging. Even for a patient that can spend as heavily as Microsoft on disease treatment.

Amazon Aims for Enterprises - Poo Poos Internal Clouds

2009-08-27T16:38:00.001-04:00

Amazon's announcement yesterday regarding an enterprise feature for linking existing datacenter operations to Amazon's AWS via a Virtual Private Network feature did not surprise me. It is an obvious extension of their value proposition, and folks had already been accomplishing a similar capability with work-arounds that were simply a bit more cumbersome than Amazon's integrated approach. The more surprising piece of news, in my opinion, is the subtle racheting up of the rhetoric by Amazon regarding their disdain for the notion of “internal” cloud. Werner Vogels blog post explaining the rational for the new VPN features is a case in point. Here are a few tasty excerpts:

Private Cloud is not the Cloud

These CIOs know that what is sometimes dubbed "private [internal] cloud" does not meet their goal as it does not give them the benefits of the cloud: true elasticity and capex elimination. Virtualization and increased automation may give them some improvements in utilization, but they would still be holding the capital, and the operational cost would still be significantly higher. . . .

What are called private [internal] clouds have little of these benefits and as such, I don't think of them as true clouds. . .

[Cloud benefits are]

* Eliminates Cost. The cloud changes capital expense to variable expense and lowers operating costs. The utility-based pricing model of the cloud combined with its on-demand access to resources eliminates the needs for capital investments in IT Infrastructure. And because resources can be released when no longer needed, effective utilization rises dramatically and our customers see a significant reduction in operational costs.

* Is Elastic. The ready access to vast cloud resources eliminates the need for complex procurement cycles, improving the time-to-market for its users. Many organizations have deployment cycles that are counted in weeks or months, while cloud resources such as Amazon EC2 only take minutes to deploy. The scalability of the cloud no longer forces designers and architects to think in resource-constrained ways and they can now pursue opportunities without having to worry how to grow their infrastructure if their product becomes successful.

* Removes Undifferentiated "Heavy Lifting."The cloud let its users focus on delivering differentiating business value instead of wasting valuable resources on the undifferentiated heavy lifting that makes up most of IT infrastructure. Over time Amazon has invested over $2B in developing technologies that could deliver security, reliability and performance at tremendous scale and at low cost. Our teams have created a culture of operational excellence that power some of the world's largest distributed systems. All of this expertise is instantly available to customers through the AWS services.

Elasticity is one of the fundamental properties of the cloud that drives many of its benefits. While virtualization has tremendous benefits to the enterprise, certainly as an important tool in server consolidation, it by itself is not sufficient to give the benefits of the cloud. To achieve true cloud-like elasticity in a private cloud, such that you can rapidly scale up and down in your own datacenter, will require you to allocate significant hardware capacity. While to your internal customers it may appear that they have increased efficiency, at the company level you still own all the capital expense of the IT infrastructure. Without the diversity and heterogeneity of the large number of AWS cloud customers to drive a high utilization level, it can never be a cost-effective solution.

OK. Let's examine Werner's sales proposition without the pressure to sell anything (as I am not currently trying to sell anyone anything). Clearly, Amazon is now attacking the vendors such as VMware that seem intent on attacking them by proclaiming that Amazon cannot give you enterprise features. Not only is Amazon delivering features targeted at the enterprise, but they are also scaling up the war of words by poo pooing the value proposition of these classic vendors – namely the notion of an internal cloud. Werner makes two assertions in dissing internal clouds:

First, he asserts that an internal cloud is not elastic. Well, why not? Just because your IT department has historically been labeled the NO department doesn't mean that it always must be that way. Indeed, the very pressure of Amazon providing the terrific services they provide without the mind-numbing procurement and deployment friction of your IT department is going to lead to massive changes on the part of IT. They are going to virtualize, provide self provisioning tools, and more closely align business application chargebacks to actual application usage. If the application owners are thoughtful about their architecture, they will be able to scale up and scale back based upon the realities of demand, and their IT transfer costs will reflect their thoughtfulness. Other business units will benefit from the release of resources, and server hoarding will be a thing of the past. All this is not to say that an IT department should “own” every bit of compute capacity they use. They don't. They won't. And there will probably be an increasing shift toward owning less.

But Werner claims that ownership is generally a bad thing in his second assertion that capex is bad and opex is good. Werner writes that cloud eliminates costs by eliminating capital spending. Well, it might - depending on the scenario. But his insinuation that capex is bad and opex is good is silliness. They are simply different, and the measurement that any enterprise must take is one relating to risk of demand and cost of capital. For a capital constrained startup with high risk associated with application demand, laying out precious capital for a high demand scenario in the face of potential demand failure makes no sense at all. However, for a cash rich bank with years of operating history relative to the transaction processing needs associated with servicing customer accounts, transferring this burden from capital expense to operating expense is equally senseless. Paying a premium for Amazon's gross profit margin when demand is fairly deterministic and your cost of capital is low is certainly a losing proposition.

The challenge and the opportunity of cloud for any enterprise is moving applications to an architecture that can exercise the cloud option for managing demand risk while simultaneously striking the right balance between capex and opex relative to the cost of capital. I find it funny that Amazon's new VPN feature is designed to make this opportunity a reality, while the blog post of their CTO announcing the feature proclaims that internal operations are too costly. Maybe they are viewing the VPN as a temporary bridge that will be burned when capex to opex nirvana is attained. Personally, I see it as the first of many permanent linkages that will be built to exercise the cloud option for managing demand risk. Lower costs associated with a proper portfolio balance of capex and opex is just icing on the cake.

VMware Springs Big for SpringSource

2009-08-24T22:25:00.001-04:00

In a blog post back in May, I described why I believed a SpringSource and Hyperic combination was a good thing. In the new world of virtualized infrastructure and cloud computing, the application delivery and management approach is going to be lightweight and lean. At the time, however, I never imagined lightweight and lean would be worth $420M to VMware. While I have no doubt that a lightweight and agile approach to application delivery and management is going to replace the outdated heavy approach of J2EE and EJB, I am not quite convinced that VMware is getting in this deal what they want us to believe they are getting – general purpose operating system irrelevance.

VMware has done an incredible job abstracting the hardware away from the general purpose operating system. Now they have moved to the other end of the stack in an attempt to abstract the application away from the operating system. If the operating system is not responsible for hardware support and it is likewise not responsible for application support, then it is irrelevant, right? It is a good theory, but it is not quite true.

While the majority of application code will certainly be written in languages that can be supported by SpringSource (java, grails), there will remain lots and lots of application utilities and services that are provided by various programs that are not, and will never be, written in Java or the related languages supported by SpringSource. All of these various programs will still need to be assembled into the system images that represent a working application. And while I absolutely believe the general purpose operating system should die an ugly death in the face of virtualized infrastructure and cloud computing, I do not believe that operating systems can be rendered irrelevant to the application. I simply believe they become lighter and more application specific. I also believe that we are going to see a proliferation of application language approaches, not a consolidation to Java alone.

Acquiring SpringSource puts VMware on the path to providing not only Infrastructure as a Service technology, but also Platform as a Service technology. From what I have seen to date in the market, PaaS lags far, far behind IaaS in acceptance and growth. I have written multiple posts praising the Amazon approach and decrying the Google and Salesforce approach for cloud because the latter requires developers to conform to the preferences of the platform provider while the former allows developers to exercise creativity in the choice of languages, libraries, data structures, etc. That's not to say that PaaS cannot be a valuable part of the application developer toolkit. It's just that the market will be much more limited in size due to the limitations in the degrees of freedom that can be exercised. And if developers love one thing more than anything else, it is freedom.

VMware's acquisition of SpringSource moves them into the very unfamiliar territory of developer tools and runtimes. It is a different sale to a different audience. Developers are notoriously fickle, and it will be interesting to see how a famously insular company like VMware manages to maintain the developer momentum built by the SpringSource team.

Microsoft Embraces Linux Virtual Appliances

2009-07-22T10:05:00.002-04:00

In a very savvy move aimed at gaining competitive advantage in the cloud computing space, Microsoft yesterday announced that they were contributing source code to the Linux kernel in order to optimize the performance and management of virtual appliances on Microsoft's hypervisor, Hyper-V. One of the goals of cloud computing is elasticity – applications scale up and down based upon the hour by hour demand for the application. Well, you cannot have hour by hour elasticity for an application when it takes days/weeks/months to install, provision, and instrument the application onto the infrastructure. Virtual appliances eliminate this challenge by allowing the application owner to pre-configure the application as a set of virtual machines that are ready to respond to demand. The “set-up” is done “off-line.” Microsoft, realizing that Linux is the de facto underpinning for virtual appliances that run on Amazon's EC2, is now contributing code to Linux that will optimize the performance and management characteristics of Linux-based virtual appliances on Hyper-V – the virtualization technology that underpins Microsoft's Azure cloud.

Microsoft's new status as a Linux kernel contributor sheds light on the amazing shift that is occurring in the battle-lines for IT infrastructure. The operating system is going to split, with one half becoming the control software for the hardware (the hypervisor) and the other becoming the control software for the application (virtual appliances). Given their huge advantage with developers based upon the installed base of applications that run on Microsoft-only application frameworks (.Net, etc.), Microsoft has determined that they need to pull out all of the stops in order to be certain they do not get ripped off the hardware in favor of VMware (the dominant hypervisor) or Xen (the hypervisor that supports Amazon's market leading cloud service). Linux is no longer the biggest threat to Microsoft in the datacenter when datacenters begin embracing a cloud architecture such as Amazon's in order to enable IT-as-a-Service.

If indeed this move enables higher elasticity and simpler management of Linux based virtual appliances that run atop Hyper-V, the competitive pressure might force VMware to follow suite and make their drivers and tools available as source code that is included in the Linux kernel. To be clear, Microsoft does not currently plan to support Linux virtual appliances on Azure, but that position may be shifting with changes of this type. With Amazon currently holding the dominant position in cloud and with VMware holding the dominant position in the datacenter for virtualization, Microsoft might have lots of crafty tricks up its sleeve to re-assert themselves in this new theater of datacenter war where hypervisors and virtual appliances rule the day.

IBM Cloud Fizzles

2009-06-30T13:51:00.003-04:00

Based on my positive review below of IBM's CloudBurst technology for building internal clouds, I tuned into the IBM webinar for the external cloud companion product with high hopes. I was hoping to hear about a consistent architecture across the two products that would allow an enterprise to federate workloads seamlessly between the internal and external cloud. Boy, was I disappointed.

It seems the IBM external cloud is nothing more than an IBM hosted capability for running virtual appliances of IBM Rational Software products. Among my many disappointments:

- no ability to run virtual appliances defined by me. They don't even publish a specification.

- no federation between internal and external. They are not even the same architecture because one runs Xen and the other runs VMware, and they do not provide a conversion utility.

- private beta (alpha maybe?) for invited customers only. Why make an announcement?

- no timetable for general availability of a product. Why make an announcement?

This announcement was a terrible showing by IBM to say the least. It is obvious to me that the CloudBurst appliance folks (call them “left hand”) and the Smart Business cloud folks (call them “right hand”) were two totally different teams. And the left hand had no idea what the right hand was doing. But each was intent not to be outdone by the other in announcing “something” with cloud in the title. And they were told to “cooperate” by some well meaning marketing and PR person from corporate. And this mess of a situation is the outcome. Good grief!

IBM CloudBurst Hits the Mark

2009-06-29T11:35:00.002-04:00

IBM rolled out a new infrastructure offering called CloudBurst last week. Aimed at development and test workloads, it is essentially a rack of x86 systems pre-integrated with VMware’s virtualization technology along with IBM software technology for provisioning, management, metering, and chargeback. I believe IBM, unlike Verizon, has hit the cloud computing mark with this new offering.

First, IBM is targeting the offering at a perfect application workload for cloud – development and test. The transient nature of development and test workloads means that an elastic computing infrastructure with built-in virtualization and chargeback will be attractive to IT staff currently struggling to be responsive to line of business application owners. The line of business application owners are holding the threat of Amazon EC2 over the head of the IT staff if they cannot get their act together with frictionless, elastic compute services for their applications. By responding with a development and test infrastructure that enables self-service, elasticity, and pay-as-you-go chargeback capability, the IT staff will take a step in the right direction to head off the Amazon threat. Moving these dev/test workloads to production with the same infrastructure will be a simple flick of the switch when the line of business owners who have become spoiled by CloudBurst for dev/test complain that the production infrastructure is not flexible, responsive, or cost competitive.

Second, IBM embraced virtualization to enable greater self-service, and elasticity. While they do not detail the use of VMware’s technology on their website (likely to preserve the ability to switch it out for KVM or Xen at some future date), IBM has clearly taken an architectural hint from Amazon by building virtualization into the CloudBurst platform. Virtualization allows the owners of the application to put the infrastructure to work quickly via virtual appliances, instead of slogging through the tedious process of configuring some standard template from IT (which is never right) to meet the needs of their application – paying for infrastructure charges while they fight through incompatibilities, dependency resolution, and policy exception bureaucracy. CloudBurst represents a key shift in the way IT will buy server hardware in the future. Instead of either a bare-metal unit or pre-loaded with a bloated general purpose OS (see the complaint about tedious configuration above), the systems will instead come pre-configured with virtualization and self-service deployment capability for the application owners - a cloud-computing infrastructure appliance if you will. Cisco has designs on the same type capability with their newly announced Unified Computing System.

Third, it appears that IBM is going to announce a companion service to the CloudBurst internal capability tomorrow. From the little information that is available today, I surmise that IBM is likely going to provide a capability through their Rational product to enable application owners to “federate” the deployment of their applications across local and remote CloudBurst infrastructure. With this federated capability across local (fixed capital behind the firewall) and remote sites (variable cost operating expense from infrastructure hosted by IBM), the IBM story on cloud will be nearly complete.

The only real negatives I saw in this announcement were that IBM did not include an option for an object storage array for storing and cataloging the virtual appliances, nor did they include any utilities for taking advantage of existing catalogs of virtual appliances from VMware and Amazon. While it probably hurt IBM’s teeth to include VMware in the offering, perhaps they could have gone just a bit further and included another EMC cloud technology for the object store. Atmos would be a perfect complement to this well considered IBM cloud offering. And including a simple utility for accessing/converting existing virtual appliances really would not be that difficult. Maybe we’ll see these shortcomings addressed in the next version. All negatives aside, I think IBM made a good first showing with CloudBurst.

Verizon Misses with Cloud Offering

2009-06-18T10:43:00.004-04:00

About two weeks back, I was excited to see a headline about Verizon partnering with Red Hat to offer their customers a “new” cloud computing offering. I was hopeful that the details would reveal a KVM hypervisor based elastic compute capability coupled with an OVF based specification for virtual appliances to run on the service. I was also hoping to discover some details on storage as a service, with all of the services accessible via a management capability exposed via RESTful APIs. Boy, was I disappointed. Turns out the new Verizon cloud offering is just the old Verizon hosting offering with a new name.

Why is it so difficult for all of these old school infrastructure providers to understand the new path being blazed by Amazon AWS? Why can't they offer even a reasonable facsimile of the capability provided by Amazon? Surely it is the threat of Amazon that is leading them to re-name the old hosting stuff as the new cloud stuff. Why not go all the way and actually offer something that is competitive? Here is a recipe for any that are interested:

First, provide a X86 hypervisor based, virtualized compute service that allows the customer to bring their applications with them as pre-packaged, pre-configured virtual machines (virtual appliances). Don't ask them to boot a “standard OS” and then spend hours, days, weeks, months configuring it to work for them (because what you specified as the “standard” is certainly not what they have tested with their applications, and the whole purpose of elasticity is defeated if you can't quickly put images to work on the network in response to application demand). Better yet, let them boot existing Amazon Machine Images and VMware virtual appliances. Providing this capability is not rocket science. It is just work.

Second, provide a simple storage service (see Amazon S3 for what it should do) for storing unstructured data as well as for storing their virtual appliances that boot on the virtualized, elastic compute service. If you don't want to take the time to develop your own, follow AT&T's lead and go buy the capability EMC offers as part of the Atmos product line. You don't even have to think, you just need to write a check and viola – an Amazon S3 type capability running on your network. What could be easier?

Third, provide a block storage capability for attaching to virtual appliance images that must store state, such as database images. Most of the hosting companies already provide this type of SAN offering, so this part should be a no-brainer. Just price it with a very fine grained, variable cost approach (think megabyte-days, not months).

Fourth, provide access to the infrastructure management services via simple, RESTful APIs. You don't have to go overboard with capability at first, just make certain the basics are available in a manner that allows the services to be run effectively over the Internet without any funky protocols that are specific to your network implementation.

Finally, go sign up partners like rPath and RightScale to offer the next level of manageability and support for the virtual machines that will run on the network. These are the final touches that indicate to your customers that you are serious about providing a terrific capability for the complete lifecycle of your cloud computing offering. Instead of asking them to be patient with you while you re-name your hosting offering as a cloud offering in the hopes that it will assuage their bitterness that Amazon-like capability is not available on your network.

Federation - The Enterprise Cloud Objective

2009-06-02T15:34:00.003-04:00

I know the title to this blog post sounds a bit like a Star Trek episode, but I believe I have an useful point to make with the term federation - even at the risk of sounding a bit corny. I have been watching with interest the lexicon of terms that are emerging to describe the architecture and value of cloud computing. VMware uses the terms Internal/External/Private to describe the distribution of application workloads across multiple networks in a coordinated fashion. Sun uses the terms Private/Public/Hybrid, respectively, to describe the same architecture (although they would argue for Sun branded components in lieu of Vmware/EMC branded components). I think both of these term sets as descriptors for a cloud architecture that distributes workloads across multiple networks are flawed and confusing. Rather than simply complaining, however, I am willing to offer a solution.

The term Federation describes the end state of an effective cloud architecture perfectly, and I think we should all begin using it when we attempt to sell our respective goods and services to enable the enterprise cloud. Whether part of a Internal/External/Federation combination or a Private/Public/Federation combination or Network1/Network2/Networkn/Federation, the common term accurately describes the end objective of cloud computing.

First, some attribution. This term was presented to me as a descriptor for cloud value during my work with the cloud infrastructure group at EMC (the folks that own the Atmos product line) over a year ago. It is now my turn to put some greater structure on this enviable original thought that belongs to EMC.

A good general definition for Federation (independent of an IT context) is a union of member entities that preserves the integrity of the policies of the individual members. Members get the benefits of the union while retaining control over their internal affairs.

In the case of a technology infrastructure federation (aka a cloud architecture), the primary benefit of the union is the lower cost and risk associated with a pool of technology assets which are available across a diversified set of independent networks. In other words, application workloads should be distributed to the network with the lowest risk adjusted cost of execution – i.e. based upon the risk policies of the enterprise. If the risk of running a mission critical, enterprise workload on Amazon's AWS network is deemed high (for whatever reason, real or perceived), that workload might stay on a proprietary network owned by the enterprise. Likewise, a low risk workload that is constantly being deferred due to capacity or complexity constraints on the enterprise network might in fact be run for the lowest cost at Amazon or a comparable provider. For a startup, the risk of depleting capital to purchase equipment may dictate that all workloads run on a third party network that offers a variable cost model for infrastructure (Infrastructure as a Service, IaaS).

Independent of the proprietary calculus for risk that must be undertaken by every enterprise relative to their unique situation, it should become clear to all that the distribution of application workloads across multiple networks based upon the cost/capability metrics of those networks will lower the risk adjusted cost of enterprise computing. The same diversification theories that apply to managing financial portfolio risk also apply to managing the distributed execution of application workloads. The historical challenge to this notion of application workload federation is the lack of an efficient market – the transaction cost associated with obtaining capacity for any given application on any given network were too high due to complexity and lack of standards for application packaging (de facto or otherwise). Now, with virtualization as the underpinning of the network market, virtual appliances as the packaging for workloads, high bandwidth network transit and webscale APIs for data placement/access, the time is coming for an efficient market where infrastructure capacity is available to applications across multiple networks. And Federation is the perfect word to describe a cloud architecture that lowers the risk adjusted cost of computing to the enterprise. Enterprise. Federation. Clouds. Star Trek.

Oracle Lowers Expectations

2009-05-15T10:21:00.005-04:00

I was floored when Oracle announced the acquisition of Sun after Sun's deal with IBM fell apart. I never saw Oracle buying Sun in a million years – too much reliance on hardware revenue. Then, Oracle announced this week that they intend to purchase Virtual Iron Software. It seems that Oracle is serious about lowering their influence in the software technology ecosystem. They are going after the lower layers of the infrastructure, and they are putting their money where their mouth is in order to put some real assets into the battle for the next generation, virtualized datacenter.

Sun provides some terrific infrastructure assets in the form of the Solaris operating system, the ZFS file system, the Java programming language, and several other lesser known projects that are nonetheless useful technology in assembling a world class datacenter infrastructure. I actually believe that Solaris is not so relevant as an operating system (too difficult to do the driver work for all the variations in X86 hardware) as it will be relevant as a collection of useful system software and engineering expertise for delivering innovation and support. If you have not seen Sun's investment in this area, do some research on the Image Packaging System. It is Sun's implementation of the rPath approach for tailoring the operating system to the needs of an application (JeOS) and providing robust lifecycle management – they even wrote it in python, not java, to make it easier to mimic rPath's features.

Virtualization is going to change the notion of the operating system, and Solaris will get torn apart into a series of useful components and support libraries that will be attached as JeOS to various applications that will run on a cloud of virtualized infrastructure (witness Amazon EC2). The lines between Linux, Solaris, and other open system components will blur. This outcome is a good one for Oracle because it is very disruptive to the existing providers of one size fits all, general purpose operating systems. Especially if Oracle follows through on their commitment to virtualization as evidenced by their acquisition of Virtual Iron.

Virtual Iron was an early entrant into the hypervisor space – behind VMware but ahead of XenSource. They never really got out of the gate from a marketing perspective, but they always had some useful technology for managing virtual infrastructure. If Oracle makes a strong commitment to Xen as well as the management interface for controlling virtual infrastructure, they could definitely emerge as the strong contender to take on VMware in this market. I do not believe Citrix has a strong commitment to the datacenter infrastructure market (they prefer the desktop with a strong Microsoft alliance), Red Hat is far behind in market adoption with their late commitment to KVM in lieu of Xen, and Microsoft has so much anxiety over Google that worrying about VMware is likely lower on the list of priorities.

The race to the bottom of the software stack just became more interesting yet again. With Oracle continuing to raise the bar with lower expectations, it will be fun to watch the feeding frenzy for the assets that will win the hearts and minds of those datacenter customers that are certain to embrace virtualization and cloud as the new architecture for scalable, elastic computing.

Cloud Application Management - Agile, Lean, Lightweight

2009-05-06T22:32:00.003-04:00

The acquisition of Hyperic by SpringSource got me thinking about the next generation of application delivery and management for cloud applications. At first, I was cynical about this combination – two small companies with common investors combining resources to soldier on in a tough capital environment. While this cynical thinking probably has a kernel of truth to it, the more I thought about the combination the more I thought that it makes sense beyond the balance sheet implications. Indeed, I believe the future of application delivery and management will combine agile development with lean resource allocation and lightweight management. This new approach to application delivery and management is one that complements the emerging cloud architecture for infrastructure.

Agile development, with its focus on rapid releases of new application functionality, requires a programming approach that is not overly burdened with the structure of J2EE and EJB. Spring, Rails, Grails, Groovy, Python all represent the new approach – placing a premium on quick delivery of new application functionality. Application functionality takes center stage, displacing the IT infrastructure dominance of the legacy application server oriented approach. Developers will use what works to deliver the application functionality instead of using what works for the IT organization's management framework. The new approach does have implications for scalability, but we will get to that issue in a moment.

Lean is one of the newer terms emerging to describe the future of application delivery. I first referenced lean as an IT concept by relating it to the lean approach for manufacturing operations in a blog post about a year ago. With lean application delivery, applications scale horizontally to consume the infrastructure resources that they require based upon the actual demand that they are experiencing. The corollary is that they also contract to release resources to other applications as demand subsides. This “lean” approach to resource allocation with dynamic scaling and de-scaling is what a cloud architecture is all about – elasticity. Rather than optimizing the code to “scale up” on an ever bigger host, the code remains un-optimized but simple – scaling out with cheap, variable cost compute cycles when the peaks in demand require more capacity. Giving back the capacity when the peaks subside.

With the lean approach for resource allocation, a lightweight management approach that measures only a few things replaces the old frameworks that attempt to measure and optimize every layer in an ever more complex infrastructure stack. If the service is under stress due to demand, add more instances until the stress level subsides. If the service is under extremely light load, eliminate resources until a more economical balance is struck between supply and demand. If an instance of a service disappears, start a new one. In most cases, you don't even bother figuring out what went wrong. It costs too much to know everything. This lightweight approach for management makes sense when you have architected your applications and data to be loosely coupled to the physical infrastructure. Managing application availability is dramatically simplified. Managing the physical hosts becomes a separate matter, unrelated to the applications, and is handled by the emerging datacenter OS as described by VMware or the cloud provider in the case of services like those provided by Amazon AWS.

Take a look at the rPath video on this topic. I think it reinforces the logic behind the SpringSource and Hyperic combination. It rings true regarding the new approach that will be taken for rapid application delivery and management in a cloud infrastructure environment. Applications and data will be loosely coupled to the underlying infrastructure, and agile development, lean resource allocation, and lightweight management will emerge as the preferred approach for application delivery and management.

McKinsey Recommends Virtualization as first step to Cloud

2009-04-20T16:49:00.003-04:00

In a study released last week, the storied consulting company, McKinsey & Company, suggested that moving datacenter applications wholesale to the cloud probably doesn't make sense – it's too expensive to re-configure and the cloud is no bargain if simply substituted for equipment procurement and maintenance costs. I think this conclusion is obvious. They go on to suggest that companies adopt virtualization technology in order to improve the utilization of datacenter servers from the current miserable average of ten percent (10%). I think this is obvious too. The leap that they hesitated to make explicitly, but which was called out tacitly in the slides, was that perhaps virtualization offers the first step to cloud computing, and a blend of internal plus external resources probably offers the best value to the enterprise. In other words cloud should not be viewed as an IT alternative, but instead it should be considered as an emerging IT architecture.

With virtualization as an underpinning, not only do enterprises get the benefit of increased asset utilization on their captive equipment, they also take the first step toward cloud by defining their applications independent from their physical infrastructure (virtual appliances for lack of a better term). The applications are then portable to cloud offerings such as Amazon's EC2, which is based on virtual infrastructure (the Xen hypervisor). In this scenario, cloud is not an alternative to IT. Instead, cloud is an architecture that should be embraced by IT to maximize financial and functional capability while simultaneously preserving corporate policies for managing information technology risk.

Virtualization as a step to cloud computing should also be viewed in the context of data, not simply application and server host resources. Not only do applications need compute capacity, they also need access to the data that defines the relationship of the application to the user. In addition to technology such as VMware and Citrix's Xen technology, enterprises also need to consider how they are going to abstract their data from the native protocols of their preferred storage and networking equipment.

For static data, I think this abstraction will take the form of storage and related services with RESTful interfaces that enable web-scale availability to the data objects instead of local network availability associated with file system interfaces like NFS. With RESTful interfaces, objects become abstracted from any particular network resource, making them available to the network where they are needed. Structured data (frequently updated information typically managed by a database server technology) is a bit trickier, and I believe solving the problem of web-scale availability of structured data will represent the “last mile” of cloud evolution. It will often be the case that the requirement for structured data sharing among applications will be the ultimate arbiter of whether an application moves to the cloud or remains on an internal network.

The company that I founded, rPath, has been talking about the virtualization path to cloud computing for the past three years. Cloud is an architecture for more flexible consumption of computing resources – independent of whether they are captive equipment or offered by a service provider for a variable consumption charge. About nine months ago, rPath published the Cloud Computing Adoption Model that defined this approach in detail with a corresponding webinar to offer color commentary. In the late fall of last year, rPath published a humorous video cartoon that likewise offered some color on this approach to cloud computing. With McKinsey chiming in with a similar message, albeit incomplete, I am hopeful that the market is maturing to the point where cloud becomes more than a controversial sound-byte for replacing the IT function and instead evolves into an architecture that provides everyone more value from IT.

Outsourcing gives way to Now-sourcing via Cloud Technology

2009-04-13T19:18:00.003-04:00

The theory behind the value of outsourcing, aside from labor arbitrage, was that the outsourcer could deliver IT resources to the business units in a more cost effective manner than the internal IT staff due to a more highly optimized resource management system. The big problem with outsourcing, however, was the enormous hurdle the IT organization faced in transitioning to the “optimized” management approach of the outsourcer. In many cases this expensive hurdle had to be crossed twice – once when the applications were “outsourced” and then again when the applications were subsequently “in-sourced” after the outsourcer failed to live up to service level expectations set during the sales pitch. Fortunately, the new architecture of cloud computing enables outsourcing to be replaced with “now sourcing” by eliminating the barriers to application delivery on third party networks.

The key to “now sourcing” is the ability to de-couple applications and data from the underlying system definitions of the internal network while simultaneously adopting a management approach that is lightweight and fault tolerant. Historically, applications were expensive to “outsource” because they were tightly coupled to the underlying systems and data of the internal network. The management systems also pre-supposed deep access to the lowest level of system structure on the network in order to hasten recovery from system faults. The internal IT staff had “preferences” for hardware, operating systems, application servers, storage arrays, etc., as did the outsourcer. And they were inevitably miles apart in both the brands and structure not to mention differences in versions, release levels, and the management system itself. Even with protocols that should be a “standard,” each implementation still had peculiarities based upon vendor and release level. NFS is a great example. Sun's implementation of NFS on Solaris was different than NetApp's implementation on their filers, leading to expensive testing and porting cycles in order to attain the benefits of “outsourcing.”

I believe a by-product of the “cloud” craze will be new technology, protocols, and standards that are designed from the beginning to enable applications to run across multiple networks with a much simpler management approach. A great example is server virtualization coupled with application delivery standards like OVF. With X86 as a de facto machine standard and virtualization as implemented by hypervisor technology like Xen and VMware, applications can be “now sourced” to providers like Amazon and RackSpace with very little cost associated with the “migration.”

Some will argue that we are simply trading one protocol trap for another. For example, Amazon does not implement Xen with OVF in mind as an application delivery standard. Similarly, VMware has special kernel requirements for the virtual machines defined within OVF in order to validate your support agreement. Amazon's S3 cloud storage protocol is different than a similar REST protocol associated with EMC's new Atmos cloud storage platform. And the list of “exceptions” goes on and on.

Even in the face of these obvious market splinters, I still believe we are heading to a better place. I am optimistic because all of these protocols and emerging standards are sufficiently abstracted from the hardware that translations can be done on the fly – as with translations between Amazon's S3 and EMC's Atmos. Or the penalty of non-conformance is so trivial it can be ignored – as with VMware's kernel support requirements which do not impact actual run-time performance.

The other requirement for “now sourcing” that I mentioned above was a fault tolerant, lightweight approach to application management. The system administrators need to be able to deliver and manage the applications without getting into the low level guts of the systems themselves. As with any “new” approach that requires even the slightest amount of “change” or re-factoring, this requirement to re-think the packaging and management of the applications will initially be an excuse for the IT staff to “do nothing.” In the face of so many competing priorities, even subtle application packaging and management changes become the last item on the ever lengthening IT “to do” list – even when the longer term savings are significant. But, since “now sourcing” is clearly more palatable to IT than “outsourcing” (and more effective too), perhaps there is some hope that these new cloud architectures will find a home inside the IT department sooner rather than later.

Maintenance Woes Handicap Microsoft's Azure

2009-03-30T16:42:00.002-04:00

Last week, Microsoft product manager Steve Martin indicated in his blog that Azure would be a Microsoft-only service – hosted only by Microsoft in Microsoft data-centers. It seems that maintenance challenges, or the ability to “innovate freely,” are limiting the availability of Azure to a pure service play. To be fair, Microsoft has indicated that key technology developed to support Azure will find its way into Microsoft Windows server products, which, of course, can be purchased and deployed by anyone. But I have serious doubts about Microsoft as a service provider because I do not think they have the operational mentality to succeed on the margins that are available to service providers – they are not cheap and efficient like Amazon. Microsoft is already demonstrating that the ties to their past technology architecture are already handicapping the potential success of their future cloud endeavors.

Anyone that pays attention to my musings in this blog knows that I am a huge fan of Amazon's web services – particularly the elastic compute cloud (EC2). And I believe Amazon is going to be very successful in this space because they have led the way with an architecture that effectively de-couples the definition of the application from the definition of the compute resources. Plus, they have an operational mentality – they are cheap and efficient. Aside from Amazon, I believe the really big winners are going to be the technology companies that enable existing service providers to respond to Amazon with a technology model that loosely couples applications to the infrastructure - eliminating the complex maintenance challenges that have historically precluded elasticity and enabling seamless application delivery across public and private clouds. Here are some of the potential players and my handicap for their success.

VMware – I believe virtualization technology is going to play a critical role in cloud due to its ability to de-couple the applications from the host computers – enabling elegant maintenance and true elasticity. VMware is certainly the leader in this space, and there is little doubt that they have relevant technology. The key question is going to be their willingness to respond to the “cheapness” and “openness” requirement of the service providers. VMware has historically sold technology to enterprises in a typical perpetual software licensing model. Cloud will require much more sharing of risk/reward in the sales model and a willingness to be flexible in the technology requirements to co-innovate with the service providers and ecosystem partners. VMware is not known for its partnering mentality.

Citrix – Xen is currently the incumbent technology for virtualization at Amazon, and Citrix is clearly aligned with Xen due to their $500M purchase of XenSource. However, cloud to date seems to be all about Linux and related open source infrastructure due to the flexibility of the licensing model, and Citrix has historically been very tightly aligned to Microsoft and its associated technology and licensing model. I think it is an open question whether Citrix is willing to embrace a radically different business model to monetize Xen as part of a service provider cloud ecosystem.

Cisco – the announcement earlier this month about the Cisco unified computing system is all about a new brand of infrastructure that is unencumbered by the legacy approach which tightly couples applications to the compute host resources. I believe Cisco “gets it” much more so than the current mega-vendors HP and IBM – each of which is going to be hampered with the legacy approaches for application delivery and management. But, as is always the case with new ventures like Ciscos unified computing system, the sky is always bluest when there is nothing yet to lose. We'll see if they can actually execute in the market with a new model.

Microsoft – While Microsoft certainly has many strengths, I do not think any of them lend themselves particularly well to cloud success. Of all vendors, they have the most to lose, and the least to gain, with this new approach. Their sales and distribution model discourages elasticity, they are way behind the leaders in the market for virtualization technology, and they do not have a mindset to be operationally efficient as a service provider. I believe their biggest opportunity in cloud is to provide a new desktop approach that seamlessly integrates all of the services a user would expect to be effective and productive in a networked world. If Apple doesn't continue to run away with that opportunity, maybe Microsoft will make some hay in this space.

Google – I have criticized Google again and again for their AppEngine model that essentially requires application providers to recode their applications to fit Google's infrastructure. I actually do not think Google wants to be a big cloud player for “infrastructure” services in the manner that Amazon is currently defining the market. Instead I think Google is going to be a next generation application platform provider more in the mold of Apple with its consumer products, but geared more to the needs of the business user. As a giant SaaS platform for applications like email, office productivity, VOIP, calendaring, geographical services, etc., I think Google has a great play with their current approach. Just don't expect them to compete with Amazon in the infrastructure space.

Salesforce.com – I think Salesforce.com is completely out in left field for everything other than applications that need to reside close to their CRM application. I personally feel that Marc should just go ahead and take on SAP and Oracle in the business applications space and forget about trying to morph their highly proprietary application platform into a general purpose application delivery platform. They have a terrific sales team, a terrific customer base, and the competing applications from Oracle, SAP, and others are just awful in terms of the grief customers endure to maintain and support them.

EMC – Some folks may be a bit puzzled by the inclusion of EMC in this list of potential cloud players, but remember that EMC owns nearly 90% of VMware which means that cloud is definitely top of mind at EMC. And they actually have been thinking about the data problem associated with cloud. Namely, how do I ensure my data is available to my applications when they become portable across multiple compute service providers? Their Atmos product is effectively Amazon S3 in a box with some interesting features around policy management and data federation. As an equity play, EMC may be a cheap way to own VMW, and they might make some hay themselves if they execute in the cloud market ahead of the other storage providers.

The rest of the A-list technology providers – HP, IBM, Oracle, Red Hat, Sun – either do not get cloud, are actively bashing it (witness Oracle), have no assets that are currently relevant, or are so far behind in investing that, short of a series of acquisitions, I believe they will remain effectively irrelevant. Of course there will be lots of lip service and cute marketing tricks such as IBM's recent open cloud manifesto, but until there is some substance, I stand by the above handicaps.

Killing the OS Octopus

2009-03-16T11:18:00.004-04:00

The inspiration for this blog post title comes from a Paul Maritz (CEO of VMware) quote, during his presentation to financial analysts last week. Paul used the phrase "severing the tentacles of complexity" multiple times when referring to the new level of business flexibility that is possible when applications are liberated from their physical host by encapsulating them inside of a virtual machine with “just enough operating system (or JeOS).” They can be provisioned much more quickly because there is no need to provision physical assets. They can be moved from datacenter to datacenter more quickly because there is no onerous installation and validation process required. Indeed, virtualization enables cloud computing because the applications are no longer defined by the physical computers upon which they run. But, until VMware truly embraces a JeOS approach with their operating system support matrix, they are simply recommending “isolating” the tentacles of complexity. And the result will be a perilous and expensive condition often referred to as VM sprawl.

So what is the difference between “isolating” the tentacles of complexity and “severing” them? Isolating the tentacles means shoving the previous definition of your application running on a physical server into a virtual machine box. When you put a virtual machine box around the octopus, it can no longer create mischief with other application octopi running on the same physical host. Its tentacles are “isolated,” and utilization on the physical host can be much higher. This approach is valuable, and it has catapulted VMware into the spotlight as one of the hottest technology companies on the planet.

However, the octopus is still alive and well inside the box, and system administrators must continue to feed that hungry animal in exactly the same way they did when it was living on a physical server host. The level of maintenance has not been reduced. The level of security vulnerability has not been reduced. Although isolated, it is still a resource hog because those crazy tentacles demand CPU, and memory, and disk to flail and flap as they do. This condition of ever expanding system administration grief associated with the frictionless deployment of virtualized applications whose tentacles of complexity have simply been isolated and not severed is known as VM sprawl. And it will be a nightmare of system administration expense for those that embrace it.

In order to avoid the nightmare of VM sprawl, the tentacles of the complexity octopus must actually be severed, not simply isolated. Application developers and system administrators alike must re-think the category of the operating system in the context of a virtualized datacenter. Since the operating system is no longer the conduit for managing the hardware, it should become a simple shared library for system services required by the application. Two great example of this approach are rPath and BEA's (now Oracle) liquid VM technology. With both of these platforms, the operating system is specified in a manner that explicitly supports the needs of the application – without any extra bloat associated with the typical general purpose OS approach. As a result, the OS in both of these cases is 10X or more smaller than the smallest installation option offered by a general purpose OS. In theory, this should lead to a 10X reduction in the scope and scale of administration activities. Severing the tentacles of complexity by re-thinking the OS eliminates the perils of VM sprawl.

But VMware does not currently support this approach. They only support the legacy vendors of general purpose OS technology. Sure, these new approaches have terrific performance and value, and VMware is happy to have them contribute to the value of their virtual appliance market, but their support statement pretty clearly favors “isolation” of complexity over true elimination of complexity. But the winds of change are steadily and surely blowing in favor of this new approach in the market. Red Hat, for example, just announced that they are going to market with a bare metal hypervisor that is directly competitive with the VMware approach in lieu of their historical product architecture - where virtualization was simply a feature of the general purpose operating system. And Paul Maritz was pretty clear in his presentation that “severing the tentacles of complexity” and a “just enough operating system” approach are important to VMware. Perhaps we are drifting toward the precipice of an all out war for the definition of the future datacenter operating system. I said it back in 2006, and I'll say it again today -- let's fry up that OS octopus polvo frito and serve it with some spicy mango chutney and cold beer.

Will Agile drive a Hybrid Cloud Approach?

2009-03-04T10:10:00.002-05:00

Some workloads are perfectly suited for cloud deployment. Generally, these are workloads with transient or fluctuating demand, relatively static data (lots of reads, few writes), and no regulated data compliance issues (i.e. patient healthcare records). Test fits this description perfectly – especially with the growing popularity of Agile methods. With its focus on rapid iteration and feedback to achieve faster innovation and lower costs, Agile demands a flexible and low cost approach for testing cycles. I have no doubt that developers will begin using variable-cost compute cycles from services like Amazon EC2 because of its flexibility and pay-for-what-you-use capability. But I am also willing to bet that testing with Amazon will put further pressure on the IT organization to respond with a similar, self-service IT capability. I think a hybrid-cloud architecture with complementary internal and external capability will emerge as a productive response to the demand for true end-to-end agility.

Some time ago, I authored a blog post titled “When Agile Becomes Fragile” that outlined the challenge of implementing Agile development methods while attempting to preserve the legacy IT approach. What good is rapid development when the process for promoting an application to production takes several months to absorb even a few weeks of new development? If developers take their Agile methods to the cloud for testing (which they will), it becomes a slippery slope that ultimately leads to using the cloud for production. Rather than the typical, dysfunctional IT response of “don't do that – it's against policy,” I think the IT organization should instead consider implementing production capacity that mimics and complements cloud capability such as that offered by Amazon.

Along with all of the cool technology that is emerging to support Agile methods, new technology and standards are also emerging to support the notion of a hybrid-cloud. The new Atmos storage technology from EMC and the OVF standard for virtualizing applications are two good examples of hybrid-cloud technology. Atmos gives you the ability to describe your data in a manner that automatically promotes/replicates it to the cloud if it has been approved for cloud storage/availability. Whether applications run on an external cloud or on your “internal cloud,” the supporting data will be available. Similarly, OVF has the potential to enable virtualized applications to run effectively externally on the cloud or internally – without significant manual (and error prone) intervention by system administrators (or those developers that play a sysadmin on a TV show). In both cases, the goal is to enable greater flexibility for applications to run both internally and on the cloud – depending on the profile of the application and the availability of resources.

Agile is yet another important technology change that is going to pressure IT to evolve, and rPath is sponsoring a webinar series that dives into this topic in some detail. Whether you are a developer, an architect, or a system administrator, these webinars should be interesting to you. For the IT staff, the series may offer a glimpse at an approach for IT evolution that is helpful. In the face of Agile and cloud pressure, the alternative to evolution – extinction – is much less appealing.

Red Hat Goes Streaking - Dumps Xen

2009-02-24T13:26:00.004-05:00

Yesterday Red Hat announced their revised virtualization strategy. Most interesting to me was Red Hat's declaration that the hypervisor should indeed lie "naked" on the metal. Red Hat also announced end-of-life for Xen support and some stuff regarding desktop virtualization and protocols that I don't pretend to understand. While it was expected that Red Hat would end-of-life support for Xen after their acquisition of the KVM technology from Qumranet, the fact that Red Hat went streaking into the market with a bare metal approach to the hypervisor was a pretty significant strategy reversal.

Until yesterday, Red Hat had always adopted the Microsoft line on the hypervisor - it is simply a feature of a general purpose operating system. Red Hat historically claimed that the hypervisor should not lie naked on the bare metal, but instead it should be wrapped up inside the general purpose OS - a little extra bloating that never hurt anybody. It looks like some combination of market forces - likely VMware financial success and Amazon's adoption of Xen for their Elastic Compute Cloud - have forced Red Hat to consider a new approach. Now Microsoft stands alone in their contention that a hypervisor is just a new general purpose OS feature, and the rest of the market can move on to the market share land grab for the large-scale Linux datacenters. It should be an interesting race, because all three players - Xen (Citrix), KVM (Red Hat), and VI (VMware) are all coming from different positions of strength - and weakness.

In almost every case I have observed, Xen is the incumbent virtualization technology among the Linux datacenter consumers that have virtualized any production workloads. However, not many have actually virtualized because the historical compelling case for virtualization - server consolidation - falls on deaf ears among the Linux crowd. With Linux, as opposed to Windows, it is possible to run multiple application workloads on a single server without significant instability. Server utilization can be quite high without virtualization - hence no requirement for Linux server consolidation. But, the new benefits of virtualization - flexibility, security, and elasticity - apply equally, if not especially, well to the Linux workloads. If application workloads are separated into unique, small footprint, virtual machines (virtual appliances, if you like) that run atop a bare naked hypervisor, then:

- flexibility is better because one application can be managed/administered without interfering with another workload running on the same host

- security is better because vulnerable services like DNS are isolated and easier to maintain/secure due to a smaller footprint

- elasticity is better because application workloads can be scaled quickly when demand increases and also retired quickly when demand recedes (cloud, anyone?)

Because of these new business benefits, Xen is beginning to take hold in the Linux datacenters. Citrix, however, has not historically been a strong brand among the Linux savvy crowd, so it is unclear if they have the stomach for pursuing a new market segment. VMware is the 800 pound gorilla in hypervisors generally, but they too have shown very little Linux savvy (their management console only runs on Windows) - probably because 90% of their revenues come from virtualizing Windows workloads. Then there is Red Hat, the 800 pound Linux gorilla who has been in denial about the importance of a bare metal approach for the hypervisor - until now.

Based on this set of circumstances, I would say that the virtualization opportunity for the Linux datacenter market is a wide open race. Now that Red Hat has decided to run the race naked, it should be more fun to watch.

Is the Cloud Game Already Over?

2009-01-25T20:43:00.004-05:00

This is the thought that crossed my mind a few weeks back as I pondered Amazon's beta release of the Amazon Web Services Console. The reason the game might be over is because Amazon is apparently so far ahead of the competition that they can now divert their engineering attention to the management console instead of core platform functionality. To me, this signals a competitive lead so vast that absent quick and significant re-direction of resources and potential strategic acquisitions of capability, Amazon's competitors are doomed in the cloud space.

I saw this dynamic once before during my time at Red Hat. Red Hat had such a lead in the market with almost total mind share for the platform (Red Hat Linux, now Red Hat Enterprise Linux), that the company could launch a strategic management technology, Red Hat Network, while others were grasping for relevance on the core platform. Note that in the case of Red Hat, no one else has come close to their lead in the Linux market space. And no one else has really gotten around to building out the management technology that was offered by Red Hat Network 8 years ago.

Consider these other challenges facing Amazon's competitors:

1. Lack of machine image definitions - Amazon published the AMI spec for EC2 about 2 years ago. To my knowledge, all of the competitors that use virtualization (Amazon uses Xen) are still requiring customers to boot a limited set of approved "templates" which must then be configured manually, and subsequently lose their state when retired.

2. Proprietary versus open - when you require the customer to program in a specific language environment that is somewhat unique to a particular "cloud" platform (ala Google with Python and Salesforce with Apex), you dramatically limit your market to virtual irrelevance out of the gate. Amazon doesn't care, so long as you can build to an X86 virtual machine.

3. Elastic billing model - until you have a platform for billing based upon the on-demand usage of resources, you don't have a cloud with the key value proposition of elasticity. You simply have hosting. To my knowledge, most competitors are still on a monthly payment requirement. Hourly is still a long way away for these folks.

Perhaps I am wrong, but I bet I am not. If I am right, the day will come in the not too distant future (after the equity markets recover) when Amazon spins out AWS as a tracking stock (similar to the EMC strategy with VMware) with a monster valuation (keeping this asset tied to an Amazon revenue multiple makes no sense), and the valuations on the technology assets that help others respond to Amazon go nutty (witness the XenSource valuation on the day VMware went public). I say "Go, Amazon, Go!"

Cloud in Plain English

2008-12-23T11:09:00.002-05:00

I must take my hat off to Jake Sorofman, who runs marketing for rPath. Jake has done an incredible job distilling a bunch of complex stuff into a consumable and entertaining video. Do yourself a favor, and check out his Cloud Computing in Plain English video. Al Gore never looked so good.

And Happy Holidays!

Is JeOS a Tonic for VM Sprawl?

2008-12-12T14:56:00.000-05:00

It seems that everyone is worried about VM sprawl these days. When system capacity is easy to consume because the application is prepackaged as a virtual machine (a virtual appliance in the case of an ISV), the virtual infrastructure capacity is quickly gobbled up by those that have been waiting for IT to get around to provisioning systems for them. No more friction due to virtualization means no more waiting and wanting. It also leads to VM sprawl. But why is VM sprawl bad?

VM sprawl is bad because the scale of the management problem used to be throttled by the capital spending associated with the size of the infrastructure. Now, the scale of the management problem is equal to the true demand for application capacity as represented by the number of application images, or virtual machines, that get deployed. This scale factor associated with application images throws the old yardstick of X system admins per Y server machines out the window. What are we going to do to lower the work profile associated with so many new systems that now need to be managed?

I think at least part of the tonic for VM sprawl is the new acronym coined by Srinivas Krishnamurthi of VMware – JeOS. JeOS stands for Just enough OS and it is pronounced “juice.” In my mind, this liquid pronunciation is appropos given my view of its potential potency as a tonic for VM sprawl. A huge part of the burden of system management is the patching and associated regression testing for maintaining the security and functionality of the general purpose OS. If you can shrink the size of the OS by 90% (which is where we have measured the typical size for most applications built with our rBuilder technology) by eliminating any elements not required by the application, you can eliminate about 90% of the patching burden. More importantly, you eliminate the bigger burden of regression and stability testing that is coupled to the patching process.

With a JeOS approach, the number of virtual machines can theoretically grow by 10X the legacy approach without any impact to the cost structure associated with patching and testing the OS changes. I suspect a 10X reduction represents a real win for most shops as the OS patching and testing associated with security and infrastructure performance is a very sizable portion of their management spending. So if you are feeling the pangs associated with VM sprawl, I strongly suggest a healthy slug of JeOS each morning and once again in the afternoon to clear your system of the painful bloating that is brought on by virtualizing the general purpose OS.

Will Managing VM Sprawl lead to Rogue Cloud Deployments?

2008-12-05T15:34:00.002-05:00

I just read an interesting article regarding the potential cost pitfalls associated with VM sprawl. Jett Thompson, an enterprise computing architect from Boeing, has developed a cost model regarding the benefits of virtualization and the related pitfalls of VM sprawl. It seems that virtualization is easy to justify, so long as you don't give the users everything that they want. Here is the money quote from the article:

However, all of those savings [from virtualization] can be eliminated if sprawl isn't controlled. With virtual servers easy to spin up, users may ask for large numbers of new virtual machines and it's up to IT to hold the line, Thompson says.

"If you don't have demand management and good governance in place you're actually going to cost your company money," he says. "Virtual server sprawl can wipe out any savings."

Gartner analyst Thomas Bittman also says virtual server sprawl can be tough to control and is harder to measure than physical server sprawl. "Fundamentally, we believe virtualization sprawl can be a much bigger problem than physical sprawl," Bittman said

I believe that the unintended consequences of "IT hold[ing] the line" will be rogue cloud deployments. Rogue cloud deployments describes the phenomenon of business unit developers taking matters into their own hands when IT "holds the line" on making computing resources available. Once the business units understand that resources can be made available on-demand, either internally or via services such as Amazon EC2, they are simply not going to take "no" for an answer. Deploying applications as virtual machines, or virtual appliances in the case of an ISV application, removes all of the friction from the deployment process. This same friction was formerly the tonic that IT sprinkled about in order to "hold the line" on availability (and the subsequent management costs) of computing resources. The instant gratification culture that we are cultivating with SaaS and cloud will not be held in check if IT "holds the line" by saying "no" to requests for capacity/capability.

I have a recommendation for Jett and the folks at Boeing and elsewhere who are fearing the unintended consequences of frictionless system capacity brought about by virtualization. Push the control point for deployment policy upstream via automated build, release, and management processes for applications released as virtual machines, manage the scale problem by going vertical with a JeOS architecture, and build a seamless bridge managed by IT to cloud offerings like Amazon EC2. Charge the users with the costs for deployment and management, but give them the technology to do it the right way. Check out our cloud computing adoption model and the webinar that accompanies it. Rogue cloud deployments can be avoided, even in the face of VM sprawl control measures, when you say "yes" to your users while holding them accountable for building manageable system images.

IT Management Goes Vertical

2008-12-01T17:17:00.001-05:00

About 6 months ago, I had the privilege of engaging with the CTO of a large Wall Street investment bank on the topic of virtual machine management. One of his staff who was also participating in the conversation informed me that they had done a broad survey of their management technology providers regarding a new approach for system management -- vertical instead of horizontal. He said they polled the vendors with the following question:

Q: When does the unit of management shift from the physical host (horizontal) to the virtual machine (vertical)?

A: About 3 - 5 years

He said that all of the vendors agreed that this shift was inevitable, and the time frame for it to be in full swing was "about 3 - 5 years." He said they followed on with a second question after they received the response above:

Q: When do you plan to begin providing us with technology to manage vertically via virtual machines instead of horizontally via the physical hosts?

A: We'll worry about that in 3 - 5 years.

Amazing. The problem with this response, aside from the obvious, is that "about 3 - 5 years" is quickly becoming here and now. The term "vm sprawl" is on the tip of every analyst tongue, and the IT blogosphere is beginning to buzz with the imperative for managing the deployment of applications as coordinated collections of virtual machines (or virtual appliances in the case of applications delivered by a third party provider).

Lori MacVittie from f5 DevCentral writes:

When applications are decoupled from the servers on which they are deployed and the network infrastructure that supports and delivers them, they cannot be effectively managed unless they are recognized as individual components themselves.

Traditional infrastructure and its associated management intrinsically ties applications to servers and servers to IP addresses and IP addresses to switches and routers. This is a tightly coupled model that leaves very little room to address the dynamic nature of a virtual infrastructure such as those most often seen in cloud computing models . . . .

That model is broken in a virtual, dynamic infrastructure because applications are no longer bound to servers or IP addresses. They can be anywhere at any time, and infrastructure and management systems that insist on binding the two together are simply going to impede progress and make managing that virtual infrastructure even more painful.

The impending challenge is that the horizontal approach will become prohibitively expensive when the number of instances to be managed "horizontally" increases by an order of magnitude due to the liquidity of system capacity provided by server virtualization. Because it is so "easy" to provision a VM, lots of them will be provisioned for lots of very valid reasons. Managing 100 hosts becomes managing 1000 virtual machines. When this happens to you, be prepared to go "vertical."

Going vertical means defining and managing the contents of the virtual machines based upon the needs of the application. When a "host" is defined as the minimal software that is required to support the system functions of the application within the VM, 90% of the operating system software falls away because it becomes irrelevant. This smaller, Just Enough OS (JeOS or "juice") approach which considers only the needs of the application is more scalable because 90% of the management burden disappears. If the number of VMs increases by an order of magnitude, at least the footprint has shrunk by a corresponding order of magnitude.

Going vertical also means extending the discipline of Application Lifecycle Management (ALM) into the realm of the virtual machine. Since all of the system software that supports an application is now based upon the needs of the application rather than the needs of the infrastructure (which is, after all, the point of virtualization), it is now possible to put the entire software content of the VM under strong version and release management. This version management discipline, which is classically associated with ALM, enables change management on a much larger scale than the ad hoc approach of the legacy horizontal model.

Historically, the boundary between applications and IT operations was fuzzy because it was impossible to discern the portion of the operating system that was responsible for supporting the application from the portion that was responsible for supporting the infrastructure. They were intrinsically coupled due to the duality of the operating system role - hardware support and application support. With virtualization, the fuzziness becomes a bright, clear line. The hypervisor supports the allocation of hardware resources, and the host inside the VM supports the application.

With this new bright line separating application development from IT operations, all of the change management advantages associated with years of prior art in the ALM space become available as a scalable mechanism for managing production releases of the application as a set of VMs. The expensive testing burden for promoting an application from development to production is dramatically reduced when every software component of every VM falls under a unified change management system. The unintended consequences of change are largely eliminated because version management provides visibility and pinpoint accuracy regarding the deployment of changes across the universe of VMs - those in production as well as those in the library repository.

When coupled with JeOS, strong version management provides the scalability for IT management to go vertical. If you stay horizontal in the face of the impending VM tsunami, you might drown in the technical equivalent of the 1000 year flood.

The Virtual Machine Tsunami

2008-11-14T13:38:00.001-05:00

Over the past 2 weeks, I have had a number of very interesting conversations with partners, prospects, customers, and analysts that lead me to believe that a virtual machine tsunami is building which might soon swamp the legacy, horizontal system management approaches. Here is what I have heard:

Two separate prospects told me that they have quickly consumed every available bit of capacity on their VMware server farms. As soon as they add more capacity, it disappears under the weight of an ever pressing demand of new VMs. They are scrambling to figure out how they manage the pending VM sprawl. They are also scrambling to understand how they are going to lower their VMware bill via an Amazon EC2 capability for some portion of the runtime instances.

Two prominent analysts proclaimed to me that the percentage of new servers running a hypervisor as the primary boot option will quickly approach 90% by 2012. With all of these systems sporting a hypervisor as the onramp for applications built as virtual machines, the number of virtual machines is going to explode. The hypervisor takes the friction out of the deployment process, which in turn escalates the number of VMs to be managed.

Amazon EC2 demand continues to skyrocket. It seems that business units are quickly sidestepping those IT departments that have not yet found a way to say “yes” to requests for new capacity due to capital spending constraints and high friction processes for getting applications into production (i.e. the legacy approach of provisioning servers with a general purpose OS and then attempting to install/configure the app to work on the production implementation which is no doubt different than the development environment). I heard a rumor that a new datacenter in Oregon was underway to support this burgeoning EC2 demand. I also saw our most recent EC2 bill, and I nearly hit the roof. Turns out when you provide frictionless capacity via the hypervisor, virtual machine deployment, and variable cost payment, demand explodes. Trust me.

Bottom line, we are all facing an impending tsunami of VMs unleashed by an unprecedented liquidity in system capacity which is enabled by hypervisor based cloud computing. When the virtual machine becomes the unit of application management, extending the legacy, horizontal approaches for management built upon the concept of a physical host with a general purpose OS simply will not scale. The costs will skyrocket.

The new approach will have vertical management capability based upon the concept of an application as a coordinated set of version managed VMs. This approach is much more scalable for 2 reasons. First, the operating system required to support an application inside a VM is one-tenth the size of an operating system as a general purpose host atop a server. One tenth the footprint means one tenth the management burden – along with some related significant decrease in the system resources required to host the OS itself (memory, CPU, etc.). Second, strong version management across the combined elements of the application and the system software that supports it within the VM eliminates the unintended consequences associated with change. These unintended consequences yield massive expenses for testing and certification when new code is promoted from development to production across each horizontal layer (OS, middleware, application). Strong version management across these layers within an isolated VM eliminates these massive expenses.

The VM tsunami is coming. I'm just happy to have my trusty rPath surfboard tethered to my ankle. It's going to be the ride of a lifetime.

Can You See the Clouds from Windows?

2008-10-24T18:49:00.000-04:00

During the course of our webinar entitled "The Pragmatist's Guide to Cloud Computing: 5 Steps to Real ROI," several of the attendees submitted questions regarding the status of Windows as an environment for cloud applications. In a partial answer to the question, Jeff Barr, a speaker during the webinar and a member of the Amazon Web Services team, responded that a beta implementation of Windows for EC2 was now available. The problem with the notion of “Windows for EC2” is that it perpetuates the broken, legacy model of tying your application to the infrastructure upon which it runs.

In the legacy model, applications became artificially tied to the physical server upon which they ran, and server utilization was low because it is very difficult to run multiple applications on a single instance of a general purpose operating system. The reason it is difficult to run multiple applications on a single instance of a general purpose operating system is because each application has unique needs which conflict or compete with the unique needs of other applications. Virtualization technology, such as that provided by VMware or Citrix with XenServer, breaks the bond of the application to a physical server by placing a layer of software, called a hypervisor, on the physical hardware beneath the operating system instances that support each application. The applications are “isolated” from one another inside virtual machines, and this isolation eliminates the conflicts.

Amazon embraces this virtualization model by using Xen to enable their Elastic Compute Cloud (EC2) service. So what's the problem? If the OS instances are not tied to the physical servers any longer (indeed you do not even know which physical system is running your application on EC2, nor do you need to know), why am I raising a hullabaloo over a “broken model?” The reason this new model of Windows for EC2 is broken is because your application is now artificially coupled to EC2. When you begin with a Windows Amazon Machine Image (AMI), install your application on top, configure-test, configure-test, configure-test, configure-test, configure-test to get it right, and then save the tested configuration as a new AMI, the only place you can run this tested configuration of your application is on Amazon's EC2. If you want to run the application on another virtualized cloud, say maybe one provided by RackSpace, or Terremark, or GoGrid, or even your own internal virtualized cloud of systems, you have to install the application yet again, configure-test, configure-test, configure-test, configure-test, configure-test to get it right again, and then save the tested configuration on the other cloud service. Why don't we just stop the madness and admit that binding the OS to the physical infrastructure upon which it runs is a flawed approach when applications run as virtual machine images (or virtual appliances) atop a hypervisor or virtualized cloud of systems like EC2?

The reason that we are continuing the madness is because madness is all we have ever known. Everyone knows that you bind an operating system to a physical host. Operating systems are useless unless they bind to something, and until the emergence of the hypervisor as the layer that binds to the physical host, the only sensible approach for operating system distribution was to bind it to the physical host. When you buy hardware, you make it useful by installing an operating system as step one. But if the operating system that you install as step one in the new virtualized world is a hypervisor in lieu of a general purpose operating system, how do we get applications to be supported on this new type of host? Here's your answer -- what we previously knew as the general purpose operating system now needs to be transformed to just enough operating system (JeOS or “juice”) to support the application, and it should bind to the application NOT THE INFRASTRUCTURE.

Virtualization enables the separation of the application from the infrastructure upon which it runs – making possible a level of business agility and dynamicism previously unthinkable. Imagine being able to run your applications on-demand in any data-center around the world that exposes the hypervisor (any hypervisor) as the runtime environment. Privacy laws prevent an application supporting medical records in Switzerland from running in an Amazon datacenter in Belgium? No problem, run the application in Switzerland. Need to run the same application in Belgium in support of a new service being offered there next month? No problem, run it on Amazon's infrastructure in Belgium. The application has to support the covert operations associated with homeland security and it cannot be accessed via any Internet connection? No problem, provide it as a virtual appliance for the NSA to run on their private network. Just signed a strategic deal with RackSpace that provides an extraordinary level of service that Amazon is not willing to embrace at this time? No problem, shut down the instances running on EC2 and spin them up at RackSpace. All of this dynamic capability is possible without the tedious cycle of configure-test -- if we will simply bind the operating system to the application in order to free it from the infrastructure and let it fly into the clouds.

So why doesn't Microsoft simply allow Windows to become an application support infrastructure, aka JeOS, instead of a general purpose operating system that is bound to the infrastructure? Because JeOS disrupts their licensing and distribution model. Turning a ship as big as the Microsoft Windows licensing vessel might require a figurative body of water bigger than the Atlantic, Pacific, and Indian oceans combined. But if they don't find a way to turn the ship, they may find that their intransigence becomes the catalyst for ever increasing deployments of Linux and related open source technology that is unfettered by the momentum of a mighty business model. Folks with valuable .Net application assets might begin to consider technology such as Novell's mono project as a bridge to span their applications into the clouds via Linux.

I can tell you that there are lots of folks asking lots of questions about how to enable Windows applications in the “cloud.” I do not believe the answer is “Windows for EC2” plus “Windows for GoGrid” plus “Windows for RackSpace” plus “Windows for [insert your data-center cloud name here].” If Microsoft does not find a way to turn the licensing ship and embrace JeOS, the market will eventually embrace alternatives that provide the business agility that virtualization and cloud computing promises.

Will the Credit Crunch Accelerate the Cloud Punch?

2008-10-14T09:14:00.001-04:00

It's no secret that the days of cheap capital might be over. While it is obvious that startups with lean capital structures are already embracing cloud offerings such as Amazon EC2 for computing and S3 for storage, it seems to me that this trend might accelerate further for both startups and even enterprise customers.

Cloud consumption in the startup segment is poised to accelerate as investors like Sequoia Capital warn their portfolio companies to “tighten up” in the face of this credit crunch. Even the well capitalized SaaS software providers might begin re-considering the “ridiculous” expense of building out their offerings based upon the classic salesforce.com model of large scale, proprietary datacenters with complex and expensive approaches to multi-tenancy. They might be better served by a KnowledgeTree model where on-demand application value is delivered via virtual appliances. In this model, the customer can deploy the software on existing gear (no dedicated server required) because the virtualization model makes for a seamless, easy path to value without setup hassles. Or they can receive the value of the application as a SaaS offering when KnowledgeTree spins up their instance of the technology on Amazon's elastic compute cloud. In both cases, the customer and KnowledgeTree both avoid the capital cost of acquiring dedicated gear to run the application.

Large enterprises as well will be re-considering large scale datacenter projects. When credit is tight, everyone from municipal governments to the best capitalized financial institutions must find ways to avoid outlays of precious capital ahead of the reality of customer collections. More and more of these customers will be sifting through their application portfolio in search of workloads that can be offloaded to the cloud in order to free up existing resources and avoid outlays for new capacity to support high priority projects. Just as the 9/11 meltdown was a catalyst for the adoption of Linux (I witnessed this phenomenon as the head of enterprise sales at Red Hat), a similar phenomenon might emerge for incremental adoption of cloud associated with the credit crunch of 2008. All new projects will be further scrutinized to determine “Is there a better way forward than the status quo?”

As enterprises of all sizes evaluate new approaches to minimize capital outlays while accelerating competitive advantage via new applications, rPath is offering a novel adoption model for cloud computing that might serve as a convenient bridge to close the credit crunch capital gap. For those that are interested in exploring this new model, pleas join us in a webinar along with the good folks at Forrester, Amazon, and Momentum SI on October 23rd. If necessity is the mother of invention, we might be poised for some truly terrific innovations in the cloud space . . . . and we will owe a debt of gratitude to the credit crunch for driving the new architecture forward.

Larry Rains on the Cloud Parade

2008-09-30T17:02:00.002-04:00

At Oracle world last week, Larry Ellison derided the current “cloud” craze, likening the technology industry's obsession with “fashion” to the women's apparel industry. In a sense, he is right. Everything is being labeled cloud these days. New datacenters from IBM – cloud. New browser from Google – cloud. New strategy from VMware – cloud. I myself commented to Ben Worthen of the Wall Street Journal that I too feel the cloud craze is a bit “nutty.” At the same time, I believe there is some real change underfoot in the industry, and I believe that Amazon's Elastic Compute Cloud (EC2) is leading the way in capturing the imagination about what is possible with a new approach.

The reason EC2 has captured the imagination of so many people in the industry is because it offers the possibility of closing the painful gap that exists between application development and production operations. Promoting applications from development to production has typically been a contentious negotiation between the line of business application developers and the IT production operations management crew. It is a difficult process because the objectives of apps and ops run orthogonal to one another. Apps is about new features to quickly respond to market demand, and ops is about compliance, stringent change control, and standardization to assure stability.

With EC2, developers don't negotiate with operations at all. They simply package up the innovations they want inside a coordinated set of virtual machines (virtual appliances in the case of the ISV vernacular), and deploy, scale, and retire based upon the true workload demands of the market. No requisitions for hardware. No laborious setup of operating environments for new servers. No filling out waivers for using new software components that are not production approved yet. No replacement of components that fail the waiver process and re-coding when the production components don't work with the new application features. No re-testing. No re-coding. No internal chargebacks for servers that are not really being used because the demand for the application has waned. No painful system updates that break the application – even when the system function is irrelevant to the workload. No. No. No.

The on-demand, self-service datacenter architecture of Amazon's EC2 is going to put huge pressure on the operations organization to respond with an internal “cloud” architecture – or lose the business of the developers who would rather “go to the cloud” than negotiate with ops. Here at rPath, we believe that the ops folks are going to need to provide the apps folks with a release (rBuilder) and lifecycle management system (rPath Lifecycle Management Platform) that enables the self-service capability and rapid promotion of EC2 while preserving compliance with operating policies that assure stability and security. And, if an application really takes off, you don't have to build a new datacenter to respond to the demand. Just scale out the workload onto Amazon, or another provider with a similar cloud architecture. IT operations now has a way to say “yes we can” instead of “no you can't.” Getting to “yes” from your IT ops provider by closing the gap between apps and ops is what the excitement of cloud is all about.

The Ties that Bind

2008-09-28T14:21:00.000-04:00

Last week I attended the MIT Emerging Technology Conference in order to listen in on the panel on cloud computing. The panel included participants from salesforce.com, Google, Amazon, as well as Mendel Rosenblum, the founder of VMware and a professor at Stanford University. The broad sentiment among the group was that cloud computing is an extension of the trend where network ubiquity and performance enables a transition of IT capability from fixed costs with a slow rate of change to variable costs and a rate of change that reflects true demand. Software as a service, platform as a service, and infrastructure as a service all qualify under this broad definition. Technology innovations such as rich browser interfaces, virtualization, and SOA are further hastening this trend.

Sitting at rPath, my favorite quote was one from Mendel Rosenblum regarding the changing role of the operating system in response to a question regarding the technological transformations that will follow these trends. Mendel's quote went something like this:

"An operating system is not very useful by itself. It needs to bind to something to provide value. Historically, you would bind the OS to hardware in order to expose the hardware to the applications. Now, with the widespread adoption of virtualization as the technology that binds to the hardware, the OS needs instead to bind to the application. Every time I say that you guys in the press claim I said the OS is going away. I'm not saying that. I'm just saying that it is going to change."

And this is what we have been saying at rPath for the past 3 years. When you bind an OS to the hardware in the legacy approach, the tie that binds is the OS installer that detects and loads drivers along with the utilities that are useful in managing the hardware lifecycle. When you bind an OS to an application to create a virtual machine, the tie that binds is technology that detects the needs of the application and loads the right JeOS (Just enough OS) elements while also bringing utilities that are useful in managing the application lifecycle. These are two very different approaches to operating system technology, and Mendel rightly points out that the future OS (and the related business model around the OS) will be different.

The great news is that this difference will further the move toward a more flexible datacenter - ultimately enabling seamless movement of application workloads across datacenters ala cloud computing. Binding the OS to the application as part of the application release process means that applications can be delivered to the infrastructure as a pre-defined, pre-tuned, coordinated set of virtual machines. Anyone with any experience with virtual machines knows that it is much easier to deploy a virtual machine (or a virtual appliance in the case of an application vendor) than to provision a new server and configure the application atop a new host OS. Metadata standards like OVF coupled with packaging innovations from companies such as rPath will free applications from the infrastructure and allow them to run in the clouds.

Needless to say, I was grinning ear to ear when I heard Mendel's comments. I hope they get repeated again and again.

VMware Strikes Back

2008-09-16T14:47:00.001-04:00

The tech industry has been all abuzz lately about the competitive hullabaloo surrounding the new hypervisor technologies that are emerging to take on VMware's dominant hypervisor product. Microsoft launched Hyper-V with a party last week to upstage VMware's VMworld event this week. Red Hat purchased Qumranet to solidify its control of the KVM hypervisor technology. Now, VMware is striking back at the legacy OS vendors by labeling their new product category the Virtual Datacenter Operating System – a direct attack on the entrenched category of the general purpose operating system. The cold war of spies and covert operations to grab mindshare while outwardly promoting a message of peaceful co-existence has officially escalated to a hot war for the future architecture of the datacenter.

I, for one, am happy to see this rise in hostilities because I believe it will carry the industry to a much better place – and customers will be the primary beneficiary of the new approach. In the legacy datacenter, a general purpose operating system attempts to serve both the hardware infrastructure with device drivers while also serving the applications with system services. This approach has the disadvantage of artificially coupling applications to physical servers. Any attempt to move the application to another physical server typically requires that the configuration and validation process begin anew because it is extremely unlikely that the new server is absolutely identical to the previous one. This lack of flexibility leads to extreme overspending on capital equipment because an application with a period of low demand cannot relinquish its hardware resources to an application experiencing high demand. The hardware resources become application specific, and each application owner must size hardware capacity to meet peak demand. Server utilization in the datacenter averages 15 – 20%, and the general purpose OS is the culprit.

VMware has now declared that they offer an alternative approach to the general purpose operating system. The technology is not new, but marketing it under the category of an operating system is a very different tactic in this war for the datacenter. The conflict is now overt instead of covert, and this change was inevitable as VMware attempts to expand its footprint beyond its bread and butter business of Windows server consolidation and test lab operations. The new objective is the elastic datacenter and ultimately cloud computing.

The datacenter becomes elastic when applications are released by developers as coordinated sets of virtual machines (or virtual appliances in the case of a vendor release), each with Just Enough Operating System (JeOS or “juice”) attached to provide the system services required by the application. These applications can expand or contract on-demand because there is no onerous configuration process to ready the general purpose OS for a specific application. Instead, the hypervisor accepts the virtual machine and allocates it resources as specified by the OVF meta-data that is included with the image. Applications are up and running in a matter of seconds, and the process is totally repeatable to assure stability, security, and compliance as workloads scale, de-scale, and re-scale to meet the ever changing demand profiles of the enterprise application portfolio. Infrastructure can become a variable cost via this architecture because the scaling cycle can include hardware resources provided by third parties via a hypervisor layer – aka cloud computing as popularized by Amazon's Elastic Compute Cloud (EC2) via the Xen hypervisor.

The reason I label this new competitive tact by VMware as “warfare” is because the concept of a hypervisor as the infrastructure management layer with JeOS as the system services layer for the applications delivered as virtual machines destroys the value of the general purpose OS. If a hypervisor provides access to the infrastructure via device drivers, and applications receive system services from JeOS, and the flexibility of the datacenter improves, and the management of applications is simplified, and I can embrace cloud computing for variable cost infrastructure, why would I ever again buy a general purpose operating system? I won't. And customers won't either.

Unleash the dogs of war. Let's get to it so that we can all live happily ever after on the other side of history.

Red Hat Escalates Hypervisor Wars

2008-09-04T12:50:00.000-04:00

Red Hat today announced the acquisition of Qumranet, the company behind the Kernel-based Virtual Machine (KVM) bare metal hypervisor. With this acquisition, Red Hat is escalating the already fierce battle that is raging for control of the software layer that is rapidly replacing the general purpose OS as the access layer for hardware infrastructure. Qumranet is a very savvy acquisition by Red Hat because it plays to their strength as the primary maintainer of low level Linux kernel technology. The Linux kernel is a mature, high performing provider of hardware driver capability, and there is no doubt in my mind that it can become a significant competitor in the bare metal hypervisor space.

Given all of the competitive noise surrounding hypervisors these days – the Microsoft Hyper-V launch is next week followed by VMworld the following week – the stakes in this game are enormous. It represents a fundamental shift in the architecture for both server applications as well as desktop applications. No more will the general purpose OS be the table stakes for releasing or “certifying” an application to run in the customer's environment. Instead, the hypervisor is going to be the target, and applications will arrive pre-configured and ready to run as virtual machines with Just Enough OS (JeOS or “juice”) attached to provide system services and a connection to the hypervisor. Indeed this new architecture is one of the driving forces behind the concept of cloud computing. Amazon's Elastic Compute Cloud (EC2) is enabled by the Xen hypervisor, and the new CEO of VMware, Paul Maritz, has sounded off time and again about the importance of a cloud computing architecture since taking the helm of VMware.

Red Hat's new, aggressive move in this space is good news for customers who will inevitably embrace this cloud approach for enabling the flexible, elastic datacenter. It means that there will be more competition for the hypervisor design win, which translates to better features, better performance, and lower cost. This type of bare-knuckle competition is what the software market is all about, and customers are the big winners in this fight.

Single Minute Exchange of Applications (SMEA) - The Cure for Server Hoarding

2008-08-17T21:44:00.002-04:00

I recently had an interesting conversation with an IT executive that has built a self-service datacenter capability based upon virtualization. He described for me a system whereby business units can request “virtual server hosts” with a pre-set system environment (i.e. Linux and Java), and within an hour or two they receive an email notification informing them of the availability of the “virtual machines.” The goal of this system, as it was explained to me, is to “cure server hoarding” by the business units.

The theory is that if the business units are confident that they can get new capacity “on-demand,” then they will not request more systems than they really need. And since they are billed based upon the actual amount of capacity deployed, they have incentive to “give back” any systems that are not necessary to meet production demands. I asked how it was working:

IT Exec – Great. We have over 1500 virtual machines actively deployed in production in support of business unit demand.

Billy – Wow! That's terrific. What do the statistics look like for server returns?

IT Exec – What do you mean?

Billy – I mean how many systems have the business units returned to the pool of available systems because their demand was transitory?

IT Exec – No one has ever given back a single machine ever. They have the economic incentive to do so, but so far not one machine has ever been given back to the pool.

And therein lies the problem. The reason no one gives systems back is because the setup costs associated with getting them productive are simply too high. Even in this case, when the setup of the operating environment is accomplished within an hour or two of the request, the process of “fiddling around with the system” to get the application installed, configured, and stable is so expensive that no one ever gives a productive system back when demand falls. This situation leads to tons of waste in the form of over deployed capital and over consumption of resources such as power. I am reminded of the early days of the lean production revolution in the world of manufacturing.

In the late eighties, Toyota was whipping Detroit's fanny because they had implemented a system that the folks in Detroit did not think was possible. The folks at Toyota got much higher utilization out of their capital investment with much lower levels of waste and work in process because they had implemented a system that assured the expensive production equipment was always engaged in producing parts and vehicles that closely reflected true demand. A big part of this system was a capability known as the Single Minute Exchange of Dies, or the SMED system, which was pioneered by Toyota and evangelized by the legendary manufacturing engineer, Shigeo Shingo.

With SMED, expensive body stamping machines (or any machine for that matter) are kept productively engaged building the exact parts that are required to meet true demand by reducing the setup time for a “changeover” to less than 10 minutes. This is accomplished primarily by precisely defining the interface between the machine and the stamping dies such that the dies can be prepared for production “off-line.” While a machine is productively engaged building Part A, the dies for Part B are setup for production in a manner that does not require interfacing with the production machine. When it is time for a changeover from Part A to Part B, the machine stops, the Part A dies are quickly released and pulled from the machine, and the Part B dies are quickly engaged using a highly standardized interface. No fiddling around to get it right. The machine starts up again in less than 10 minutes and down the line roll the perfect output for Part B.

Contrast this approach with the standard approach in Detroit in the late eighties. The economy of scale theory in Detroit was to set up the line for long runs of a single part type and build inventory because changing over the line was filled with setup costs. Fiddling around with the dies to get the parts to come off according to specification might take a day or even a week. So instead of building for true demand, Detroit over-deployed resources, both capital equipment and work in process, in an attempt to compensate for poor setup engineering. We all know how this story ends. The Toyota system is still the envy of the manufacturing world.

Now is the time for the technology world to take a lesson from Toyota. Virtualization will provide the standard interface for production, but it is almost worthless without “setup” technology that enables the applications to be defined independent from the production machine. The resources of the datacenter should reflect “true demand” for production output instead of idling away – suffering from a miserable case of server hoarding because setup is so expensive and error prone. The time has come for SMEA – Single Minute Exchange of Applications.

At rPath, we are working towards SMEA every day. We have high hopes that the complementary trends of virtualization and cloud computing will highlight the possibility for an entirely new, and more efficient, approach for consumption of server production capacity. An approach where applications are readied for production without consuming machine cycles “fiddling around” to get the application stable. An approach where expensive machines running application A are given back for production of application B when true demand indicates that B needs the resources instead of A. The Department of Energy and CERN are already on board with this approach, but it will be curious to observe who in the technology world emerges as “Toyota” and how long it takes the status quo of “Detroit” to wake up and smell the coffee.

VMware Accelerates Cloud with Free ESX

2008-07-29T11:01:00.000-04:00

The new CEO of VMware, Paul Maritz, seems to be committed to establishing VMware technology as the basis for emerging compute cloud offerings that enable shared, scalable infrastructure as a service via hypervisor virtualization. With Amazon EC2, the poster child for the successful compute cloud offering, being based upon the competing Xen technology from Citrix, Maritz is losing no time staking claim to other potential providers by meeting the Xen price requirement – zero, zilch, nada, zip. I love it. Low cost drives adoption, and free is as good as it gets when it comes to low cost and adoption.

As the economics of servers tilt more and more toward larger systems with multi-core CPUs, the hypervisor is going to become a requirement for getting value from the newer, larger systems. Developers simply do not write code that scales effectively across lots of CPUs on a single system. The coding trend is toward service oriented architectures that enable functions as small, atomic applications running on one or two CPUs, with multiple units deployed to achieve scalability. Couple the bigger server trend with the SOA trend with the virtualization trend with the cloud trend, and you have a pretty big set of table stakes that VMware does not want to miss. If a hypervisor is a requirement, why not use VMware's hypervisor if it is free?

The only challenge with free in the case of VMware is going to be lack of freedom. Xen currently offers both free price and freedom because of its open source heritage. If I run into a problem with VMware's ESX, my only recourse is to depend on the good will of VMware to fix problems. With Xen, I have the option of fixing my own problem if I am so inclined and capable. It will be interesting to watch the hypervisor choices people make as they build their cloud infrastructures, both internally and for commercial consumption, based upon the successful Amazon EC2 architecture.

The CIO is the Last to Know

2008-07-24T09:04:00.001-04:00

A recent Goldman Sachs survey of CIOs indicates that these executives do not plan to spend much money on cloud computing in the coming year. Indeed, most of their stated plans involve reducing the amount of consulting services and hardware that they are buying. I'm certain the predictions are accurate, and this scenario will lead to even more rapid growth in cloud computing. And the CIO will be the last to know.

How does this work? If Goldman has correctly measured the intentions of the CIOs, then they will not be spending money on cloud computing. Instead it will be the business units that they are supposed to serve that will be spending the money because the service level of the IT department will not meet their needs. Recall the reduction in consultants and service personnel? When a fixed income group at an investment banking house needs to stand up 50 servers to run a set of Monte Carlo simulations to test a hypothesis, the over-stressed IT department response is going to be “we'll get to that request after we fill the 25 that are in line ahead of it. It will probably be next quarter.”

The “swoosh” sound you just heard is the developer of the simulation code swiping his credit card to set up his Amazon Web Services account. Three days later, he has 100 systems standing up on Amazon's Elastic Compute Cloud pumping back the information he needs to help his traders make money. The credit card bill is only about $5000 per month – much cheaper than the IT chargeback for similar capability. The head of fixed income hears about the profits due to the extra simulation capacity, and the developer gets a promotion and is encouraged to spin up another 100 to 200 machines to get even more aggressive with the strategy. Relative to the millions in profit, the cost is peanuts and the IT department just can't respond to these type requests anyway. The CIO is the last to know.

It always happens this way with new technology. As the leader of North America sales for Red Hat in 2002, I remember calling on the CIO of a company in the financial services industry that processed millions of transactions daily in support of the equities market. I sat in his office while he explained to me that his operation was mission critical – the markets depend on this operation. He would never consider using Linux and open source. “Why don't we take a tour of the datacenter,” he asked. I was game, so I replied “Sure.”

As we walked the floor, I noticed several machine consoles indicating they were attached to Red Hat Linux 7.1 servers. Here is the conversation that ensued:

Billy: What's this?

CIO: Huh? I don't know. Steve, what's this all about?

Steve the Admin: Yeah, we're running Red Hat Linux for most of our network services.

CIO: What do you mean?

Steve the Admin: You know, Apache, BIND, SendMail, a few transaction servers and log crunchers mixed in here and there.

CIO: How many of these are we running in this datacenter?

Steve the Admin: About 25% of the machines, I would guess. About 800 servers in total.

Billy: Why don't we go back to your office and have another conversation about how much value you are getting out of Linux and open source and how Red Hat can help you.

The CIO is always the last to know about new technology. The head of engineering brought UNIX into the enterprise for CAD/CAM and analysis applications, and the CIO was the last to know. Department managers brought in PCs and Windows for personal productivity and desktop publishing, and the CIO was the last to know. System administrators brought in Linux for network services, and the CIO was the last to know. The sales force brought in salesforce.com and introduced the enterprise to SaaS, and the CIO was the last to know. Developers in the business units will use cloud computing, and the CIO will be the last to know.

The good news is that CIOs know where their bread is buttered, and eventually supporting the business units becomes the top priority. In this case, I would guess that all of that spending that Goldman noted as being earmarked for virtualization will pave the path for a hybrid approach to cloud computing. The enterprise IT function will begin to model the services that they provide after Amazon, with hypervisor virtualization as the basis of the compute capacity. Then, with a single, corporate architecture for cloud computing, applications will be able to scale seamlessly across the internal cloud infrastructure and also out into the external clouds when necessary for extra capacity. In this scenario, everyone gets what they want, and the CIO is a hero for reducing the fixed costs and operating budget associated with data center capacity. Being the last to know isn't necessarily a bad thing.

Citrix Management Land Grab - Project Kensho

2008-07-18T21:07:00.002-04:00

In an effort to secure the management technology high ground as hypervisors proliferate and become more of a ubiquitous commodity than a premium point product, Citrix has announced Project Kensho. The strategy to enable portability and scalability across heterogenous hypervisors is so obvious and correct, that my only question after reading the release was “What is the provenance of the name 'Kensho'?”

Given Citrix' headquarters deep in Florida, my first speculation was that it was a phonetically correct implementation of an affirmative response with a southern accent:

“I ken sho' yew zackly how dis new technology iz gonna be betta dan anything yew eva saw in yo' life.”

It seemed reasonable at first, given my own tendency to display a southern flair. But given the likelihood of a strong influence by Simon Crosby, an Englishman who is the Citrix CTO for all things virtualization, perhaps the name is not based at all upon the southern heritage of Citrix. I sent Simon a note asking about the provenance of the name, and he replied:

“Kensho is a Zen Buddhist term (pun on Xen) for enlightenment experiences . . . . Now go deep into your Zen mind and figure those out!”

With the mystery of the name solved, let's make some commentary on the obvious. I have no doubt that the future of application release and lifecycle management is going to be based upon virtual machine images. By releasing and managing applications as virtual machines, it is possible to define the application independent from the infrastructure upon which it runs. In so doing, applications can be deployed and scaled on-demand - on any virtualized cloud of machines - without complex, costly, ad hoc setup procedures. More importantly, you can de-scale on one infrastructure and re-scale on different infrastructure as demand fluctuates because the setup and configuration information is not unique to the infrastructure. Unlike VMware's VADK technology which is unique to the VMware hypervisor, Project Kensho is aiming higher by embracing this obvious, management focused, hypervisor independent architecture for cloud computing.

Well, I say “Welcome to the party, Citrix!” The more voices we have proclaiming the benefits of this new architecture for cloud computing, the better. I have spoken with not less than 12 CTOs and CIOs over the last 3 weeks who have proclaimed to me the importance of multi-hypervisor support for any application release and lifecycle management system based upon virtual machines. The ability to scale, de-scale, and re-scale seamlessly and repeatably across multiple infrastructure targets is critical if cloud computing is to move from promising hype to bankable reality. Kudos to Citrix for moving the ball down the field on this critical goal with Project Kensho.

Thank you, Diane Greene

2008-07-10T16:17:00.001-04:00

“Hello, this is Diane Greene.” Such was my introduction to Diane back in 1998 when she joined a conference call with me and Matthew Szulik. I had just reviewed the VMware technology with one of VMware's business development managers, Reza Malekzadeh, as part of a partnership opportunity between Red Hat and VMware. Red Hat, although still a very small company with only 70 employees and about $12M in revenue, was a hot target for alliances, and VMware wanted us to distribute their product with our Red Hat Linux product as part of the “extras” CD. Our engineers thought the technology was “very cool,” so shipping it as part of the CD made sense because it would create more demand for our product. I also thought it was cool, but at the time I was very skeptical of the business model.

Reza had shown me a diagram of the different permutations of how someone might use VMware. He indicated that it would be used immediately as a host environment atop an existing OS (such as Windows or Linux) to enable developers to rapidly develop and test for many platforms atop their workstations. But, he indicated to me that the big vision was for VMware to be the bottom layer, right against the hardware, with multiple other OS implementations running as guests atop that layer. My response: “I don't understand why anyone would ever want to do that.” Now we understand why Diane got SDForum's visionary award a few weeks back and Billy Marshall was lucky to be on the guest list.

Not one to be left behind, it only took me 6 years to determine that this new approach indeed represents one of the biggest opportunities to improve the efficiency and capability of information technology. As the hypervisor replaces the general purpose OS as the layer that exposes the hardware, the applications that ride atop that layer become much more portable and the datacenter resources become much more efficient. As Diane leaves VMware to explore her next opportunity, I owe her a big debt of gratitude for shining so much bright light on this revolutionary approach to computing. Thank you, Diane Greene, for giving all of us that play in this market an opportunity to do something wonderful for our customers.

Shut Down the Datacenter

2008-07-07T10:47:00.000-04:00

Or at least power down significant pieces of it during periods of low demand. This message always draws funny looks from IT types when I suggest a seemingly simple answer to the problem of extreme costs for datacenter resources. I push on:

Billy – If utilization is around 20 – 30%, aren't there periods of time when you could just shut down about 50% of the systems? Or at least 25%?

IT – We can't just shut the systems down. . .

Billy – Why not? You aren't using them.

IT – You don't understand.

Billy – What am I missing?

IT – Well, it just doesn't work that way.

Billy – How does it work?

IT – It takes a long time to lay the application down atop a production server.

Billy – Why?

IT – Set up is complicated. Laying down the application and bringing it online can take several days, typically 2 to 4 weeks.

Billy – So part of the application definition is described by the physical system it runs on?

IT – Yes, that's right. If I shut down the physical system, I lose part of the definition and configuration of the application.

And therein lies the culprit. The “last mile” of application release engineering and deployment is a black art. Applications become tightly coupled to the physical hosts upon which they are deployed, and the physical hosts cannot be powered down without losing the definition of a stable application. Bringing the application back up is expensive due to the high costs of expert administration resources, and it is fraught with peril because the process is not repeatable. Enterprises are spending billions of dollars on datacenter operating costs because the risk of bring applications back on-line is not worth the savings of taking them off-line.

Of course I blame most of this mess on the faulty architecture of the One Size Fits All General Purpose Operating System (OSFAGPOS). OSFAGPOS is typically deployed in unison with the physical hosts because OSFAGPOS provides the drivers that enable the applications to access the hardware resources. To get an application to run correctly on OSFAGPOS, the system administrators then need to “fiddle with it” to adjust it to the needs of any given application. This “fiddling” is where things run amok. It's hard to document “fiddling,” and it is therefore difficult to repeat “fiddling.” The “fiddle” period can last for up to 30 days, depending on the complexity of the “fiddling” required.

So how do we get away from all of this “fiddling” around, and deploy an architecture that allows the datacenter to scale up and down based on actual demand? Start with a bare metal hypervisor as the layer that provides access to the hardware. Then extend release engineering discipline to include the OS by releasing applications as virtual machines with Just Enough OS (JeOS or “juice”) in lieu of OSFAGPOS, complete with all of the “metadata” required to access the appropriate resources (memory, CPU, data, network, authentication services, etc.). By decoupling the definition of the application from the physical hosts, a world of flexibility becomes possible for datacenter resources. Starting up applications becomes fast, cheap, and reliable. As an added bonus, embracing cloud capacity such as that provided by Amazon's EC2 becomes a reality. Instead of standing up application capacity in-house, certain peak demand workloads can be deployed “on-demand” with a variable cost model (in the case of Amazon it starts at about $.10/CPU/hr).

With oil trading at around $140 per barrel, the cost of allowing datacenter resources to “idle” during slow demand periods is becoming a real burden. “Fiddling around” with applications to get them deployed on OSFAGPOS is no longer just good clean fun for system administrators. It is serious money.

Red Hat oVirt(ly) targets VMware and Citrix

2008-06-23T09:49:00.002-04:00

Red Hat announced last week that they are developing technology that will one day become a product to challenge the offerings from VMware and Citrix. The technology is showcased at a website sponsored and maintained by Red Hat as part of an “emerging technology” initiative. Aside from the fact that Red Hat is announcing technology projects and not products, the most noteworthy detail of this approach is the emergence of KVM (kernel-based virtual machine) as the hypervisor approach that Red Hat intends to back. Taking a page from the Microsoft HyperV playbook, Red Hat is claiming virtualization as a “feature” of the OS they already control in order to maintain their investment in the server distribution channel they have established with OEMs such as Dell, HP, IBM, Fujitsu, and others. Bare metal hypervisors and their associated infrastructure management frameworks are threatening the entrenched status quo of the operating system vendors, and Red Hat does not intend to “go gentle into that good night.”

Historically, a general purpose operating system performed two key functions: 1) provide drivers so that applications can access the hardware, and 2) provide system services (file system, libraries, etc.) to the applications. The trouble with this approach is that applications become artificially coupled to the hardware. Have you ever experienced pain upgrading hardware when the application that ran just fine on the old hardware no longer runs on the new hardware because of changes in the operating system? Do you find that you are continuously porting and testing applications just to upgrade hardware? Why? Because a one-size-fits-all general purpose operating system (OSFAGPOS) couples hardware support with application support due to the architecture of the product. Aside from the application portability issues, the OSFAGPOS approach also leads to the enormous management cost associated with bloating. Most applications run atop an operating system in the datacenter that is 10X the size actually required by the application, which leads to a patching nightmare.

To be fair to Red Hat, KVM could certainly be implemented and maintained by them as a skinny bare-metal hypervisor that only concerns itself with managing the hardware infrastructure. The Linux kernel is a fine provider of hardware support, and Red Hat has more Linux kernel expertise than any other vendor in the world. Their technical credibility in this space is terrific. The trick will be the marketing challenge of offering a product that has a very different value proposition than the OSFAGPOS. The goal of a bare-metal hypervisor is to support as many application OS variants as possible in order to enable application providers to choose the system software that works best for their application. This flexibility for the application flies in the face of the OSFAGPOS argument to “standardize” the OS to gain economies of scale in management and support. With OSFAGPOS, at least you know what systems need to be patched – all of them . . . all the time. Embracing a bare-metal product approach would mean that Red Hat also needs to face up to OSFAGPOS management challenges and embrace virtual appliance value for applications that run with Just enough OS (JeOS or "juice") optimized for their workload.

I believe that the separation of the duties of the historic OSFAGPOS architecture into two separate product domains - hypervisors for drivers and JeOS for application support - is inevitable. Separating hardware support from application support simply provides too many benefits. The popularity of both VMware and Amazon's emerging EC2 service are great examples. Watching both Red Hat and Microsoft navigate this change will be interesting. The marketing gurus at both companies will be strained to the breaking point as they “rage, rage, against the dying of the light.”

When "Agile" becomes "Fragile"

2008-04-30T17:04:00.000-04:00

Last week one of my board members, Andrew Nash, shared a conversation with the CTO of one of the ten biggest software companies in the world. The subject of the conversation was agile development. The CTO complained to Andrew that agile development is worthless if you cannot extend the stream of innovation to customers in a timely manner. What does it matter if the application developers deliver code to release engineering and QA on a monthly basis if the subsequent delivery and consumption by the customer takes between 6 months and a year? The valuable concepts of rapid feedback and minimal work in process get blown away because navigating the matrix of pain and distributing innovation to customers is so inefficient. If application providers are to truly gain benefit from agile development, they are going to be forced to embrace appliances, virtual appliances, or SaaS as the delivery model. Without control over the the system software as part of the application delivery and consumption cycle, agile development rapidly deteriorates to fragile development and becomes worthless.

Agile development is a close cousin to the lean manufacturing concepts pioneered by Toyota in the mid-80's and early 90's. The concept is simple – keep work in process to a minimum in order to discover and improve mistakes with a minimal amount of waste and rework. Agile development likewise seeks to keep work in process to a minimum in order to avoid large scale mistakes in architecture and feature design by facilitating rapid feedback throughout the value chain – from the developer, to QA and release, and ultimately to the customer. It has the side benefit that products that rapidly evolve to deliver ever greater value to the customer become “stickier” and less prone to competitive displacement. It often happens that a competitor replaces a product because the upgrade to new features in the incumbent product was more expensive than the switching costs to the competitive product. Just ask the folks that used to work at Baan about this problem. SAP made a killing switching out Baan customers due to extraordinary upgrade costs.

In order to prevent software “work in process” from piling up at the proverbial shipping dock (i.e. release engineering), the complexity of the release engineering matrix must be dramatically simplified. Fortunately, appliances, virtual appliances, and SaaS all provide this goal of simplification, and they are rapidly becoming the de facto delivery model for much of the new software being consumed in the market today. These improved distribution models enable application providers to do something good for their customers and their developers in one fell swoop. Customers get more innovation with fewer hassles, and developers get to work on new features instead of grinding away to solve the problems created by the context of release engineering. If you hope to embrace agile development and avoid fragile development, you better sort out your distribution strategy as well. The legacy approach will not get the job done.

Cloud Computing Casts Shadow on Walled Gardens

2008-04-17T16:09:00.000-04:00

As a technology provider that helps application companies embrace cloud computing by virtualizing the applications to run on any cloud, I was a bit disappointed with Google's appengine announcement. It appears that Google is embracing the “walled garden” approach of salesforce.com and Microsoft instead of the cloud approach of Amazon. I believe that walled gardens will ultimately be overshadowed by clouds because you cannot achieve webscale computing if every application has to run on a server owned by Google.

Historically, Google has been very good about providing APIs that enable applications to access its web services independent of the computer on which they run. This is an important concept because it is often the case that an application needs to run on a particular network or network segment in order to preserve some critical aspect of performance or security. It is also important because it provides developers with the broadest choice of system and programming tools when developing or maintaining their applications. If you must program the application in the Python implementation specified by Google and run it on a Google server in order to take advantage of services like BigTable and Sawzall, a huge segment of the application market has just been eliminated from consideration (note that it is unclear to me at this time if Big Table and Sawzall can be accessed independent of appengine).

Why not simply expose a virtual machine API (such as Amazon Machine Image) along with the API for the web services (such as Amazon's S3, SQS, etc.)? Application instances that require minimal latency to Google services are provisioned as virtualized appliances on a Google server. For applications that need to run on a different network, you can provision the same system definition to that network while accessing the web services over the Internet. Write the program in any language you choose. With any set of system components that you choose.

The problem with walled gardens is that they ultimately restrict the growth of the market. While it is true that an attractive and well manicured walled garden will result in asymetrically large economic rent for the owner of the garden (witness Microsoft), the size of the market is nonetheless constrained. It seems to me that Google would reap the greatest benefit from maximizing the market for cloud applications quickly – independent of their ability to collect an asymetrically large portion of the rent from that market. Even their marketing of the current implementation of appengine indicates this hypothesis is correct – it is free. Success with cloud computing will no doubt lead to a decline in the value of the Microsoft system software franchise (the ultimate walled garden). Why not accelerate that decline with broad market capability instead of yet another walled garden (YAWG)?

Let me provide a concrete example. rPath was approached by a SaaS application provider to help them release their on-demand application as an on-premise application – without sacrificing management control of the system software. They want on-premise capability in order to meet the data security requirements of a certain segment of the market which they have been unable to penetrate with their SaaS offering. Their current application runs on Microsoft server technology, but it is written in Java so skipping out of the Microsoft walled garden was pretty trivial. We provided them with a virtualized implementation of their application, and we demonstrated how it could run on a local network atop a hypervisor, or as a variable cost implementation on Amazon's elastic compute cloud (EC2). Their reaction was so positive that they are now planning to gradually migrate their entire infrastructure from Microsoft to virtual infrastructure in order to seamlessly deliver the application via SaaS, variable cost cloud (Amazon), and local network (virtual appliance). Without changing their preference for programming language. Without sacrificing control of the system software layer.

To be fair to Google, appengine is a beta service. I have no doubt that they made compromises in architecture in order to get the service out the door more quickly. I hope they follow Amazon's lead and expose all of their great services as true web services while enabling any application to run close to those services via a simple virtualization spec such as Amazon's AMI. The faster we take the market to cloud computing, the sooner we can kill off the walled gardens through webscale shadows that deprive them of economic sunlight.

The Patching Dilemma

2008-03-21T21:25:00.000-04:00

As virtual appliances pick up steam as an alternative approach to delivering SaaS value, I have seen a few analyst proclaim that the burden of patch delivery and management makes multi-tenancy via virtualization unattractive. They are correct . . . . if you attempt to build the customer virtual appliances using a legacy approach with a general purpose operating system without an integrated approach for lifecycle management. They are wrong if you consider a Just enough Operating System (JeOS or “juice”) approach with robust lifecycle management such as that offered by rPath. Let me provide an example.

rPath maintains a reference implementation of a full featured distribution of the Linux operating system as part of our rPath Appliance Platform offering. Compared with Red Hat and Novell, we probably offer about 80% of the software packages they provide as part of this reference. The remaining 20% represent desktop technology and other packages that do not matter for our target market, but are certainly important to their respective go-to-market strategies. As part of our commitment to maintain a full featured distribution, we have released about 200 security patches over the past 2 years. That is a lot of patches, but it is the reality of maintaining an OS. Keep this number in your head for a moment.

rPath also delivers our products to our customers as virtual appliances. Our ISV customers receive rBuilder and the rPath Appliance Platform as turn-key server applications completely managed by rPath on their network. Due to the unique packaging technology we pioneered, the operating system footprint to support rBuilder and the rPath Appliance Platform is about 50 Mb. When you use a JeOS architecture you eliminate any package that is not required by the application. Why is this important? Remember the 200 security patches released by rPath over the last 2 years? Only 3 were required to support our product implementation at our customers. That's correct – only 3.

Furthermore, because we deliver our patches to a specific implementation of the product (i.e. the customer did not assemble it themselves from multiple third party components, rendering clean application of patches virtually impossible), all of our customers received and applied the patches with no testing burden for them and no customer support burden for rPath.

Returning to the analysts that claim patching makes multi-tenancy via virtualization untenable, using a legacy approach with a general purpose operating system inside of snapshot virtual machine would ruin the economics of multi-tenancy via virtualization. With the rPath approach, coupled with rPath's technology for distributing patches to large numbers of systems with minimal administrator labor, you can host 66 customer virtual appliances for the same administrator effort as one virtual machine with the legacy model (3 patches vs. 200). And you avoid the expense of re-architecting and re-writing your code to support multi-tenancy – a VERY expensive proposition. And you avoid changing your business and sales model because customers can run the virtual appliances on-premise – but without the headaches of technology integration and multiple party maintenance management.

Virtual appliances deliver all of the value of SaaS to your customer base without all of the vendor hassles associated with changing your technology and changing your business model. However, just snapshotting an implementation of legacy components and ignoring the lifecycle management issues will not scale. Taking that approach would be crazy and unprofitable. rPath gives you the best of both worlds.

A Big Switch or a Gradual Shift?

2008-03-11T13:21:00.000-04:00

I just finished reading Nicholas Carr's new book, The Big Switch. I enjoyed the read, but I found the conclusions just a bit sensational. Not surprising, as all such books seek to be titillating and a bit controversial in order to hold our attention from cover to cover. The basic premise of the book is that there will be a “big switch” from internal application development, deployment, and management to external procurement of application services. The losers will be the skilled developers and IT staff that currently toil away inside the development centers and datacenters of corporations, and the winners will be the application providers such as Google and salesforce.com that provide applications on demand. I do not believe the "big switch" will be so black and white, but I do believe a gradual shift is underway.

The historical metaphor that Carr effectively uses to demonstrate the likelihood of this pending change is the switch from locally produced electrical power to regionally produced electrical power delivered via a high performing electrical grid infrastructure. In Carr's metaphor electricity is analogous to applications and the electrical grid is analogous to the Internet. There are clearly some parallels, but I believe the metaphor is flawed because information applications are more analogous to hair dryers, drill presses, and die stamping machines (i.e. applications that consume electricity) as opposed to the electricity itself.

Here is a simple example. Both a paper mill and a steel mill have a need for high voltage electricity, but the paper mill applies that electricity to an application that involves digesting wood chips into a slurry suitable for making paper while the steel mill applies that electricity to the transformation of molten iron ore into various steel products. The paper mill has no use for an application that transforms iron ore, and the steel mill has no use for an application that digests wood chips. Their application requirements are very different, but they do use very similar electrical inputs.

It is true enough that all businesses have a need for certain applications that are somewhat universal. Salesforce.com has certainly demonstrated that a single implementation of a customer relationship management and sales force automation application can be applied across a variety of businesses and delivered effectively via the Internet. Perhaps Google will indeed accomplish the same result for basic professional productivity application such as word processing and spreadsheet analysis. But what is the fate of proprietary applications? Is salesforce.com going to deliver chip design and analysis simulations to Intel? I doubt it. Is Google going to deliver portfolio and risk analysis applications to Goldman Sachs? Unlikely. If these applications are not candidates for the “big switch,” how might their delivery still be improved according to Carr's theory?

Carr identifies two key technology developments in the “big switch” from local to regionally produced power - alternating current (AC) and reliable transformers. Alternating current makes it possible to distribute high potential voltage over large distances while transformers reliably “step down” this voltage to levels where it can be safely and reliably consumed by a variety of applications (hair dryers, drill presses, etc.). Clearly fiber optics and broadband switching are the IT equivalent of alternating current by enabling efficient delivery over long distances. I believe that hypervisors coupled with virtual appliances are analogous to the transformer technology of the power system. When applications can reliably plug into a grid to receive “power” in a standardized and repeatable manner, it will be increasingly popular to let someone else deliver the power of the grid while the individual companies focus on the “design of the application” (i.e. the drill press, the chip digester, the ore smelter).

Currently, applications used by Goldman Sachs to perform portfolio and risk analysis are not easily portable to a band of computers that Intel uses for chip design and simulation. The only way Goldman could reliably “move” these applications to another “power” provider would be to literally unbolt the racks of machines from their datacenter, truck them to another datacenter, and rebolt them to the floor and re-attach them to power and network. The definition of the applications is hard-coupled to the machines that run the applications because there was never any thought of running them on a different “grid.” It takes effort to design applications to be totally independent from the computers they run upon.

If, however, Goldman Sachs were to “transform” these applications into a coordinated group of virtual appliances, then they could literally “plug” the applications into any set of computers that exposed a standard hypervisor. As standards emerge for reliable “transformation” of applications to virtual appliances, opportunities will emerge for utility providers of variable cost datacenter capacity (aka cloud computers such as Amazon's EC2) to supply the “power” to these applications. I do not believe it will occur as a “big switch,” but I am convinced that we are witnessing the beginning of a gradual shift in the division of labor for application delivery. Companies will increasingly focus their scarce resources on the definition of the application, and the machines that provide “power” to the application will increasingly be purchased as variable cost computing cycles. But I have to agree with Carr that The Big Switch is a much better title for a book than The Gradual Shift.

EMC means Even More Clouds

2008-02-19T09:43:00.000-05:00

Last week, Doug Merritt, an executive vice president at SAP, informed Information Week that EMC is planning to offer a computing utility service suitable for running enterprise applications. This “cloud” (a term popularized by Amazon's Elastic Compute Cloud) would offer SAP an ability to deliver on-demand applications to enterprise customers without investing the significant fixed costs of large scale datacenters ala Salesforce.com. The cloud capability is likely to make heavy use of the VMware hypervisor technology owned by EMC (Amazon uses the competing technology called Xen) such that providers like SAP can define their applications as virtual appliances (or a set of virtual appliances, known as a virtual appliance network or VAN) and launch them on-demand into the cloud. ISVs such as KnowledgeTree are already doing this using virtual appliances and Amazon's cloud. Why not SAP with an EMC cloud? What is the likely scenario for enterprise adoption?

As enterprise customers increasingly virtualize their datacenter computing resources, it is going to become more and more common for them to rely on computing clouds outside of their datacenter for certain application workloads. Once you have virtualized the infrastructure and defined the applications you run as virtual appliances (i.e. they are defined independent of the computer that they run upon and are loaded on the virtualized infrastructure pre-configured with the OS), moving the application from datacenter to datacenter (or cloud to cloud) becomes much easier.

Rather than buying all the servers required to deliver the application at peak load, it makes more sense to launch new instances in the cloud during peak demand and pay a variable cost charge until the peak subsides. Similarly, batch oriented analysis that occurs once per day/week/month (perhaps closing the books and logging transactions at the end of a month) is an ideal application scenario for cloud computing with virtual appliances. Why own the servers when you only use them periodically? Is it becoming clearer why SAP and EMC might be collaborating on this type of service? It's pretty clear to me.

But what's missing? Why is this obvious scenario not already mainstream? To begin with, production implementation of virtual infrastructure is only now beginning to take hold. Historically, virtualization was used extensively in test and development or to curb Windows server sprawl (a condition that occurs because Windows cannot effectively host more than one application per OS instance). Now, enterprises are beginning to understand that virtualizing every application makes sense because it separates the evolution and maintenance of the application from the evolution of the hardware infrastructure.

For example, it makes much more sense to create a virtual appliance for an existing single-threaded application and run several instances on a dual socket, quad-core eight way machine than to attempt to re-write the application to take advantage of the quad-core threading model. There is no value in the re-write, and virtualization provides a means to scale the application to take advantage of modern hardware. Another obvious benefit of this separation of application and hardware is server maintenance. You can shutdown one virtual appliance to patch it and then spin it back up without disturbing any of the other virtual appliances running on that server. This approach minimizes application to OS conflicts, and it eliminates application to application conflicts.

If virtualized production infrastructure is a cloud requirement that is slowly becoming a reality, what other obstacles lay on the path to cloud nirvana? Well, the current most popular approach for virtualizing applications is building a working system from legacy components and then “snapshotting” it with your favorite physical to virtual (P2V) tool. This approach will not scale to cloud computing. First, it is unlikely that your cloud provider has a virtualized infrastructure that is identical to your own. Going through a manual, ad-hoc process to build snapshot images for each cloud is going to be slow, expensive, and prone to errors. Also, it is inevitable that cheap, variable cost computing is going to tempt enterprises to define ever increasing numbers of application images. However, the administration and maintenance burden of the legacy OS system layer is going to result in skyrocketing administration costs or a “don't patch it and hope for the best” security scenario. A new approach to system software that couples the application with just enough OS is a requirement to make cloud computing a scalable reality without untenable administration costs or security risks.

Finally, and perhaps most importantly, there needs to be some “context” language by which networks of virtual appliances communicate to the cloud their requirements for system resources, configuration, and service level. How does an application server communicate to the cloud to boot it on the same network segment as the database server it requires? How does a critical application that is CPU intensive communicate a requirement for 2 CPUs and 4 gigabytes of RAM? How do redundant application servers communicate a requirement to run on separate physical servers to preserve a failsafe capability?

An application architecture for cloud computing will take into consideration all of these current deficiencies. The application and all of its associated context information will be defined as virtual appliances and virtual appliance networks independent from the physical infrastructure. The general purpose OS will give way to a just enough OS approach to enable a more simple and scalable maintenance model. And there will emerge cloud release and service brokering systems for launching applications into the clouds at acceptable service level without error prone manual processes and extraordinary administration expenses. These changes are coming. What are you doing to get your enterprise on the path to cloud computing?

The Software Mission

2008-01-23T10:00:00.000-05:00

Every year as part of our operational planning exercise, we review the rPath mission statement. We want to make certain that the mission statement is still meaningful, and we want to align our annual plans with the broader objective of the company. Ideally, the alignment of the operating plan with the mission statement is good, AND the mission remains relevant in the industry (the alternative is painful). We also try to improve the wording of the mission statement for greater simplicity and clarity without altering the original meaning. It's an exercise that always sparks a lot of debate regarding the “absolute best” way to convey the mission in words.

As part of the exercise, I like to review the mission statements of other successful technology companies. Before I tell you how our exercise ended, let's take a look at a couple of the ones we reviewed:

Google – To organize the world's information and make it universally accessible and useful.

This is a useful mission statement. It is actionable (“organize the world's information”) and it expresses a specific benefit (“make it universally accessible and useful”). Google sets themselves apart by accomplishing their mission.

Microsoft – To enable people and businesses throughout the world to realize their full potential.

This statement is less specific than Google, and the benefit statement is very broad. Given that Microsoft is such a large company with so many product categories, perhaps over reaching is unavoidable. How do you describe the mission of a company that operates at such a large scale? rPath isn't that big yet, so the question goes unanswered . . . .

Salesforce.com – To create an on-demand information management service to replace traditional enterprise software technology [lifted from Benioff's bio as the company does not explicitly call out a mission anywhere on the website].

Like Google, this statement is specific and actionable (create an on-demand management service”). It also assumes that “replacing traditional enterprise software” is a benefit that is self evident. Given the success of the company, this benefit assumption seems correct - an incredible indictment of enterprise software. The summary statement for the mission,“The End of Software,” has been an effective rallying cry against the complexity and waste of the tradition software model. Overall, I like this statement. It provides the context in which salesforce.com created a new category and differentiated themselves from the competition.

Given this review, where did rPath land? Were we able to describe some noble undertaking that sets rPath apart? Is it actionable? Is it beneficial? Does it matter?

rPath - To simplify the delivery and consumption of software innovation

This is a broad mission statement, but it is actionable and the benefit is self evident – less complexity, more innovation, higher value. We considered being more prescriptive like salesforce.com – calling out virtual appliances, hypervisors, and cloud computing as an architecture to deliver software applications on-premise or on-demand without the hassles of traditional software. We decided that new architecture ideas will present themselves in the future, and being overly prescriptive today might discourage us from embracing change when it comes along tomorrow.

Does it matter? We believe there is no end in sight to the opportunity for simplifying software delivery and consumption. Salesforce.com has blazed a trail that demonstrates that customers will abandon the legacy approach when presented with something that delivers more value with less complexity. IDC estimates that in 2006 over $400B was spent on services for integrating and managing software – almost 2 times the value of the combined spending on licenses and maintenance. And this number does not account at all for the spending on internal staff, which probably doubles an already preposterous number.

If we are correct, the traditional software model is on the verge of an enormous transformation. The value of SaaS demands a response from all ISVs, but the multi-tenant, hosted architecture presents enormous technical challenges for the legacy vendors. Hypervisors replace the general purpose operating system, and ISVs abandon their multi-OS strategy in favor of virtual appliances to avoid an incredible escalation in certification costs across myriad OS/hypervisor combinations. Cloud computing offers a variable cost approach to deliver applications on-demand, relieving ISVs of the expensive burden of building out a salesforce.com-like infrastructure (Jeff Bezos calls it “muck”). And virtual appliances are the new application architecture for delivering applications to customers on-premise (via the hypervisor) or on-demand (via cloud computing).

Given the momentum of virtual appliances, SaaS, and cloud computing, I think it is going to be fun delivering on the rPath mission. Simplifying the delivery and consumption of software innovation is the focus of the entire software world these days. It's better to be in the ring when the bell sounds for round 1 of the prize fight than to be hauling water in the warm-up gym across the street. . . .

Maintenance Sucks

2007-12-20T14:53:00.000-05:00

Over the past week, I have been involved in multiple debates regarding the role of “maintenance” in establishing an effective software business. In one conversation with a key analyst at a top tier research firm, I listened to the analyst proclaim:

“Given the last 15 years of end user maintenance experience, customers have concluded that maintenance sucks. Avoid it if you can because something always breaks.”

Contrast that sentiment with Aneel Bhusri's (President of Workday) comments on a panel hosted by David Dobrin of B2banalysts:

“As a software vendor, the SaaS model is critical because it enables Workday to deliver updates and upgrades in a cost effective manner.”

If “maintenance sucks” and it should be avoided, why is Aneel so keen on efficient maintenance as a central tenant of the Workday business model? I believe I can bridge the gap between these statements by adding a couple of clarifying clauses:

“Given the last 15 years of end user maintenance experience [with legacy multi-platform server applications running on a bloated general purpose OS], customers have concluded that maintenance sucks [because, despite claims of certification, the patches from each vendor are never really tested in the exact configuration that we have deployed]. Avoid it [maintenance] if you can because something always breaks. [Unfortunately, customers that avoid maintenance create a miserable situation for a software vendor because the value of the application becomes marginalized while the vendor cost of maintaining legacy code on multiple platforms escalates]. As a vendor, the SaaS model [or a virtual appliance model] is critical because it enables Workday [or any other application vendor] to deliver updates and upgrades in a cost effective manner.”

When you pull these 2 sentiments together with a bit of editorial glue, they stick. SaaS is booming because it enables the vendor to deliver the application in a more efficient manner, and the customer gets application value without IT hassles. Salesforce.com is able to deliver their application to customers with just 8% of revenue for R&D compared with the more typical 16% that is standard for the application industry. I can only assume that Workday will achieve similar results due to the efficiency of maintaining a single code base running on a single platform for all customers.

Virtual appliances, done correctly, yield the same benefit. The vendor assumes responsibility for all software maintenance. The customer accepts maintenance (and presumably value) without hesitation because it is tested by the vendor before it is released to the customer – in the exact configuration used by the customer. The vendor and the customer get the added benefit (compared to SaaS) of flexibility regarding the network upon which the application is hosted. A virtual appliance is portable to any network that exposes a hypervisor host. The vendor can host it. The customer can host it. Or it can run on a service like Amazon's EC2 or IBM's pending Blue Cloud.

The concept must be gaining traction because the IT alarm sirens are shrieking:

WARNING! WARNING! VIRTUAL APPLIANCES ARE COMING! THEY MIGHT BE ON YOUR NETWORK ALREADY! THEY ARE BAD BECAUSE YOU LOSE CONTROL OVER THE TECHNOLOGY! THERE ARE SECURITY ISSUES! THERE ARE MANAGEMENT ISSUES! STOP THEM BEFORE USERS EXPERIENCE HOW EASY THEY ARE TO USE!

These are the same warnings that sounded regarding SaaS about 3 years ago. Anyone want to bet that the outcome will be different? My advice to the IT status quo – don't fight virtual appliances. Just like SaaS, customer value will trump IT control. Work with the emerging vendors to get what you need while giving your customers what they want -- application value without IT hassles. Maintenance doesn't have to suck.

Hypervisor Certification Crossroads

2007-12-10T08:19:00.000-05:00

My last blog post, Certification “aka Some Assembly Required,” stirred up an interesting brew of fan mail. It seems that I wrote what lots of people have been thinking when I proclaimed that certification is a weak promise at best -- a hoax at worst -- with most customers spending 6X their license costs on installation, maintenance, and administration of “certified” software. The pity of this whole situation is that software vendors are also paying a high price for the myriad of customer preferences regarding middleware and operating systems. And the price is about to get a lot steeper.

Applications vendors will soon find themselves at a crossroads. They must either embrace virtual appliances (with certification to the hypervisors as the critical element), embrace SaaS ala Salesforce.com as their only distribution approach (where customers get no choices regarding infrastructure or even features for that matter), or brace for an enormous expense uptick in certification costs when they must begin including all of the hypervisor products as additional customer preference items required for testing.

The certification gauntlet that application vendors have historically run in order to deliver their application to the widest possible market is often referred to as “the matrix of pain.” For every element in the stack that might be a customer preference – OS, application server, web server, database – and for every release of these elements, the application vendor creates a row and a column. Each intersecting cell above the diagonal represents a configuration to be tested as part of the R&D expense associated with release engineering. Of course the testing never occurs in the exact configuration of elements for a given customer situation, but some testing is better than none.

And, theoretically, each maintenance update of each element requires re-testing. Of course, no application vendor actually does this testing because they simply cannot afford to do it. As it is, certification gobbles up 40 – 50% of R&D expenditures (not to mention the customer service burden associated with customer variability). Don't believe me? Why do you suppose salesforce.com spends 8 – 9% of revenue on R&D for a pretty rich CRM application when most software companies spend 15 – 18%? Salesforce.com only certifies to ONE infrastructure – their own datacenter infrastructure. And they bring every customer along with every release of software so they do not have the testing expense associated with maintaining legacy code on new platforms or vice versa.

Now, with hypervisors providing customers with so much value and flexibility as the layer that abstracts the hardware from the application, vendors are going to be pushed by their customers to certify their applications against all of the popular hypervisors. And, with such a hot category, there will be lots of technology development and lots of releases. That means lots of NEW columns and rows for the “matrix of pain.” If you thought 40 – 50% of your R&D budget was painful because it sucked the wind out of new feature development, wait until that number climbs to 60 – 70% when you throw in another layer for the hypervisor. Add some more to those costs when customers start demanding that you also support Amazon's EC2 and other similar services like IBM's Blue Cloud.

Had enough? Ready to cry “UNCLE?” Better get busy figuring out how to deliver your application as a virtual appliance. For those virtual appliances, you will STILL need to release and test against the hypervisor products -- VMware, Citrix' XenSource, Microsoft's pending Hyper-V -- along with all of the “cloud” computing services that are going to emerge to compete with Amazon's EC2. Better choose your virtual appliance infrastructure provider wisely to accommodate these customer choices in the market (shameless plug for rPath).

No doubt customers will initially object to some limitation of choice regarding the elements that support the application when you wrap it all up and deliver it with a bow on top as a virtual appliance, but these are the same customers that are going to drive you to embrace hypervisors in your testing matrix. Give them a tradeoff – virtual appliances that are cheap and easy to install, maintain, and administer, or the old fashioned way with a $250,000 per year support uplift to cover your expense for navigating their “matrix of pain.” Standing at this expense crossroads, I bet they make the same decision you should make as you stand at the hypervisor certification crossroads today -- “Let's give that virtual appliance a try.”

Certification aka "Some Assembly Required"

2007-11-08T13:59:00.000-05:00

Red Hat announced yesterday that it intends to provide a product for application vendors to use as a basis for virtual appliances. The product will be the same as the general purpose release of Red Hat Enterprise Linux in order to maintain "certification." It will, however, have a new name – Red Hat Appliance OS. According to Red Hat, the product will be a valuable alternative to rPath because it preserves application “certification.” Apparently this means that customers will still need to assemble, configure, and maintain the components inside the virtual appliance. After all, “certification” is only valuable when the components are not provided as an integrated, optimized, and tested unit.

Given that most customers only buy “certified software solutions,” is it surprising to anyone other than me that they spend 6X the cost of software licenses on assembly, test, administration, and maintenance? If certification guarantees that things work together, why do software vendors currently spend as much as 50% of their customer service time troubleshooting the relationship between their application and whatever “certified” OS the customer has decided to use to support the application? If each release of the components is "certified," why do customers drag their feet when it comes to updates and upgrades? Maybe because "certification" does not really guarantee anything other than "some assembly required."

Fortunately for all of us, “certification” will be a thing of the past when applications companies distribute their applications as virtual appliances. Just like the auto industry, the car will come ready to drive. The notion of Toyota “certifying” the body to work with the frame, or the engine to work with the transmission, is nonsense. All the components work together because that is the definition of a car. Toyota integrates and tests all of the components from all of the vendors in a manner that best meets the needs of the customer.

Given that “certifications” and all the headaches associated with “some assembly required” will soon be a thing of the past, I thought we would take a fond look back at all of the responses that customer service reps provide customers when they run into troubles assembling the “certified” components:

Top Ten Responses to Certification Problems

10 – Re-install and call me back if you are still having problems.

9 – Can you send me a test case that reproduces that problem?

8 – It works for me.

7 – Have you been to any of our training classes yet?

6 – This is obviously not an application problem. Call the OS vendor.

5 – My shift is about to end and I am going to need to transfer you to someone else

4 – Did the sales guy talk to you about our consulting services?

3 – I'm going to need to escalate this one to engineering

2 – Your support contract doesn't cover this type of issue

1 – Take a picture of your screen and email it to me because I have never seen anything like this

The Legacy Code Triple Threat

2007-11-05T11:50:00.000-05:00

Over the past 3 weeks I have had several conversations with CTOs and CEOs of software application companies where the support of legacy systems at their customer base has occupied the top spot among the multitude of their competitive concerns. Specifically, these executives are worried about what I now describe as the legacy code triple threat:

Higher Support Costs – customers that will not migrate to new versions of an application are more expensive to support because they expect their basic maintenance contract to provide bug fixes and incremental features that do not rise to the level of new release functionality. Updating a legacy code base is much more expensive than maintaining the newer releases because the “minor updates” inevitably create regressions in areas of the code that were never intended to be maintained or updated but which now must be attended to in order to provide even the most basic of services to the customer. Providing these updates to a legacy code base also siphons off scarce engineering resources that could better serve the company by providing features in the new release that drive new license sales, which is the metric that most interests investors and provides the fuel for the business.

Lower Revenue – customers running legacy code are often unable to implement add-on modules that would provide the application vendor with new license revenue. When the IT “switching costs” of implementing a new module include development and testing costs for existing application functions to be migrated to a new code base, the customer will often say “no” to the new functionality. The possibility of “breaking” something that is “working” in order to get something “new” can be a bitter pill for many customers to swallow.

Competitive Risk – customers that face a significant cost to receive the latest and best code of their existing vendor are prime candidates for the competition to lure away to an entirely new solution. “If you are going to spend a ton of money to upgrade, you might as well consider our competing technology as part of your evaluation” is a classic pitch that works well against any vendor that lacks an elegant approach for delivering updates to their customers. SAP used this to terrific advantage against Baan in the late '90s. Customers that are a “flight” risk lead to very expensive sales and service retention costs.

All of these conversations with executive leaders at software application companies were prompted by their interest in virtual appliances as a way for them to resolve their legacy support issues. They were actually far more interested in this value proposition for virtual appliances than in the more obvious advantages associated with simplified distribution and installation. Of course, in order to realize this benefit of elegant and seamless upgrades, their approach to a virtual appliance architecture has to be much more than a simple snapshot of a running installation delivered as a virtual machine.

The virtual appliance objective for these executives is twofold:

1. Walk into the customer base and give them a working instance of the latest and greatest code running against the customer dataset as a replacement for their legacy instance. A virtual appliance where the application vendor assumes responsibility for the entire technology stack down to the hypervisor is a great way to accomplish this “forklift migration” objective.

2. Create a new relationship with the customer where the application vendor is able to easily deliver new functionality to the customer without ever again requiring a “forklift migration.”

If a virtual appliance architecture can accomplish both of these objectives, the legacy code triple threat can be eliminated. The vendor gets more revenue at a lower costs with a reduced threat of competitive replacement, and the customer gets the best technology the vendor has to offer. It's not often in the software industry that both the vendor and the customer get exactly what they want, but virtual appliances might be one of those rare magical opportunities.

A Walk in the Clouds

2007-10-09T08:13:00.000-04:00

Yesterday, Google and IBM announced that they plan to invest $30 million to promote the concept of “cloud” computing. In the coming months, I believe we will see more and more infrastructure companies follow the Amazon lead and announce or deliver “cloud” services similar to the Elastic Compute Cloud (EC2). The timing for these initiatives is right because high performing hypervisor technology from companies like VMware and XenSource (Amazon uses Xen to enable EC2) makes it possible to separate the infrastructure definition from the application server definition. The infrastructure is the “cloud” and the application server definition is a virtual appliance or a Virtual Appliance Network (VAN).

I believe the availability of high quality, low cost infrastructure provided as an on-demand, variable cost service with no restrictive assumptions regarding the application server platform (such as those required by Sun's languishing grid compute offering with its 198 page developer guide) is going to dramatically improve the quality and cost of server applications.

The quality will be higher because application companies will no longer invest huge portions of their research and development budget slogging through the “muck” (Jeff Bezos' term) of infrastructure planning (multi-platform porting/testing, SaaS enablement build-out, custom infrastructure deployment at customer sites, etc.). Instead, they can focus on application features that improve the value of the application for the customer. The cost will be lower because customers can “right scale” their infrastructure for the basic needs of their business with an ability to “expand” into the cloud for the high demand cases. For some customers, this will mean zero investment in fixed assets. For others, some mix of on-premise plus “cloud” capacity will be correct.

There are still problems to be solved, but I believe they are manageable. For example, there is currently no well defined way for a VAN of virtual servers (security, web, app, data, etc.) that form a complete application solution to be defined in a manner that makes the VAN completely portable across “clouds” while preserving the “rules” that govern the interrelationships among the various virtual appliances in the VAN. The OVF specification and the capability in the rBuilder platform from my company rPath are both early steps in the right direction, but there is still much work to be done in this area of “contextualization” and cloud portability. Eliminating the “seams” between clouds will be a key piece of work that enables the benefits of this new model.

In spite of the barriers, new solutions will still arrive. Just today, KnowledgeTree, a small, innovative company based out of South Africa, announced the availability of their on-demand offering, KnowledgeTreeLive, which utilizes Amazon's EC2 for application availability. KnowledgeTree has architected the solution using multiple virtual appliances running on EC2 in a manner that provides high availability and data protection for the customer's document management solution. Their only “infrastructure” investment was the “brain power” to design the solution. No more playing around in the “muck” for KnowledgeTree or their customers. Sounds like a “walk in the clouds” is going to be good for all of us.

Be Like Mike

2007-10-04T16:54:00.000-04:00

Based upon the title of this blog entry, and further based upon the fact that I live in Chapel Hill, North Carolina, you may be inclined to think I am referring to North Carolina's favorite son, Michael Jordan. Many of us aspire to “be like Mike” - dominant in our field of endeavor. In this case, however, I am not referring to Mr. Jordan, but instead I am writing about another famous Mike that is equally dominant in his vocation – the vaunted Mike Rosoft, the top player in the software game.

I am writing about Mike today because I was reminded of the pending growth in new application revenue that Mike is soon to realize by a just published blog entry by Bruce Richardson, the Chief Research Officer of AMR Research. It seems that Mike is about to get even more aggressive in the applications space, and he possesses a considerable competitive advantage over the rest of the field – far greater than the advantage Mr. Jordan brought to his dominance of the NBA.

You see, Mike Rosoft not only gets paid on the price of admission to the game (the application license), but he also gets paid on all of the concessions because he owns the arena where the games are played (the platform). And in his arena, he sets all of the rules of the game such that he has an enormous advantage over the competitors. He knows the location of all of the wet spots on the floor that will slip up the competition and land them on their ass, looking and feeling foolish for taking such a spill. He turns off the air conditioning in the guest locker room so the visiting team does not benefit from refreshment during halftime. He generates white noise from several unknown locations in the stands such that communications between the coach and the players gets confused. Yes, competing with Mike in his arena is a daunting task. The solution? Get your own arena. By the way, many of the best building materials are free.

What in the world am I talking about? Microsoft has a huge advantage in the applications space because they only deliver their applications on their platform. This approach cuts their R&D expense and customer service expense in half compared with application providers that attempt to deliver on every platform. But Linux coupled with hypervisor technology such as that offered by VMware is about to change all of the rules of engagement in the application sale. Application providers can wrap their application up with Just Enough OS (JeOS) to optimize application performance and deliver a complete experience to their customers in the form of a virtual appliance. No more “slipping on the wet spot” (an undocumented “feature” in the OS or a maintenance “patch” that just happens to conflict with your application), or “sweating in the overheated locker room” (watching application performance slide because the OS is consuming all of the hardware resources), or “screaming to be heard from the sidelines over the roar of the arena” (being pre-empted on application calls by a cacophony of OS cross talk). In your arena, you set the rules.

Linux and open source infrastructure provide a terrific set of building materials for your application “arena.” The hypervisor allows the customer to continue to exercise their preference for hardware and system infrastructure. Are you ready to “be like Mike?” I think you will like the way the outcome looks on your income statement.

JeOS - Product or Architecture?

2007-09-15T00:49:00.000-04:00

JeOS (pronounced “juice”) is a concept first described in writing by Srinivas Krishnamurti of VMware in this blog entry. With the pronouncement this week by Canonical that the Ubuntu distribution of Linux is now a JeOS product, I thought I would make the argument that JeOS is a packaging architecture, not an operating system product. I also believe that the hypervisor is the replacement for the product formerly labeled as the general purpose operating system.

Historically, and general purpose operating system served 2 important roles:

1) provide drivers that abstract the hardware and make the hardware resources and network available to the software applications, and

2) provide system services (file system, program libraries) to the applications such that they can execute their code in a predictable and efficient manner.

The 1st role above is now being relinquished to the hypervisor. Srinivas argues that the second role should be accomplished using JeOS. However, the actual configuration and components of JeOS, by definition, should vary considerably from application to application as each application needs different resources to operate efficiently. So, can JeOS be a product category sold in the same manner that general purpose operating systems were historically sold? As a “one size fits all” collection of general purpose utilities and libraries? Well, let's examine how the general purpose operating system is sold and determine if the same approach can be applied to the new JeOS category.

As background, I ran product marketing at Red Hat from early 1999 until midyear 2000, and I was the first enterprise sales leader for Red Hat. I was Vice President of North America sales for Red Hat from April 2001 until January 2005. During that time, we were very successful in growing sales of Red Hat Enterprise Linux by mimicking the Microsoft server OS product approach. Red Hat provided a stable release of operating system technology with a very broad set of utilities and libraries included. This release was accompanied by a commitment to five years (now seven, I think) of maintenance and regular updates for new hardware drivers, bug fixes, and security fixes.

It was also accompanied by a restriction that released Red Hat from its support obligation if the customer made any changes to the components that defined the operating system. Without this restriction, Red Hat could not possibly afford the engineering and customer service expense associated with supporting every customer's preference for utilities, program libraries, and system services. Further, the operating system itself required so many components to support so many components to support so many more components that changing any one of them often left the operating system in an unstable and unmaintainable state. This problem is often referred to by system administrators as “dependency hell.” But, since the operating system was often distributed by the hardware server providers as the mechanism to attach an application to the hardware, it was unavoidable that this restrictive model be strictly enforced to enable the industry to get on with shipping hardware that could be used by applications.

The restriction did not prevent customers from making changes to the operating system and subsequently seeking support from Red Hat. They simply absorbed the risk and much of the costs associated with the changes. If Red Hat turned them away because their changes did not conform to Red Hat's definition of the operating system, they soldiered on and did their best to make the operating system work for their applications. It never really occurred to them or to Red Hat that it might be a good idea to change the underlying packaging technology of the operating system to make it easy for the customer to change the things that did not work for them. Such a solution was unthinkable because it would leave the server hardware vendors with an undefined bill of materials. The server business is a high volume business that requires some level of standardization among the components to achieve price competitiveness through economies of scale. Changing the operating system for every server depending on the application workload it is intended to serve does not make any economic sense.

Now that the hypervisor is going to become the new operating system that supports the hardware, should the JeOS that supports any given application be a product with the same architecture as the legacy general purpose operating system? i.e. a collection of components defined by the operating system vendor as supportable and maintainable? so long as you don't change the assumptions the operating system vendor made when the collection was assembled and tested and released? so long as you don't change any of the components because that makes the collection unsupportable? How is that possible, when by definition, the collection of components that the application vendor is going to use is going to change depending on the needs of the application? Think about it.

JeOS cannot be an operating system product. JeOS must be a packaging architecture (and perhaps a testing architecture as well). JeOS must answer the question “What does the application need?” based upon the application it is presented. JeOS must then assemble around the application the components that represent the smallest possible closed loop set of components to host the application. It must then confirm that this “set” is reasonable based upon the testing scenario for the application (hence my comment above about a testing architecture). It must subsequently enable the integration of the various maintenance streams for the components with the server set to provide an elegant lifecycle experience for the application vendor and its customers. In this scenario, the application provider takes on a much broader responsibility for the support of the operating system, and at the surface level this can seem very scary.

On further inspection, however, the application vendor or their customer was already assuming this burden in the legacy model where the general purpose OS was modified on premise to support the application. With a fight through dependency hell followed by an ongoing war with the maintenance stream from the OS vendor (which was often ignored because it could not be reconciled with the needs of the application). With JeOS as a packaging architecture (instead of a “one size fits all” product), the component set that must be maintained is much smaller and much more intimately related to the application. By definition, the application vendor will be in a good position to determine and resolve problems given this tight definition and technical affinity with the application. No doubt they will still want the technical expertise of a vendor with deep operating system skills as a backstop, but the product they acquire from that vendor will look very different than the product historically labeled as a general purpose operating system. It will be a packaging architecture with a system software reference platform and a build/test methodology. Anything else defies logic in a world where every server has a hypervisor and every application arrives as a virtual appliance with JeOS.

Symantec Virtual Appliance Stymied by Windows Licensing

2007-08-27T15:22:00.000-04:00

Last week Symantec declared that it would delay delivery of its security virtual appliance due to Windows CE licensing restrictions. As part of the announcement, product manager Gary Sabala indicated a desire to “move to more open source components” in order to “ease our ability to work around some of the licensing issues.” Not surprising.

Microsoft has no reason to endorse virtual appliances as an application delivery format because they eliminate the general purpose operating system's value proposition. Also not surprising that Symantec has reached the conclusion that they need more control over the components of a solution that ships as a virtual appliance. Having your product shipments subject to the licensing whims of an erstwhile “partner” is not a good idea.

I could drone on and on (as I typically do) regarding the deficiencies of the general purpose operating system as a platform for virtual appliances, but today I will point you to some of the hits from the past that hammer home this point:

Your Application is Your Avatar – why settle for the limitations imposed on your application by the requirement to connect to a physical server?

Microsoft Attempts Software Appliances – the key word is “attempts,” as the Microsoft packaging approach for the operating system is woefully inadequate due to its general purpose operating system legacy.

Yrtsudni Erawtfos – Putting the operating system first among customer application considerations is the business equivalent of spelling “software industry” backwards.

Microsoft Seeks Pepto-Bismol Patent – Although not yet delivering applications as virtual appliances, they are nonetheless seeking intellectual property that paves the path to the future.

Now it's time to place those sales calls to Intel and Symantec. I think they might be in the market for a real virtual appliance platform. . .

VMware IPO Sizzle

2007-08-15T11:27:00.000-04:00

The financial markets have voted with their wallet, and the VMware phenomenon is for real. With a market capitalization at the close of trading yesterday of about $17B, the VMware franchise is more valuable than Sun and about 5 times more valuable than Red Hat. If you thought Linux was a hot trend, the hypervisor trend is going to be even better (note that XenSource was acquired by Citrix for $500M in the wake of the VMware IPO). But what makes VMware so valuable?

VMware offers the possibility for the technology industry to escape the miserable complexity of the general purpose operating system. Microsoft and Red Hat have established valuable franchises by offering a “standard” as the solution to general purpose operating system misery. The theory of the “standard” is that complexity is reduced by eliminating variability. The problem with this “standard” theory is the painful reality that each software application and hardware combination actually requires a different implementation of the “standard.” It is impossible to run multiple applications on a single copy of a general purpose operating system because each application requires different configurations of the “standard” in order to work properly. It is also impossible to quickly move applications to a new hardware platform because the “standard” that ran the application on the old server needs to be “upgraded” to run on the new server because the drivers are different. That “upgrade” inevitably “breaks” the application, resulting in an expensive re-stabilization process just to move the application to a new server.

In the new world where VMware is the king of the hill (and therefore collects the kings ransom for owning the hill), applications are completely de-coupled from each other and from the hardware. Each application is a virtual appliance with Just Enough OS (JeOS or “juice”) to support the application and attach to the hypervisor. Want to run multiple applications on the same machine? No problem. Each virtual appliance runs autonomously without affecting other virtual appliances on the machine. Want to upgrade the server? No problem. Fire up the new box and “vmotion” the old application over to it. Less than a millisecond for the migration. No porting. No complexity. No more general purpose operating system.

VMware is worth $17B because their technology enables the dream of utility (or grid) computing. By eliminating the operating system as a bottleneck for attaching applications to computers, applications can now run flawlessly on any computer that has the VMware hypervisor as the bottom layer -- with no expert resources required for testing, porting, stabilization, etc. How big can VMware get? What is the value of owning the operating system market PLUS the value of removing wasted administration effort associated with stabilizing applications to run on the “standard” general purpose OS? Big. Very big.

Why rPath?

2007-07-17T16:25:00.000-04:00

I have received several requests lately to comment on the relative strengths of the rPath platform as compared with other alternative approaches to delivering an appliance experience. In particular, I have been asked to contrast the rPath approach with Solaris, BSD, and Ubuntu/Debian/Red Hat/SuSE/Mandriva/(pick your favorite Linux and insert it here) Linux. I will throw Windows into the mix as well because many of the same arguments apply.

If you are going to deliver an appliance-like experience for your application, you must deliver all of the infrastructure for the customer to receive AND use the application without any of the typical hassles of installation, integration, maintenance, and administration. That means that you need to engineer a solution that eliminates all of the complexity of these hassles throughout the lifecycle of the application. Engineering that lifecycle capability can be very expensive, or it can be very cheap depending on the decisions you make regarding your platform. Let's take each element of the solution and comment on the capability of each platform.

Hardware

Solaris – if you choose Solaris on SPARC your hardware will be expensive. Perhaps you can get a deal on the first shipments, but if you are successful your hardware will be costly. If you engineer a solution that is tightly integrated with Solaris on SPARC, your switching costs to another platform will be high. The Sun sales team knows that high switching costs means NO DISCOUNTS. If you choose Solaris on X86, your hardware will be expensive for the same reason. No X86 hardware vendor but Sun works hard to assure that Solaris works well on their servers. It might work, but it might not. You will be shipping Sun hardware, and you will be paying more than you should.

BSD – if you choose BSD, your hardware choices will be better than Solaris, but you will spend more engineering cycles in validation and test. BSD does not generally have the large community of development that benefits Linux, so you will have to augment those efforts if you want significant hardware flexibility.

Linux – all of the Linux platforms will enable good driver support for X86 hardware.

Windows – Windows has the best driver support because it is the leading platform for X86.

rPath – rPath benefits from the very good driver support provided by Linux.

Installation

Solaris – Sun does not provide an installation toolkit suitable for an integrated installation of the system software and your application on hardware or within a virtualized environment. By choosing Solaris, you have already marginalized your hardware choices, and lack of installation technology just adds insult to injury when it comes to flexibility of hardware partners.

BSD – Like Sun, no real installation toolkit is available.

Linux – There are various utilities for unattended installation, but none of them is very robust nor do they provide for the configuration of the application as part of a unified installation process. If you choose Linux without rPath, you will likely settle on a single hardware platform. Switching hardware platforms will entail some incremental engineering effort - limiting flexibility in choosing hardware partners.

Windows – Microsoft does not provide any scriptable installation technology for an appliance experience, to my knowledge. They have announced 9 “server core” roles that enable specific functionality to be installed for Microsoft applications, but nothing has been announced to extend that capability to third parties. Switching hardware platforms will entail some incremental engineering - limiting flexibility in choosing hardware partners.

rPath – rPath provides installation technology that allows the application vendor to configure and brand the installation experience to be specific to the needs of the application. Installation technology is provided both for standard X86 server hardware as well as for the various hypervisors (VMware, Xen, etc.). This capability is a core competency of rPath because it provides our ISV customers with the greatest platform flexibility in delivering their application to their customers. Flexibility in choosing hardware partners is important because it gives the ISV choices for go to market strategy. A great partner in North America may not be suitable for delivering appliances in Europe. You may want to provide the customer with the ability to install the appliance on server hardware they prefer. And while hypervisor support may not seem important now, the rise of virtual appliances is inevitable. Engineering support for each hypervisor spec and for every “cloud” computer such as Amazon's EC2 will be an expensive proposition for application companies in the future. rPath eliminates these expenses by providing systematic support for all of the hypervisors and “cloud” computers.

Integration

Solaris – packaging technology for Solaris is non-existent. You will be on your own figuring out how to systematically build and maintain your tailored appliance image.

BSD – same situation as Solaris.

Linux – there are essentially 2 competing package standards that Linux projects use to create and maintain a general purpose OS release – rpm and dpkg. Both are reasonably well supported with a community of development, but neither make it easy to maintain an application specific system image. Because they were engineered to support a general purpose OS, they are both unwieldy and inflexible as platforms for integrating applications with the various system software components required for a complete appliance experience. See the Zimbra whitepaper for a comparison of standard Linux packaging vs. the rPath approach.

Windows – same situation as Solaris.

rPath – rPath's platform was engineered to support the integration of applications with just enough OS (JeOS or “juice”) to support the application. It is designed to enable the application vendor to integrate (or “package”) their application in a manner that eliminates integration hassles for the customer while also eliminating those hassles for the application vendor. Additionally, rPath provides a brandable web interface for the customer to use in configuring the application on first boot. All of the “knobs” the customer needs to “turn” to setup the application are available in the web interface. No complex, command line actions are required by the customer – ever.

Maintenance

Solaris – Sun provides an engineered maintenance stream for Solaris, but due to the lack of packaging technology (see above), Solaris maintenance will be difficult. Sun provides very little infrastructure for delivering maintenance to the ISV customer base with an appliance type experience. It is all command lines and complexity. It seems that they are planning to change it to something they describe as a network packaging system that might be more similar to rPath, but we will need to wait several months to understand what that means. And then wait several more months or years for it to be a stable, production quality product.

BSD – same situation as Solaris but without the support of a vendor.

Linux – there are a number of utilities for Linux maintenance that are associated with the various packaging technologies listed above. There is the Yellow Dog Updater (yup), the Yellow Dog Updater Modified (yum, which Red Hat uses), apt (Debian and Ubuntu), as well as commercial offerings from Red Hat and SuSE. Unfortunately, none of these is engineered to make it simple for the ISV to provide their customers with a maintenance stream that is tested and released by the ISV. They are all geared at providing general, broadcast updates to the end user base for the general purpose OS. In many cases, the maintenance is not even required by any particular application, but simply serves to keep various utilities associated with the OS secure and bug-free. The burden of maintaining a general purpose OS can be a significant one, independent of the applications running on the OS, because the OS is so bulky and bloated.

Windows – Microsoft provides a utility for receiving updates, but they do not allow the ISV to control the promotion of those updates to the customer. If an update is unnecessary, or even harmful to the application, that is simply the price you pay to receive the value of the Microsoft platform. Microsoft determines what is needed for the OS, and the applications must simply deal with the consequences. And, like general purpose implementations of Linux, the OS requires a large amount of maintenance work because it is so bulky and bloated.

rPath – rPath provides the technology and the process for an appliance-like maintenance experience. Since there is only JeOS to support the application, the maintenance requirement is reduced by an order of magnitude as compared to a general purpose OS. rPath also promotes all maintenance to the ISV instead of broadcasting it to the end customer. The ISV then manages the release of system software maintenance in coordination with application maintenance to minimize unnecessary disruptions at the customer site. Finally, rPath provides the infrastructure for maintenance delivery and installation - both the customer facing web interface (branded by the ISV) as well as the server infrastructure the ISV uses to authenticate maintenance requests and deliver the maintenance payload. rPath also provides for a robust logging and rollback capability in the very unusual circumstance that a tested, certified update goes awry. None of the other platforms offer this capability.

Administration

Solaris – Sun does not provide an appliance specific, web based user interface for Solaris and applications that might be attached to it. Sun provides great utilities for managing an operating system, but most of the functions provided by these utilities are completely unnecessary for an appliance experience. The ISV is on their own to engineer an appropriate web based administration interface.

BSD – Same situation as Solaris.

Linux – Same situation as Solaris.

Windows - Same situation as Solaris.

rPath – rPath provides a scriptable, brandable web interface to the ISV for use by the customer to configure and administer the appliance. Configuration utilities for backup, networking, user management, maintenance, and license management are all included. rPath also provides professional services to write the application specific configuration utilities, or the ISV can write those modules using the documentation provided by rPath. In either case, it is simple for the ISV to provide their customers with a simple experience for administering their application.

In summary, none of the other alternative appliance platforms has been engineered to support an ISV in delivering an appliance experience. They are all engineered as one-size-fits-all, general purpose operating systems, which are difficult to conform to the needs of a specific application. Further, because of the one-size-fits-all engineering, they are all bloated and bulky. This bloating yields inefficient application performance and creates a significant incremental maintenance burden. Until these platforms are significantly re-tooled to acknowledge the appliance market requirements, rPath is the only efficient approach for delivering an appliance experience.

Intel Invests Heavily in the Future of Operating Systems - VMware

2007-07-09T16:08:00.000-04:00

Intel and VMware announced today that Intel Capital is making a $218M pre-IPO investment in VMware. This investment would give Intel a 2.5% ownership stake in VMware, with a board seat, but no real voting rights whatsoever (less than 1%) relative to VMware's parent company, EMC. So why might Intel be making such a significant investment in VMware? I believe it is because Intel plans to influence the outcome of the investment through its X86 distribution chain.

Sales of computers, especially server computers, are currently constrained by the complexity of software. Larry Ellison estimates that for every $1 in license spending for applications customers spend $6 in integration and maintenance expense. Not only is this complexity a drag on new license sales of software, it is also a drag on new sales of servers to run that software. What if a technology existed that could eliminate the drag of integration and maintenance such that customers could spend more money on new applications and servers? Maybe Intel would be interested in seeing such a technology proliferate?

Well, VMware has that technology, and I believe Intel plans to integrate VMware's technology with it's chipsets such that applications can arrive as virtual appliances, attach to the system in a matter of seconds, and then be managed by the application provider as a complete system image instead of being managed by the customer as a set of “lincoln logs” to be assembled in a unique manner each time. With VMware hypervisor technology as the replacement for the bloated, “one size fits all” general purpose operating system, customers can actually take advantage of Moore's law instead of being held back by the lack of installability, maintainability, and portability of software applications.

How many times has the hardware sales rep been stymied in the sale of a hot new machine because the customer absolutely refused to touch the machine that was currently running their ERP/CRM/SCM/(pick your alphabet soup) application? They refused to touch it because once it was installed, configured, and stabilized no one wanted to repeat the hellish process that led to that stability.

Imagine the new world order with the hypervisor embedded at the chipset level as the replacement for the bloated general purpose operating system. You simply hook up power and networking to the new box, and then “vmotion” the system image on the old box over to the new box. And magically you now have X% more processing power, I/O, memory, or whatever you need on that fancy new box. Life doesn't get much better than that for a hardware sales guy or gal.

My bet is that you see Intel pushing their downstream server partners to adopt Intel chipsets that enable VMware hypervisor functionality at the level of the BIOS. Intel and its downstream partners such as IBM, Dell, and HP will develop system management technology along with vendors like Cisco, Symantec, and McAfee that enables “value added” services to also be delivered to the machine without the general purpose operating system getting in their way. You will see these supply chain partners develop virtual appliance catalogs that contain applications that can be added to the system in the same manner that you add songs to your iPod through iTunes. When software ceases to be a drag due to operating system complexity, hardware sales accelerate tremendously. All this for a measly investment of $218M. I think Intel got a great deal.

Microsoft Attempts Software Appliances

2007-06-23T15:21:00.000-04:00

Microsoft has been slowly revealing the roles that will be available as Server Core installation options for the upcoming Windows Server 2008. These Server Core installations are effectively Microsoft's attempt to provide a software appliance packaging option for customers in order to address the nightmare of administration and stability issues created by the general purpose OS approach (i.e. “one size fits all”). So how are they doing?

In a recent blog post, Scott Fulton replays some of the messaging that Microsoft product manager Andrew Mason delivered at TechEd 2007. According to Scott's post, Microsoft has achieved the following:

- a reduction in the attack surface of the OS from 5GB to about 1.5GB

- a corresponding reduction in the patching burden of 60%

- command line administration (although without “Powershell” - whatever that is)

- 9 server roles available sometime in 2008 or 2009, including a hypervisor role

Well, I suppose this is progress. However, if the goal is simplicity and a reduction in the burden of administration for security and patching, Microsoft still has a very long road to travel. Let's do a quick comparison with rPath's rBuilder capability:

- attack surface for “group core” is about 50MB (no, that is not a typo)

- by proxy, a reduction in the patching burden of 85% vs. the classic Windows approach (this figure does not consider at all the lesser patch requirement for Linux overall, just a straight ratio)

- command line plus graphical, Internet enabled rPath Appliance Agent administration

- infinite roles available based upon the applications generally available for Linux

As the world adopts server virtualization for X86 en masse, these critical packaging differences are going to become a huge challenge for all of the general purpose OS vendors. When hypervisor virtualization such as that offered by VMware and XenSource replaces the general purpose OS as the mechanism for managing infrastructure and attaching applications to the infrastructure (via virtual appliances), the critical packaging requirements for the OS that hosts applications will be:

- tight dependency management for the smallest possible attack surface

- configuration flexibility to optimize the system software to the application workload

- flexible kernel tooling to optimize performance across the various hypervisor management systems

- user friendly interfaces for creating and maintaining the software appliance definition

Given the technology, business, and cultural hurdles that the incumbent general purpose operating system vendors face in implementing these capabilities, it is unclear to me that they will commit to this new approach in a timely fashion. Although Microsoft acknowledges the need for these “slimmer” server core roles, they only plan to achieve a surface area reduction to 1.5GB, with limited roles, and virtually no user enabled packaging options for further optimization for the variety of applications and hypervisors. If that is all that Microsoft can achieve in the face of this obvious demand in the market for hypervisors with virtual appliances, the relevance of the incumbent players in the OS space may be diminished more quickly than anyone currently imagines.

Amazon and the CIO Nightmare

2007-06-12T11:39:00.000-04:00

If you watch the behavior of tech-savvy consumers closely enough, you will garner clues regarding the hot trends that will hit the world of enterprise computing in the next 1 – 3 years. It happened with Microsoft, as individuals brought their PC mentality from home to work to enable desktop publishing productivity long before the IT department agreed to support the PC. It happened with Linux, as computer geeks brought Linux to work to enable web infrastructure long before the IT department acknowledged the benefit of Linux. In fact, Red Hat created a $20M business selling boxes of CDs and documentation at CompUSA and Frye's before the first IT customer paid a single dollar to the company. I think Amazon is now driving the vision for the future of enterprise computing, and the digital consumer is declaring that they have got it right.

With it's Simple Storage Service (S3) and Elastic Compute Cloud (EC2), Amazon is demonstrating that it is possible to dramatically lower the cost and complexity of “web scale” computing to a price point where even a kid can get into the game. With S3, storage is only $.43/gigabyte*month, with an interface that is simple to use. Likewise with EC2, computing infrastructure costs only $.10/CPU*hour, and there is no litany of complicated rules for promoting your application to production (there are some rules, but relative to the promotion process in most enterprises, they are an order of magnitude less complex). Imagine the difficult conversation between the CIO and the CEO of a company when the CEO realizes what Amazon has accomplished:

CEO: Remind me again what we charge our business units for storage and computing in order to fund your department?

CIO: Well, for storage, we charge the business units about $12/gigabyte/month, and for computing it is about $9/CPU*hour. We re-coup about 60% of the IT budget with these chargebacks.

CEO: So I see these credit card charges from Amazon on my statement, and I asked my 14 year old kid what he was doing to rack up this bill. He tells me that he is running a wiki on Amazon where all the kids in his soccer league post information, stats, links, etc. regarding their teams. I take a look, and this site has everything, including full streaming video, forums, etc. He tells me it is $.43/gigabyte/month for storage, and $.10/CPU*hr for computing. He tells me not to worry about the charges, because they are running a hot dog stand at the games these days to pay me back for the Amazon bill. And the hot dog stand is going to turn a profit to boot. What do you think we should do about this situation?

CIO: Uuuhhhhh, I don't know? Maybe we should open a hot dog stand?

Bottom line, the CIO's that are not getting prepared to be the “Amazon” for their company are getting prepared to be a hot dog vendor. They must simplify the cost and complexity of delivering compute infrastructure to the business units by at least an order of magnitude, or the business units are going to take their business to Amazon or some other provider that “gets it.” I am not saying that Amazon has everything the enterprise needs today, but they have the right idea about what it should cost and how simple it should be to use.

The suppliers see it, because their business is threatened by Amazon. EMC is talking about it in USA Today, and Sun is promoting black box computing. The future of the datacenter is a “simple storage service” and an “elastic compute cloud.” The days of impossible complexity and “wait 6 months to get a server provisioned for your application” are definitely numbered.

Steve? Darl? All of the Above?

2007-05-15T18:30:00.000-04:00

Which of the software CEO's below expressed the following general sentiments, either directly or via their counsel:

“Linux clearly violates our intellectual property rights”

“We choose not to disclose the exact nature of the violations at this time”

“Someone is going to pay us for using our intellectual property without our permission”

a. Steve Ballmer of Microsoft?

b. Darl McBride of SCO?

c. All of the above?

If you chose option c, you would be right. Albert Einstein supposedly defined insanity as repeating the same procedure over and over again and expecting a different result. Perhaps the folks in Redmond have temporarily lost their marbles, because this tactic of claiming intellectual property violations without disclosing the nature of the violations is doomed to fail. It failed miserably for SCO, and it will likewise fail for Microsoft.

My best guess on the rationale behind this strategy is that it is so very similar to a strategy that has worked very well for Microsoft in the past. When faced with a competitive product that poses a threat, Microsoft will pre-announce a new product that addresses the threat in order to slow adoption of the competing product while they scramble to catch up. The market, faced with uncertainty, freezes for a time, thereby allowing Microsoft an opportunity to respond. Microsoft is using this same tactic with allegations of intellectual property abuse directed at the competitive threat of Linux and open source. The weakness with this approach as it relates to Linux and open source is that the timescale and the nature of the competitive forces are dramatically different. Microsoft may achieve a short reprieve in a few accounts, but the fuse has now been lit on the bomb that will change the industry.

The timescale and competitive forces for Linux and open source are very different than the timescale and competitive forces for a proprietary threat to Microsoft because the popularity of the open source works are not achieved through corporate profit motive. Given that profit motive did not drive adoption, Microsoft cannot easily "cut off the oxygen supply" by attacking issues that are related to corporate profit motive (i.e. the expense of a lawsuit).

The timescale of the corporate world is a fiscal quarter or two (perhaps a fiscal year at most), and the nature of competition is measured in revenue, bookings, gross margin, and net earnings. The timescale of a piece of community property is a function of the utility that the collective users of the property receive relative to the utility they receive from alternate property. The only way to “choke” a community property is to provide a more useful alternative (note that I did not say “free” alternative, as the value relative to the payment must simply be in the correct proportion). For many application workloads and for many types of users, Microsoft has failed to respond to the utility challenge presented by many open source alternatives. I do not believe that the current intellectual property threats will stem the tide of that failure to innovate.

By throwing down the gauntlet with allegations of IP violations, Microsoft will now be hounded by all interested parties to disclose the nature of the alleged violations in order to “clear the air.” I suspect that Microsoft might even be compelled to disclose the nature of their complaints if an interested party (such as the Free Software Foundation or Sun or IBM or Red Hat) determines to sue Microsoft under the Lanham Act and under other deceptive trade practices acts. Or Microsoft might be compelled to disclose the nature of the alleged violations to avoid losing their rights to collect rents through non-enforcement (i.e. thereby granting some sort of implicit license). In any case, the fuse has been lit and the bomb will blow. I do not believe Microsoft will like the nature of the collateral damage.

It's Amazing . . . It's Ridiculous

2007-05-11T16:05:00.000-04:00

Earlier this week I attended the Sandhill.com Software 2007 trade show, and I was delighted to hear Marc Benioff make the following proclamation during his keynote address:

“It's amazing the amount of money we spent to be able to deliver the salesforce application in a SaaS model. In fact, it's ridiculous the amount of money we spent.”

Marc is a consummate salesman, and the point he was driving home was that application providers shouldn't think about building all of the infrastructure for SaaS on their own because it is just too expensive. Instead, they should leverage the infrastructure investments that salesforce.com has already made . . . and pay Marc to run their applications on Apex. Marc has moved beyond selling the salesforce.com CRM applications to selling the Apex SaaS platform. I think this is a smart strategy for salesforce.com, but I also believe there is a cheaper and better way to deliver SaaS value that will ultimately make the Apex approach a difficult one to swallow for the application vendors.

Marc raves about the importance of multi-tenancy so often that you would think this is a feature that customers actually care about. Well, they don't. Providers care about it because it gives them a cost effective way to deliver an on-demand application. Multi-tenancy allows sharing of hardware resources across the customer base, and it also enables a scalable system management model that is cost effective in terms of the administration labor required to maintain the software (i.e. one maintenance process serves all customers). But, as Marc points out, the engineering investment to design such a capability for any given set of applications is very expensive. Also, the first investment in the underlying infrastructure to make such an architecture robust and redundant is very steep. Fortunately, there is an alternative to Marc's very expensive design for SaaS: virtual appliances.

With a virtual appliance architecture, an application provider can use their current, single-tenant application architecture while gaining the benefit of high system utilization by running multiple customer instances of the application on the same system. Virtualization is a poor man's approach to multi-tenancy which provides all of the utilization and redundancy benefits without the “ridiculous” expense of re-engineering the application for salesforce.com style multi-tenancy. In addition to being a cheap way to enable multi-tenancy, virtual appliances offer the following additional advantages over a typical SaaS architecture:

portability – a virtual appliance can run on any machine in any location, without expensive configuration and maintenance procedures. The application can run locally on the customer's network in the event that data, latency, or security issues are important.

flexibility – if an important customer requires a unique feature, it is easy to provide that feature to one customer without forcing all customers to receive the feature also. This flexibility avoids the one-size-fits-all mentality that ultimately limits the markets that can be served with the application.

I am a big fan of salesforce.com for blazing a trail to a world where software value is delivered without the technical headaches of the legacy software approach. However, I also believe that salesforce.com's architecture is a first generation approach to the problem of software complexity, and innovations in the area of virtualization and virtual appliances are going to trump the Apex architecture for other software vendors to deliver the value of SaaS to their customers. Pay close attention to what is happening with Amazon's EC2 service, the growth of VMware, and the software appliance experience being pioneered by my company, rPath. The salesforce.com approach to SaaS is just the beginning of a wave of innovations that will transform the legacy application delivery model.

The High Cost of Open Source

2007-04-16T11:32:00.000-04:00

In the past 3 weeks, I have been approached by three separate companies seeking advice regarding how to implement an open source strategy for their products. In each case, I have asked them a few simple questions:

What problem are you trying to solve for your customers by giving them access to the source code?

How much money are you willing to spend to create a broad community of development around your product?

Mostly I am receiving blank stares as the answer to my questions. For the uninitiated, open source seems to be a good way to make more noise about your product. Unfortunately, in most cases it is just noise, not commerce. In order for commerce to be the key outcome, customers need to be the beneficiary of the change, and the supplier needs to preserve the ability to charge a premium for providing more value. If not managed correctly, the reverse will occur. The ability to charge a premium disappears because customers believe “open source” equates to “free,” and providing the source code creates more problems than it solves.

Let's examine the correct answers to the questions above in order to gain some insight into the real value and cost of an open source product.

What customer problems might be alleviated with access to the source code?

Application Integration – If your customers are having difficulty integrating your application with other key applications because of weak APIs, and this difficulty is costing you lots of unbillable time in customer service, perhaps open source would help the customer (or their contractors) with integration while alleviating some of your customer service expense.

OEM Trepidation – If you are trying to convince solution providers to include your technology as part of their offering, a license that relieves their worry regarding future disputes that might jeopardize product shipments could be a good idea.

Vendor Viability – If your prospective customers are very concerned about the future of your company and your ability to service the technology, an open source license that allows them to become familiar with the technology as they consume it will lower concerns about large scale deployments that are later jeopardized by a failed vendor. Note that code escrow is often not a suitable answer to this problem because of the fear of the unknown (i.e. what awful surprise might I find in the event I need to retrieve the code from escrow).

The primary value of open source for the customer is a reduction in cost and risk. These customer values may or may not result in value for the vendor. Also, there may be some other customer problems that can be solved with open source, but they are not particularly obvious to me.

What are the costs for creating and supporting a good open source project?

Architecture for Collaboration – Software packaging and systems for collaboration become important if third parties are to become interested in participating in your project. The developers on your payroll will tolerate some packaging and system inefficiencies (they may indeed be the source of them), but outsiders will not. Count on an extra administrative overhead of 10 – 15% of your typical developer expenses for better code packaging and systems for collaboration.

Code Documentation – If your code was historically developed and maintained by a small cadre of internal developers, it is unlikely that you have spent many cycles documenting the code so that others may use it (after all, others did not use it). Plan on spending 10 – 15% of your core developers time for a year on documentation and clean-up.

Customer Service – In the near term (think 2 – 3 years), your developers are going to be the only experts on your code. If you do not do a good job responding to questions from the newly interested developers, you will never reach critical mass. Plan for at least 20 – 30% of your core developers time to be spent in customer service regarding developer issues.

In short, the near term additional development overhead for your previously proprietary product will likely be in the range of 40 – 60%. For software companies that historically spend about 20% of revenue on application R&D, count on this number rising to about 30% or higher as expenses climb and revenue potentially declines as a result of customers expecting the software to be a bargain now that it is open source (i.e. “free”). Over a period of 3 – 5 years, these numbers could improve significantly if the project catches on with a developer community and additional customers adopt the product because it has become easier to integrate and all issues of vendor viability disappear.

Open sourcing your software product is a very big commitment with an outcome that is uncertain at best. If the problem you are hoping to solve with open source is one of broader market awareness and consumption, you might be better served with a SaaS or a software appliance approach. The difficulty with most software applications is that they are too difficult to setup and maintain relative to the value they provide. By eliminating the barriers to consumption with SaaS or software appliances, you can offer the world free trials to your software that they will actually consider. With software appliances and virtualization, you don't even need to re-architect the product as you do with SaaS. The good news about this approach versus open source is if customers like the technology after trying it, they cannot get it for free, or worse, from a competitor that supports your product (witness Oracle vs. Red Hat).

Open source is no cure all. Consider carefully all alternatives and ramifications before heaving your code onto the Internet. You might not like the result.

Yrtsudni Erawtfos

2007-03-15T14:02:00.000-04:00

“Yrtsudni erawtfos” is “software industry” spelled backwards. This ridiculous word is the image that popped into my head when I reviewed Red Hat's exchange concept. According to what I have read, Red Hat intends to sell and support third party, open source business applications. In my mind, this is akin to Emerson Electric announcing that they intend to sell and service Whirlpool washing machines, dryers, and refrigerators. Why would a consumer buy a Whirlpool appliance from a manufacturer of appliance motors?

I understand the importance of the motor to the appliance function, and it is my impression that Emerson engineers motors that are quiet, energy efficient, and reliable. The only reason I can fathom for someone wanting to purchase the Whirlpool appliance from Emerson is because their prior experience with appliances leads them to believe the motor is going to be noisy, inefficient, and prone to breakdowns. And by God they want Emerson on the hook to be responsive when the inevitable breakdown occurs.

If Red Hat is truthful when they say that RHX is what their customers have asked them to deliver, it is a sad indictment of the software industry in general and of the operating system category in particular. Have customers been so conditioned to expect technical breakdowns in system software that they now want the OS vendor to be the primary point of contact for application support? I think it is likely that this is the software buyer condition, but I believe Red Hat's proposal of “more, better technical support” is the wrong answer.

I believe it is time to set the software industry straight. The operating system and related system software should not be the center of attention for software customers. Customers should have higher expectations for things to work correctly WITHOUT “more, better technical support.” I doubt Marc Benioff is sitting around today asking himself:

“Gee, I wonder if I got it right? Maybe what customers really want is for Red Hat and Oracle to sell and support my application products. After all, they do provide me with two very important pieces of the salesforce.com infrastructure? When things go wrong, the customer can call them instead of calling me. Maybe I should try that model.”

No. Marc is right when he says customers should not be exposed to the hassles of software infrastructure. SaaS and software appliances are both better answers to the software dilemma than Red Hat's proposal for “more, better technical support.” Customers don't want more support. They want software that provides application value without hassles. The perfect software product is one that requires an absolute minimum of technical support, and now is the time for application vendors to step up to that challenge.

Black Box Computing

2007-03-12T15:23:00.000-04:00

I remember reading about Sun's Black Box project a few months back and scratching my head and wondering “Why would someone want a tractor trailer load of proprietary hardware with an air conditioner attached?” With the emergence of Amazon's EC2and S3 services, the growing pervasiveness and promise of virtualization, and Sun's further speculation that the world will only need 4 – 5 computers, I think I am beginning to understand why Sun wants to be synonymous with black box computing.

In the future, what is in the box (i.e. the hardware and its OS) will be irrelevant because your applications will not be defined by the systems that they run atop (as they are today). Applications will be defined as software appliances, and they will run atop the black box that offers the best service level/price performance ratio at that particular point in time. The best service level could indeed be influenced by proximity to the point of consumption (latency), security of output (network access characteristics), performance capacity (throughput per unit of time), or price capacity (throughput per $). Applications will be completely portable due to virtualization and software appliance technology. Compare the promise of such a future with an anecdote regarding the burden of our past experience.

Early in my tenure as the leader of Red Hat's enterprise sales effort, I once asked a large enterprise customer how they defined the systems to be deployed to support a given application. The answer that came back in an absolute deadpan was:

“If it is to be deployed in less than 6 months, we look under the developers desk to see what he is running, and we go buy as many of that exact system as meets the projected application requirement.”

I bet there is still quite a lot “developer system cloning” by procurement and IT staff going on today. I also bet that there won't be much of it happening a few years from now because the system that runs the application will be a black box, and developers will define their applications, sans hardware, as software appliances.

Licensing Hullabaloo

2007-03-01T14:15:00.000-05:00

I love using the word “hullabaloo.” I don't get to use it often, but it is a great word to describe the cacophony of voices currently debating virtualization licensing practices.

It began with a New York Times article by Steve Lohr in which he described the competitive challenge that virtualization (and VMware) poses to Microsoft's operating system business. Then VMware posted a white-paper describing the licensing practices and their potential negative impact on customers. And Microsoft responded with a statement from Mike Neil, the General Manager of virtualization platforms for Microsoft. No doubt the analysts will begin weighing in on the matter soon. Why all the buzz? Why now? Because the impact of virtualization on the software industry promises to be similar to that of the Internet, and everyone wants to find a control point from which to steer their fortunes.

Virtualization is disruptive because it fundamentally changes the relationship between the hardware, the operating system that runs on the hardware, and the applications that run on the operating system. The hypervisor (virtualization platform) becomes the new layer on the hardware, and the operating system simply becomes an extension of the application as part of a software appliance. If the hypervisor becomes the “new” standard operating system, and any application can run on the hypervisor as a software appliance, it stands to reason that Microsoft and the other big operating system vendors such as Red Hat would be threatened by this change because they lose their point of control – how applications get attached to computers.

But what does all this have to do with licensing? Historically, there has been a strong correlation between the number of physical computers owned by a customer and the number of operating system licenses that a customer required. With virtualization, it is conceivable that every application is a software appliance with its own operating system attached. Moreover, these software appliances might come and go on the network based upon demand for the application. For example, a payroll application might run for a couple of days every month, but otherwise, it is not needed. With software appliances, the payroll software appliance would be deployed to a computer (atop the hypervisor) to run during the days before payday, and then be removed from the machine to make room for other applications to run more speedily during the rest of the month. Should the customer pay for a “full time” license to the operating system that is inside the payroll software appliance? Or should they have a “part time” license that more closely reflects the manner in which they use the payroll software appliance?

There is a lot at stake here for Microsoft. If the hypervisor becomes the “full time” operating system on the hardware that enables the “part time” software appliances to arrive and run as needed, Microsoft surely wants that hypervisor to be a Microsoft product, not one from VMware. If the operating system in the software appliances is simply an extension of an application, Microsoft may experience price erosion due to the minimalist nature of the operating system and the “part time” usage scenario.

In any case, Microsoft has a right to license their technology in any manner they see fit, so long as it does not break the law. If the licensing is not in the best interest of customers, customers will vote with their wallets and seek alternatives. Best of all for me, it gives an opportunity to use a great word like “hullabaloo” to describe the ruckus.

Certification Tax Protest

2007-02-15T15:58:00.000-05:00

Has anyone other than me ever wondered why software integration and administration is so expensive considering that all the components are supposedly "certified" to work together? The conservative estimate of the ratio of administration expense to new license expense is about 6:1 -- it costs you 6 times as much money to integrate and maintain the "certified" parts as it does to buy the parts themselves. Let's put this "certification tax" into perspective with an example from the auto industry.

Assume that you buy a new automobile for $40,000, drive it for 100,000 miles, and then sell it for $10,000. The cost of the "new license" for that car was $30,000 over a useful life of 100,000 miles. Conventional wisdom regarding the expenses for gas, oil, tires, maintenance, etc. for an auto is about $.25 per mile, or $25,000 for 100,000 miles. Therefore, the ratio of administration expense to "new license" expense for an auto is roughly 1:1. In order for a car to have the equivalent administration to license expense ratio of the software industry, we would need to increase the price of gasoline from $2.50/gallon to about $30/gallon. This $27.50/gallon difference is the tax you pay for driving a car that you assembled from "certified" parts. Imagine how different the auto industry would be if there was a “certification tax” on gasoline of $27.50/gallon? The auto industry would be tiny.

Now, imagine how much bigger the software industry could become if we could eliminate the $27.50/gallon “certification tax” that represents the administrative burden of assembling and maintaining components? Of course, there is nothing wrong with the PROMISE of certification, it is simply that the REALITY of it really stinks. Instead of a loud, technical guarantee of interoperability, it has devolved to a tinny, marketing whisper of “well, it ought to work,” or “it works for me,” or “cover your ass -- buy certified.” Maybe it is time for a new approach that truly lowers the administration costs.

Fortunately, the promise of software appliances and virtualization does just that. The application provider delivers the actual “automobile” instead of a “pallet of parts,” and provides a Lexus type maintenance experience – taking full responsibility for the outcome. Support becomes a latte in the lobby while you wait for your lube job instead of frequent rides in the front seat of the tow truck after each break down among the “certified” components. Which would you rather be doing? Sipping latte? Or continuing to pay the “certification tax” with little reward other than your first name relationship with the AAA call attendant?

Microsoft Seeks Pepto-Bismol Patent

2007-02-02T16:05:00.000-05:00

It seems that Microsoft may be searching for some relief from the pain and bloating of the general purpose operating system. A June 8, 2005 patent filing by the Redmond giant describes, among other things, the following:

QUOTE: Operating systems are available as monolithic blocks that incorporate all the above functions and often more. Purchasing these large operating systems can represent a significant portion of the cost of an overall computer system. After purchase, the user may find significant functionality available that is not required, or some functions that would be desirable that are not available.

TRANSLATION: General purpose operating systems often have lots of features that are not useful and in fact often prevent useful applications from being deployed due to conflicts.

QUOTE: The small basic kernel, may be used for simple operations and for basic application support. A user of the computer system may then have the opportunity to add specific modules supporting the functionality required for his or her particular interests. While some add-on modules may be free, others may be available for a fee or as part of a subscription.

TRANSLATION: Instead of a general purpose operating system, we think the future is about a simple hypervisor that talks to the hardware, and new applications arrive as optimized virtual appliances.

Microsoft has always been able to catch up quickly just when it seems the technology future might pass them by. It appears that they have been quietly planning for the end of the general purpose operating system for a couple of years already. Pass the Pepto Bismol and get on with the relief from the pain and bloating.

Marc Knows Marketing

2007-01-28T22:06:00.000-05:00

In a recent interview with Victoria Murphy Barret of Forbes, salesforce.com CEO Marc Benioff elaborated on his marketing prowess. And indeed Marc has done a great job selling the vision of salesforce.com. His company's stock carries the highest multiple on revenue and earnings of any comparable company in its category. Possibly in any category. But despite all of this success (and I am a big fan of salesforce.com), in the final analysis, it remains to be seen whether or not Marc knows economics.

I heartily agree with Marc that the application software business is broken. I disagree, however, that on-demand applications are necessarily the best way to cure what ails the industry. I believe that software appliances may prove to be more economical than Software as a Service (SaaS).

Just 2 weeks ago, I was in the office of a very successful senior executive of an application software company. He was very bullish on their use of salesforce.com, and he was going to give me a demonstration of the application. Well, the pages were too slow to load for the demonstration to be effective, so we skipped it. In all likelihood, it was not a salesforce.com server problem, but instead some latency problem on the Internet at that moment in time. It got me thinking that high speed LAN technology is much cheaper and much more reliable than the comparable Internet connection.

Consider that you can deliver 100Mbits wireless connectivity to an entire office with a couple of Linksys routers that will run you a grand total of about $100. These are solid state devices that never break. What would 100Mbits dedicated connection to your SaaS provider cost you? Thousands per month, or more.

"No matter," you say, "A little latency is a small price to pay to be shed of all of the misery of integrating applications with infrastructure and managing the resulting snarl of code and maintenance from multiple vendors that don't seem to communicate with one another." Indeed, the legacy approach of integrating and maintaining an application atop a general purpose operating system lacks the elegance of an engineered solution. But what if an application was as easy to use as a song downloaded from iTunes onto your iPod? That day may be coming sooner than you think.

With a software appliance, the software vendor integrates the application with just enough operating system to support the application. A browser interface is provided to configure the application, just like with SaaS. Installation of the application is as simple as downloading a file onto a server, because server virtualization technology from a vendor such as VMware turns the server into a "player" like an iPod. Have you ever had one song "crash" another song on your iPod? Of course not. When you load your servers with virtualization as the base layer, you get the reliability of the iPod player for applications that are delivered to the server in much the same manner as songs are delivered to the iPod.

The result? An application with zero integration and maintenance hassles (the application vendor does all that work for you), with a browser interface, running on a server that costs less than $2000, delivering high bandwidth output to everyone in the office (and equivalent bandwidth to everyone outside the office as any other SaaS application).

I think I'll take Marc's lesson in marketing, and stick with the economics of the software appliance approach to build my business.

Virtualization is a Poor Man's Multi-Tenancy

2007-01-21T20:35:00.000-05:00

A few years back I recall reading several analyst comments about how Siebel was going to eat salesforce.com's lunch just as soon as they could get that sucker online. I just laughed. I laughed because I had some experience with both salesforce.com and with another application provider who "put their app online," and I understood thoroughly the difference between an application that is architected for the Internet and multi-tenancy, and one that is not.

My experience with salesforce.com was both as a user (Red Hat was a very early customer in 2001)and as a supplier (my team called on salesforce.com as part of our Linux sales effort). I knew that salesforce.com had a pretty lean infrastructure due to multi-tenancy (serving lots of customers with just a small number of Linux servers). I also knew that they designed the application and the interface to tolerate the uncertainties of the Internet. By contrast, I had witnessed several cases where the ERP/CRM vendor simply put up a couple of machines dedicated to serving customers from their data-center. It was rarely successful, and the applications typically came back in house quickly. These legacy applications simply were not designed for the uncertainties of the Internet. Nor did they have a multi-tenant design to serve many customers from a set of shared systems (efficiency being the key to the SaaS model).

Well, part of that problem may be going away soon. High performing virtualization coupled with virtual appliances that are tuned to be efficient (i.e. no overhead from a bloated, general purpose OS) may provide the first hope of multi-tenancy for legacy applications. Add the soon-to-be-introduced quad-core processors from AMD and Intel, and you may just have a recipe for efficiently serving lots of customers from a common infrastructure. Add Amazon's EC2 and S3 services, and you may not have to endure fixed costs at all to effectively serve your customers with an on-demand version of your application.

None of these developments solve the latency problem of the Internet, but they do promise to make the costs of entering the SaaS space much lower for software application companies. Separating the infrastructure from the application architecture will be a very good thing for application companies that are looking for ways to better serve their customers and grow license revenue.

The Traditional OS Goes Away

2007-01-17T15:33:00.000-05:00

I have said it before, but someone else said it this time. Tony Iams, senior analyst with research firm Ideas International Inc., said it in an interview with Patricia Pickett of eChannelLine. Here is the money quote:

"The traditional OS that we know no one loves basically goes away and becomes part of the application itself. The application and OS code are combined and wrapped up in a virtual machine and shipped."

Looks like Tony is now a student of the virtual appliance university. Welcome to the club, Tony. I'll send you the "No OS" pin you see at the top of my blog site.

IDC Predicts Software Appliances

2007-01-03T18:12:00.000-05:00

In mid-December, IDC released its 2007 predictions for infrastructure software. The number 2 prediction on the list is:

"Software appliances will become a household word in 2007. The convergence of virtual machine technology and a new initiative by several tool vendors is giving birth to this new form of software packaging."

Just 12 months ago, the concept of a software appliance was a very nascent trend when Steven Hamm wrote an article in BusinessWeek describing the concept. VMware had just decided to give the player away for free, and they were offering 3 software appliances from their site as demos. The WikiPedia definition for Software Appliance was created in December of 2005.

Now, in January of 2007, IDC has predicted household familiarity is coming soon. VMware offers more than 350 virtual appliances from VMTN. rBuilder Online now has over 2400 software appliance projects, with over 1000 software appliances available as downloads. And Gartner has predicted that Windows Vista will be the last of the monolithic operating system releases because the software appliance approach will make the general purpose OS obsolete. What a difference a year makes.

Why should all the folks in the server application business care about this trend? Because software is too complex, the ratio of services to license sales (6:1) is ridiculous, and license sales have become stagnant due to the excessive cost of deploying and maintaining software solutions. Software appliances coupled with virtualization offers the possibility of market expansion and license sales growth through dramatic simplification of deployment and maintenance. Yes, SaaS is also a fine answer, but it requires a total architecture overhaul and operational expertise that most software companies do not possess. Software appliances are a better answer for most server application vendors.

Want more proof? Read Digium's latest press release about their software appliance, AsteriskNOW. Simplification, market expansion, a better solution for customers. We should all be experiencing the bliss that Digium feels at this moment.

BEA Delivers Software Appliance Platform

2006-12-19T21:53:00.000-05:00

Last week in Beijing, BEA announced that it intends to deliver a platform for running software appliances atop VMware's virtualization technology. The software appliance approach eliminates the hassles of maintaining a general purpose operating system, and when coupled with virtualization, virtual appliances are delivered pre-configured and ready to run on the hypervisor.

For any application that is 100% pure Java, BEA's Liquid VM provides a great way for an application vendor or application developer to deliver an application from test straight to production without worrying about incompatibilities or conflicts with the operating system. The company claims a reduction in deployment time from 45 minutes to mere seconds. Performance is also supposed to be significantly higher.

It is likely that both of these claims are absolutely correct. Virtualization offers the possibility of streamlining the deployment and management of applications by eliminating the general purpose operating system. When coupled with the hardware acceleration in the latest chipsets from Intel and AMD, virtual appliances should also run faster than native applications because much of the baggage of the general purpose OS is eliminated. And given that there are fewer components and services loaded into memory, virtual appliances should also consume far fewer system resources.

BEA's announcement is one more proof point that higher server utilization was just the beginning of the benefits of server virtualization, and the best is yet to come.

Support is NOT a Software Business Model

2006-12-10T22:59:00.000-05:00

I shudder every time I read a blog post or article by some “expert” that proclaims that open source is a “business model” predicated on providing customers “good support” and that open source is fundamentally different from proprietary software. Hogwash.

Open source is not a business model, it is a development model. The software business, open source or not, is about providing customers with a product that is better than the competing product. This concept is fundamental to business, and yet somehow a bunch of young software companies have it in their head that there is something new and magical about open source that exempts it from this fundamental truth. Maybe that is why there is only one successful software company underpinned by open source -- Red Hat. And if you look at the financials of Red Hat, it looks like a great software company, period. No “open source” modifier required.

Let’s do a quiz for the “experts” and see how much they really understand about building a great software company based on open source:

Who spends more money as a percent of revenue on “support and service”? Oracle or Red Hat? It has to be Red Hat, right? They are all about “support and service.” Wrong. Oracle spends 23% of revenue to “service” the customer while Red Hat spends only 18%. I guess open source isn’t necessarily just about selling “support and service” after all.

Who spends more money on engineering their product as a percent of revenue? Oracle or Red Hat? It has to be Oracle, right? Red Hat gets so much R&D leverage from the community. Wrong. Red Hat spends 15% of revenue on R&D while Oracle spends only 13%. I guess customers want more than just a download re-direct from redhat.com to kernel.org.

Who spends more money on sales as a percent of revenue? Oracle or Red Hat? It has to be Oracle, right? Selling really expensive proprietary software has to cost more than selling open source. Wrong. Red Hat spends 47% of revenue on SG&A while Oracle spends 25%. I guess there must be more to selling “free software” than putting up a download site and setting up a PO Box to collect the checks that roll in to turn on support.

A great software business is a great software business, independent of open source. Furthermore, customers will pay a premium for great software, if they cannot get the same great software cheaper from somewhere else. Therefore, if you have a great software business based upon high performing software that is only available from you, there is little reason to open source your product. The corollary – if you have a non-existent software business or mediocre software that cannot command a premium, you are probably a candidate for open source. This basic truth could sink open source software as a category because so many young companies are using the term “open source” like a badge of honor without paying any attention to the fundamentals of the software business.

Red Hat and Oracle are BOTH great software businesses because they invest in engineering and sales in order to deliver a premium product to their customers. Support is a bad business model for software because it misaligns the customer and the vendor. Customers don’t want to pay for support or services, they want software that works WITHOUT support. Vendors that generate revenue from support only scale their business if the software is buggy and difficult to configure – driving support calls/incidents/whatever in order to scale revenue.

The biggest problem I had while running sales for Red Hat was overcoming the customer objection that Red Hat’s software should be very cheap, or that Red Hat’s value should be based upon how much “support” the customer consumed (incidents, callers, whatever). I responded by banning the term “support” from the sales lexicon, setting high quotas and demanding million dollar deals, and requiring the reps to sell on the value of solving a difficult problem – providing Unix functionality on industry standard hardware with a 10X price performance advantage. Additionally, Red Hat moved to the RHEL model where the enterprise product was only available if you agreed to pay a premium for it.

Guess what happened? We sold the value of 10X price performance vs. Unix, and the reps took down million dollar deals and made their big quota numbers and then some. Customers stopped talking about “support” and “free software” because we convinced them that engineering is what really matters . . . . and they could only have the product if they paid for it. If you want to be successful in the software industry, go and do likewise.

Your Application is Your Avatar

2006-12-03T22:52:00.000-05:00

Given that “higher system utilization” is the most commonly cited server virtualization benefit, does anyone other than me find it funny that none of the benefit language surrounding Second Life relates to “utilization.” Why is it that we do not hear about the unleashing of formerly idle cycles when one enters the virtual world of Second Life? Why is no one talking about the greater level of productivity achieved when they are able to lead several parallel lives in the virtual world? Is it simply that the business value of computer servers is far removed from the fantasy value of avatars? Or is there something that the server virtualization industry can learn from the popularity of Second Life that will inspire a higher bar for value than the pragmatic, but boring, benefit of “utilization?”

In a recent interview with Zdnet’s Dan Farber, Mendel Rosenblum, chief scientist and founder of VMware, hypothesized the following:

“The biggest misconception [about server virtualization value] is server consolidation [i.e. higher system utilization]. It was the first big application in the enterprise. Part of the problem with being the first successful application is that it misses a lot of the benefits. A virtualization layer is a different way of thinking about hardware, and server consolidation is just one tiny sliver of it.”

I believe Mendel is correct, and I believe we can take a lesson from Second Life to provoke new thinking about what we ought to be attempting to achieve with server virtualization.

In my opinion, the popularity of Second Life stems from the freedom and creativity that is possible when the collective constraints of physical and social boundaries are left at the threshold of the virtual world. Second Life residents are only constrained by their ability to project their imagination into their avatar and their environment, and their ability to represent that projection to others in the virtual world. If freedom and creativity are the dual engines powering Second Life’s popularity, when will these benefits begin to be realized through server virtualization?

The reason that freedom and creativity are not yet the force behind server virtualization is because developers are currently dragging all of the baggage of the legacy world into their virtual machines. How popular would Second Life be if the first step in building an avatar was to do a full and exact body, social, and intellect scan of the player in order to project that exact image into the avatar? And all of the rules and environment of the game were exactly the same as the rules and environment of the physical world? It doesn’t take a genius to figure out that Second Life would be a colossal flop if these were the ground rules. It’s time that developers took charge of their applications in the virtual world and leave the constraints of the legacy server world at the threshold of the virtual environment.

The first step is to get rid of the bloated, general purpose operating system. If you are a developer, your application is your avatar. Its characteristics are unique to the purpose you are attempting to serve with its existence. Its libraries, system functions, and resources should be uniquely crafted to fulfill its mission, without regard for the libraries and functions used by the avatar next door. In the virtual world, your avatar can live side by side with vastly different avatars with never a worry about a server crash associated with conflicts over shared resources.

So why are you using all of the stale stuff provided in a general purpose operating system instead of branching out to create the perfect avatar? It’s akin to getting the girl of your dreams to say “yes” for an outing, and then wearing clothes from your dad’s ‘70s wardrobe, splashing on his Old Spice aftershave, driving his 1974 Buick Electra 225, and using all of his corny lines as your rap. Why would you do that? You wouldn’t do that, yet you accept the technical equivalent of this dismal scenario with the way you are building your virtual appliance – your avatar.

Don’t settle for that weak rap. Virtualization gives you the freedom to express your application creativity. Arm your avatar with the good stuff, and forget about what the general purpose OS neanderthals say you should use. Clip-on ties, Old Spice, and Buick just won’t get it done in the virtual world of tomorrow.

Amazon is Utility Computing

2006-11-13T10:26:00.000-05:00

The cover story for last week's BusinessWeek magazine described how Amazon is launching a set of new technology services that enable other businesses to use Amazon's computing infrastructure. The tone of the article was a bit skeptical, but we use the services here at rPath, and we like what we see. We are particularly excited about the Elastic Compute Cloud (EC2) service because it is the first large scale utility computing offering to come to market based upon Xen virtualization technology.

Our first experience with Amazon's computing services was when we migrated all of our virtual appliance images from our NAS at our datacenter site to Amazon's Simple Storage Service (S3). We transferred 1.7 terabytes of images (it has since grown to over 2 terabytes), redirected all of our web services that control access to the images, and our rBuilder Online users began receiving their disk images from Amazon instead of from rPath. Our first monthly bill from Amazon was about $300, which includes the cost of the bandwidth to serve the images. We eliminated the need to purchase a very expensive NAS disk array (about $80K), and freed up our existing NAS storage for other uses. Amazon is amazing.

EC2 promises to be even more interesting than S3 because it has historically been very difficult to offer CPU capacity in a utility model due to the inflexibility of the general purpose operating system. Unlike file storage, where it is easy to decouple the application from the data due to well codified access protocols and file system standards, CPU services have been hamstrung by the lack of standards for application system services at the level of the operating system. For example, in order to use the Sun Grid Compute Utility, your application must be ported to run natively on Solaris 10 and be scripted to work with N1 Grid Engine Software. And the service cost $1/CPU-hr. With EC2 from Amazon, your application can run any Xen enabled, x86 guest OS at any version level. And the service costs just $.10/CPU-hr. No operating system constraints and one-tenth the cost. Another bummer for the folks at Sun. . . .

The Red Hat Paradox

2006-10-29T20:29:00.000-05:00

When Oracle announced its entrance into the general purpose operating system market, I was most surprised by the endorsements provided by Red Hat’s partners. AMD, Intel, IBM, NetApp, HP, Dell, and several others endorsed Oracle’s move into the space. On the surface, this endorsement would seem to indicate that Red Hat has done a poor job meeting the needs of these partners. If you dig a bit deeper, however, it is inevitable that any general purpose operating system company will eventually fail their partners and customers if they are successful in the market.

How to explain this paradox of failure through success? Let’s examine the economics of the general purpose operating system business. The goal is to sell as many copies of the OS while spending the least amount of money engineering the OS. The easiest approach to maximize sales while minimizing expense is to engineer a single, one-size-fits-all implementation of the OS targeted at the sector of the market that is willing to pay the most for such an OS. The benefit of this approach is that it is possible (with a bit of luck, some good engineering, and a bit of a lead at the beginning of the race) to pull off the holy grail of software and create a de facto “standard.” The downside is that no one gets what they really want.

The reason that no one gets what they want is because the OS becomes a huge compromise - the lowest common denominator across all requirements. Even mighty Microsoft struggles with this issue of attempting to serve all requirements with a single, one size fits all, product. The period between releases of products grows and grows. The amount of new features that distinguish the OS shrinks and shrinks. So much effort goes into simply trying to make all of the old programs and drivers continue to work with all of the new programs and drivers coming into the market that the OS schedule and features become casualties of this work. Another casualty is the relationship with your partners and customers.

When attempting to make everyone’s drivers and code work with a single release of the OS, no one’s drivers and code work really well. When the schedules and features slip, your partners’ businesses are impacted. Your customers planning schedules are impacted. No one is happy. Which explains why Red Hat appears to be a victim of its own success when all of its partners stand up to endorse a competitor such as Oracle.

So what is the answer? Well, I don’t think the answer is Oracle using the same technology and the same approach to the market and promising lower prices. Nor do I believe the answer is for Red Hat to simply “try harder.” Something needs to change. I believe the change will occur when pervasive virtualization technology splits the operating system into the separate categories of hardware OS and application OS. Then, the only standard that needs to be maintained is the virtual machine format. Each application can arrive as a virtual appliance with its own unique, tailored OS wrapped inside a standard virtual machine. All applications work seamlessly together on the same machine without the requirement for a general purpose, one-size-fits-all operating system.

The signposts of this change are already there if you care to look for them. VMware’s revenue and growth rate are off the charts. The hype surrounding Xen is deafening. Microsoft just announced they are opening up their virtualization spec in an attempt to avoid falling further behind VMware. And the virtual appliance hype has gone from non-existent to a continuous, ever louder thunder roll in a matter of only 9 months. And, finally, all of Red Hat’s partners stood up to endorse Oracle as a Red Hat competitor because they believe Red Hat should have done a better job serving them. It’s time for all of us to take responsibility for this problem and move to a better model. A model where everyone can have what they want, and the operating system providers can be successful without failing their customers or their partners.

Oracle Offers Linux

2006-10-26T00:12:00.000-04:00

Today at Oracle World, Larry Ellison announced that Oracle would offer Linux maintenance and support services. This offer supports Charles Phillips' earlier keynote message (in reference to software generally, not Linux specifically):

"We want to change the ownership experience completely. We are trying to systematically reduce risk, systematically reduce cost, and improve the experience of owning software."

We will all have to wait and see if Oracle emerges as a credible provider of a general purpose operating system. They are clearly the market leader for general purpose database technology, so they definitely have the skills to handle complex technical products. However, Oracle has not distinguished themselves in other similar general purpose infrastructure categories. In the case of application servers, Oracle mostly provides technology in support of their application deployments. How many deployments of Oracle application servers exist absent a deployment of Oracle applications?

There are also many unanswered technical questions regarding how Oracle will provide Linux maintenance. Oracle claims they are going to follow the Red Hat maintenance stream, but they also claim they are going to issue patches to customers independent of Red Hat. When Oracle patches a system for a customer, and Red Hat subsequently provides a patch that conflicts with the Oracle approach, how will the conflict be resolved?

Red Hat systems rely on a maintenance packaging approach (the Red Hat Package Manager or rpm) that assumes a single maintenance stream from a single vendor. The naming convention for rpm combined with the approach for handling dependency metadata will make it very difficult for Oracle to insert themselves into the maintenance process in a manner that provides the best Linux technology offered by both Red Hat and Oracle separately. It is more likely that a customer move to the Oracle maintenance stream will be a permanent move because the resulting system will quickly become incompatible with Red Hat maintenance. Perhaps the Oracle system will be better. We will have to wait and see.

Independent of the technical challenges that Oracle’s approach will present for customers, Oracle’s embrace of Linux demonstrates the power of open source as a competitive weapon. Any company can use Linux to serve their specific application or business purpose. Here are just a few examples of companies that provide Linux to their customers in support of a complete software experience:

- Google provides Linux as part of the Google mini search appliance

- EMC provides Linux as part of its Centera content addressed storage solution

- Cisco provides Linux as part of its Unified CallManager product

- Agami Systems provides Linux as part of its Information Server to achieve record breaking NAS throughput

- Digium provides Linux as part of their Asterisk Business Edition VOIP solution to enable hassle-free deployment and maintenance by their customers and channels.

- Ingres provides Linux as part of their project Icebreaker database software appliance.

These companies all provide Linux in order to deliver greater value to their customers. Oracle has now decided that they also want to ship Linux in support of their unique business objectives. The lesson from all of these examples is the same – the freedom of Linux can provide competitive advantage. How will you use it to improve your business?

Microsoft Responds to Virtualization Threat

2006-10-18T09:55:00.000-04:00

At risk of falling farther behind in the increasingly important virtualization race, Microsoft indicated yesterday that it would open up its Virtual Hard Disk format under the terms of its Open Specification Promise. VMware opened its VMDK specification almost a year ago, and they continue to barrel forward, delivering $188M in revenue in the current quarter - up 86% from last year. I suppose Microsoft has determined that they cannot set the agenda with a proprietary format when they are so far behind such a significant competitor.

Why should anyone in the open source space care about this announcement? It is important because it will enable software vendors to build Linux applications that run on Microsoft systems. When all Microsoft servers provide virtualization services via VHD, a Linux software appliance can install on the system by copying a VHD file onto the system and starting the VHD service. The software appliance simply needs to be "wrapped" in a VHD container, thereby creating a virtual appliance. Any utilities, such as failover, SAN connectivity, etc, that Microsoft provides via the VHD service will now be available to Linux based software appliances. The OS that runs the hardware is now de-coupled from the OS that runs the application.

Why should software vendors care about this announcement? Currently, software vendors spend far too much engineering expense, up to 50% of the R&D budget in some cases, on multi-platform porting, maintenance, QA, installers, customer service, etc. With virtualization services as the interface between the application and the "hardware" operating system, ISVs can simply do a single Linux port to create a Linux based software appliance that will plug-in and run in any customer environment. Because the "application OS" is de-coupled from the "hardware OS," the ISV is in total control of their environment without sacrificing any market share associated with customer platform preferences.

Why should customers care about this announcement? Customers will be able to dramatically reduce the expense of software deployment and maintenance when applications are provided as virtual appliances. The burden of integration, testing, and maintenance shifts to the application vendor, and all of the conflicts that historically required you to run "one app to one box" are eliminated because each application is isolated in a virtual container. Additionally, the number of servers required for the infrastructure will be cut in half when utilization rates climb from the high teens to something in the neighborhood of 70 - 80% because multiple apps can now run on the same server without crashing.

When the leader (a monopolist no less) in the operating system space has to provide open specifications that enable other operating systems to control the applications, it is a strong indicator that the OS business is about to undergo massive changes. The days of the standard, general purpose operating system are definitely coming to an end.

The Standard OS is Virtually Gone

2006-10-07T20:19:00.000-04:00

Last week, Intel announced that it plans to certify its motherboards to run VMware's operating system. The VMware operating system is marketed under the brand name VMware Infrastructure, but it is effectively a "hardware" operating system. I use the "hardware" modifier because this product is not a "standard", general purpose operating system, but instead it is a platform for running virtual appliances. Each virtual appliance brings its own, unique "application" operating system with it, eliminating the requirement for a "standard" operating system to run multiple applications on the same server.

"Why is this important?" you might ask. Well, it is important because historically applications were artificially bound to the hardware infrastructure because the standard operating system had to support BOTH the hardware AND the application. Separating the operating system into independent "hardware" operating systems and "application" operating systems allows the hardware infrastructure to evolve independently from the application infrastructure, and vice versa. With this announcement, any application running any version of any operating system can be wrapped in a virtual machine container (creating a virtual appliance) and deployed to an Intel server by simply copying a file onto the "hardware" operating system.

Imagine, application installation that is as easy as copying a file to the server . . . every time. Imagine never having any conflicts among applications due to competing system service requirements. Imagine server utilization percentages in the high 80s because you no longer need to run "one app to one box" due to the impossible conflicts among applications. Imagine skipping at least one step of the DEV, QA, TEST, PRODUCTION application promotion cycle because incompatibilities among the app and the infrastructure are a thing of the past. If you are a software vendor, imagine eliminating all of the hassles of multi-platform porting, maintenance, and customer service, allowing you to drive 6 - 10 points of pure margin to the bottom line.

When virtualization eliminates the "standard", general purpose operating system and creates the new categories of "hardware" and "application" operating systems, we will all wonder why we tolerated so much pain for such a long time.

GPLv3: Innovation will Trump Narrow Agendas

2006-09-27T10:53:00.000-04:00

The hullabaloo surrounding the current GPLv3 debate makes for great theater and good cocktail conversation, but it is unlikely to create any real serious trouble for Linux or the open source software movement. Any extreme attempts to promote a narrow agenda will be crushed by the greater weight of our collective craving for innovation. A simple historical case study serves as a great example of what happens to narrow agendas that stand in the path of the relentless march of innovation.

In August of 1997, a group of disgruntled compiler developers forked gcc2 because the FSF was no longer responsive to their needs to innovate. FSF had a "stability" agenda around the compilation of C code for GNU. The Cygnus crowd, Linux crowd, and other constituents of gcc2 needed innovations to promote their products that were not being accepted by FSF. They forked gcc2 and created the Experimental/Enhanced GNU Compiler System, EGCS (pronounced eggs). They proceeded forward with innovations that were important to their user base, and they left the FSF branch of gcc2 behind.

In April, 1999, FSF abandoned their branch of gcc2, and re-united with the EGCS crowd because their branch had become largely irrelevant to users because of their narrow agenda. In less than 2 years, the agenda of FSF was buried by the greater good because of the original license innovation of the FSF, the GPL. The GPL allowed the EGCS crowd to create the branch that ultimately became the trunk of the code tree because they were delivering value to a broader constituency than the FSF compiler maintainers.

I believe the same scenario will play out with GPLv3. If the FSF attempts to promote a narrow agenda with the new license, all the code that is created under that license will be marginalized and the competing branch that continues under GPLv2 will become the trunk of the tree. Ideology and agendas are the stuff of interesting theater and intriguing cocktail conversations. Large scale adoption of innovation is what makes our lives better, and narrow agendas that stand in the way of a good life quickly become marginalized.

Golden Bits or Fool's Gold

2006-09-17T22:40:00.000-04:00

One of the more popular open source business models is the "golden bits" model. In this model, the term "certified" or "certification" comes up often during the sales cycle. But what does "certification" really mean?

Generally it means that there is a 100% probability that a certain configuration of technology has passed a certain set of tests. Unfortunately, the probability of that technology configuration matching your system preferences multiplied by the probability that the test cases mirror exactly your application workload is equal to 0%. This reality explains why IT shops that only buy "certified" solutions, open source or not, still have four layers of application promotion: development, test, QA, and production. These layers are required to "certify" that the "certified" solutions will actually work for the customer. It also explains why the services segment of the IT industry is the largest by a significant factor, even in the face of so many "certified" solutions.

The problem with "golden bits" is that gold is useful for certain things like jewelry and corrosion resistant electrical contacts. But, for example, carbide steel is better for milling and drilling. Diamonds are better for making grinding wheels. It really depends on what you are attempting to accomplish with the materials you are provided. Fortunately, the current trends in the software industry are favorable to the needs of the customer with respect to software becoming more modular and malleable to workload specific requirements.

The first favorable trend is open source itself. With open source, at least you have the option of changing a particular component to suit your needs. It certainly requires an investment in skilled developers and software architects to carefully choose a sustainable path, but it is likely that this investment could be cheaper than the alternative -- being stuck in a technological dead end with the proverbial helicopter airlift to better ground as the only way forward. Be certain your open source vendor supports your needs to modify the components to suit your unique application requirements.

The second favorable trend is service oriented architectures (SOA). With SOA, integration among "loosely coupled" components occurs based upon a network access standard. Certification is irrelevant because the provider can run the "service" as a black box if the system software approach for a vendor's component is not consistent with the system software for components provided by the internal company developers.

The third favorable trend is server virtualization. With server virtualization, the system software that supports the application can be decoupled from the system software that abstracts the hardware and exposes the network. Each application component has its own operating system and related system software, and multiple components are integrated on the same hardware via a virtual machine wrapper. Certification simply means that each component can be delivered as a virtual appliance, which eliminates all potential for conflicts among components. Certification is real because the entire unit is integrated and maintained by the application provider. If it breaks, it is their problem.

"Golden bits" will never be a substitute for great software architecture. Be wary of any vendor hawking "golden bits" or "certification" because the reality is there are likely lots of hidden costs if they have not implemented a solid software architecture.

Gartner Prescribes Software Appliances to Relieve Windows Bloating

2006-08-28T12:44:00.000-04:00

In a recent report covered in techworld, Gartner analysts Brian Gammage, Michael Silver and David Mitchell Smith declared:

""The current, integrated architecture of Microsoft Windows is unsustainable - for enterprises and for Microsoft."

Said another way:

"The era of the bloated, 'one size fits all' operating system must come to an end."

The article goes on to explain how virtualization and software appliances are the future of the enterprise operating system, even for the desktop. Drivers and basic system services will be provided by a streamlined OS that is tightly integrated with the hardware. New applications will arrive as software appliances, with their own tailored operating systems. These software appliances will integrate with the hardware OS via a virtualization layer such as VMware, eliminating all of the conflicts that arise when applications that install natively on the hardware OS begin competing for OS system resources. With software appliances, applications will truly "plug and play."

According to the article:

"Microsoft doesn't agree with this vision, saying it's identified problems with integrating data across partitions and creating a consistent user experience."

Of course they "don't agree with this vision" because it eliminates the innovation stranglehold of the standard operating system that perpetuates Microsoft's monopoly power. The software appliances that carry the value of the application don't even have to run Windows because the virtualization layer makes the OS irrelevant. The power of Microsoft is their abilty to charge more money for their applications because they own ALL the layers from the hardware up through the user interface. When the operating system functions get split between a hardware OS that provides drivers and system management and an application OS that provides system services such as file systems and libraries, the stranglehold is broken.

You may still need a Windows application OS to run Office, but if Microsoft does not tailor it to simply support Office, it will not be competitive. The footprint will be 10X what is required to run Office, and customers will balk at this gross inefficiency foisted on them by Microsoft's unwillingness to abandon the practices that were once so profitable but now are impractical given the changes brought on by virtualization. Customers always win . . . . . eventually.

Ingres beats Oracle to Linux

2006-08-20T20:35:00.000-04:00

Matt Marshall (no relation) of the San Jose Mercury News sounded the alarm this past week at Linuxworld, declaring that Oracle was going to ship their products pre-configured with a Linux operating system based on Red Hat's product. It seems that Matt's prognostications were a little premature. The biggest news on the floor actually came from an Oracle rival, when Ingres announced Project Icebreaker. It seems that while Larry Ellison has been talking about Linux, Ingres has been working on their own software appliance, which is currently in beta and will ship in a few months.

I fully expect Oracle, and almost every other software vendor for that matter, to eventually ship their products pre-configured with a Linux operating system. The benefits of the software appliance approach to both customers and the software vendors themselves are indisputable, and the open source license of Linux makes it a perfect OEM component for the software vendors. As virtualization becomes pervasive with Intel VT and AMD Pacifica, the argument for the standard, general purpose OS is going to get weaker and weaker.

Although I don't doubt that Oracle will someday embrace software appliances, I would be suprised to see them ship their products pre-bundled with the Red Hat distribution. Red Hat does a great deal of work to create and maintain a general purpose operating system. Because of the "one size fits all approach" of a general purpose operating system, an extraordinary number of patches and components are included in order to serve the broadest possible needs of the market. If Oracle chooses Red Hat Linux as the basis of their software appliances, then they will be forced to maintain a much larger footprint for Linux than their products actually require. For a company that seems obsessed with operating efficiency, adopting an inefficient approach to providing software appliances seems unintuitive.

I guess we will all wait with baited breath for the news from Oracle Openworld in October. . .

TCO with a Baseball Bat

2006-08-06T15:02:00.000-04:00

A recent Forbes cover story profiling Larry Ellison and Oracle proclaimed:

"For every dollar corporate customers spend on new software, they spend $6 implementing it and getting it to talk to other programs."

Put another way:

"If you think it costs a lot to buy a license for software, wait until you see how much it is going to cost you to use it."

What a terrible indictment of the software industry. Even the open source vendors who clang and bang about the expense of proprietary software licenses are often missing the big picture of the user experience. And customers are complicit in this debacle when they accept that they are going to have 4 levels of server infrastructure associated with promoting software: development, test, QA, and production. Why did you pay so much money for a "certification" if nothing is really ever certified until you, the customer, put it through its paces?

It is not surprising that SaaS vendors such as salesforce.com and appliance vendors such as Barracuda Networks are now the rage of the software industry. Both of these companies assume responsiblity for component integration, and they build their software to interoperate with every other required software element via standard protocols like TCP/IP. It is also not suprising that VMware is growing 73% annually by providing technology that enables virtual appliances, decoupling the infrastructure layer from the application layer which reduces "certification" to copying a file to a system.

I still chuckle every time I read a Microsoft funded study that reveals the TCO benefits of Windows server relative to Linux. In most cases, Windows is proclaimed to be anywhere from 10 - 25% cheaper to own for some certain workload - the equivalent of counting gnats on a rhino's ass. The point of Linux from a TCO perspective is that for Unix based applications (which won't run on Microsoft Windows anyway), the customer gets 3X the application performance on hardware that costs half to one third the price of proprietary Unix, yielding a price performance benefit of 6 - 9X. This breakthru performance for the customer is TCO that swings like a baseball bat.

Like Linux in the Unix market, SaaS, appliances, software appliances, and virtualization all swing TCO like a baseball bat. If you are a software vendor, whether open source or not, you better start warming up in the on-deck circle with a similar capability, or you will soon be headed for the showers.

Innovation Trumps Consistency

2006-07-26T15:05:00.000-04:00

During my last days at Red Hat, I recall making a sales call on a very large customer that had completely transformed their customer service costs and application performance by switching from proprietary Unix to Linux on X86. We were laying out the technology roadmap for the next Linux release, which offered yet again more improvements in performance vs. Unix, when a senior IT manager interrupted the presentation:

IT Manager - (loudly) - I really wish we could just go back to the good ol' days of Unix where nothing ever changed. How am I supposed to maintain any consistency when all of these performance innovations keep showing up in the code, and the application developers just can't resist adopting them to rachet up performance.

Red Hat SE - (calmly) - But isn't that the point? When a new innovation makes the customer experience better or the cost of service lower, adopting that innovation into the application just makes sense.

IT Manager - It's just more work for me. I care about performance and cost, but no one measures the cost of our efforts to support innovation. A slow moving, standard OS lowers the costs for the administrators who have to make the applications work with the rest of the infrastructure.

Red Hat SE - Well, I think innovation is always going to trump consistency. I wish there was more we could do to make it easier for you to stabilize multiple streams of innovation in your datacenter.

This interaction came to mind when I read an interview with the CTO of Amazon, Werner Vogels, about how Amazon enables innovation by giving the developers complete responsibility for delivering a working application service. The developer can adopt any innovation that makes their service more effective, but they have to manage the availability of that service in the datacenter. At some level, I am certain Amazon has guidelines, such as server preferences, OS preferences, management utility preferences, etc. My guess, however, is that these are pushed to the lowest possible level to enable flexibility in the upper portions of the stack in order to maximize application performance.

I am sympathetic to the workload of the datacenter staff, and I have high hopes that virtualization is going to separate forever the management of the box from the delivery of the application. Maintaining consistency of the OS across applications at the expense of application performance is simply unacceptable in a hyper-competitive world.

I think it was Ralph Waldo Emerson that said "A foolish consistency is the hobgobblin of little minds." I think Ralph was a closet application developer before application development was cool.

Slaying the Microsoft Octopus

2006-07-15T00:28:00.000-04:00

I laughed and I cried when I read Robert Spivack's response to the Microsoft blog entry that announced its intention to enter the hosted CRM space 9 - 12 months from now (scroll to the bottom of the post to see Robert's comments). Here is the quote that made me laugh:

"The intrincate interdependencies between IIS, SQL Server, Reporting Services, Active Directory, and Security (double-hop Kerberos) create an installation nightmare with octopus-like tentacles reaching into every nook and cranny of server infrastructure almost impossible to get running correctly."

I visualized an octopus with a Microsoft branded head wrapping itself around a server and then reaching up and down the rack to torment other servers that had the misfortune of being placed in close proximity to the one that the octopus was intent on destroying. Frightening visualizations that accurately reflect Microsoft application architecture often make me chuckle.

I cried because it seems that Robert's sentiments regarding the complexity of Microsoft CRM application software are becoming generally accepted as the "software condition." All of the software conferences that I have attended in the last 90 days have been dead, dead, dead. It seems that the general wisdom around software these days is that it is a crappy business unless you have an application that fits the SaaS mold, but not all applications lend themselves to an on-demand form factor.

I think software business can be great business, but software application companies absolutely have to abandon the old methods of licensing and distribution. After all, if Microsoft cannot get it right when attempting to deliver their various products on their platform because of the impossible interdependencies that arise when running multiple applications on a single, controlled instance of their operating system, what hope is there for an independent software vendor to get it right when dealing with the multiple OS (short for Octopus, I think) variations they invariably encounter across their customer base?

In a by-line article recently published in Enterprise Open Source Magazine, I elaborate extensively on the benefits of "losing your independence" for the independent software vendors (ISVs). Instead of fighting the octopus at each customer, embrace Linux and open source components to "wrap your application" and create a software appliance. Use virtualization to make the software appliance a virtual appliance in the case where a customer simply cannot bear to remove the octopus from an existing server. Your application will be safe from the octopus because of the protective layer of the virtual machine container.

The software business can be a great business if you have the right weapon to slay the octopus. SaaS is one weapon, but don't discount the software appliance as an approach that is cheaper than SaaS re-architecture, and may indeed be far superior for many types of applications. It is certainly more channel friendly as it allows your partners to add hardware and customer service value when delivering your new, simple to install, simple to maintain software appliance.

But what about that octopus carcass? Serve it up Polvo Frito, but don't overcook it as it can become tough when over-exposed to heat.

Software Lessons from High Fuel Prices

2006-07-04T22:38:00.000-04:00

While we were watching the dollar meter spin on the gas pump that was re-fueling the boat after an offshore fishing trip this weekend, a couple of my fishing buddies commented on the devastation of the boat and SUV market brought about by high fuel prices. Buyers that would have jumped into the market for a Chevy Suburban, Ford Expedition, Grady-White, or Boston Whaler just one year ago are now on the sidelines, perhaps permanently, because it is simply too expensive to operate these vehicles and sea going vessels with gasoline prices in the $3/gallon range. The concept of price elasticity of demand applied to the total cost of operating a SUV or boat is obvious to these good ol' boys from South Carolina. Funny that the geniuses that are the executive class of the software industry often don't get it.

It is obvious that rising prices shrink markets. The corollary is that falling prices expand markets. Price is not just the cost to acquire a product; it also applies to the cost to operate that product. The lesson for the software industry is that lowering the cost of integration and maintenance for a product will expand the available market for that product. The on-demand SaaS companies and the appliance companies are currently reaping a windfall from this simple lesson in economics while most software application company executives stand on the sidelines, knees knocking over the prospect of changing their distribution, licensing, and revenue model. It's time to get over it. The invisible hand of Adam Smith is either going to shove these companies forward to embrace the on-demand SaaS or software appliance concept, or that same invisible hand is going to shove them over a cliff into competitive oblivion.

I do not believe anyone questions the turnaround prospects for GM and Ford in the face of $1/gallon gasoline. Funny that this same logic does not lead more folks to the obvious conclusion that the current malaise of the software industry could be cured by a large dose of simplicity.

Virtualize ASAP, Then What?

2006-06-25T00:13:00.000-04:00

This week, VMware and Intel announced a program called Virtualize ASAP to help ISVs turn their applications into virtual appliances. The value of virtual appliances is obvious for customers - no installation or configuration hassles. But how can that value be sustained? After all, a virtual appliance, although contained inside a virtual container, still has lots of components (such as an operating system, application server, web server, database, etc) that will invariably suffer from bugs and obsolesence in the face of technological progress. If the customer is supposed to manage lifecycle issues such as operating system support and maintenance associated with the components of the virtual appliance, is the convenience of a simplified installation really much of a bargain?

Virtual appliances will truly become an interesting form factor when application companies shoulder the burden of lifecycle management for all of the components that make the virtual appliance a complete solution. If a virtual appliance only provides the value of a snapshot in time for the system components, it is only really valid for demonstrations and trials. Fortunately, Linux and open source offer application ISVs a perfect OEM license that enables absolute control of the customer experience throughout the application lifecycle.

With Linux and open source, an application company can tailor the components to suit the performance characteristics of its application. Then, as the components evolve, they can be passed along to the customer -- if and when the changes add value for the customer in the context of the application. Because the application company tests and certifies all components before a customer receives them, the customer headaches and hassles of coordinating and testing asychronous maintenance streams disappear.

If ISVs are going to Virtualize ASAP, we can only hope that Intel and VMware are going to provide them with some guidance on how to harness Linux and open source to offer a complete value proposition for the customer.

Is VMTN the next AppExchange?

2006-06-10T22:01:00.000-04:00

VMware held its industry analyst day this past week in Boston at the Charles Hotel. I was invited to speak on a panel about virtual appliances, and I was pleasantly suprised by how aggressively VMware is embracing the concept of virtual appliances. Diane Greene and Brian Byun gave a presentation that described virtual appliances as one of three strategic pillars for the company. As part of the panel presentation on virtual appliances, Srinivas Krishnamurti proclaimed that VMware intends to create a marketplace for virtual appliances on VMTN.

It makes perfect sense. When software can be downloaded, installed, and fully operational as easily as buying a new hit tune from iTunes and loading it on the iPod, why not create a market where such software is readily available to the buyers? We are not just talking about simple software like screensavers and games folks. We are talking full blown server side applications, such as an Ingres database, a Zimbra messaging server, or a Digium Asterisk VOIP PBX. All of these companies have publically demonstrated their commitment to providing their software in a software appliance or virtual appliance form factor. Why? Because it makes it drop dead simple for a customer to receive value from their application quickly. The operating system, and all of the headaches associated with it, disappears into the application. The application vendor takes responsibility for integration, test, and maintenance. Sound familiar? Just like the SaaS offerings on AppExchange.

The major difference in this case is that the technology that makes everything work nicely together is not a hosted datacenter infrastructure maintained by someone like salesforce.com. Instead, it is the virtualization layer provided by VMware's products. Each application vendor integrates their application with the operating system that is tailored and configured to deliver high performance for their application workload (in nearly every case it is Linux, just like the SaaS offerings all use Linux). Then, they seal it with the VMware virtual machine container. When the customer downloads the virtual appliance, they simply copy it to any server that supports VMware virtual machines (which includes most Linux, Windows, and VMware server OS offerings). For highest performance, it should be deployed on VMware's ESX server, which is optimized to run VMware virtual machines.

Service Oriented Architectures, SaaS, and Virtualization are all the rage now because they give us hope that the innovation chokehold of the standard OS is about to be broken. When applications can work together on the same server quickly and easily without days, weeks, and months of integration and untold days, weeks, and months of care and feeding as part of the maintenance cycle, all of us will get more value from our software application vendors. I hope that VMTN is remarkably successful as a virtual appliance marketplace because that success will reflect the death of the innovation drag created by the "standard OS."

Open Source with few Open Benefits

2006-05-29T15:22:00.000-04:00

Several months ago I was attending a conference on open source when a senior executive from IBM who was participating on a panel declared:

"Approximately 0% or our customers that use Linux ever modify any source code."

This statement is probably a bit of a commentary on IBM's customer base (they are IBM customers because IBM tends to serve companies that want IBM to do everything for them), but I think it is also a commentary on the current status of commercial open source software products. For as much as open source has shocked the software industry, the software market is essentially the same as before open source, but with a slight change in the players on the sell side of the equation. The open source companies are using open source as a signal to the market that their software is low cost (no pricing leverage through proprietary rights) and high quality (anyone can inspect the code), but they are not signaling to the market that open source is a way for the customer to create the unique innovations that matter to the customer. Generally speaking, commercial open source software is still "one size fits all" and de facto proprietary. Let's take the example of a couple of poster children in the open source space - Red Hat, JBoss (now Red Hat), and MySQL.

Red Hat is the great citizen of open source. The amount of R&D spending that Red Hat pours into Linux and other related projects to which it exercises no rights of ownership is extraordinary. However, if you are a Red Hat customer and you want commercial support for some small tweak to Red Hat Enterprise Linux, forget about it (unless of course you are one of a handful of $million plus annual customers, then the policy is similar to gays in the military - "don't ask, don't tell").

Then there is JBoss, "the professional open source company." When I first met Marc, he took great pride in explaining to me that the only contributors to JBoss products were the JBoss employees, which put JBoss a commercial cut-above most open source companies who rely on developers not on the payroll for large portions of the code. I never really understood why JBoss was open source. If you are not building a community of developers to create leverage in R&D, why not just give away the binaries and sell the product for a low price? Also, the tagline "professional open source" implies by contrast that every other company in open source is an amateur hack (which Marc supported with his frequent attacks on other open source projects and companies, especially Red Hat).

As for MySQL, I was on a panel with Marten Mickos when Robert Leftkowitz stood up in the audience with a red clown nose on his face (which he also sported during a contract negotiation with Red Hat while working for Merrill Lynch) and asked about commercial support for "bozos that want to modify the code." Marten responded, cleverly, "we have a special bozo contract for those customers." Again, if you want to be a bozo and modify the code, we will treat you like a bozo in our commercial relationship. Perhaps Marten was just being clever for the panel, but generally speaking open source software tracks very closely to proprietary software as it relates to business model assumptions.

I mean no disrespect to the commercial success of any of these companies, but they have really changed very little about the software industry (except perhaps the pricing expectations of the customers) through their success. There is so much more work that needs to be done to make software truly "soft" such that it can literally be molded to the requirements of each customer. Current software business models are still in a phase of maturity that is reminiscent of Henry Ford's commentary about customers that wanted colorful autos, "You can have any color you like so long as it is black." Open source is a great step towards "mass customization" for software, but there is still much more work to be done before the industry can truly claim that a customer centric transformation has been achieved.

Small Percentages make a Big Difference

2006-05-16T17:23:00.000-04:00

I had the good fortune of having a very entertaining conversation about the economics of the software industry with a bright senior software executive this week. We were talking about the economics of a subscription model as compared to the economics of a perpetual license model, and she had all of the fundamentals down cold:

- R&D is cut in half (from 16% to 8% of revenue, on average) when you can bring all customers along with every bit of maintenance so there are no legacy platform or release issues.

- Renewal rates are higher when customers see the software vendor continuously delivering innovation that makes their application more valuable

- It is difficult to transition to a subscription model when you are so dependent on perpetual licenses for quarterly revenue

She explained to me that her company historically had a very high maintenance renewal rate of 90% compared to the industry average of 85%. I thought about those numbers and how they contrast with Red Hat's most recently reported renewal rates of nearly 100% for the top 100 customers, which I wrote about in a previous blog post. I decided that I wanted to do a little test of the power of subscription renewals for topline performance of a software company.

Let's take a hypothetical case of a company that books $100 in new subscription bookings each year, with some percentage of the prior years new bookings renewing each subsequent year. In year 2 and year 3, for example, the company bookings would be:

- Year 2 Bookings = $100 + (Year 1 Bookings)*X%

- Year 3 Bookings = $100 + (Year 2 Bookings)*X%

and so on for each subsequent year. Over a ten year period, how much more annual bookings would a company have if they are able to improve their subscription renewal rate from the average of 85% to 90%? to 95%? to 98%?

Here are the numbers and the percentage improvements for each case relative to the industry average of 85%:

- 85% renewal yields $555 in Year 10 Bookings, base case

- 90% renewal yields $686 in Year 10 Bookings, 24% improvement

- 95% renewal yields $862 in Year 10 Bookings, 55% improvement

- 98% renewal yields $996 in Year 10 Bookings, 79% improvement

I was shocked to discover that a mere improvement of 13 points on renewals yields a topline improvement of nearly 80% over 10 years. Then again, I think it was Albert Einstein who claimed "The most powerful force in the universe is compound interest." Why should I be suprised when Einstein is proven correct yet again?

Sun's Gift to Linux

2006-05-05T10:58:00.000-04:00

According to a recent article on ZDNet, Sun plans to remove the distribution restrictions from its Java Runtime Environment. Now developers and solution providers that use Java programs can provide the golden standard of Sun's JRE to their customers without the customer burden of annoying licensing and download routines. Without these cumbersome restrictions, Linux developers that were formerly cautious on Java might find themselves making a different decision in light of Sun's JRE beneficence.

As with the many gifts that Sun has provided to Linux in the past (OpenOffice, sponsoring GNOME development, buying Cobalt for $2B, making SPARC expensive and slow so customers adopt Linux instead), I have not quite figured out how this latest gift benefits Sun. I suppose that fewer restrictions on Java resulting in more Java programs in the market makes for a bigger market overall for Sun's wares, but I don't know if Java growth still correlates to Sun growth in the current environment.

I shared this sentiment with a friend and colleague of mine, Nathan Thomas, and he asked me if I had ever heard of the Underpants Gnomes from the Comedy Central show Southpark. I had not, so Nathan pointed me to wikipedia to fill in the details. It seems that the Gnomes steal underpants as part of a three step business plan:

1. Collect Underpants

2. ???

3. Profit!

Rather than looking a gift horse in the mouth by questioning the motives of the giver, let's just thank Sun for helping advance the cause of Linux one more time and hope the market trend lines associated with Linux get yet another boost from Sun's actions. Perhaps the questions regarding motivation and business models that perplex all of us that watch Sun from a distance are more clear to those inside the Sun organization.

For Whom the (IT) Bell Tolls

2006-04-27T14:57:00.000-04:00

I recently read a post by Jeff Kaplan, a SaaS expert with THINKStrategies, about being Nick Carr for a day. It seems that some IT folks are still stewing over Nick Carr's assertion that IT doesn't matter, and they directed their venom at Jeff when he piled on the Carr bandwagon by demonstrating that SaaS is a natural evolution in the commoditization of IT. Personally, I believe that IT matters, but the question is "for whom does IT matter?" and its corollary "for whom the (IT) bell tolls?"

Lou Gerstner recently gave a presentation to the Cognos sales team where he asked that team the following questions (or something very similar, as I was not there):

"How many of you have purchased an electric motor in the past year?"

Two people raised their hand.

"How many of you have purchased any one of the following in the last year: a refrigerator, a car, a fan, a dishwasher, a washing machine or a dryer, a laptop, PC, or a vacuum cleaner"

Half the audience raised their hand. The point is obvious. You do not care about buying motors, you care about the functionality they deliver when integrated into an application. A faster, cheaper, quieter, lower power motor matters VERY much to the appliance vendor, as it potentially gives them competitive advantage. The customer sees these benefits as longer batter life, a quieter appliance, cheaper electric bills, and so on. But the motor doesn't matter by itself, at least not to the consumer.

IT, therefore, matters very much to technology application providers. The ability to consume better components into their solutions so that customers get a better result matters a great deal. The ability to create unique combinations that deliver a new innovation matters a great deal. IT certainly matters to Apple, and Microsoft, and IBM, and salesforce.com, and Red Hat. If IT does not begin to matter a lot less to the customers of these companies, then the executives of these companies all deserve their ultimate low stock value fate.

But what about the IT staffer that has so long labored to overcome the inefficiencies of her technology suppliers? If her company's basic value is not achieved via the intellectual capital contained in the application technology that she produces, "never send to know for whom the bell tolls; it tolls for thee."

Oracle Ponders Linux Software Appliances

2006-04-17T17:19:00.000-04:00

In an interview this past weekend with the Financial Times, Oracle CEO Larry Ellison pondered the notion of Oracle providing Linux as an integrated component of its software offerings. It makes perfect sense. Not because the world needs another Linux vendor, but instead because customers need a simplified approach for consuming complex software applications such as those provided by Oracle. With a single decision to ship Linux, Larry Ellison could transform all of the Oracle software portfolio into Linux software appliances that eliminate the customer hassles of integration and mismatched maintenance streams, delivering the value of software as a service (SaaS) for on-premise application deployments. Oracle could become the leader in SaaS overnight through the software appliance concept.

Linux and open source offer software application providers the historic opportunity to transform their business from selling licenses to delivering integrated solutions -- software appliances. Before Linux and open source became mainstream, delivering an integrated solution meant complex OEM agreements and being beholden to the technical and economic agenda of multiple third party component providers. Linux and open source, on the other hand, offer the perfect OEM license - freedom of distribution. And the technology is flexible to accomodate the technical agenda of the application provider. And customers get hardware vendor choice because Linux supports industry standard hardware instead of proprietary architectures. Customers get an integrated solution with "one throat to choke" -- the hallmark of SaaS, but for on-premise application deployments.

Larry's comments are less about Linux and more about customers. Linux doesn't matter. The customer experience with software applications does. Simplicity is good.

Red Hat Pays Big for JBoss

2006-04-11T16:31:00.000-04:00

In a brilliant move to own the hearts and wallet of their datacenter customers, Red Hat yesterday announced that it would pay up to $420M for JBoss, the market leader for open source Java. After spending just less than 2 years attempting to build their own Java implementation with JOnAS, Red Hat decided it was time to go buy the leadership position in this critical technology area. With the purchase, Red Hat moves one step closer to the stated strategic goal of "owning the stack" for all infrastructure components their datacenter customers utilize to deploy applications.

What is the next item that Red Hat will add to the shopping cart in this buying spree? Well, several weeks ago Red Hat announced that they were going to create, maintain, and support "certified stacks" for their customers. If they are not successful building these stacks, perhaps they will go buy SpikeSource, mirroring the behavior in the Java space. And, given that a LAMP certified stack also includes PHP and MySQL, maybe Red Hat's next buy will be Zend or MySQL, or maybe Enterprise DB?

In any case, the acquisition is great validation of the value of open source, and it is now more certain than ever that Red Hat is going to be an even bigger force in the datacenter world.

Microsoft Embraces Software Appliance Market

2006-04-07T10:57:00.000-04:00

At Linuxworld this week, Microsoft announced that the upcoming release of Virtual Server 2005 is going to provide a platform for software appliances. Well, they didn't actually use the term "software appliance," but they did say Linux would be supported on their platform. The underlying message, however, is consistent with the Microsoft objective of having all applications run on Windows - "If your application only runs on Linux, you can now run it on Windows with zero porting, installation, or configuration hassles."

Running Linux on Microsoft is not interesting at all. Giving application developers the flexibility to do a single port to a single platform without excluding any part of the available market due to OS incompatibilities is VERY interesting. Microsoft's announcement further cements the argument for application providers to choose Linux and open source as their preferred platform because they can eliminate the expense and time to market penalties associated with multi-platform support without sacraficing any market share associated with legacy "Windows Only" IT shops. Embracing the software appliance concept, with Linux and open source as the system software platform, just got easier.

Most legacy software vendors support multiple releases of multiple platforms in order to offer their product to the widest possible market. These companies typically spend about 14% of revenue on R&D expense. Salesforce.com only spends about 7.5%. Why the difference? Salesforce.com only supports a single platform, so they eliminate an enormous amount of engineering, QA, and test associated with installers, multi-platform ports, configuration scripts, etc. Imagine the benefits to shareholders and customers if software companies were able to drive 6 points on revenue to the bottom line or to better R&D (i.e. "core" issues instead of "context" issues).

Microsoft may have just helped the rest of the software world become more efficient by embracing the software appliance concept. Linux and open source becomes the universal platform and Microsoft provides the bridge for the legacy Windows users to avoid being left behind.

Red Hat Demonstrates Subscription Power

2006-03-29T08:55:00.000-05:00

Red Hat announced full results for FY06 yesterday, and the results were very impressive. The most notable statistic, in my opinion, was the strength of the renewals. 25 of the top 25 customers and 99 of the top 100 customers up for renewal in 4Q06 renewed their subscription contracts with the company. These renewals are not for some percentage of an initial license sale, but for the full value of the subscription. In most cases, it was probably higher due to an increasing footprint within each customer.

The important lesson here for the rest of the software industry is that getting your product in the door at a reasonable price (i.e. a "pay as you go" subscription) leads to a great business model if you continue to fulfill on the value of the product over time. I was on a call yesterday with Jason Maynard, a CSFB equity analyst, where Jason declared that he was seeing a definite transition to the "pay as you go" subscription model. He claimed that the pendulum of negotiating power was swinging toward the customer, and software vendors are reacting by accepting some of the risk of software success by changing the license terms from perpetual to subscription.

During the early days at Red Hat, it was difficult to sell the customer on the notion that the payment each year was the total subscription value instead of a bigger payment in year one for a license and lower payments in subsequent years for "maintenance." The logic that ultimately prevailed with the customers was that a subscription to the total value of the supplier provides a healthier foundation for a relationship. Customers can consume the technology that works best for them independent of the "version" or "release" because the subscription entitles the customer to the total value stream for a particular product line. Additionally, the supplier has to "earn the value" everyday or be faced with the prospect of cancellation at the end of the subscription term.

One final lesson for the software vendors that are still addicted to the adrenaline rush of the perpetual license model - customers do not replace software that works for them. The switching costs are too high. Lower the barrier to entry, get in the door, do a good job, and maybe you too can build a model that looks as good as RHAT.