Cloud Computing has taken center stage in 2008 as one of the promising new technologies that will drive the next level of innovation in enterprise datacenters and infrastructure outsourcing. In 2008 we saw many players inlcuding the Internet heavyweights – Google (Apps), Amazon (EC2 and S3) and SalesForce (Force.com); and the traditional infrastructure players -  IBM (Blue Cloud), HP (Adaptive Infrastructure) and even Microsoft (Azure) making big announcements. Lately Tier 1/2 telcos and hosting providers have also joined in the fray offering cloud services. Like in any area of disruptive innovation – a host of startups are already in the mix enabling enterprises to build cloud infrastructures (e.g. Elastra), support hybrid public-private datacenter models (e.g. 3Tera) and  manage presence on major public cloud infrastructures (e.g. RightScale).

The business press has also been abuzz with the potential of the new cloud economy. While adoption rates are still in its infancy as this analyst report on VMBlog indicates, most analysts and industry watchers see a huge ramp up in the use of cloud services in the next few years. Inc. Magazine wrote a nice report on how SMB’s can leverage cloud services. Interest from large enterprises and mainstream media and analysts in cloud platform evolution is also rising.

For mass adoption, Cloud Computing has still to cross many organizational (ownership, control), technological  (security, compliance, deployment) and management (monitoring, SLA) challenges. In fact, the ascendance of cloud computing also raises the question whether it can upset the current IT management industry structure of the Big 4 players at the top (HP, IBM, CA and BMC), a host of best-of-breed mid-range players in the middle, and,  a vast majority of smaller firms selling into the SMB and mid size enterprise market rounding up the bottom.

The possibility of IT management market disruption is real – because of a few critical differences in cloud monitoring as opposed to traditional datacenter management.

Cloud technologies are relatively new and draw sustenance from diverse virtualization stacks as well as many open source components for parallelization and queuing. New standards from the DMTF like WBEM /CIM XML and WSMan have deeper representations than the simplistic SNMP and are far more suited to capture dynamic application dependencies. Further, cloud operations are essentially remote, requiring secure data collection and reporting from across the Internet – which SNMP and proprietary data collection mechanisms from traditional management players do not support.

This will require a large shift in the architecture, protocol support and management mechanisms in IT management vendor solutions – which given their customer base and steady stream of maintenance revenue is not easy to accomplish. Using traditional methods of monitoring for cloud services is like using a hand-held thermometer where you require a responsive and automated sensor. The result will be that readings on even simple metrics like CPU and memory consumption will be erroneous or irrelevant (especially since now one is measuring an elastic resource).

More on the next generation cloud monitoring plays in a later post.

- Ronnie