With the recent Hadoop World event hosted by Cloudera on October 2, 2009, Cloudera and Hadoop have been getting quite a bit of attention from the media, and the visibility for open source software in the cloud has increased along with them. I didn&apost attend the Hadoop World event, but I heard that it was well attended with solid content. Stephen O&aposGrady from RedMonk did a great summary of the event along with his analysis of the key trends if you want a little more information. The Hadoop World event is just a single point in time; however, the more interesting story in my opinion comes from the Hadoop / Cloudera combination.
Hadoop is an Apache project focused on open source software for reliable, scalable, distributed computing. One of the largest contributors to Hadoop is Yahoo, and Hadoop is part of the solution powering Microsoft&aposs new Bing search engine. According to the Who Uses Hadoop page, other companies using Hadoop include Amazon, Adobe, AOL, Facebook, Google, Hulu, IBM, The New York Times, several universities, and many more.
Cloudera was founded to provide enterprise class support for organizations using the open source Hadoop product, and they recently announced the availability of a new product, Cloudera Desktop, a graphical interface for Hadoop that can be used with internal clusters or clusters running in a public cloud. Cloudera has also put together a strong team of people with deep expertise in open source, data management, search and cloud computing. One of the most recent additions to the Cloudera team is Doug Cutting, a search engine specialist from Yahoo and one of the founders of the Hadoop project. This is a big loss for Yahoo and a huge gain for Cloudera.
Om Malik recently weighed in on Cloudera to compare them to Red Hat. He sees many parallels between what Cloudera is doing now for Hadoop and what Red Hat was doing for Linux in the early days from similarities in the executive team, venture capital funding, technologies and more. Here&aposs how he summarized his comparison:
The big change came this past August, when Doug Cutting left Yahoo and joined Cloudera. Cuttingˇ¦s involvement is like the icing on the cake, giving the company the ability to corner all the Hadoop talent out there. It also helps that Cloudera has started to make inroads into newer markets, including biotech and retail. ˇ§Hadoop is going to find potential markets in any industry where there are large data sets that need complex analysis,ˇ¨ CEO Olson told me.
I remember talking to Red Hat executives back in the day and listening to their pitch about Linux everywhere, how they were going to go beyond the web community and help drive Linux into other corporate environments and eventually, build a services business around it.
Cloudera is following that same path. Itˇ¦s developed its own version of Hadoop, one thatˇ¦s optimized for the needs of large corporations, especially those that prefer a little hand-holding from their suppliers. By giving them this version of Hadoop, Cloudera hopes to make revenue from services. And the timing ˇX the company unveiled Cloudera Desktop at Hadoop World (we are media partners) in New York, an event it organized ˇX is perfect.
Game, set, match for Cloudera. (Quoted from GigaOM)
With cloud computing taking off, the open source Hadoop project is well positioned to become a key part of the many clustered solutions with large, complex data requirements, and Cloudera will be right there for those organizations that need help implementing Hadoop.