The Channel logo

News

By | Gavin Clarke 18th November 2011 09:44

Microsoft Dryad trampled in Hadoop stampede

Redmond flees wild big-data crunching elephant

Microsoft is surrendering to the Google-inspired Hadoop it once sought to challenge with its own big-data crunching architecture.

There will be no further updates to Dryad, officially called "LINQ to HPC", and Dryad won't be developed as a product, Microsoft has said.

The news was tucked in a blog post announcing availability of the Windows Azure HPC Scheduler and HPC Pack 2008 R2.

Microsoft will instead make Hadoop - an open-source project represented by a cartoon yellow elephant - run better on Windows Sever and Windows Azure, Microsoft's cloud architecture, the company said.

The company in October announced it would integrate Hadoop with the forthcoming SQL Server 2012 database following development of a set of bi-directional connectors.

The work has been undertaken in partnership with Hortonworks, the Hadoop-engineering service and support start-up spun out from Yahoo!'s engineering group.

Dryad had been in development at Microsoft Research since about 2006 and was only released to beta in May this year.

One Microsoft-friendly consultant claimed Dryad had been used within Microsoft for "several years".

Hadoop is an open-source Java implementation from Doug Cutting of Google's MapReduce, with Cloudera but formerly with Yahoo!. Cutting was hired by Yahoo! around the same time Dryad emerged inside Microsoft Research.

Hadoop processes large amounts of data over networks of distributed, clustered servers. Unlike Dryad, Hadoop has seen widespread adoption within its mother company and the broader community: users include Amazon, Facebook and Twitter – while Hadoop is also a project with Apache.

Microsoft came to Hadoop inadvertently, through the acquisition of semantic search specialist Powerset in 2008. Powerset ran on Hadoop, but a year and a half after the acquisition, Powerset was taken off Hadoop by Microsoft because it didn't run well on Windows.

Powerset is now part of Bing, and it's not clear whether Microsoft's return to Hadoop means that the big-data architecture inspired by Google is now running on Microsoft servers underpinning Bing against Google. ®

alert Send corrections

Opinion

Chris Mellor

Drives nails forged with Red Hat iron into VCE's coffin
Sleep Cycle iOS app screenshot

Trevor Pott

Forget big-spending globo biz: it's about the consumer... and he's desperate for a nap
Steve Bennet, ex-Symantec CEO

Chris Mellor

Enormo security firm needs to get serious about acquisitions

Features

Windows 8.1 Update  Storeapps Taskbar
Chinese Buffet self-service
Chopping down the phone tree to scrump low-hanging fruit
An original member of the System/360 family announced in 1964, the Model 50 was the most powerful unit in the medium price range.
Big Blue's big $5bn bet adjusted, modified, reduced, back for more
Microsoft CEO Satya Nadella
Redmond needs to discover the mathematics of trust