Wednesday, March 4, 2009

Dryad and DryadLINQ

Problem: Companies like Microsoft, Google, Yahoo, etc have large data sets that can only be effectively processed on large clusters of PCs. The queries being applied to these data sets are growing more complex and a generalized version of MapReduce is needed.

Solution: Microsoft attempts to create its own language for a more generalized version of MapReduce in the line of Yahoo's Pig Latin.

Future Influence: It is unclear if Dryad and anything built upon it will gain acceptance outside of Microsoft. However, the need for a simplified method of running complex queries on large data sets on clusters of PCs will remain and there will be many efforts to develop solutions.

No comments:

Post a Comment