Apache Hadoop ecosystem

It’s a tremendous tribute to the Apache ecosystem that they were able to work together and have Cloudera, which was the earliest and best-funded, and Hortonworks and MapR, all working on the open source project to some extent and cooperating but also competing to create a productized Hadoop distribution. But Hortonworks went public and their business performance wasn’t clothed in glory. Cloudera got the huge investment from Intel and then eventually went public. MapR never went public.

Eventually Cloudera and Hortonworks merged and MapR just recently wasn’t able to get funding. This all slowed Hadoop’s progression, along with the cloud, which meant the need for the Hadoop file system to store big data became much less urgent because you could store all that data at scale in the cloud. Then other programming engines could process the data, whether it was in HDFS or whether it was in the cloud. The idea of using MapR or Yarn as an application development environment became less and less attractive, especially due to Spark and now there is a lot of bad financial news coming out around the Hadoop ecosystem. That’s my summary—is there anything you’d like to add?

Leave a Comment