Datacenter Agility of Hadoop

The current demands on the datacenter require a dynamic environment to succeed. The legacy approach of static clusters limits overall cluster utilization and does not allow for the growth demands of different versions of the analytic clusters. There is an increasing need to integrate big data and related Apache Hadoop applications with other datacenter services, ideally, by co-locating the data in HDFS/HBase with the services that need it.

In this talk, Ken (one of the committers on project) will show off Apache Myriad, which integrates Apache YARN with Apache Mesos. Mesos enables efficient resource sharing and isolation across a variety of distributed applications. This allows Apache Hadoop workloads to run alongside other datacenter applications, while sharing the same cluster, and hence breaking silos. This multi-tenancy strategy improves overall cluster utilization and operational efficiency.