Scaling and Fault Tolerance Workshop

There was a day when it was common to see the twitter fail whale! This imagine, extinct today, was the sign that scaling at twitter was broken in some way. What did Twitter do in order to increase their ability to scale, be more fault tolerant all while growing significantly. The answer is a move to Apache Mesos and leveraging container technology.

Google in 2009 had a top secret project we now know as the Google Borg project. That technology was reincarnated in an open source project out of UC Berkley known as Apache Mesos. Mesos has grown up significantly while at Twitter providing production tested solution for scaling applications and containers. In addition Solomon himself stated at DockerCon EU in December 2014 that Mesos is the only reliable way to scale docker in a production environment today.

This session is a hands on workshop. It is broken into 3 separate but related parts.

The first part is an 1.5 hour lecture of the challenges of the datacenter today. It will provide an overview of

* containers, 
* static vs. elastic partitions 
* schedulers.

The second part is a dive into docker with time for hands on labs. You will need access to docker for 1/2 of this session. We will have some discussion on the issues of running Java in a container with a focus on needs of production.

The third part will be on Apache Mesos and several different schedulers. We will discuss:

* different types of containers
* stateful service solution
* service discovery
* typical failures at twitter and how to avoid them

This also has a hands on component. You will need either:

* A Google Compute Engine account
* A AWS account
* Software local on your laptop to run vagrant

The last shorter part will include an overview of technologies on the horizon in this fast paced micro-services space.