Deployment with Mesos

Practical introduction to Mesos frameworks

by Dmitriy Volk dmitriy.volk@gmail.com  @xbo

Why PaaS didn't win

Traditional PaaS tried to do too many things:

• containerization (app → executable)
• provisioning of common infrastructure resources
• coordination
• provisioning
• scheduling and execution
• monitoring and management

Separation of concerns

• containerization ⇐ containers
• common infrastructure resources ⇐ containers
• coordination
• provisioning
• scheduling and execution ⇐ Mesos
• monitoring and management

Commoditization of computing

Virtualization, Cloud, Big Data, etc. abstracts away the execution platform for computing tasks.

Instead of configured machines, all that matters is available resources: CPU, memory, disk, ports.

Apache Mesos

• “Distributed systems kernel”
• Execution fabric
• Turns datacenter into a single pool of resources

Mesos architecture

Mesos Doesn't do much

Slave

• Offers resources to master
• Launches tasks
• Reports task status

Master

• Forwards slaves' offers to frameworks
• Forwards tasks to slaves
• Routes messages

What Mesos doesn't do

• Create task
• Assign task
• Enforce task

Demo

Deploying an application using various pieces of Mesos ecosystem.

Usage of Docker and Mesos is indicated for each step

“I suppose it is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail.” Abraham Maslow

The application

Finally, a truly distributed software system to calculate square roots!

Server holds work items (approximation iterations)

Client checks out a work item and calculates next approximation using Newton's formula:

\[newApproximation = \frac{oldApproximation + \frac{square}{oldApproximation}}{2}\]

Marathon

Marathon framework manages long-running tasks on Mesos.

upstart / systemd for the cluster

Makes sure n instances of a task (application) are running in the cluster

Exposes rudimentary UI and rich REST API

Deployed on demo cluster

REST call to Marathon

{
  "id": "sqrt-server",
  "cmd": "",
  "cpus": 1,
  "instances": 1,
  "mem" : 1024,
  "container": {
    "type": "DOCKER",
    "docker": {
      "image": "registry.dmitriyvolk.net:5000/sqrt/sqrt-server",
      "network": "HOST"
    }
  },
  "constraints": [
    [ "hostname", "CLUSTER", "registry.dmitriyvolk.net"]
  ]
}
          

Eureka  Sqrt Server UI  Marathon

Deploy SQRT Client

Also on Marathon:

{
  "id": "sqrt-client",
  "cmd": "",
  "cpus": 1,
  "instances": 1,
  "mem" : 1024,
  "container": {
    "type": "DOCKER",
    "docker": {
      "image": "registry.dmitriyvolk.net:5000/sqrt/sqrt-client",
      "network": "HOST"
    }
  }
}
          

After a square root has been successfully calculated, the client keeps getting launched. Not what we want, let's undeploy client. Marathon

Keep the inputs coming

We emulate constant stream of inputs with scheduled calls to generate random input.

Chronos framework is cron for the cluster: distributed and fault-tolerant scheduler for Mesos.

Exposes rudimentary UI and rich REST API

Deployed on the demo cluster: Chronos UI

{
  "name": "random-square-input",
  "schedule": "R50/2015-05-27T19:00:00Z/PT30S",
  "command": "/usr/bin/curl -X POST 
    'http://registry.dmitriyvolk.net:31080/newinput/random'",
  "epsilon":  "PT15M",
  "owner": "dmitriy.volk@gmail.com"
}
          

Problem: Client scheduling

Marathon will keep restarting client, even after all the work is done.

Chronos will launch client regardless of the work availability.

The alternative is obvious...

Yep.

Writing your own framework

SDKs available in C++, Python, Java

Communication with master through native library

Implement reactions to:

• connect / disconnect
• offers presented / rescinded
• task status updated (launched, running, failed, finished)

Custom Scheduler

public void resourceOffers(SchedulerDriver schedulerDriver, List<Protos.Offer> list) {
  for (Protos.Offer offer : list) {
    if (checker.hasMore()) { // makes REST call to app to check for available work items
      ...
      Protos.ContainerInfo.Builder containerBuilder = Protos.ContainerInfo.newBuilder()
        .setType(Protos.ContainerInfo.Type.DOCKER)
        .setDocker(Protos.ContainerInfo.DockerInfo.newBuilder()
          .setImage(sqrtClientImageId).setNetwork(Protos.ContainerInfo.DockerInfo.Network.valueOf("HOST")).build());

      Protos.TaskInfo task = Protos.TaskInfo.newBuilder()
        .setTaskId(taskId).setName("sqrt-task-" + taskId.getValue())
        .setSlaveId(offer.getSlaveId())
        .addResources(Protos.Resource.newBuilder()
          .setName("cpus").setType(Protos.Value.Type.SCALAR)
          .setScalar(Protos.Value.Scalar.newBuilder().setValue(1)))
        .addResources(Protos.Resource.newBuilder()
          .setName("mem").setType(Protos.Value.Type.SCALAR)
          .setScalar(Protos.Value.Scalar.newBuilder().setValue(1024)))
        .setContainer(containerBuilder).setCommand(Protos.CommandInfo.newBuilder().setShell(false)).build();
        ...
      schedulerDriver.launchTasks(Collections.singleton(offer.getId()), Collections.singletonList(task), filters);
    }
  }
}
          

Deploy custom framework

Custom framework is written as a Spring Boot application

Let's deploy it into the Mesos cluster

Watch clients not being launched when no work is available: Sqrt Server UI

Thank you!

• E-mail: dmitriy.volk@gmail.com
• Twitter: @xbo
• https://github.com/dmitriyvolk/mesos-presentation