• Clocker – Creating a Docker Cloud with Apache Brooklyn

    • Andrew Kennedy
    • 18 June, 2014
    • Posted in Blog

    In this post we introduce Clocker – an open source project which lets you spin up a Docker Cloud (hence the name). We show how under the covers this Docker Cloud is deployed and managed in an environment of your choice using Apache Brooklyn, which in turn treats it as a target location – deploying and managing applications on it.

    Docker is an open platform for distributed applications for developers and sysadmins, and Apache Brooklyn is an open source framework for modeling, monitoring and managing applications through autonomic blueprints currently undergoing incubation at the Apache Software Foundation.

    Recently we have seen Docker cluster management projects appearing which are predominantly focused on managing clusters in a single provider’s environment. Clocker is designed to deploy and manage Docker clusters in a portable and cloud provider agnostic way. Clocker can even be used on-premise exploiting an enterprise’s virtual or private cloud environment.

    Docker Cloud Capabilities

    Docker has captured widespread interest because it allows more efficient and more flexible use of infrastructure, when compared to traditional virtual machine partitioning.

    Docker makes it easy to partition a single host into multiple containers. However, although useful, many applications require resources beyond a single host, and real-world deployments require multiple hosts for resilience, fault tolerance and easy scaling of applications.

    Clocker creates a Docker Cloud capable of provisioning Docker containers across multiple Docker hosts, typically running on cloud virtual machines. Clocker implements the intelligent placement of containers, and the automatic provisioning of new Docker hosts and containers as required by the application. Clocker handles this automatically using various strategies, including depth-first and breadth-first.

    Clocker raises the bar in terms of Docker cluster management in a cloudy world.

    Clocker features:

    • Automatically create and manage multiple Docker hosts in cloud infrastructure
    • Intelligent container placement, providing:
      • resilience
      • fault tolerance
      • easy scaling
      • maximum resource utilisation of hosts
      • maximum application performance
    • Use of any public or private cloud as the underlying infrastructure for Docker Hosts
    • Deployment of existing Brooklyn/CAMP blueprints to Docker locations, without modification.

    Getting Started

    (This example uses the IBM Softlayer Cloud, but many other clouds are available, or you could create a Docker Cloud from fixed hosts.)

    Clocker uses Brooklyn to deploy and manage applications in the cloud. Configure a ‘brooklyn.properties’ file (sample) with your cloud credentials to allow Brooklyn to request cloud machines.

    Use the following commands to download, extract and run the Brooklyn server configured with the Docker entities:

    % wget --no-check-certificate --quiet \
    -O brooklyn-clocker-examples-0.4.0-dist.tar.gz https://git.io/WOhfyw
    % tar zxf brooklyn-clocker-examples-0.4.0-dist.tar.gz
    % cd brooklyn-clocker-examples-0.4.0/
    % ./clocker.sh launch
    

    You should see Brooklyn start:

    ...
    2014-06-09 22:12:51,536 INFO  Starting brooklyn web-console on loopback interface because no security config is set
    2014-06-09 22:13:04,564 INFO  Started Brooklyn console at http://127.0.0.1:8081/, running classpath://brooklyn.war and []
    2014-06-09 22:13:04,582 INFO  Persistence disabled
    2014-06-09 22:13:04,588 INFO  High availability disabled
    2014-06-09 22:13:21,767 INFO  Launched Brooklyn; will now block until shutdown issued. Shutdown via GUI or API or process interrupt.
    

    Launching a Docker Cloud

    Once your Brooklyn server has started, connect to the web console using your browser (127.0.0.1:8081), select Add Application then the Docker Cloud catalog entry.

    Select Docker Cloud in the Application Catalog

    You will then see a page with the available configuration options, and should choose the Location where your Docker hosts will be provisioned. The Location Name should be set to my-docker-cloud so we can reference this later and the rest of the configuration options can be left at their default settings. The Container Cluster Maximum Size option, set to 4, controls the number of containers per host and Host Cluster Minimum Size sets the initial number of Docker hosts to 2.

    Configure Docker Cloud parameters

    Select Finish and the Docker Cloud infrastructure will start. After the cloud VMs have been provisioned and the Docker host software downloaded and installed, the service should report successful startup and will look like this:

    Brooklyn receives and displays data about the health and usage each host

    Note the docker.machine.* sensor data for the Docker hosts. This shows CPU and memory usage for the cloud virtual machine running the Docker service.

    Deploying a Simple Application

    The following YAML blueprint describes a Tomcat web application server deployment, configured with a War file. It can be copied and pasted into the Brooklyn ‘Add Application’ dialog as-is.

    name: "Tomcat Web Application"
    location: my-docker-cloud
    services:
    - serviceType: brooklyn.entity.webapp.tomcat.TomcatServer
      brooklyn.config:
        docker.dockerfile.url:
          "https://s3-eu-west-1.amazonaws.com/brooklyn-clocker/UsesJavaDockerfile"
        wars.root:
          "https://s3-eu-west-1.amazonaws.com/brooklyn-clocker/hello-world.war"
        jmx.agent.mode: "JMXMP"
    

    The docker.dockerfile.url configuration entry points to a Dockerfile that has been configured with the commands required to pre-load the OpenJDK Java packages required for a Java application. (This is generally the slowest part of deployment, and further enhancements could be made to the Dockerfile to include the Tomcat archive itself and so on.)

    The default strategy of filling Docker hosts to capacity before provisioning the next will be used when deploying the blueprint to the my-docker-cloud location. Other fill strategies are available, including breadth-first (fill all hosts equally) and CPU-first (fill least used CPU first).

    As this is the first time this Docker file has been used on this host, a new Docker image will be built. Subsequent deployments reuse images to speed up deployment.

    Once the Docker container has been provisioned (a few seconds, rather than the usual minutes for a cloud VM) the normal Brooklyn process of creating a Tomcat server takes over (downloading a binary from an Apache mirror, expanding it and running the catalina.sh startup script).

    This process takes less than a minute, and once complete the Tomcat server will be available and serving the configured web application.

    The Tomcat Server is running. Open the mapped.webapp.url in a browser for the Hello World page.

    This Tomcat process uses port 8080. Those familiar with Brooklyn will note the additional sensor –  mapped.http.port – showing a port numbered in the 49000-65535 range. These are the mapped Docker ports on the Docker Host’s public IP address, and are used to to communicate with the running processes in the container, and to communicate across container.

    The JMX data has been obtained in a similar way, with Brooklyn connecting to the mapped.jmx.direct.port by way of the JMXMP URL sensor. These mappings all take place transparently, allowing the use of unmodified existing Brooklyn/CAMP blueprints for simple application deployments.

    Deploying a Complex Application

    This example is an elastic web application containing an SQL database, a load balancer and a cluster of application servers. It is a more complex application, uses multiple entity types, and illustrates how Docker Cloud provides more flexibility than manually configuring Docker hosts.

    This application blueprint is defined in the Java class brooklyn.clocker.example.TomcatClusterWithMySql (source) and is configured in the application catalog.

    To start it up, select Add Application from the Brooklyn console and choose Elastic Web Application from the list of blueprints.

    Select my-docker-cloud as the deployment location, and then choose Finish to start the application.

    Configure elastic web application parameters

    The individual containers are be provisioned in the Docker hosts, and appear as entities in the management tree. Each container has a sensor named docker.container.entity which links to the entity that has been deployed to that container.

    Individual containers are displayed under their host in the management tree.

    Currently Clocker uses a vanilla Dockerfile, downloading and installing software for each entity after the container has started. This takes a couple of minutes, and is an area marked for attention in the roadmap.

    The running application provides a URL that points to the Nginx load balancer, which can be opened to access the application. As this is running in a container the webapp url is the mapped public URL, available on the web application cluster as mapped.webapp.url.

    Access the cluster using the mapped.webapp.url on the Nginx load balancer sensors

    How this Works

    Brooklyn uses Apache jclouds, a cloud API agnostic library, to provision and configure secure communications (SSH) with cloud virtual machines.

    The Docker architecture provides ‘containers’ on ‘host’ machines. Brooklyn provisions cloud machines using jclouds and uses them as Docker hosts.

    Simple Docker architecture diagram

    Brooklyn uses a Dockerfile which makes an SSH server available in each Docker container. The container can then be treated like any other virtual machine. To make this fully transparent, a driver was developed for jclouds that allows provisioning of containers on a particular host with Docker installed, using the same API as any other cloud. It’s slightly meta, but it works well in practice.

    Brooklyn receives sensor data from, and can effect change in:

    • The overall application
    • Each cloud machine (Docker host)
    • Every Docker container
    • Each piece of software making up an application

    This makes it possible to automatically manage distribution of the application across the Docker Cloud.

    Further Reading

    The AMP for Docker post explains the jclouds driver, and Implementing a Docker Cloud with Apache Brooklyn explains the Brooklyn architecture with far greater technical detail.

    Roadmap

    This is the first release of Clocker, and development is ongoing. We would greatly welcome your thoughts and contributions (github).

    Some of the features to look out for in the next releases will be:

    • More placement strategies and policies for container management. Currently these include breadth first, depth first and CPU usage aware strategies. But it will be possible to do RAM and IO based selection.
    • Implementation of affinity and anti-affinity APIs and a simple DSL to control container placement based on deployed entities and applications, and allowing blueprints to specify their affinity preferences on a per-entity basis.
    • Improvements in the jclouds-docker driver will allow more control over Docker container provisioning, such as CPU shares and memory allocation.
    • Docker image repository integration will allow images to be pulled from centrally configured locations, and shared between different hosts in a Docker cloud.
    • Adding integration with software-defined networking (SDN) services such as Open vSwitch or OpenContrail will allow isolation and control of container network traffic, as well as easier communication between containers on different hosts, by introducing a shared Docker VLAN.
    • Seamless integration of all existing Brooklyn blueprints with the Docker infrastructure.

    Summary

    Clocker allows application blueprints to be deployed to a cloud of Docker containers, spread out intelligently across Docker hosts on managed cloud VMs.

    An application blueprint is able to treat the Docker infrastructure “Docker Cloud” as though it is a homogenous set of virtual machines in a single location, and then Docker Cloud can make intelligent decisions about container provisioning and placement to maximise resource usage and performance.

    Get Involved

    The code is available on GitHub under the Apache 2.0 license, so please fork it, and contribute by creating pull requests or opening issues.

    To find out more about Clocker either contact us, look for us on Freenode IRC channel #brooklyncentral or join us on the Apache Brooklyn mailing list.

    Acknowledgements

    Andrew Kennedy @grkvlt with support from Andrea Turli @andreaturli

    Apache Brooklyn is an effort undergoing incubation at The Apache Software Foundation (ASF). Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.

Latest Tweets

ARCHIVE