Setting up DC/OS on Google Cloud Platform

dcos%2520logo%2520%2528horizontal%2529-skvoop_zc-1

googlecloud

My previous post discussed setting up your Google Cloud Platform account to run DC/OS. Now we move on to installing DC/OS. Before starting, if you are new to DC/OS, take a moment to familiarize yourself with DC/OS concepts and DC/OS architecture. Also, have a look at The Mesosphere guide to getting started with DC/OS written about setting up DC/OS on AWS. Even though those instructions use shell scripts to generate the nodes, and the Google Cloud Platform project uses Ansible scripts, the concepts are the same.

Create the DC/OS Bootstrap Machine

The bootstrap machine is where DC/OS install artifacts are configured, built, and distributed. The DC/OS installation guide includes setup instructions for many cloud platforms. The instructions for Google Cloud Platform rely on Ansible scripts from a dcos-labs project on GitHub . At the time of this writing, the GCE set-up page in the DC/OS 1.8 Administrative guide was a little out of sync with the project README file, so I followed the README file instead.

Start from the Google Cloud Platform dashboard. Make sure the Google Cloud Platform project that you created for DC/OS is selected. Click the “hamburger” menu icon in the upper left corner of the page and then select Compute Engine from the menu. You need an “n1-standard-1” instance running Centos 7 on a 10 GB disk. Follow the README instructions carefully on how to configure this new VM instance.

Connecting to the DC/OS Bootstrap Instance

Once the DC/OS bootstrap instance starts, it will appear on the GCE VM Instance page with a green checkmark to the left of its name. To connect to the machine, either download the Cloud SDK to your desktop computer or activate Google Cloud Shell from the >_  icon near the right side of the page header.

shell_icon.png

Cloud SDK runs in an f1-micro Google Compute Engine and displays a command prompt at the bottom of the console page. Open the SSH drop-down list to the right of the bootstrap instance and pick View vcloud command. When the dialog opens pick RUN IN CLOUD SHELL (or use one of the other options to ssh into the bootstrap instance).

runinshell.png

After connecting to the DC/OS Bootstrap instance, follow the instructions in the README file until you reach the configuration of the “hosts” file…

Ensure the IP address for master0 in ./hosts is the next consecutive IP from bootstrap_public_ip.

Defining the Master Node(s)

Unlike agent nodes that can be added as needed, once the master node cluster is installed, the number of master nodes cannot be changed without reinstalling the cluster. A cluster of master nodes needs a quorum to determine its leader, so you must install an odd number of nodes to avoid a “split brain” situation.

It is common to use three master nodes; however, if you are using the Google Cloud Platform free trial, it only allows eight CPUs. I tested with a single 2-CPU master node, two 2-CPU private agent nodes, and one 2-CPU public agent node.

The “hosts” file defines the number of master nodes in the DC/OS cluster. The IP addresses of the master nodes should be contiguous addresses from the IP address of the bootstrap machine. Here is an example of a three-node cluster.

[masters]
master0 ip=10.128.0.3
master1 ip=10.128.0.4
master2 ip=10.128.0.5
[agents]
agent[0000:9999]
[bootstrap]
bootstrap

Defining the Agent Nodes

The README file says:

Please make appropriate changes to dcos_gce/group_vars/all. You need to review project, subnet, login_name, bootstrap_public_ip & zone

The variables are explained at the bottom of the README file. At a minimum, you must set the following variables in the “./group_vars/all” file.

  1. Project.
  2. Login name.
  3. Bootstrap public IP.
  4. Zone.

You may also want to tweak the template settings of the master and agent nodes, but it is not required. I changed the “gcloudimage” property to match the Centos 7 image that I found in the Compute Engine –> Image listing.

Centos7.png

Install the DC/OS Cluster

You are now ready to install the DC/OS cluster using the Ansible script.

ansible-playbook -i hosts install.yml

You’ll see some error messages as it attempts to shut down instances that don’t yet exist, and then it will begin creating and configuring the new instance(s). When the script completes, the new instance(s) will appear in the Compute Engine VM Instance list.

Install the Private Agent Nodes

As previously mentioned, I created two private nodes with the following command.

ansible-playbook -i hosts add_agents.yml --extra-vars \
"start_id=0001 end_id=0002 agent_type=private"

Install the Public Agent Nodes

If you are using the free trial, things get a little tricky here. Your public agent node creation will fail because it puts you over the CPU quota. To get around the quota, I stopped agent0002 first, and then I created the public node. Once the public node was created, I shut down the bootstrap machine to free-up one CPU, and then I restarted agent0002. Here is the command to create the public node.

ansible-playbook -i hosts add_agents.yml --extra-vars \ 
"start_id=0003 end_id=0003 agent_type=public"

Once the public node is running, edit the instance configuration from the GCE VM Instance page and check the https option (temporarily do the same for the master0 instance).

https.png

Open the DC/OS Console

Now we are ready to launch the DC/OS console. Eventually, we will do this with a virtual host pointed at the Marathon load balancer running on the public agent node, but for now, we will open the DC/OS dashboard directly from the master node.

directshot.png

After signing in with the Google account that you used for Google Compute Cloud, you will see something like this.

console.png

Installing the Marathon Load Balancer

To try it out, let’s install the Marathon Load Balancer on the public agent node. Select Services from the left side menu, click Deploy Service and then click the link that says install from Universe. When the page opens, search for “marathon.” The search results should look something like this.

install_marathon-lb.png

Click Install on the marathon-lb row. When the dialog opens, click Install Package.

marathon-lb.png

Return to the Services page, and in a moment, you should see this.

installed.png

You are now ready to being using DC/OS with your own artifacts, which I will discuss more in my next post.

Collaborating on Google Cloud Platform

googlecloud.png

As described in my previous post, we are doing a bake-off between Amazon Web Services, Microsoft Azure, and the Google Compute Platform to see which platform we like best for DC/OS. I’ve used AWS off-and-on for years, but this was my first time using the Google Cloud Platform. I am very pleased with its features and especially with its user interface.

Getting Started

Getting started with Google Cloud Platform couldn’t be easier. Start out with the $300/8-week free trial. Google’s Getting Started document tells you everything you need to know. Google automatically gives you a default project when you sign-up, but I created a separate project for the bake-off to share wth my team.

team-icon.jpgCollaboration Required

Adding team members to the Google Cloud platform is simple. Just click Manage project settings from the Project card on the dashboard. Click the IAM menu item to bring up the IAM panel. Use its Add button to grant permissions to team members.

AddFolks.png

I added my co-workers with a mixture of Editor or Owner roles.  That was enough for all of us to be able to access the project and control Compute Engine resources.

card-in-use-32.png Not on my Dime

You must provide a credit card to sign-up for the free trial. Not only did I want to avoid any personal charges should we decide to move forward with Google Cloud Platform, I also wanted to avoid having to file company expense reports. Google Cloud Platform lets you do that by transferring the billing administrator duties for a project to somebody else.

To assign a new billing administrator you first need to set-up their permissions as described above. The billing administrator needs to have both Owner and Project Billing Manager roles.

ProjectBillingManager.png

Next, pick Billing from the main menu and add that person as a billing administrator. See Manage billing administrators. The billing administrator can now go into Google Cloud Platform and replace your credit card with the company credit card, plus all billing correspondence will go to them.

You’re Ready to Setup DC/OS

That’s it. You now have a collaborative cloud platform that is not on your dime. You are ready to get started installing DC/OS, which I will cover in my next post.

Diving into Microservices on DC/OS

DCOS%2520Logo%2520%2528Horizontal%2529-SkVooP_ZC (1).png

In March of 2016, I left Garmin to join former co-workers DaShaun Carter and David Kelly at a startup company. DaShaun worked for months setting up our environment and developing the software stack before David and I joined him. He built a hosted vSphere environment running across two data centers on which we run DC/OS, the Apache Mesos distributed system kernel.

small_h

We use Marathon to orchestrate microservice Docker containers. It lets us easily spin up and monitor multiple instances of each microservice. This allows us to roll out updates or restart instances without any downtime.

Modern Stack

angular.pngPart of what is so attractive about joining a startup is that there is no technology debt. Our microservices architecture is built using Spring BootSpring Cloud, and Spring Cloud Netflix. Our presentation layer is a mobile application written in Angular 2 and packaged for IOS and Android using Adobe PhoneGap. It’s fun being on the cutting edge!

Diversifying into the Public Cloud

Recently, we decided to move test and QA on to a public cloud platform to isolate our production environment, partly because it is cheaper than adding capacity to our hosted vSphere environment. This month we are doing a bake-off between AWS, GCE (Google Compute Engine), and Azure. David took AWS, I took GCE, and DaShaun took Azure.

The following series of posts will record my experience setting DC/OS on GCE.

 

Upgrading to Leaflet 1.0 Beta 2

This post is just a migration log. I’m very excited to be learning about Leaflet Hotline, but it requires Leaflet 1.0 Beta 2. The problem is that several of the other plugins that I use on http://exploringspatial.com are not compatibleLeaflet 1.0. Here is the run down of what I ran into.

Why are there problems?

I will try to keep this post up-to-date with issues as I find them:

  • Multpolygon class is gone – The Mulitpolygon class is being replaced by enhancements to the Polygon class (see thread). If your plugin used Multipolygon, then you’ll need a new version of the plugin for Leaflet 1.0.
  • GeoJSONLayer.resetStyle(layer) is gone. I switched to layer.setStyle().
  • L.FeatureGroup.EVENTS no longer exists.
  • “Event.layer” is now called “Event.target” in mouse events.

Bing, Google and  Yandex Plugins

Pavel Shramov and Bruno Bergot provide Leaflet plugins for Bing, Google, and Yandex maps. When you switch to Leaflet 1.0 the Google Tiles layer will break (I’ve just dumped Bing for the time being since it has not been addressed). When you add a Google layer it to your Leaflet 1.0 map you will see an error in Leaflet-src.js  at line 2382 because callback is undefined.

    whenReady: function (callback, context) {
        if (this._loaded) {
            callback.call(context || this, {target: this});
        } else {
            this.on('load', callback, context);
        }
        return this;
    },

Fortunately, Bruno Bergot is already on top of this and you can pull his patch down for the time being: https://github.com/shramov/leaflet-plugins/pull/175.

Leaflet Point-in-Polygon

Leaflet-pip depends on the Mutlipolygon class, so it broke http://exploringspatial.com/#demo/8 at line 15: “Uncaught TypeError: Expecting a function in instanceof check, but got undefined.” Specifically, it breaks at this line of code: “if ((l instanceof L.MultiPolygon…”

var leafletPip = {
    bassackwards: false,
    pointInLayer: function(p, layer, first) {
        'use strict';
        if (p instanceof L.LatLng) p = [p.lng, p.lat];
        else if (leafletPip.bassackwards) p = p.concat().reverse();

        var results = [];

        layer.eachLayer(function(l) {
            if (first && results.length) return;
            if ((l instanceof L.MultiPolygon ||
                 l instanceof L.Polygon) &&
                gju.pointInPolygon({
                    type: 'Point',
                    coordinates: p
                }, l.toGeoJSON().geometry)) {
                results.push(l);
            }
        });
        return results;
    }
};

There is not a patch yet, but I found the solution contained in this thread: https://github.com/mapbox/leaflet-pip/issues/8.  There is a post from user “nikolauskrismer” that replaces the entire code block starting at line 6: “var leafletPip = …” That got me past the error above.

Leaflet MarkerCluster

In http://exploringspatial.com/#demo/5 I used Leaflet.MarkerCluster, but in http://exploringspatial.com/#demo/6 I switched to Leaflet.PruneCluster for performance reasons. Upgrading to Leaflet 1.0 Beta 2 broke Demo 5. Leaflet.MarkerCluster caused the error “Uncaught TypeError: Cannot read property ‘trim’ of undefined” in leaflet-src.js at line 137. The cause was line 51 from leaflet.markercluster-src.js, “this._featureGroup.on( L.FeatureGroup.EVENTS, this._propagateEvent, this).” The problem is that L.FeatureGroup.EVENTS no longer exists in Leaflet 1.0. Since I was already using PruneCluster successfully with Leaflet 1.0 beta 2, and I didn’t see any patches, I dumped Leaflet.MarkerCluster and switched to Demo 5 to Leaflet.PruneCluster.

Mouse Events

Another issue that I ran into on http://exploringspatial.com/#demo/7 was that my mouseover and mouseout events were expecting “event.layer“, but with Leaflet 1.0 that has changed to “event.target” That was easy for me to switch. I was just calling event.target.style to highlight polygons on mouseover.

Reset Style

The final issue I hit was on http://exploringspatial.com/#demo/8. In my states map view I called GeoJSONLayer.resetStyle(layer) on all the state polygons each time a new race distance was selected. That no longer worked in Leaflet 1.0. I had to switch to layer.setStyle() to get around that.

That is all that I’ve found so far. All my changes are now live at http://exploringspatial.com. Let me know if you spot any problems that I missed.

 

Color Gradient Lines: Speed Bump

I was all set to start using Leaflet Hotline when I hit a speed bump. It does not appear to work with Leaflet 0.7.7. Unfortunately, some other plugins on my site do not work with Leaflet 1.0 Beta 2.

This is where having a demo “site” with an MVC framework, as opposed to simple standalone demo pages or in JSFiddle with straight HTML/javascript, becomes a problem. I’ve brought Leaflet and all of my plugins up to the latest versions, but I cannot run Leaflet 1.0 Beta 2 without causing problems for Leaflet PruneCluster and my Google/Bing plugins.

I’ll have to either, a) figure out whether RequireJS will allow me to essential overwrite “L” with a different version of Leaflet for Demo 10, or b) build a standalone page for Demo 10.

Grrrr!

Color Gradient Lines: Begin with the End in Mind

I’ve been looking into what it would take to draw map lines with color gradients, like the red/blue line to the left. My assumption was that this would require generating raster map tiles (something I know little about), but this week Tom MacWrite pointed out the demo that he wrote for MapBox. That, in turn, led me to look for Leaflet plugins that support gradient colors.

The first one I found is Leaflet.MultiOptionsPolyline.  The screen shot below is from its demo page. It looks interesting, but it’s not quite the smooth gradient effect I was after.

A more promising Leaflet plugin I found is Leaflet.hotline, by iosSphere of Cologne, Germany. Below is a screen shot from its demo page — much closer to the effect that I was looking for. Leaflet Hotline was inspired by the Leaflet.heat plugin. That makes sense to me now since heat maps also have smooth color gradients.

Clearly, a polyline drawn with a color gradient is possible with Leaflet, so now let’s step back and begin with the end in mind.

Objective

My objective is to convey information, like heart rate or pace, on top of the race path recorded by a GPS device. I’ll use pace for my demo. The map should show the runner where her pace was faster, and where it was slower.

The race path on the map should support interaction with other components on the page. For instance, it may not be enough to see speed alone. It would be nice to hover over the line on the map and see the corresponding point on an elevation chart. A slowdown may be the result of a steep incline.

Colors

The gradient colors need to convey additional information about the runner’s race without being distracting. The colors should not make the map harder to read, for example, blending into the background.  The Leaflet Hotline example used black outlines to solve that problem. Also, the colors should take color blindness into consideration.

Below is an example of line colors Google chose for the Bart Transit map. How do those work for red/green, or blue/yellow color blindness? Perhaps those shades are okay?

Awhile back I found an article, How The Rainbow Color Map Misleads, that explains why not to use rainbow colors. For more information see Rainbow Color Map (Still) Considered Harmful (Full discloser, I didn’t pay to download this report). There is a handy tool to help find alternative colors for your map called ColorBrewer. Another tool for picking gradient colors generally is uiGradients.

I stumbled upon this set of color-blind safe colors on Paul Tol’s website. The shades of blue and green from Paul Tol’s site are somewhat similar to the colors from Google’s Bart transit map above. I’ll try to incorporate some of these colors into my soon-to-come demo.

colourpalette_small

Color-blind Safe Palette from Paul Tol

Stay tuned for a demo at http://exploringspatial.com.

Privacy Geofencing

A question was posted yesterday on the Leaflet forum about doing geofencing with LeafletJS, so I posted this demo in response: http://exploringspatial.com/#demo/9.

Screen shot of privacy geofence demo

Privacy Geofence Demo

My demo scenario is that there is a user who likes to share her runs with friends on social media but doesn’t want everybody to see exactly where she lives, so she creates a privacy geofence around her house. The starting and ending sections of her run polyline are hidden once they enter the geofenced area. If the middle section of the run passes through the geofenced area it remains visible.

Here is a shout out to Roadrunners of Kansas City. The geofence boundary in this demo is around Sport+Spine, home-base to many of the Roadrunner runs. Thanks, Coach Amy!

For my example, I turned once again to the Leaflet-pip library, as I did for Demo7 and Demo 8. I use it a couple of different ways in this demo. First, I use it to locate runs for this demo whose start point falls within the demo’s privacy geofenced area. Second, when the user displays a run (by clicking the numbered buttons on the bottom of the page) I use Leaflet-pip again to test which points at the beginning and end of the run fall within the privacy geofence.

In the real world, you would always do the privacy filtering server-side, using something like GeoTools or spatial database queries, so that only the public sections of the polyline are exposed in the GeoJSON. I did this demo on the browser-side because my site is an Amazon S3-hosted static website, and also because the forum question was about doing geofencing with Leaflet.

The code for this demo can all be found in RightSideView.js. The render method does the following things:

  1. Creates the L.featureGroup, geoFence, and adds a L.polygon defining the geofence boundaries.
  2. Displays the geofence on the map.
  3. Loads the running log GeoJSON file from Demo 5.
  4. Uses leafletPip.pointInLayer to find runs that start in the geofenced area, and then adds a paging div to the bottom of the page to open each matching run.
  5. Loads the first run.

The interesting part of the code is in the onActivityFetched function. The “hidden” parts of the polyline are shown in gray on the map, so I needed to define three polylines: the hidden start, the visible middle, and the hidden end.

I loop through the polyline coordinates and add the points to the “fencedStart” array until I hit a point falling outside the geofenced area. After that, all points are added to a “middle” array.

Screen shot of the code to find the hidden start of the run.

Find the Hidden Start of the Run and the Middle

Next, I loop backward over the points collected in the middle array looking for the points at the end of the run to be hidden.

Screen shot of the code to find the ending section of the run to be hidden.

Find the End Section of the Run to Hide

Finally, now that I have the points to hide at the start and the end of the run, I just need to find the points in between to draw on the map in red.

Screen shot of code to find the middle of the run to add to map.

Find the Visible Section of the Run to Display on Map

Go to the demo and click on a few of the runs to see how it works. Doing this demo I learned that “Leaflet.PIP” (point-in-polygon), only works on polygons (go figure). I started out trying to use a L.circle for the geofenced area. That didn’t work.

The other shortcoming of this demo is that sometimes the first point outside the privacy geofence is some distance away, causing the visible line to stop short. That could easily be fixed with point projection along the bearing of the line between the two points that span the geofence boundary.

Happy Geofencing.