Web Hosting with Amazon S3 and Route 53

S3 logoDemo 6 went on the back burner while I re-hosted my site with S3, Route 53, and CloudFront. I found that adding 99,000 GeoJSON files into a child folder of an Intellij project is problematic. Indexing all those folders and files caused Intellij to become unresponsive, so I wanted to move my content elsewhere.

I decided to move my data files to Amazon S3 into a bucket named data.exploringspatial.com. That worked great. Publishing that many files to S3 was time consuming, but since they don’t change often I decided that was okay. I updated all the web site URLs and I was off to the races.

Next, since my web site is static, I decided to move it from Amazon EC2 on to S3 too. Moving the site to S3 frees up my EC2 instance for hosting a other interesting software, like ElasticSearch. There is no sense paying for more than one EC2 instance for a demo site.

The process was pretty painless. First, I created buckets for my data and my web sites:

Next, I created a hosted zone using Route 53, mapping my URLs to my buckets.


I transferred my domain names from GoDaddy and Networks Solutions to Amazon Route 53.


Finally, I switched my CDN from CloudFlare to AWS CloudFront.


That’s all it took. You’ll find plenty of instructions the Amazon AWS web site.

Now… back to my demos.

How to Display Feature Collection GeoJson with Leaflet’s Marker Clusters

I enjoyed playing with Leaflet’s marker cluster plugin this weekend while writing a new demo, Demo 5, that shows how to load feature collection GeoJson into a Leaflet map layer.

Leaflet Marker ClusterIt was surprisingly easy to setup. First, I updated my GeoJson utility project to create feature collection GeoJson loaded with 768 of my runs from Garmin Connect. The features in the feature collection are summaries. They contain the same properties as the activity detail GeoJson from Demo 4, except that they have starting latitude/longitude instead of a polyline.   Next, I generated individual feature GeoJson files for each run with the full polyline. Finally, I added the new GeoJson files to my web site at http://www.exploringspatial.com/activity/.

A Backbone Collection, Activities.js, uses AJAX to fetch the activity summaries. The collection is passed into the ActivitiesMapLayerView.js. The code in its render function is simple:

render: function() {
    var _self = this;
    geoJsonLayer = L.geoJson(this.collection.toJSON(),{
        onEachFeature: _self.onEachFeature
    this.activitiesLayer = L.markerClusterGroup();
    this.map.on('popupopen', function(event) {_self.onPopupOpen(event);});
    $('.returnToSearch').on('click', '.returnTrigger', function(event){
  1. L.geoJson – Create a map layer by passing in the activity collection JSON along with an onEachFeature function (see below).
  2. L.markerClusterGroup – Create a marker cluster map layer.
  3. addLayer – Add the GeoJson layer to the marker cluster group.
  4. addLayer – Add the marker cluster group to the map.

The last two lines of code bind a listener to the popupopen event, and add a listener for the activity detail page event that returns to the main page.

The onEachFeature function is used to bind a popup to each feature marker on the map. It does some date and number formatting and then passes html into layer.bindPopup.

onEachFeature: function(feature, layer) {
    var date = new Date(feature.properties.startTime);
    var triggerId = feature.properties.activityId;
    var msg = [];
    msg.push('Start: ' + date.toLocaleDateString() + ' ' + date.toLocaleTimeString() + '');
    var dist = Math.round((feature.properties.totalMeters * 0.000621371)*100)/100;
    msg.push('Dist: ' + dist + ' mi');
    msg.push('Go to Activity');
    layer.bindPopup(msg.join(''), {maxWidth: 200});

The interesting bit happens in the onOpenActivity and renderActivity functions when a users clicks the “Go to Activity” anchor in the popup. The onOpenActivity function instantiates an Activity Backbone model and invokes its fetch function to go get the polyline for the selected activity id. The renderActivity function is passed in the success property of the AJAX call.

onOpenActivity: function(event, popup) {
    var location = popup._latlng;
    // Capture the current center and zoom to restore map later
    this.originalCenter = this.map.getCenter();
    this.originalZoom = this.map.getZoom();
    this.activity = new Activity({activityId: event.target.id});
    var _this = this;
        success: function () {

When the polyline is successfully returned from the server the renderActivity function creates a new activity map layer to swap with the marker cluster map layer. A feature GeoJson map layer is added to the map, along with the start and end markers.

renderActivity: function() {
    if (this.map.hasLayer(this.activitiesLayer)) {
    var props = this.activity.get('properties');
        [props.get('minLat'), props.get('minLon')],
        [props.get('maxLat'), props.get('maxLon')]
    var style = {
        color: '#FF0000',
        weight: 3,
        opacity: 0.6
    this.activityLayer = L.geoJson(this.activity.toJSON(), 
        {style: style}).addTo(this.map);
    var polyline = this.activity.get('geometry').get('coordinates');
    var startPoint = polyline[0];
    var endPoint = polyline[polyline.length - 1];
    this.activityStart = L.marker([startPoint[1], startPoint[0]], 
        {icon: this.startIcon}).addTo(this.map);
    this.activityEnd = L.marker([endPoint[1], endPoint[0]], 
        {icon: this.endIcon}).addTo(this.map);

Once the user clicks Back to Search, this is the block of code that restores the marker cluster layer. The activity feature layer, start and end markers are removed (not shown), and then the marker cluster layer is added back to the map and the original center and zoom are restored.

 if (this.originalCenter != null && this.originalZoom != null) {
    this.map.setView(this.originalCenter, this.originalZoom, {animate: true});
    this.originalCenter = null;
    this.originalZoom = null;

That’s it! Be sure to go to the demo, http://www.exploringspatial.com/#demo/5, and click around.

Rendering a GeoJSON layer in Leaflet

The addition of a GeoJSON layer showing a Garmin activity polyline completes my example of using Leaflet with Bing, Google, and OSM base layers.

Static GeoJSON Leaflet map with Google, Bing and OSM base layers.

Static GeoJSON Leaflet map with Google, Bing and OSM base layers.

I did a lot work to get to this point. Some tasks were completely unrelated to the demo, like converting my website to a single-page Backbone application. Other tasks were behind the scenes, like writing a utility to convert Garmin FIT files into GeoJSON. A great deal of my time was spent recreating the Garmin-style map controls to switch between Google, Bing, and OSM. Eventually, I want to add the Google bicycle path layer and fix a bug I found today on the menu controls, but that can wait.

Once I was ready to add the polyline and custom pin icons, coding was a breeze. All that needed to be done was to extend L.Icon for the custom pin icons and drop the polyline GeoJSON into the L.geoJson function as shown below:

    var ActivityMapLayerView = Backbone.View.extend({

        initialize: function(args) {
            this.map = args.map;
            var CustomIcon = L.Icon.extend({options: {
                iconSize: [33, 50],
                iconAnchor: [16, 49]
            this.startIcon = new CustomIcon({iconUrl: 'media/pin_start.png'});
            this.endIcon = new CustomIcon({iconUrl: 'media/pin_end.png'});

        render: function() {
            var props = this.model.get('properties');
                [props.get('minLat'), props.get('minLon')],
                [props.get('maxLat'), props.get('maxLon')]
            var style = {
                color: '#FF0000',
                weight: 3,
                opacity: 0.6
            L.geoJson(this.model.toJSON(), {style: style}).addTo(this.map);
            var polyline = this.model.get('geometry').get('coordinates');
            var startPoint = polyline[0];
            var endPoint = polyline[polyline.length - 1];
            L.marker([startPoint[1], startPoint[0]], {icon: this.startIcon}).addTo(this.map);
            L.marker([endPoint[1], endPoint[0]], {icon: this.endIcon}).addTo(this.map);


The GeoJSON is passed into L.geoJson along with style properties to control the color, weight, and opacity of the polyline. That is all it took!

Now it is time to being exploring the plethora of Leaflet plugins supporting interactive maps. I’m looking forward to getting started.

Converting Garmin FIT activities into GeoJSON

My demo accomplished what I wanted to do with Leaflet regarding Bing, OSM, and Google base maps. The demo is at http://www.exploringspatial.com/#demo/4. The only base map plumbing that remains is to figure out how to add Google’s bicycle path layer.

The next step is to display the activity polyline as a Leaflet layer instead of adding it directly to Google, Bing, or OSM maps via the vendors’ map APIs. With Leaflet, the base map is simply the background for the vector graphics displayed on a Leaflet map layer in the foreground.

Using GeoJSON with Leaflet

First, I needed to produce GeoJSON as described here: http://leafletjs.com/examples/geojson.html. GeoJSON is a JSON standard for geometry types (see http://geojson.org). The geometry types are:

  • Point
  • LineString
  • Polygon
  • MultiPoint
  • MultiLineString
  • MultiPolygon

Geometry types may be combined with additional properties using Feature or FeatureCollection. Leaflet has a map layer type that can consume Feature JSON or FeatureCollection JSON.

Garmin Activity FIT File in –> GeoJSON out

Modern Garmin fitness devices store data in a compressed binary format called the FIT protocol (see http://thisisant.com).

My first task was to create a utility to read Garmin activity FIT files and produce Feature GeoJSON. The code is on GitHub: https://github.com/smitchell/garmin-fit-geojson.

I used my 2012 Little Rock Marathon activity to test the GeoJSON utility. The FIT SDK decodes the activity FIT file. The GeoTools FeatureJSON class outputs the GeoJSON.

There are two steps to creating GeoJSON using the GeoTools API:

  1. Define a feature type (schema) for the feature properties.
  2. Build the feature following the feature type definition.

Defining a Simple Feature Type

The SimpleFeatureTypeBuilder is used to layout the schema for the Feature.

public SimpleFeatureType getFeatureSchema() {
    final SimpleFeatureTypeBuilder simpleFeatureType = new SimpleFeatureTypeBuilder();
    simpleFeatureType.add("geom", LineString.class, DefaultGeographicCRS.WGS84);
    simpleFeatureType.add("name", String.class);
    simpleFeatureType.add("activityId", Long.class);
    simpleFeatureType.add("activityName", String.class);
    simpleFeatureType.add("sport", String.class);
    simpleFeatureType.add("startTime", String.class);
    simpleFeatureType.add("totalMeters", Double.class);
    simpleFeatureType.add("totalSeconds", Double.class);
    simpleFeatureType.add("minLat", Double.class);
    simpleFeatureType.add("minLon", Double.class);
    simpleFeatureType.add("maxLat", Double.class);
    simpleFeatureType.add("maxLon", Double.class);
    return simpleFeatureType.buildFeatureType();

Building the Feature

The SimpleFeatureBuilder builds the SimpleFeature following the feature schema definition.

public SimpleFeature buildSimpleFeature(final FitActivity fitActivity) {
    final SimpleFeatureType featureSchema = getFeatureSchema();
    final SimpleFeatureBuilder builder = new SimpleFeatureBuilder(featureSchema);
    builder.set("activityId", fitActivity.getActivityId());
    builder.set("sport", fitActivity.getSport());
    builder.set("startTime", fitActivity.getStartTime());
    builder.set("totalMeters", fitActivity.getTotalMeters());
    builder.set("totalSeconds", fitActivity.getTotalSeconds());
    final Coordinate[] polyline = fitActivity.getPolyline().toArray(
        new Coordinate[fitActivity.getPolyline().size()]);
    final Geometry geometry = simplifyLineString(polyline);
    final Coordinate[] boundingBox = generateBoundingBox(geometry);
    builder.set("minLat", boundingBox[0].y);
    builder.set("minLon", boundingBox[0].x);
    builder.set("maxLat", boundingBox[1].y);
    builder.set("maxLon", boundingBox[1].x);
    return builder.buildFeature("0");

The final result, formatted for readability, can be found here: http://www.exploringspatial.com/feature.json

This is a partial listing:

  "type": "Feature",
  "geometry": {
    "type": "LineString",
    "coordinates": [

... many points omitted.

  "properties": {
    "activityId": 155155867,
    "sport": "RUNNING",
    "startTime": "2012-03-04T14:02Z",
    "totalMeters": 42453.58984375,
    "totalSeconds": 15162.140625,
    "minLat": 34.73176879808307,
    "minLon": -92.34434505924582,
    "maxLat": 34.78602569550276,
    "maxLon": -92.25817699916661
  "id": "0"

Now I’m ready to create a Leaflet GeoJSON layer to display the polyline!

Relaunching Exploring Spatial as a Single Page Web Site.

After a bit of a break from my blog I have returned. There is a lot on the plate for work in 2015, and I need to get out ahead of that development. 2015 will be the year that we begin to introduce Leaflet to our maps, so it’s time that I returned to my exploration of Leaflet and its plugins.

Begin Again

Over the last couple of weeks I did several things to improve Exploring Spatial. I converted the site to a single page web application, I reworked that HTML/CSS on my site to utilize more real-estate, and I made everything available on GitHub.

Bigger is Better

The CSS template I was using had a fixed height and width. I’ve expanded it to be full page to allow for bigger maps. There are some wrapping issues that must be fixed, but overall the result was what I wanted. One of my TODO tasks is to add some JavaScript to size the height and width of the map container DIV tags from the document viewport. Some map providers don’t work well with percentage sizes, so I want to plug in the height and width in pixels and redraw the map when the browser window is resized.

“GIT”ting Better

The entire site is now available at GitHub: https://github.com/smitchell/exploringspatial. Not only can you find all the demo code there, but theoretically you can contribute your own code to the demos too.

One Page Fits All

My biggest change was to convert the entire web site to be a single page app using Backbone.js. I did this for no other reason than this is the direction we are heading at work. At first I resisted making this change because it makes the demos a little harder to follow if you are not familiar with Backbone.js.

Eventually, I decided that my website is as much a personal journal as it is a public resource, and the kind of people that might be interested in my journal are professionals who can find their way around the Backbone JavaScript files.

Enjoy the new site. You can expect a lot of activity on the Leaflet demos in 2015.

OpenSUSE on VirtualBox


After some Memorial Day weekend trial and error I’ve settled on a Linux distribution for my Oracle MapViewer lab. Neither Ubuntu 14.04 nor Fedora 20 performed well under VirtualBox. In the case of Ubuntu it was the lack of 3D support for Unity, which I think can be fixed by rebuilding the VirtualBox Guest Additions from the kernel source, but I wasn’t able get VirtualBox to find the source. Based on some other distro instructions I think I just needed to do something like this: “KERN_DIR=/usr/src/kernels/`uname -r`.”

In any case, I had good luck with openSUSE 13.1. Oracle SQL Developer connects to Oracle 12c and the OS performance is good even with both VMs running. Now, my Oracle database and Oracle client environments are both running in virtual machine images that can be moved from one computer to another. More importantly, I can run Java 7 on openSUSE and Java 6 on Ubuntu, my host OS, where I do some customer development.

This illustration depicts the resulting spatial lab environment.spatiallab

Picking a Linux distro took longer than I thought it would. I’m reading to move on now.

Is Oracle MapViewer a Fit for Me?

Recently I began evaluating Oracle MapViewer to determine if it is the right tool for me. I’m looking for a mapping API that will allow me to manage vector graphics, points and polylines, independently from the base map. Today, I write too much repetitive code for map providers like Google Maps and OSM.  Oracle MapViewer it is a good tool for people working in a Java EE environment. It also has a place as part of Oracle’s Business Intelligence Suite, and is a very handy adhoc spatial data viewer, but does it belong in my website development toolbox?

current_map_stackMy personal preference is open source and lightweight, as this graphic illustrates, with a client-side JavaScript MVC framework making AJAX calls to JAX-RS web services. The RESTful web services should run in a standard Java web container and not depend on Java EE server containers.

In order to do vector graphics and event handling for polylines and points independently from the base map, my map API abstraction layer must be replaced. Ideally, the replacement framework should be purely JavaScript and should not require any server-side changes, something like Leaflet.

MapViewer Components

MapViewer is much more than just a JavaScript framework for maps. It has several server-side components that support its client-side AJAX-based JavaScript API:

  • A core rendering engine Java library, SDOVIS, that performs cartographic rendering. A servlet exposes the rendering functions to Web applications.
  • Application programming interfaces for programming MapViewer features. These APIs include XML, Java, PL/SQL, and an AJAX-based JavaScript API.
  • A graphical Map Builder tool that enables you to create map symbols, define spatial data rendering rules, and create and edit MapViewer objects.
  • Oracle Maps, which includes map cache and FOI (feature of interest) servers that facilitate the development of interactive geospatial Web applications.

The MapViewer architecture is not targeted towards my ideal environment of simple Java web containers. The prefered environment is Oracle WebLogic or Glassfish.

That being said, MapViewer does support Tomcat, but to what extent I’ve yet to discover. The following table is from the Map Viewer J2EE Servers Support page:

J2EE Server Name and Version

 Oracle WebLogic Server  10g and Later
 GlassFish  3.1.2 or later
 Oracle Application Server  10g (10.1.3.x)
 WebLogic  9
 Oracle OC4J Standalone  10g (10.1.3.x)
 Apache Tomcat  6.x and Later
 WebSphere  6.x and 7.x
 JBoss  4.x and Later

This architecture is illustrated in this graphic from the Oracle Fusion Middleware MapViewer Primer.

mapviewerIn my current work environment, pools of servers support different domains of our site. There is nothing to say that one pool of servers cannot include Java EE servers to handle the mapping load. It would still serve to isolate the mapping services from other server pools as well reduce the workload on the other servers. The business justification for the new pool of mapping servers depends on how many MapViewer features add value to our site, things like dynamic heatmaps, for example.

I’ll start my deep dive with the tutorials that come with the project, and then will explore some of the extensive features list in MapViewer data sheet.