Processing & Open Data

top_image_bikehire_

///////////////////////////////////////////////////////////////////////
——— OPEN DATA BIKE HIRE ————-
///////////////////////////////////////////////////////////////////////

Intermediate tutorial using the Unfolding Library
developed by Till Nagel.

http://tillnagel.com

Original Tutorial by Julien Gachadoat.

///////////////////////////////////////////////////////////////////////

This tutorial is the result of work done during the Processing Workshop which took place in Rennes 4th and 5th November 2011. The original French version is available here. This English version is an adaptation of Julien’s tutorial using data from the Barclays Cycle Hire docking station.

>>>You can download the Processing sketches for this tutorial here.

///////////////////////////////////////////////////////////////////////

Contents

– Introduction
– Getting the data
– Understanding the data
– Mapping the data
– Visualisation & Interaction
– In real-time
– Notes
– Download

Introduction

The aim will be to use Processing to gain access to the Open Data database for the Tfl Cycle Hire and create a visualization of the data collected. For this, I have chosen to use a simple API developed by Sachin Handiekar. and to visualize this on a map of London.

Getting the data

To gain access to the data there is no need to sign up or generate an API key. We simply call the data using a web address (or URL). In this case it is http://api.bike-stats.co.uk/service/rest/bikestats?format=xml. We can therefore start by writing a simple Processing program that will connect us to this site and retrieve the data we want.

String url = "http://api.bike-stats.co.uk/service/rest/bikestats?format=xml";
String lines[] = loadStrings(url);
for (int i=0; i print(lines[i]);

The data request is done with a single command in Processing  by using loadStrings. This will send us back an array of characters (String), each corresponding to a line of data from the API. The format of this data is an XML file which for the moment we will just print to the Processing console using the print() function.

This is what the above code prints out for us :

xml_println
See Sketch _01_Bikes_stats_Connecting.

Understanding the data

Now, if we want to begin to use this data in a more explicit way we need to access each piece of information listed above. To do this we use Processing’s in-built methods and objects for working with XML. The procedure remains the same in that we use an URL to get the data except that we no longer call loadStrings and the function and format of the result changes.

1 XMLElement xml = new XMLElement(this, url);

If we look a little closer to the result we get above, we’ll notice the document contains tags for each piece of data: name, latitude, longitude, bikesAvailable and emptySlots. With Processing, we can navigate the XML document using these tags. For now we just want to access the number of bike stations available in London. So we are going to ask it to give us an XMLElement array of all the tags it can find with this information.

XMLElement[] stations = xml.getChildren("dockStation");
int nombreStations = stations.length;

See Sketch _02_Bikes_stats_Stations.

We now have access to each bike station represented in the XML document of our program. From here we are going to access each piece of data that englobes this object : name, address, latitude and longitude for geolocalisation, number of bikes available and the number of free docks in that station.

In order for us to save this data we are going to create a class called ‘Station’ in which each instance will be made of information held in the XML document. Each instance of our Station class will therefore be saved in an ArrayList so that we can access this data during the running of our program.

Mapping the data

We now have all our data structures in place for us to visualize our data on a map. For this we are going to use the Unfolding library by Till Nagel to load a map and position our bike stations using a simple ellipse.

// Setting up the Map of London
// London is situated at 51.500 latitude and -0.117  longitude
carte = new de.fhpotsdam.unfolding.Map(this);
carte.zoomAndPanTo(new Location(51.505f, -0.137f), 12);

See Sketch _03_Bikes_stats_Stations_Maps.

Unfolding simply lets us load maps in Processing and translate geographic positional data into onscreen pixel coordinates. The library comes with a few examples to get you up and running with loading a map and adding some data points.

london_map

See Sketch _04_Bikes_stats_Availability. This sketch parses the availability data for each station to the opacity and size of each circle giving a more explicit visual of the number of bikes in each area. The bigger and redder the circle the more bikes there are.

Visualisation & Interaction

We can take a step further and create some rollovers which will help us visualize necessary information for each bike station. To do this we will add some text that will show us the name of each station and the number of remaining bikes. We can get this information and show it directly using the text() function and by calling the necessary fields from our Station class. The dist() function lets us calculate the distance between the ellipses and our mouse position so that when the mouse hovers over an ellipse it shows us the necessary info.

if (dist(xy[0], xy[1], mouseX, mouseY) < radius+2) {
fill(255);
// Show the data value
text(s.name, 90, 55);
text(s.bikesAvailable, 140, 70);

Another technique for visualizing this kind of data on a map is to use the Triangulation library which enables us to visualize the distance between each station. By connecting these together we get an impression of the concentration of stations too. This is helpful of we want to visualize the density of stations within a city. If you visualize London during the day and compare that with the night, you see a clear difference.

london_triangulation

The image above simply shows all the stations and their distances between each in their immediate vicinity. Already from this image we can conclude that London has a pretty uniform concentration of bike stations with obvious holes where the two largest parks are. Another triangulation technique that could be used would be to associate the peak of each triangle (and therefore each station) to a colour which symbolizes the availability of bikes in a station. A dark colour (black) therefore marks less bikes available, a light color (red) marks more availability. Perhaps one to try and suss out for yourself.

Real-Time

Certain data is updated in real-time so perhaps one of the next steps would be to program a real-time app. I’ll let you do that ;–)

day-night

Notes

The sketches for this tutorial were made using version 1.5.1 of Processing and are not compatible with versions 2.0 which has just been released.

Download

Sketches
Libraries Unfolding, GLGraphics, Triangulation.