tomtaylor.co.uk

Hello, I'm Tom. I'm talking today about delighting with data.

You're probably here because you're a geek. That's cool, so am I. And if you're anything like me, you probably like data, and lots of it. I'm sure you know loads of fantastic examples of repurposing boring data for incredibly useful stuff.

But sometimes we geeks forget about all the delightful and beautiful things we can build. The things that aren't necessarily useful or purposeful, but pointless, silly and wonderful. So, I'm talking about building beautiful things out of (sometimes) boring data sources. I'll be talking less about design and visualisation, and more about projects and 'things'.

The first thing I want to talk about is making things talk.

So, this is Tower Bridge. I work near here and cycle across it twice daily. Everyone loves Tower Bridge, it's completely unique and a fantastic piece of Victorian engineering.

The bridge lifts four or five times a day, but you'd be surprised how many people live or work nearby and have never seen it do so.

So, one day over lunch, my colleague Tom Armitage decided to put it on Twitter - you can see it at @towerbridge.

It's not magic - the secret is that there's a schedule published online, and it's pulled down nightly and parsed for the boat times. It does not work, as one blogger suggested, with a manual operator inside the bridge. It's currently got 424 followers and was splashed across a few popular blogs recently.

One thing you'll notice is how it talks in the 1st person. This is really important. It projects a personality and a voice, and provides space for people to react to it and talk back to it. I've been known to twitter my annoyance at being stuck waiting for it close. I will make no comment on some people's opinions about the hidden sexuality of its constant opening and closing.

And this is the code. It's about 40 lines or something, and was built in about 20 mins. There are hundreds of ways to improve this - It's run by a cronjob every minute, which is horribly inefficient - but that's not the point. It works, and it's running. Don't let thinking get in the way of building - build, then think.

So, the thing with making things talk is that often the output is really easy. The hard bit is the input: getting the data and manipulating it into something usable.

Sometimes that input comes from dull spreadsheet-like web pages or databases, like @towerbridge, but with the price of interesting sensors dropping pretty rapidly, it's possible to build stuff that responds to touch, squeeze, sound, conductance, location or anything you can practically think of.

This is a great book. It's called Making Things Talk by Tom Igoe, and it's about connecting and communicating with physical objects to do fun and interesting stuff.

Quite a lot of the book is about this thing - the Ardiuno. It's an open source hardware prototyping platform, designed for rapid gluing together of inputs and outputs. It uses a language called Wiring, which is similar to Processing, and it's got a great development environment. It's about £18, available from Tinker.it.

And with it, you can build things like this. This is kit for a plant pot that twitters when it needs watering. It's made up of a condictivity sensor, an Arduino, an ethernet breakout for the Arduino, wiring and power. But let's face it, if you need to use twitter to garden, you're probably not on the path to horticultural greatness.

As you can tell, I really like Twitter as a delivery method. Twitter is about "what are you doing now?" - action, presence and the now. It's great for delivering timely information straight to people, and perhaps most importantly it's really easy to integrate with.

Making things listen is the next step. Putting data back into objects and for them to respond. "d towerbridge up" is the ultimate vision for @towerbridge.

So, the next bit is about discovery across space - not outer space, just discovering things around you.

Whilst we're nowhere near the semantic web, there are an increasing number of services providing data in useful, machine readable formats.

For example, some Wikipedia pages contain longitudes and latitudes. Geonames take regular dump of these pages, parse for co-ordinates, and then provide a web service allowing look up of Wikipedia entries around a specified place.

You may have heard of Fire Eagle from Yahoo. It's a location broker service, designed to sit between the user and the web services they use. You tell it your location, it can handle requests from other services for it, to the degree of privacy that you set. It can receive your locations in lots of different manners - postcode, longlat, mobile tower id. They abstraction all the difficult stuff away, leaving the web services to just get on and consume geo data.

As a developer, it's really easy to use. It uses OAuth for delegating access rights to services, which everyone seems to be moving towards nowadays.

One of the reasons I like it so much it that there can only be constant improvement in the way data gets fed into it. For example, it's a bit boring having to type in post codes and the like, but there's a hack out there for the iPhone allowing it to seamlessly update your location at regularly intervals from your current mobile tower ID or wifi point.

If you combine the Geonames Wikipedia data with Fire Eagle you can build something like this. This is wikinear.com, written by Simon Willison. It's quite simple - it shows you Wikipedia pages near you. Awesome!

Written in 200 lines of Django, he wrote up a great blog post explaining how it works. Basically, there's no database - it's just web services glue. This is the future.

I liked it so much that it was inspiration for this. This is pubs.iamnear.net. It grabs pub data from OpenStreetMap and provides a really simple mobile interface for finding your nearest pub. It can use Fire Eagle if you choose to, but otherwise it accepts any old post code or street name. Thanks to the OSM community it's got nearly 9000 pubs in.

And this is its sister site: megaliths.iamnear.net. This is perhaps the more delightful side of discovering across space. I found a KML file from the Modern Antiquarian which contained every stone megalith in the UK. You never know when you might be bored and fancy a trip to a stone circle.

And now for something comepletely different. This is Buttons from Sascha Pohflepp, part of Blinks and Buttons - a piece for his diploma in Visual Communication and the Berlin University of the Arts.

It's a camera shaped object, with a nice big red shutter button.

You hold it up the scene, press the button and... wait.

Eventually, it shows you a photo. It searches the Flickr API, returning the first photo it finds taken at the same time in a completely different place. If it find nothing, it waits and tries later. Eventually a picture pops up, but it's someone else's picture, from someone else's space.

In his thesis he says:

"[...] cameras become networked buttons that create a link between two people through the simple fact that they did the same thing simultaneously: pressing a button. The cameras create a visual trace of it, with time as a reference."

I like that.

OK, let's sneak in another dimension: time.

This is a map of the every star within 50 light years. You've got to love maps with arrows pointing to things like the "galactic centre".

This is a light cone, made famous in Stephen Hawkings' Brief History of Time, but more recently used by the Dopplr team to explain their design philosophy. You are here. This is the future, diverging in its many possibilities. This is the past, narrowing as possibilities converge into realities.

And this is a thing called Light Cone written by Matt Webb. Let's consider that your impact on the universe begins on the date of your birth. (Your mother may disagree.) This feed is an update of every star on which your ever expanding sphere of causality, travelling at the speed of light, has touched.

This is a typical feed item. Light Cone made me think a lot. I like it.

Whilst we're on space and time, I want someone to build a thing that lets me upload a GPS trace and find all the photos that were taken in nearby space + time. I'm convinced that I must be in dozens, if not hundreds of tourist photos taken at Tower Bridge. I'd love to be able to spot myself in them.

OK, last bit. Visualising the invisible.

We used to hear about news stories like this quite a lot. The Evening Standard had wonderful headlines like "IMPENDING ASTEROID DEATH DOOM", only for you to open the article and read the small print: "0.001% chance in 2056".

The media got bored of this, but the asteroids didn't. There are still millions of rocks flying past the Earth every day. Some tiny, some big. Almost all are completely harmless.

NASA keep track of all these objects - well, the big ones at least - and publish them in the Near Earth Object database, with lots of data about velocities, distances and times.

The source of that page is a horrible jumble of nested tables and font tags, but it turns out you can do quite a good job of parsing it, with a bit of Hpricot, a fantastic Ruby library for HTML munging.

And you can turn it into this. This is @lowflyingrocks, a Twitter bot I wrote that speaks whenever a rock passes within 0.2 AU of Earth. A finger on pulse of the universe if you like.

In a similar vein, this is @abovelondon, which speaks the schedule when the International Space Station is visible over London and where to find it in the sky. It was written by Paul Mison for his Hackday project in 2007, but it seems to be dead now.

I'm afraid there are no decent pictures for this. It's called Live Wire and was made by Natalie Jeremijenko in 1995 for the XEROX PARC office.

Imagine a piece of string dangling from the office ceiling. Every time a piece of data flows across the network a motor the string is attached to, pulses. As data flows more frequently and the usage on the network rises, the motor fires more frequently, jumping and spinning the string more violently.

A beautiful way of visualising something which is normally invisible: is someone downloading and slowing down my connection?

So, this is the point where I make a conclusion, recap on the key learning points, and sign all your personal development forms.

Except I won't, because there isn't really much of a thread to take away with you. I'm sure you can draw your own conclusions.

I guess the only real thread is that most of these projects are really small and really simple. Less than a day for most, some less than an hour. I have difficulty with any project that I can't release in a few hours - it starts getting too unwieldy and I never finish it off. Build first, think later. And try and squeeze making some fun stuff into your day.

I think that's it. Thanks for listening.