Building Flickr Stats at Web 2.0 New York

Paul Hammond, 1 October 2008

Building Flickr Stats

A few weeks back I spoke at Web 2.0 Expo in New York about the Flickr Stats project.

The talk covered the whole development cycle, from initial design sketches to post launch administration tools.

I focused on the problems you get when trying to engineer a system like this to work at scale, and some of the common solutions used by the Flickr engineering team, including sharding, denormalization, the flickr offline tasks system and batch processing.

I also had a chance to talk about the surprisingly large number of ways you can generate graphs online, including a little bit about the sparklines on the new Flickr homepage.

The slides are, of course, available for download in a 37meg PDF file.