MapThing v1.0 Processing Library

One of the objectives of the COSMIC project is to “extend our current techniques of visualising complex spatial systems… [to] enable a wide range of stakeholders to be involved both in understanding such complexity and using it in policy analysis.” Normally, this type of task would mean identifying a set of existing tools and producing some alternative visualisations of the same data to see what works and what doesn’t; however, as seems to be common with this type of work, I soon found that there weren’t any tools to do rapid visualisation and exploration of geodata (bar City’s excellent, but not entirely relevant to what I was trying to do, giCentre Utilities).

So, without further ado: MapThing allows you to perform a range of useful mapping (in the geographical sense) functions within Processing and offers a collection of classes for reading ESRI-compliant Shape files (a.k.a. shapefiles) and CSV point data, and then displaying them as part of a sketch. My objective here was not to implement a full-fledged GIS system, but to make it as easy as possible to take a set of geographically coded files and do something with them inside Processing without needing to think about how to map the coordinate spaces or how to read a shape file and extract useful information from it.

Geographically projected data in Processing

There are four main classes with which you want to concern yourself:

  1. BoundingBox is how you define the geographic envelope within which the sketch is displayed, in effect it maps the geographic space on to the viewable space of the sketch itself;
  2. Lines is used to read and display line-type shapefiles;
  3. Points is used to read and display point-type shapefiles; and
  4. Polygons is used to read and display polygon-type shapefiles.

You can also read in point data (tested) and line data (not tested) from CSV files and project this using the same mechanism.

The PDE sample file offers a working example of most of what is discussed above.

The library is available for download here:

Detailed feedback or suggestions are welcome.



The ARCADIA Employment Model

Duncan Smith, Camilo-Vargas Ruiz and myself (Mike Batty) have completed the employment model for the Arcadia Integrated Assessment Model. The model is unique in that it is built at the regional level for three UK regions – London, the South East and East at the level of wards – some 3202 in all, which dovetails it with Cambridge Econometrics Input –Output Model MDME3 and thus provides employment in size sectors at the fine scale for the Land Use Transportation Interaction (LUTI) Model which covers the GLA and Outer Metropolitan Area. These nested regions with the GLA shown in Black are illustrated below.

We have written a working paper on the model and this is available from the CASA web site and from

The paper covers the development of the regression model which is organised in two stages: first the prediction of floorspace for commercial, office and industrial sectors from a variety of independent variables such as land area, other floorspace and so on, and then the prediction of each of six types of employment as a function largely of floorspace and accessibility. The explanations are robust and reasonable explaining over 70 percent of the variance of each employment type. In the paper, we show how the model can be integrated as the interface between the I-O and LUTI models, focusing on how we deal with increments and decrements of employment in a comparative static framework.

Representing Social Flows

The flow analogy is central to our work in COSMIC, Mechanicity and related projects which argue that cities should be considered as first and foremost as flow systems, not as locational patterns. This is massive break with the past as it presumes that we cannot understand locations without knowing about flows. It blows up the problem dimensionally and it means that for every non trivial locational pattern, there are a countable infinity of flow patterns that can give rise to it. This is intrinsic to the derivation of spatial interaction models using entropy maximizing which is another feature of work in CASA and although this conundrum of interaction and location has been known for many years, only now is it being exploited. Social flows of course are the hardest to observe but social media may hold some clue. Here is a map of the tracing of single short text messages from Twitter taken from the London riots and plotted in Gephi, I guess,

The New York City startup SocialFlows is at the forefront of these developments although they are mainly concerned with optimising your tweets. As a side product, hey produced the map but of course this is what this kind of viral marketing is all about. Check them out.

There is a nice article by Mike Orcutt in todays Tech Review 22nd December 2012

All I Want For Christmas …

… is the biggest, fattest book I have ever contributed to, almost unknowingly. Imagine my surprise when the  large parcel, almost too heavy for the postman to carry, arrived on my desk, containing a copy of the 1465 page Technical Report of the UK National Ecosystem Assessment. What had I done to deserve this? Well I appear to have contributed to Chapter 10: Urban – 48 pages about the urban condition of the UK written by an assorted team of 25 authors for which it would appear I wrote something about population distribution in the UK and provided a couple of my own computer graphics – yes done by me, not ArcGIS, from raw code – or so I remember – well memory isn’t what it was, but it’s Christmas.

See for yourself. Click the link to download the chapter – sorry it isn’t there yet. I will sort this soon. But if you want to read the entire thing – which incidentally is the last word on Ecosystem Services, and also heavily influenced the White Paper on “Building on the National Ecosystem Assessment” – then there is a handy web site where you can download the entire volume. In fact you can get chapter 10 if you scroll down this web page and then download it directly from the link.

Beautifully produced book I must say and actually extremely useful. Even for urbanists like me who only live in the country at weekends, I learned a lot. Quite good on urban boundaries – and on land use in Chapter 10, not by me, I think – Mechanicity team please note the work on boundaries.

If you want to see the book itself, it is outside my office between my mac and my smackmac. Come and marvel at the scale and size. In a time when people say the days of the physical book are numbered, they seem to be getting larger and presumably will reach infinity in 2026. In joke for those who know. Sorry. Happy Christmas.

One Days Worth of Traffic in Milton Keynes

Nice movie of traffic moving and link volumes in Milton Keynes by Joan Serras working on our SCALE project. Notice the peak and the fact that as is quite usual for such traffic, the morning peak is more accentuated than the evening peak. Joan is working to add capacity to our ARCADIA model and itheexploring how to do this with Milton Keynes data. He also has MATSim working for MK adding to our portfolio of models having worked previously on a TRANSIMS application to the town.


Explore his movies at Vimeo by clicking on the count and link flow pictures below. Watch this space from some movies of the flow of development aid from Joan













Scaling in Social Systems

A one day EPSRC-funded workshop (at which yours truly presented) in Oxford at the Complex Agent-Based Dynamic Networks group (CABDyN) at the Said Business School (1st December 2011) had five interesting speakers all talking about the new social physics of cities and economies, focusing on regularities and irregularities in the structure and dynamics of these systems. Didier Sornette from ETH Zurich threw down the challenge telling us that scaling was boring and that all the action should be on departures from scaling – in the case of Zipf laws, departures from such regularity where the biggest cities, incomes, firm sizes and so on are substantially less or more than what Zipf’s Law might predict.

I kicked off the day with a talk on Scaling Laws for Cities, the pdf of which you can get if you click here and then Luis Bettencourt of Santa Fe talked about Scaling and Allometry with respect to what happens when cities get bigger. The essential insight from Bettencourt and West and their other collaborators is that big cities generate more than proportionate returns to scale – positive allometry or superlinearity for things like creative industries, patents, incomes and so on, linearity for local consumption at the individual level, and sublinearity for the use of physical resources such as roads and utilities. There are profound implications for bigness from these results and a lively debate ensured which involved ourselves at CASA where our own results are more controversial. It looks like cities by size do not scale superlinearly in the UK but here lies a puzzle – we need to factor out the effect of the primate city which is London and then look at residuals. Very positive suggestions from Luis Bettencourt on this from the meeting.

Didier Sornette from ETH went first after lunch talking about cascades arguing that there were distinct discontinuities caused by endogenous and exogenous factors – he presented a great classification of dynamics. Then came Tim Evans from Imperial on citations analysis which revealed the same kinds of issues related to defining the objects of interest – in this case papers – as we find in defining the size of cities. As you drill into these areas, definitional problems abound. The objects of interest in the social science are essentially blurred, ambiguous and fuzzy and this was widely recognized at the meeting.

Geoff West from Santa Fe summed up and posed twenty critical questions that pertain to how we might articulate the physics of social systems. These provided the essence of how we might tackle the future of this field and many open questions were posed. At some point, Felix Reed-Tsochas who runs the group at Oxford will post the pdfs I guess but the batting order for the day can be retrieved by clicking here.

Agent-Based Models of Geographical Systems

Provides a complete reference guide for understanding how to apply ABM to geographical systems
Presents the latest ideas and applications using ABM
Addresses the issue of applying ABM in a spatial context. This unique book brings together a comprehensive set of papers on the background, theory, technical issues and applications of agent-based modelling (ABM) within geographical systems. This collection of papers is an invaluable reference point for the experienced agent-based modeller as well those new to the area.  Specific geographical issues such as handling scale and space are dealt with as well as practical advice from leading experts about designing and creating ABMs, handling complexity, visualising and validating model outputs.  With contributions from many of the world’s leading research institutions, the latest applied research (micro and macro applications) from around the globe exemplify what can be achieved in geographical context.

This book is relevant to researchers, postgraduate and advanced undergraduate students, and professionals in the areas of quantitative geography, spatial analysis, spatial modelling, social simulation modelling and geographical information sciences.









Content Level » Research

Keywords » ABM – Agent tools techniques – Agent-Based Modelling – CHALMS – City size distributions – Computational methods models – Demographic change – Diffusion models – Education system agent based – Ex-urban residential land-use change – GIS – Geocomputational Techniques – Geographical Systems – Geography – Geosimulation – Geospatial simulation – Health inequalities – Housing choice – Land Use – Macro and microscopic simulation – Model scaling – Residential mobility – SIMPOP models – Social Sciences

Related subjects » GeographyPopulation StudiesTheoretical Computer Science


1: Mike Batty, Andrew Crooks, Linda See and Alison Heppenstall: Introduction.- Part 1: Computational Modelling: Techniques for Modelling Geographical Systems.- 2: Mike Batty: A Generic Framework for Computational Spatial Modelling.- 3: Mark Birkin and Linda Wu: A Review of Microsimulation and Hybrid Agent Based Approach.- 4: Sanna Iltanen: Cellular Automata in Urban Spatial Modelling.- 5: Andrew Crooks and Alison Heppenstall: Introduction to Agent-Based Modelling.- Part 2: Principles and Concepts of Agent-Based Modelling.- 6: David O’Sullivan et al: Agent-Based Models – Because they’re Worth it?.- 7: Steve Manson et al: Agent-Based Modelling and Complexity.- 8: Mohammad Addou et al: Designing and Building an Agent-Based Model.- 9: Bill Kennedy: Modelling Human Behaviour in Agent-Based Models.- 10: The An Ngo and Linda See: Calibration and Validation of Agent-Based Models of Land Cover Change.- 11: Shah Jamal Alam et al: Networks in Agent-Based Social Simulation.- Part 3: Methods, Techniques and Considerations when Creating Agent-Based Models (Technical Issues) 12: Andrew Crooks and Christina Castle: The Integration of Agent-Based Modelling and Geographical Information for Geospatial Simulation.- 13: Kiril Stanilov: Space in Agent-Based Models.- 14: Hazel Parry and Mike Bithell: Large Scale Agent-Based Modelling: A Review and Guidelines for Model Scaling.- 15: Andrew Evans: Uncertainty and Error.- 16: Belinda Wu and Mark Birkin: Agent-Based Extensions to a Spatial Microsimulation Model of Demographic Change.- 17: Volker Grimm and Steven Railsback: Designing, Formulating, and Communicating Agent-Based Models.- 18: Ateen Patel and Andrew Hudson Smith: Agent Tools Techniques and Methods for Macro and Microscopic Simulation.- Part 4: Applications of Agent-Based Modelling: Micro to Macro.- Micro: 19: Nicolas Malleson: Using Agent-Based Models to Simulate Crime.- 20: Paul Torrens: Urban Simulation.- 21: Anders Johansson and Tobias Kretz: Applied Pedestrian Modelling.- 22: William Rand: Business Applications and Research Questions using Spatial Agent-Based Models.- 23: Kirk Harland and Alison Heppenstall: Using Agent-Based Models for Education Planning. Is the UK Education System Agent Based?.- 24: Dianna Smith: Simulating Spatial Health Inequalities.- 25: Rene Jordan et al: ABM of Residential Mobility, Housing Choice and Regeneration.- 26: D.C.Parker et al: Do Land Markets Matter? A Modelling Ontology and Experimental Design to Test the Effects of Land Markets for an Agent-Based Model of Ex-urban Residential Land-Use Change.- 27: Nicholas R. Magliocca: Exploring Coupled Housing and Land Market Interactions Through an Economic Agent-Based Model (CHALMS).- Macro: 28: Joana Barros: Exploring Urban Dynamics in Latin American Cities using an Agent-Based Simulation Approach.- 29: Joana Simoes: An Agent-Based/Network Approach to Spatial Epidemics.- 30: The An Ngo et al: An Agent-Based Modelling Application of Shifting Cultivation.- 31: Arnaud Banos and Cyrille Genre-Grandpierre: Towards New Metrics for Urban Road Networks. Some Preliminary Evidence from Agent-Based Simulations.- 32: Yan Liu and Yong Jiu Feng: A Logistic Based Cellular Automata Model for Continuous Urban Growth Simulation: A Case Study of the Gold Coast City, Australia.- 33: Ray Cabrera et al: Exploring Demographic and Lot Effects in an ABM/LUCC of Agriculture in the Brazilian Amazon.- 34: Tom Gulden and Ross A Hammond: Beyond Zipf: An Agent Based Understanding of City Size Distributions.- 35: Joel Dearden and Alan Wilson: The Relationship of Dynamic Entropy Maximising and Agent Based Approaches in Urban Modelling.- 36: Denise Pumain: Multi-Agent System Modelling for Urban Systems: The Series of SIMPOP Models.- 37 Mike Batty, Andrew Crooks, Linda See and Alison Heppenstall: Perspectives on Agent-Based Models and Geographical Systems


Changing Rank-Size Distributions over 24 Hrs on London Tube and Rail

Inspired by Jon Reades’ great visualizations of flows using the TfL Oyster card data, we are now starting to examine the statistical properties of the data, beginning with an analysis of what goes on in the nodes. The data yields 666 hubs or nodes which have entry and exit volumes – in graph theoretic terms the in-degrees of the sources/origins and the out-degrees of the sinks/destinations. We have this data for 20 minute time segments of the entire day, giving us 72 temporal slots defining the diurnal sequence.

Now the tube and rail are closed during the dead of night and as is characteristic of such data, all 666 nodes are never active at any one time in terms of the data set. But there are records for at last one hub in every one of the 72 segments and they build up so that for the most part, most of the 666 are in use between 7am and 11pm. More or less. What we have done so far with this data is simply examine the changing profile of the hubs in terms of their volumes and an easy way to get into this is to rank them in terms of the size for each time slice and then order them, forming a rank size distribution which we fist examine as in the video below as a Zipf plot. These change at each time slice. We start the simulation at midnight and the volumes then drop dramatically until around 5 am things begin to pick up reaching a morning peal around 8am and then falling off a bit and then picking up for the evening peak at 5pm and thence declining slowly through the evening as we move back towards midnight. Click on the image below and you will get the Vimeo link where you can see the animation. I must confess that I have forgotten how to embed the Vimeo directly into the post and someone doubtless will remind me

You can see the peaks more or less as we color the plots from white at midnight to red at midday and back down to white again in the rest of the day. The color balance can be improved and we need to do this and probably this would look really good if we used a more powerful graphics language but as a first pass, it suffices. Of course the next step is to examine the local volatility of the hubs and to put all this into the rank clocks software which in fact would be our first application where the clock where truly a clock and the diurnal cycles the dominant driver. More soon.

The idea is that we present a paper on this at the forthcoming meeting organised by the Urban Design Studies Unit of the School of Architecture in partnership with ICSS Institute of Complex System on the subject of the Evolution of Complex Transportation Networks (ECTN) Workshop to be held at University of Strathclyde, Glasgow, on 29-30 August 2011.

Pulse of the City

I’ve been working with the TfL data over the past few weeks to better understand the geographical distribution of journeys and how this affects the load on individual segments across the transport network. In other words, we all know that some parts of some trips are particularly unbearable, and that disruptions in those areas (e.g. cancellations, delays) can seem to bring the entire system to a halt in a matter of 15 minutes.

The visualisation below shows how some links in the network are particularly heavily-loaded at various times of the day because of the way that people are travelling around the network based on the probable shortest path that they would take between points A and B. In other words, if someone boards the Tube at Bethnal Green at 9am and exits at Shepherd’s Bush then, in all likelihood, they have travelled along the Central Line and the map below would increment the counter for each segment in the 9am time interval between stations by one. Summing up over all travel for working days produces this time-based visualisation.

Personal experience suggests that the prominence of the leg between Highbury and Kings Cross is right on the money — the Victoria line at this point becomes nearly unbearable, but the train empties out rapidly by the time you approach Oxford Circus. The busiest segment of all looks to be between Kings Cross and Euston, but it’s followed rather closely by Kings Cross to/from Farringdon. I want to emphasise, however, that I’m not saying that most people are starting their journey at Kings Cross and finishing at Farringdon; rather, they are taking a trip that is likely to pass through those stations, and that’s why those lines become so heavily-travelled.

Screen shot from Heart Beat movie

Because of the way that London’s transit system works, for now only the rail and Tube portions of our journeys are mappable. But in the longer term I hope to come up with ways to infer data about bus journeys based on the ways that people surge on to alternative routes or modes when delays pass some threshold of bearability.

A Society of Simulations

An Essay by Koert van Mensvoort, from his blog where he has a lot of interesting material on Simulacra and much else besides. He begins this essay with the following quote

An interviewer once asked Pablo Picasso
 why he paints such strange pictures 
instead of painting things the way they are. Picasso asks the man what he means. The man then takes out a photograph
from his wallet and says, “This is my wife!”
 Picasso looks at the photo and then says: 
“isn’t she rather short and flat?”.

Says it all – download the paper either from his blog or from my own site.

Older posts «

» Newer posts