The buzz surrounding big data is enough to convert the most ardent sceptic but once the initial enthusiasm subsides, you begin to cut through the fog of marketing hype and evaluate practical benefits.  Many people are at the second stage when they are beginning to question big data benefits.  Working in business intelligence and data analytics for some time now, I’ve never had occasion to doubt its positive influence. In fact the biggest and most widespread impact is already shaping our towns and cities and turning them into smart cities, a modern urban landscape where big data is radically changing both where we live and the way we live.

According to the 2011 revision of the United Nations’ World Urbanization Prospects report[1], nearly 70 per cent of the world’s population will be urban by 2051. The world population is expected to surpass nine billion and urban dwellers to surpass six billion. Two in three people born in the next 30 years will live in cities. This growth in urban living poses tough challenges to those in local government, administration and town planning offices as they strive to ensure that the environment, transportation, residents’ safety, the provision of utilities as well as economic and social activity can continue to be improved unhindered.  It is clear that responding to these challenges and improving people’s lives, towns and cities will require local planning teams and administration bodies to think differently, where much more emphasis will be placed on the consumption and analysis of large data volumes generated by day-to-day life in our towns and cities.

And it is staggering just how much data towns and cities generate. At a rough estimate, we will generate 4.1 terabytes per day per square kilometer of urbanized land area by 2016.  In fact, you could say that cities are the true big data systems of our age. From geolocation data collected by smart phones to data generated by cars and their GPS instruments, from the contact sensor payment cards we use to ride the subway to the data we offer when we want to make use of a bike or a car in the city.  From the data generated by our health ID cards to that from our loyalty and store cards, our bank cards and every time we make use of QR, bar or flash codes to access content.

Data generation does not stop there – think about the data created by traffic management systems, from traffic lights to the sensors on our roads; from the provision of utilities such as gas, electricity and drinking water; when delivering refuse collection and waste management services; from the provision of healthcare in our doctors’ surgeries and hospitals to the data generated by schools and colleges educating our children.

Data is not being generated in isolation either, there an increasing appetite for real-time and interactive information – where we were once content to use a map provided on a street display, we now turn to our smart phone devices and tablets to interact much more dynamically.  Before, we were glad when we found a good restaurant.  Now we want to research it, see what others think of it, take photos, post content and share our reviews.  No longer do we want to wait patiently for the bus to arrive, wondering whether an alternate transport might better serve our purpose.  Now we want to know where it is, how late it is running and whether it will get us to the station in time for our train.  Before, we just got into our cars and headed off to where we needed to go.  Now, we want to research our route, see if there are any traffic jams or incidents, plot our journey using our sat nav, and see how the weather might impact our journey.  Before, we were happy if we could get our children into a local school.  Now we want to know what the school is like, how it rates in the league tables, what the teachers are like, how easy it is for our children to get to school, what other parents think about it.

You could even say that where once we used to talk to our neighbours and family to get their thoughts and opinions, now we are consumed by the digital age and look to online resources instead, creating and consuming vast amounts of content that others can use and add to, vast amounts of data that – once shared – civic authorities and town planners can use to their advantage.

Faced with this deluge of data in a wide variety of forms and formats, it may be hard to know where civic authorities and town planning organization can start.  But the answer is clear: every step made towards improving the quality of life begins by first analyzing it and making sense of it.  For me this represents a great opportunity for the city authorities and urban developers – it gives them a powerful tool to tackle rapid and unprecedented urbanization by making better informed decisions, operate more efficiently  and even predict the future to ensure resources can be organized in time.

We have already seen a number of practical implementations from utility providers exploring how information from smart meters can encourage water and energy users to change behaviour to civic authorities using available technologies, including mobile phones, sensors and closed-circuit television to improve the flow of road traffic. One of the most successful example of this was during the 2012 Olympic games in London when Transport for London, the public authority responsible for running the London public transport network, prepared and ensured smooth transport despite experiencing a 25 per cent increase in customers using real-time information collected from CCTV cameras, subway cards (Oyster card), mobile phones and social networks to ensure limited disruptions to trains and bus routes.

Data is all around us – it’s not just growing but multiplying and what the civic authorities and town planners need is fast and easy-to-use technology which can digest the data quickly and give them the answers that they need.  They don’t need to invest in large, expensive storage or data processing solutions; specialized solutions are not needed here.  There are newer analytic solutions out there in the market, ones that leverage the performance features of the latest off-the-shelf servers and hardware that can crunch through large volumes of data of all shapes and sizes and render the results on devices that we all use anyway in a matter of just seconds: on smart phones, tablets, our desktop PCs.

Cities are areas where big data is having a real impact.  Town planners and administration bodies just need the right tools at their fingertips to consume all the data points that a town or city generate and then be able to turn that into actions that improve people’s lives.  In this case, big data is not just a passing fad or marketing hype, it is definitely a phenomenon that has a direct impact on the quality of life for those of us that choose to live in a town or city.

Tomorrow’s towns and cities are being built today, and they’re being built by using big data.