Making big data deliver

The Australian Urban Research Infrastructure Network (AURIN) is a unique computing resource allowing researchers to make sense of the masses of data that they are now having to deal with related to the urban settlements (cities) of Australia.

Researchers at the world-famous Peter MacCallum Cancer Centre in Melbourne, Australia recently had a medical conundrum that couldn’t be solved in a test tube or at a lab bench.

They wanted to know why there were an increasing number of patients – about a third of them women – being diagnosed with certain types of lung cancer when none of them had smoked and their families had no history of cancer. And why were they showing up in clusters in different parts of Victoria?

Turning to big data analytics

To find the answer, these specialists turned to scientists offering innovations in capturing, analysing and visualising diverse data sets through so-called big data analytics.

Using the AURIN platform – developed through a national cooperative effort led by the University of Melbourne – the Peter MacCallum scientists were able to draw on detailed data from a wide range of sources and combine it with their own lung cancer data.

This information included data on car pollution on Victoria’s roads and factory pollution through the National Pollutant Inventory, weather data from the Bureau of Meteorology and detailed population data from the Australian Census.

Professor Richard Sinnott, Director of eResearch at the University and Professor of Applied Computing Systems in Computing and Information Systems in the Melbourne School of Engineering, led the team that built the AURIN platform, which commenced in 2010.

“We aimed to build a system for urban and built environment researchers across Australia to provide access to definitive data,” he says. “If you were interested in crime in Melbourne, you would ask Victoria Police. Or if your interest was in alcohol consumption in Perth, you’d go to an organisation that recorded who sold alcohol and how much, and so on.

“Many agencies across Government, industry and academia have such definitive data sets related to urban settlement but they are fragmented and usually not immediately ready for researchers to begin their analysis.

“The AURIN platform interacts with all of them and allows researchers to discover, access and, use data at many spatial levels in a single unified and user-oriented environment.”

Visualising relationships

AURIN’s platform, believed to be unmatched in the world, allows seamless and secure access to data from the definitive organisations of Australia. This data can be combined, compared and visualised through maps and charts that can show relationships that might otherwise be missed.

Prof. Sinnott says supporting live access to large volumes of federated (distributed) data depends completely upon the highly reliable and performant networking infrastructure. AURIN has benefited enormously from connectivity to AARNet.

“We transfer gigabytes of data to thousands of end users and have had no issues on the networking side,” he says.

There have been more than 75,000 accesses and usages of the AURIN platform from over 5,000 regular customers. Many of these customers are also connected to AARNet.

AARNet will continue to play a vital role as the AURIN system is extended to other domains and disciplines, including air quality and biodiversity amongst others.