Victoria's environmental regulator overhauled its AirWatch air quality monitoring app and website just in time for an influx of traffic during the catastrophic ‘Black Summer’ bushfires earlier this year that surpassed its previous peak 48 times over.
The Environment Protection Authority Victoria, which is two years into a transformation program, is rebuilding its technology estate from the ground up, including many services and the systems behind them, in response to a 2017 inquiry.
Led by consulting giant Deloitte, the overhaul has focused on making EPA Victoria’s services more citizen friendly, starting with the real-time AirWatch capability and website that went live in October 2019.
AirWatch sources data from sensors in 47 fixed locations around Melbourne, as well as mobile units that are deployed in response to major disaster events, to provide an hourly average reading on air quality.
Chief information officer Chris Moon told iTnews the rebuild – which is built on Sitecore’s Experience Platform CMS and sources data from a Microsoft Azure-based data management platform – had made the agency “much easier for citizens to engage with”.
“We wanted it to be a bit like the weather. You can pick up your phone, you can look at the symbol and you know it’s going to be rainy or sunny, and we wanted that from an air quality point of view,” he said.
While the EPA had previously had a website providing air quality data, Moon described it as “more of an academic exercise”, providing “very technical data” such as “how many parts per million of PM2.5 (particulate matter) … was in your area”.
“The [old] website originally was not built in a way that was intended for these emergency situations where we were never anticipating anything like the number of users we had,” he said, adding that the data can also be surfaced to third-parties via an API.
The EPA also wanted to use the rebuild to bolster the resilience of AirWatch, so that it wouldn’t be “swamped or overwhelmed” in the event of a large bushfire – like the Black Summer bushfires in January – or an industrial fire.
“From the ground up when we built this new AirWatch, we designed it to have redundancy and reliability, so virtually the whole thing runs in Melbourne and Sydney and can flip-flop between the two,” Moon said.
“In fact, during the bushfire scenario we actually were doing things like upgrading the Sydney instance, seamlessly failing the entire website over to Sydney, upgrading the Melbourne instance and then failing it back, and we did it a few times during the day."
He added that the platform is entirely SaaS and PaaS based, and the agency would continue to “push as far up the stack as we can” in a bid to create organisational efficiencies and focus on collecting air quality data.
With a relatively small IT team, EPA Victoria “leaned on Deloitte to bring technical expertise, but also real thought leadership” to help the agency adopt new ways of working.
Deloitte’s digital government lead Jason Hutchinson said the redundancy and [stress] testing performed during development came to the fore during the bushfires on January 16, when Melburnians “flocked to the website”.
Traffic to the EPA website and AirWatch application during the bushfire was 48 times the previous record traffic peak and 21 times the volume of traffic that Deloitte tested as a worst case scenario.
The worst case scenario was devised by looking back through two years of website traffic to “determine the peak volume” during two previous major fires in Melbourne, and then doubling it and rounding up.
“What that meant from our point of view was we were at over 5000 users per minute on the website, and the thing is, unlike the weather, it’s not the kind of thing you look at once and then that’s it for the day … so a lot of the traffic was consistently going to the website,” Moon said.
The level of traffic also forced EPA Victoria to uplift the denial-of-service cap to allow “more traffic to flow from the web servers through to the actual site”, which was made possible due to the modular nature of the platform and infrastructure-as-code.
Moon described the timing of the refresh as “incredibly fortuitous”, especially as the agency had wanted to show the value of the broader transformation with Deloitte “really quickly and easily to the public”.
Hutchison added that the bushfire event was a perfect example of the kind of 'black swan' event that would have traditionally downed a government service or information source just at its time of need.
“It’s [about] making sure that you can meet those expectations at the point of need … Often you don’t hear about these things until people really need the information, but then the service isn’t working or it can't meet the demand,” he said.
“That’s often when we sit down and have differing opinions about the quality of the service being provided or the quality of the information being provided.
"But what resonates is that this information was there, it was available.”
A number of “additional technical capabilities” also developed as part of the Deloitte overhaul are now expected to go live ahead of the introduction of Victoria’s Environment Protection Act in July 2021.
