How WaterNSW tamed its data 'beast' to protect state's dams

WaterNSW has created a sophisticated data integration and analytics solution to safeguard the state’s dams and the people and buildings near them.

The solution, DamGuard, has enabled WaterNSW to make better use of "massive" data sets generated by the Internet of Things (IoT) and other data sources to improve dam safety.

Utilities are swamped by data. For example, WaterNSW uses data about everything from leakage and movement to pressure, seismic activity, weather and other factors, some of which is recorded using telemetry.

Multiple external organisations collect the data, and in the past they didn’t store it in the same system. Another problem was that some data needed for safety analysis was recorded on paper during manual inspections.

This slowed down dam safety analysis – it could take up to six weeks to analyse data. WaterNSW also had doubts about data quality.

So, it built DamGuard. The platform combines field inspection and telemetry data in a data lake, automates quality assurance, provides dashboards and alerts for rapid assessment and includes an analytical workbench for deeper modelling analysis.

It’s a bespoke solution, with key components including a Microsoft Power Apps-based data collection platform and hand-held devices to replace pen and paper. Azure SQL stores the data and Streamsets integrates it and transforms data flows between systems. Microsoft Power BI provides visualisation and Jupyter Notebook delivers advanced analytic models and machine learning to model dam safety risk scenarios.

Combining these components required an agile collaboration effort involving the dam safety team, engineers who did risk modelling, an internal IT team who provided architecture, data source definitions, Streamsets capability and Azure access, and the Spanish water analytics company ADASA.

DamGuard has accelerated data collection and freed up WaterNSW to spend more time on analysis. Now analysis takes minutes, not weeks.

The platform has also significantly reduced safety monitoring expenditure. And it has created a path to data modelling and machine learning, using a complete set of historical data.

Today, WaterNSW views data as the “bedrock of a risk informed, data-driven decision support system for dam safety”. It says its data is high quality and timely – two essential requirements when the state’s water supply is at stake.

“This solution shows that by starting with the end in mind, taking a use case driven approach and consolidating data from multiple sources into a centralised data hub, the beast of data management can be tamed,” WaterNSW said.

This project is a finalist in the IoT category of the iTnews Benchmark Awards 2020.