Victoria to run AI over CCTV footage to pinpoint road incidents

Victoria’s Department of Transport is piloting machine learning to automatically analyse CCTV camera feeds across the state’s road network and alert operators to incidents in real time.

Transport operations director Emily Lodder revealed the project, aimed at helping DoT adopt a “proactive” approach to congestion, at Amazon Web Services’ Sydney Summit this week.

Using Deloitte’s AWS Panorama-powered video analytics solution AutoEye, the project – which will be operational next year – will allow DoT to use its “existing CCTV camera network as sensors”.

“Our most critical situational awareness tool on the road network are our 1200 CCTV cameras,” Lodder said.

“They enable quick validation of traffic conditions to support real-time decision making, but our use of these assets is limited.”

Lodder said there were now more than 1200 CCTV cameras across the state’s 23,000 kilometres of freeways and arterial roads, which had started to improve “situation awareness” for her team.

At least 700 of the cameras have come online since the pandemic as part of the government’s $340 million smarter roads program, which is initially focused on Melbourne’s west, east and south-east.

DoT also has access to data from “thousands of sensors on the network, including Bluetooth, in-road sensors, roadside, laser and radar”, as well as TomTom data.

But despite the additional cameras providing “visibility of roads where [there was] no data or vision before”, there are only so many operators available to monitor the network at any given time.

“The only way to glean information from a CCTV camera is for an operator to manually select and watch the live stream,” Lodder said.

“During peak periods, less than 15 percent of CCTV cameras are used by the operations team, so at any point in time, there’s around 1000 cameras just gathering dust.”

Data is also “accessed through multiple operational systems, and many can’t be accessed in real-time”, further complicating the work of operators.

“This inconsistent and incomplete access to real-time information means our operators are working reactively, manually overlaying information sources and relying on phone calls,” she added.

Lodder said that by using AutoEye and, by extension, AWS Panorama, the department could become “smarter about how we use these assets” by “autonomously detecting and prioritising incidents”.

“Using AI at the edge, AWS Panorama will analyse all 1200 live video feeds in real-time. This hasn’t previously been possible at this scale,” she said.

“AWS Panorama devices are providing the required computational power within the DoT network, ensuring data speed and privacy.”

Following detection, the solution would use machine learning to “enable identification of specific incidents of interest and general abnormal behaviour from the enriched data and baselines”.

But for Lodder, the real “game changer” is the “notification to our operators”, enabling “more focused, proactive incident response”.

“With this capability, we will know about more incidents, more quickly, which will result in faster responses and recovery,” she said.

“For example, with a truck rollover... congestion builds extremely quickly around the scene with road closures in place for hours, and there is a high risk of secondary incidents occurring on the approach and in the surrounding network.

“These are really difficult to detect while our operators are actively coordinating incident response at the scene with on-road teams and emergency, and can compromise safety and compound delays.

“Automated detection of incidents will enable operators to act more quickly and focus on the highest impact incidents, which will reduce the impact and delays, speed up network recovery and save people time on the road.

“The uplift in operational capacity as a result of AWS Panorama and machine learning capability will be significant. This is the foundation to enable the department to move from reactive to proactive network management.”