Domino’s Pizza Enterprises built a proof-of-concept using machine learning to personalise vouchers and deals for customers in Australia.
The master franchise, which operates in nine countries, is an early tester of an AWS service called Personalize, which was only made publicly available last month.
Speaking at the recent AWS Summit in Sydney, Domino’s lead data scientist Thomas Atkins said the pizza maker wanted to find a way to scale its ability to personalise deals using a wider array of "variants", such as time of day, pickup or delivery, price, and type of pizza.
“Making communication via SMS and all our marketing channels personalised is a real challenge because of scale,” Atkins said.
“To get from segmented marketing to personalised marketing requires a really deep level of automation.
“With the typical level of automation that we have, we can do campaigns with four to six variants relatively easily, but there are a number of manual steps that make this a little bit cumbersome.
“With this proof-of-concept, we were able to automate and scale it up so that we could do a lot more variants.
“This is particularly important if we're doing millions of SMSs and we're sending multiple campaigns a week.”
A particular subset of users’ purchase histories were used to train a feedforward neural network model.
Personalize needed to be trained using information that was “detailed enough so that Personalize could learn about the products, the relationships between the people ordering them, their purchase frequency and purchase patterns.
“So this is typically ‘Was it a pickup order?’ ‘Was it a delivery order?’ ‘Was it two pizzas?’ ‘Were they value or traditional range?’”
The company used the output of Personalize to determine the content of offers and the best time to send them to customers. It then executed the campaigns via another AWS service, Pinpoint.
For comparative purposes, a control group was also sent non-personalized deals or offers around the same time, with the results compared.
In the proof-of-concept, Domino’s managed to personalise offers with “74 variations”.
“So they were things like different times of day, different order types like pickup/delivery for the deals, different price points, different combinations of pizzas, whether it was a New Yorker range or three pizzas, and different bundles,” Atkins said.
“So we had a much larger number of variants that we were able to achieve - and this is just the beginning.”
Users in the proof-of-concept were generally more responsive to messages.
“We were able to reduce the time when the SMS was sent, and the user clicked on it by 38 percent,” Atkins said.
“So the control group was all sent the message at the same time, and the personalised group received it at a personalised time for them.
“While this might not directly drive conversion [into pizza sales], it is important because in my view it’s [confirmation] the messages are more relevant. People are less likely to opt out, and I think that's very powerful.”
However, “on the deal side” - where the money is actually changing hands - Atkins noted “the results were a bit more mixed.”
“Personalize was able to recognize and identify some cohorts of customers for some deals that worked very, very well,” he said.
“For some other deals, however, it didn't perform so well.
“My hypothesis is that either the deal itself intrinsically wasn't as perceived as being as valuable to the customer, or potentially it was that we didn't give enough information to Personalize for it to learn about the customers that might be interested in that deal.”
During the proof-of-concept, only selected customers saw personalised deals.
“We estimate that if we had scaled up the results from this particular proof of concept to our entire user database, we would get an incremental 1000 orders per SMS [campaign] sent, which is a huge win,” Atkins said.
“Overall, despite the mixed results, the net net result was still positive.
“We were able to improve conversion rate by 0.1 percent - and when we're sending millions of SMSs each week, this is a big win.”
