Creating ML-powered products that deliver competitive advantage

How Experience Design and Machine Learning can join forces to create powerful new products that exceed customer expectations to drive growth, revenue, retention and advocacy. 
With the push to put human needs first, the accelerated pace of technology, and increased competition for people’s attention, customer expectations are at an all-time high. Customers expect the same level of personalisation, speed and quality of service from every organisation they engage with. This is where Experience Design (XD) and Machine Learning (ML) can join forces to create powerful new products that exceed customer expectations to drive growth, revenue, retention and advocacy.
In this blog, we take a look at what this means for product development and how to integrate ML. You can also watch as a Lunch & Learn on demand here.

How ML is being used in products already

AI and ML are already being used to enhance products and experiences around us every day. A lot of it happens in the back end – measuring, capturing, computing and learning. Every interaction you take online as a consumer can help ML become much more effective and relevant. 

Even your smartphone uses every interaction to build its own model around you and your behaviour, and can predict what you need based on your habits. If you try searching for an app, you’ll notice it starts by recommending your most frequently used ones. Online streaming services like Netflix also use AI to recommend shows and movies based on algorithms that track the types of shows you watch, for how long, what you rewatch and also comparing viewing habits of people within your demographic.

They say that “data is the new oil”, which is a great way to understand how critical it is for ML. Data is the fuel that powers the algorithm for ML models to generate results. And generally speaking, the more data you have, the better your algorithm will be. Even more importantly, your ML model will only be as good as the data it gets. Like food for humans and soil for plants, without data, your AI will not thrive. With the right food, the right soil, your AI will learn faster and better. 

The ultimate goal of utilising ML in product development is creating a virtuous success cycle based on gaining better recommendations to build a better product to drive more usage, which provides more data for training which, in turn, creates further recommendations.

Virtuous Success Cycle
Virtuous Success Cycle

How to consider ML in product development

There are three main ways ML can be used in your product development:

  • Improve existing features e.g. using recommendation engines to personalise suggestions for an existing product.
  • Enable new features e.g. unlocking your phone with facial recognition instead of a passcode.
  • Enable new products  e.g. creating a new product around ML like bolttech’s Click-to-Protect, a computer vision-based screen diagnostic that helps bolttech to understand device health when offering insurance protection for a wide range of devices.

Let’s drill further by using the ‘pirate metrics’ model which iIllustrates the problems that products must resolve throughout a customers journey, and some common examples of where ML is already being used to help.

  • Acquisition
    The problem? Users aren’t aware of your product, so how do you build awareness within your target audience.
    How ML can help? Digital advertising and promotion that uses targeted methods to reach specific consumers.
  • Activation
    The problem? How to onboard your customers when they’re interested in purchasing your product or services so they start off on the right foot.
    How ML can help? Personalised onboarding process that ensures users can join simply and easily.
  • Retention
    The problem? Retention of the customer and their continued use of your product.
    How ML can help? Prediction, enhancement or recommendations.
  • Referral
    The problem? To the delight or value of your product to be referred to friends or family.
    How ML can help? Holistic experiences.
  • Revenue and profit
    The problem? Revenue and reducing cost to increase revenue/profit.
    How ML can help? Automation.

Your business goal may touch on one or more of these, and the solution will be slightly different for each.

Working out your ML objective

Working out what you want ML to achieve is the most critical part of this process and the hardest thing to get right. You need to work out what you want the algorithm to do and what data you need to get the results. 

It’s best to start with your overall business goal and what problem you’re trying to solve for end users. Once you have brainstormed different ideas and solutions, you can work through the ML objective. Your objective will start to dictate the next steps, such as your model selection, your data engineering and the type of answer you’re looking for. An oversimplified view of what the objective should be could lead your algorithm down the wrong path. Use your intuition first – then experiment through prototyping and iteratively improving your result.


Framing the business problem with an ML lens
Framing the user's problem with an ML lens

How to develop an ML solution

From an ML perspective, depending on the dataset, the problem at hand, and the time available, there are a few different options for creating ML models. 

  • MLaaS are the off-the-shelf tools available to us by cloud providers. These services are usually very easy to use but are generic so might not perform well enough for what you need. However, using them requires very little knowledge of ML so they’re a good way to start.
  • The next level is AutoML services which are available as open source tools or on the cloud. These require some knowledge of ML but take away the model design from the user. For these, you pass your dataset or inputs/features to your AutoML tool, and it trains a model on your data. For such services we require an average amount of effort but the performance and accuracy can be improved if you involve a ML Specialist who can implement a custom model.
  • In a custom model option, you need ML expertise, but this allows you to apply state of the art research and ML models to your problems. This obviously requires more effort but the highest performance is expected.  

Bringing an ML product to life

Once you’ve determined your business goal and ML objective, the next step is to bring it to life. There are four key stages that a combined XD and ML team can work through together.


Bringing an ML product to life
The four stages of bringing an ML product to life
1. Understand & prioritise 

The first stage is to understand both the business requirements and how ML can help. It’s important to prioritise ideas by business value and complexity, such as whether you have the right data needed, the technical risks involved and how ML can be implemented.

2. Experiment & iterate

It’s best to start small, focusing on one or two specific ML objectives, then agree on performance objectives (such as success metrics, model accuracy). By experimenting through a Proof of Concept (PoC),  you can confirm whether all the data you need is available, how to prepare the data, demonstrate technical feasibility and evaluate the performance of your ML Model.  The focus of a POC is to implement the model and find what works and what doesn’t.

You can do this by:

  • Defining the right metrics that measure the success.
  • Developing an iterative process to train the right model.
  • Defining a time-boxed engagement. 
  • Analysing the data and coming up with the right features and inputs to the model.
  • Choosing the right model that fits the data and is suitable for the problem at hand.
  • Training the model on the dataset.

Once you’ve implemented the POC, the next step is to evaluate and iterate. In different iterations you can either:

  • Iterate based on a model-based approach: focus on the model and try to change the model in such a way that a better performance is a achieved (this is within the time-boxed effort required for a PoC), or
  • Iterate based on a data-centric approach: where you consider the model to be fixed, look closely into the model predictions, and try to collect or generate more data that covers the cases where the model makes mistakes. Since this might require more data collection, it is a longer term process, and therefore, might go beyond a time-boxed PoC.
3. Build & launch

Once you have tested the approach through a POC, you can then move into production. This includes developing the software application or integration, the data infrastructure and pipelines, model deployment and scale. This will enable you to launch a Minimal Viable Product (MVP) to get started. 

There are different dimensions that need to be taken into account when moving from POC to production:

  • Choosing the right infrastructure, which allows your model to be responsive with reasonable latency and throughput, and is not very expensive to run. 
  • Trying to automate the model retraining gradually. This allows the data science members of the team to work on the model improvement or retrain end-to-end with the least amount of friction. 
  • Implement data pipelines so that an end-to-end training or retraining of the model is possible. After this, you’ll have CI/CD pipelines in place so that any change in code or data will automatically trigger the whole pipeline, train the model, evaluate, and in case a better model is achieved, replace the model in production.
  • Gradual rollout. You need to gain confidence that the ML model works well in practice. Therefore, try to roll out the model gradually. In this process, you might start from a fully-manual pipeline where a person makes the judgement call on the data, and compare the human judgement with the model prediction. As you gain more confidence in the model, you can send less and less data to the human and rely more and more on the model. 
  • Once you’re confident with the model, remove the human intervention from the pipeline and fully replace it with the model. You still need to implement monitoring on the model to make sure it stays performant and no data or model drift would take place as time goes by. 

In ML applications, some risks might arise because the model might be biased due to the dataset, or due to the fact that the model might not give an accurate prediction and how damaging a wrong prediction might be. For instance, in applications such as predicting the credit-worthiness of a client applying for a loan, or driverless cars, a mistake could have a massive impact.

The solution to this problem is to start off with a human in the loop pipeline. In this process, an example is passed to the model, and in scenarios where the model is not very confident about its prediction, the data is directed to a person who would make their judgement on the case. In this process, the aim is to de-risk the consequences of a bad judgement by the model. As another advantage of this pipeline, a more comprehensive dataset will be gradually collected, especially in scenarios where the model lacks a good judgement. This data is very valuable and can be used for retraining the model. 

This loop continues in practice as the model will learn to correct itself in wrong or unconfident cases, and you can decrease the engagement of human intervention. After the model reaches an acceptable maturity, you can move to automatic monitoring of the model and automatic health checking and triggering for manual review. This process is time-consuming but is the right approach to reach a reliable ML, and hence brings you closer to a fully-automated feature backed by ML models.


Human in the loop
How a human in the loop pipeline works
 4. Monitor & Improve 

Once an MVP model is deployed, you can then monitor performance, retrain with new data, manage versions and re-deploy. It’s a constant cycle.

An example using a DiUS client: bolttech

bolttech is an international InsurTech with a mission to build the world’s leading technology-enabled ecosystem for device protection and insurance. With a full suite of digital and data-driven capabilities, bolttech makes it easier and more efficient to buy and sell insurance and protection products. Using the business goal and ML objective below, two ML models were applied:


bolttech example
Business goal and ML objective used for bolttech
Model 1: Real-time phone detection
  • Assist user with correct positioning of the phone in front of the mirror
  • Optimised to run on the smartphone (edge model)


Model 2: Screen classification
  • Categorising as cracked or not-cracked
  • Larger more complex model hosted on the cloud
  • A combination of instructions and visual cues to help guide the user. ML allows for real-time phone tracking while users are aligning their phones
  • Recognising a phone is present
  • Positioning the phone in the right spot for the right time at the right angle
  • Removing as much interference as possible
  • A red bounding box indicates that phone is either too far away or too close and the green box indicates the position is good with additional instruction to hold the phone still while the capture is taken
  • Introducing an auto capture also meant we could remove the need to tap the screen to take the photo.

    You can find out more about the bolttech story here.

Combining ML and Experience Design to help guide the user to take the right photo

We can help

For more help or support with your product development journey, don’t hesitate to contact DiUS. We can take stock of your current circumstances and situation before advising on the next steps.

Get in touch


Complimentary two-hour Machine Learning workshop

We can help you through all the stages of this journey and would like to offer a complimentary two-hour ML workshop. We’ll help you get started with Machine Learning or review and recommend strategies to improve an existing ML-powered solution. 

Benefits of our ML workshop include:

  • Practical recommendations to improve machine learning performance, quality and reliability.
  • Expertise across the machine learning product lifecycle and solutions from MLaaS and
  • AutoML to custom state-of-the-art models.
    Holistic approach that considers data-centric and model improvements.
Our specialist ML Practice can help you leverage the latest advancements in ML both internally and externally to improve operational efficiency or build new customer experiences through personalisation and ML-powered solutions.
Complete the form and we'll contact you to book in your workshop.

Want to know more about how DiUS can help you?



Level 3, 31 Queen St
Melbourne, Victoria, 3000
Phone: 03 9008 5400

DiUS wishes to acknowledge the Traditional Custodians of the lands on which we work and gather at both our Melbourne and Sydney offices. We pay respect to Elders past, present and emerging and celebrate the diversity of Aboriginal peoples and their ongoing cultures and connections to the lands and waters of Australia.

Subscribe to updates from DiUS

Sign up to receive the latest news, insights and event invites from DiUS straight into your inbox.

© 2024 DiUS®. All rights reserved.

Privacy  |  Terms