As Data Product Owners, our mission is to deliver a data platform that empowers teams to make smarter, faster decisions and ultimately unlock the potential for advanced analytics and AI.
Recently, the market has been abuzz with the demand for modern data platforms that not only centralise data but also set the stage for AI-driven tools like forecasting and machine learning.
This blog walks through how we leveraged Snowflake, dbt, and Fivetran to build a scalable data product and enable AI to help us leverage ML insights.
Our journey starts with strong Data Foundations, evolves into Analytics, and paves the way for AI. Tools like Snowflake, dbt, and Fivetran make this progression possible.
Understanding the tools
Snowflake (the data warehouse)
Think of Snowflake as the “home” for your data. It’s a cloud-based storage and computing solution where all your organisation’s data—sales figures, customer info, marketing metrics—lives securely and can be queried quickly.
Why it’s great: It scales easily (no need to worry about running out of space), works across cloud providers like AWS, Azure, or Google Cloud, and lets multiple teams access data without stepping on each other’s toes.
It also contains inbuilt tools to allow the easy access to Machine Learning tools to assist with areas such as forecasting into the future (https://docs.snowflake.com/en/developer-guide/snowflake-ml/overview)
Fivetran (the data mover)
Fivetran is like a courier service that picks up data from all your sources—think CRMs like Salesforce, ad platforms like Google Ads, or even spreadsheets—and delivers it to Snowflake.
Why it’s great: It automates the tedious process of connecting systems, so your team doesn’t have to build custom pipelines from scratch.
dbt (the data transformer)
Here, dbt takes the raw data placed into Snowflake by Fivetran and turns it into something useful—like clean, organised tables or insightful reports.
Why it’s great: It lets your data team define how data should be transformed using simple SQL, making it accessible and collaborative.
Managing the build of your data platform
Step 1: Define the goals and scope
What to do: Start by sitting down with stakeholders—executives, analysts, marketing, and finance—to understand what they need from the data platform. Are they looking for real-time sales dashboards? Forecasting insights utilising ML to predict sales? Customer behaviour insights? Compliance reporting?
Data Product Owner: Translate these needs into clear project goals (e.g., “Deliver a dashboard showing daily sales by region within 8 weeks”) and document the scope (e.g., which data sources to include).
Tip: Avoid scope creep—agree upfront on a manageable first phase (like 3-5 key data sources) and plan for future expansions later. Even in the scenario of a large-scale migration, it’s still best to split these out into testable pieces rather than finding out later in the journey that there were some translation issues between the layers.
Step 2: Assemble the team
What to do: Identify who’ll do the work. You’ll likely need:
- Data engineers to set up Snowflake and Fivetran.
- A data analyst or data scientist to handle dbt and define transformations that should occur.
- Any relevant SME’s to validate outputs if it ventures outside your usual scope of data that you manage.
Data Product Owner: Coordinate resource allocation, clarify roles, and ensure everyone knows who’s accountable for what. Also ensure to prioritise the work and clarify how you will measure success once you deploy, this could be something akin to matching all previous reports if running a migration, or allowing additional insights/reports that did not exist before.
Tip: If your team is new to these tools, budget time for a quick training session—a lot of vendors offer great onboarding resources (i.e. https://docs.snowflake.com/user-guide-getting-started).
Step 3: Set up the foundation with Snowflake
What to do: Have Snowflake up and running as the central hub. This means choosing a cloud provider (AWS, Azure, Google Cloud) and setting up user access.
Data Product Owner: Work with your data engineer to define timelines and budget (Snowflake pricing is usage-based, so set limits). Ensure security requirements (e.g., who can access what) are clear from the start.
Tip: Start small—set up a “sandbox” environment for testing before going live with production data.
Step 4: Connect data sources with Fivetran
What to do: Use Fivetran to pull data from your chosen sources into Snowflake. For example, connect your CRM, ERP, or marketing tools.
Data Product Owner: Coordinate with source system owners (e.g., the Salesforce admin, Finance Team etc) to secure access credentials and prioritise which data to ingest first. Track progress as connectors are built and tested. From an AI perspective, it’s also good to start gathering expectations as to what we believe the outcomes to be, this will help us determine how accurately the predictions may be.
Tip: Fivetran’s pre-built connectors are plug-and-play, but custom sources might need extra time—so we want to flag these early.
Step 5: Transform data with dbt
What to do: Have your data team use dbt to clean and organise the raw data in Snowflake into usable formats—like merging customer data from multiple sources into one table.
Data Product Owner: Facilitate collaboration between analysts and business leads to define what usable looks like (e.g., a report-ready dataset). Then monitor testing to ensure transformations work as expected.
Tip: We suggest encouraging version control in dbt—it prevents chaos if something breaks.
Step 6: Test and validate
What to do: Run end-to-end tests to confirm data flows from sources through Fivetran to Snowflake, gets transformed by dbt, and matches stakeholder expectations.
Data Product Owner: Set up a testing schedule, assign responsibilities (e.g., analysts check numbers, business leads verify insights) and document results.
Tip: Build in a buffer—data issues like missing records or duplicates often pop up here and need fixing, we spent a good deal of time following deployment confirming the reconciliation across many reports often differing by a few values.
Step 7: Build machine learning models with Snowflake ML
What to do: Leverage the validated data in Snowflake to build and deploy machine learning models using Snowflake ML capabilities. Use the snowflake-ml-python library or Snowflake’s built-in ML Functions to create models for forecasting (e.g., predicting future sales trends), anomaly detection (e.g., spotting unusual customer behaviour), or other advanced insights tailored to your stakeholders’ goals. Data scientists can utilise Snowflake Notebooks or the Container Runtime for ML to preprocess data, engineer features, and train models at scale, taking advantage of distributed processing with CPUs or GPUs if needed.
Data Product Owner: Collaborate with data scientists or analysts to identify high-value use cases (e.g., forecasting sales for the next quarter) and ensure the models align with business objectives defined in Step 1.
Tip: Start with a simple use case, like forecasting, to demonstrate value quickly—Snowflake’s out-of-the-box ML Functions can help non-experts get started without writing complex code, while custom Python workflows offer flexibility for advanced users. Note that once built, you may need to make adjustments to the model to account for various fluctuations in the data (e.g. seasonality, trends, price increases/decreases etc).
Step 8: Deploy and train
What to do: Launch the platform and train end-users (e.g., analysts or execs) on how to access and use it—whether through Snowflake directly or a BI tool like Tableau or Looker.
Data Product Owner: Coordinate the rollout, schedule training sessions, and gather feedback to tweak the setup.
Tip: Make sure to celebrate the win! You have successfully helped roll out a new data foundation for your company and this is a great achievement and will bring a lot of new value to members internally.
Step 9: Plan for maintenance and growth
What to do: Establish a process for monitoring performance, adding new data sources, and updating transformations as needs evolve.
Data Product Owner: Assign ongoing ownership (e.g., a data team lead), set review cadences (e.g., monthly check-ins), and plan for phase two enhancements.
Tip: Keep stakeholders engaged—show them quick wins to secure buy-in for future investments as this can be tricky to do during the development of the platform. Many less technical members of the team may find it hard to see progress until supplying them with new foundational reports.
Why this matters
A well-built data platform is a game-changer for your organisation. With Snowflake, Fivetran, and dbt, you’re delivering a foundation that turns raw data into actionable insights, all while keeping the process manageable and scalable.
It also empowers you and the business to find many more insights and predictions while using the power of ML to analyse upcoming trends, forecast possible sales, and understand when you may need to expand certain areas of your business before it reaches a bottleneck.
Allow your organisation to make data-driven decisions and the results will speak for themselves.
