Problem

A leading online wholesaler in the food and beverage industry faced significant challenges in maintaining high-quality product catalogue information. Poor product data quality and inconsistency, common issues for e-commerce platforms, hindered the retailer’s ability to offer targeted services, features, and analytics. Additionally, a recent acquisition significantly increased the number of product categories and subcategories, exacerbating the problem.

Solution

DiUS employed an innovative combination of traditional machine learning and generative AI for product classification and data matching to standardise product data structure. The approach involved training machine learning models in Amazon SageMaker to accurately identify and categorise new and existing products. Regex, a sequence of characters used as a search pattern, and LLMs from OpenAI, GCP, and AWS were evaluated for their effectiveness in extracting attributes from product catalogues to enhance classification.

Result

The solution demonstrated that machine learning and generative AI are well-suited to address data quality and consistency issues, thereby improving the overall quality of product catalogue information and enabling the provision of targeted services and analytics. The machine learning classification process for incoming data achieved 94% accuracy on the validation dataset. LLMs outperformed the regex approach, with the most effective model achieving 95% accuracy in product data matching using brand names and units of measure.