Visual Product Search Engine for Retail & e-Commerce

Artificial Intelligence & Machine Learning

Case Study

Overview

PROJECT: delaPlex partnered with Plakkr to deliver a cloud-based, visual search engine for customers to more easily find their products. The visual search technology was integrated into a mobile app to enable shoppers to find products simply by taking a photo.
CLIENT: HNTRISM & Plakkr
INDUSTRY: eCommerce Retail

Client Need

The client wanted customers to have the ability to visually search across inventory in their three digital marketplaces by capturing an image real-time or uploading an image of a desired product. The visual search engine should present ranked results displayed visually and semantically based on certain criteria based on machine learning intelligence.

Product Challenges

Using Google Cloud Vision API to detect product objects in images, the biggest challenge was training custom machine learning models to understand, classify, and match labels in the client’s product dataset. Another challenge was determining the primary product when more than one product object was present in an image.

“delaPlex researched the best open source software tools to fit our needs and then used those tools to build an enterprise e-commerce solution. They are a tremendous value-add to our startup process.”

Beryl Mokros, COO
API, eCommerce, Retail

delaPlex Solution


delaPlex partnered with Plakkr to deliver a cloud-based, visual search engine for customers to more easily find their products. The visual search technology was integrated into a mobile app to enable shoppers to find products simply by taking a photo. The visual search engine uses AI & ML to analyze objects in an image and finds results that relate to visual cues with information on the product (or very similar ones), cost options and how to purchase it. The technology provides convenience, saves time, and provides product search options not previously available.

How We Did It

Developed web application in React JS and consumed a Python web service.

Web Application

Selected Magento as the cloud commerce platform to create each marketplace, and jQuery and Prototype for the JavaScript Framework.

Visual Product Creation

Created a set of products each containing reference images that visually describe the product from a set of viewpoints.

Image Data Processing

Created Python based web services to process image data and display the search results.

Image Optimization

Integrated image optimization for product search results.

Core Stack

Magento | Plakrr| Cloud Vision API
Python | Bitbucket| React

How Google Cloud Vision Works


About the Client

HNTRISM is a content-driven digital lifestyle platform providing customers an insightful journey to discover, learn and purchase innovative and emerging brands in home décor and entertaining, niche skincare, haircare, and fragrances, and cutting-edge fashion all in one place.

The platform has four distinctive brands:
HNTRISM | Berchard’s Fine Living | FOUR/POINT/EIGHT | Asbury Apothecary