Development and support of the cloud data analysis solution
Industry
Retail, E-commerce
The customer
Large automobile retailer and auction house (North America)
Project description
The customer came up with the necessity of developing of new analytiсal platform which would provide for broader use of collected data. Our team helped the customer to choose and migrate to the new MPP (massively parallel processing) system and developed the new analytical platform from the scratch. We designed the architecture, developed the data model, and embedded the ETL process in all the layers of the warehouse. It allowed us to gather, store and analyze data from several sources in an efficient manner. Significant work was done on introducing new DQ (data quality) instruments helping to easily form accurate KPIs, management, and accounting reports.
the outcome
As a result, the client has cut in half its costs on the support of the DWH due to the lesser need for interference from the IT staff. The efficiency of data analytics has raised while the workload of data analysts has significantly decreased, allowing them to perform more tasks with no additional costs. Due to the new DQ system, the client was able to provide accurate reports to the regulatory bodies in time with no mistakes connected with the human factor, which allowed to minimize the resources spent on settling disputes with the government.
tech stack
AWS, Snowflake, AWS Lambda, AWS Glue, EMR-clusters, PostgreSQL, MicroStrategy
