E-Commerce Medallion Pipeline
A data engineering pipeline for an e-commerce dataset, organized into Bronze, Silver and Gold layers. The project's goal is to demonstrate end-to-end data engineering skills: transformation with Polars, analysis with DuckDB and orchestration with Airflow. Currently in development — orchestration with Airflow and the Gold layer haven't been reached yet, so this page covers only the project's general approach for now.
This case study is under construction. The idea is to take e-commerce data and process it in layers following the Medallion architecture: Bronze (raw data as it arrives), Silver (cleaned and validated data) and Gold (curated, analysis-ready data). Transformation is done with Polars, analysis with DuckDB, and the whole flow is orchestrated with Apache Airflow.
As of this writing, development hasn't reached Airflow orchestration or the Gold layer yet — detailed content (business context, technical decisions, results) will be added as the project progresses.