An end-to-end data engineering pipeline that extracts real-time flight data from an API, loads parquet files into AWS S3, and applies the Medallion architecture using DuckDB and dbt for robust analytics models and automated testing.
Python
AWS S3
DuckDB
dbt
CI/CD