What architecture did you follow for your data pipeline? Did you use a traditional data warehouse structure or a medallion architecture, and how did you implement incremental loading?

I followed a hybrid approach that combined a data warehouse for reporting and a medallion architecture for raw and curated data. The bronze layer ingested raw logs into S3, the silver layer transformed them into cleansed tables using Spark, and the gold layer produced business‑ready fact tables in Redshift. Incremental loading was achieved with change‑data‑capture (CDC) from source systems and watermarking in Spark jobs. Each layer had its own schema and lineage, and we used dbt to manage transformations and tests. This design allowed us to keep the warehouse lean while still providing a flexible lake for exploratory analytics.