In the recent project you worked on, how did you handle data ingestion from multiple source systems into your data warehouse?

In a recent project I was responsible for building a data ingestion pipeline that fed multiple source systems into a Snowflake data warehouse. The first step was to identify the source data formats and connectivity options; we used Azure Data Factory to orchestrate the workflow and leveraged its built‑in connectors for SQL Server, Salesforce, and flat files in Azure Blob Storage. For each source, we performed an extract step that pulled the latest data into a staging area in Snowflake using the COPY command. To handle schema drift, we used a schema registry and applied a transformation layer in Snowflake that mapped source columns to a canonical schema. Incremental loads were implemented with change data capture (CDC) using timestamps and hash columns, so only new or updated rows were processed. Data quality checks were added as Snowflake tasks that ran nightly to flag nulls, outliers, and duplicate keys. Finally, we loaded the cleaned data into the fact and dimension tables using MERGE statements to upsert records. The entire pipeline was scheduled via Azure Data Factory, monitored with Snowflake’s query history, and alerts were set up in Azure Monitor. This approach ensured reliable, repeatable ingestion, maintained data lineage, and allowed the analytics team to query fresh data within minutes of source updates.