Explain a project where you faced challenges using Azure Data Fabric and Azure services, and how you addressed them. Additionally, describe the difference between a lakehouse and a data warehouse, and how it differs from Azure Data Lake.

In a recent project I used Azure Data Fabric (Synapse Analytics) to orchestrate data ingestion from on‑premises sources into a lakehouse architecture. The main challenge was handling schema drift and ensuring data quality while keeping the pipeline automated. I leveraged Azure Data Factory pipelines to copy data into Azure Data Lake Storage Gen2, then used Synapse Spark pools to clean and transform the data. To manage schema drift, I enabled schema inference in the Spark job and wrote a custom UDF to map new columns to the existing schema, storing the mapping in a Delta table. For workflow, I used Synapse pipelines to schedule the Spark job, monitor its status, and trigger downstream analytics. I also set up alerts in Azure Monitor for failures. A lakehouse combines the best of data lakes and data warehouses: it stores raw data in a lake (e.g., ADLS) but provides ACID transactions, schema enforcement, and SQL‑like querying via Delta Lake or Synapse SQL. A traditional data warehouse stores structured data in a schema‑on‑write fashion, optimized for OLAP queries. The lakehouse allows schema‑on‑read and supports both batch and streaming workloads, whereas a warehouse is more rigid. Azure Data Lake is simply a storage layer; it does not provide the transactional guarantees or query engine that a lakehouse or warehouse offers.