What is Delta Lake in Databricks, and what are its main components?

Delta Lake is an open‑source storage layer that brings ACID transactions, schema enforcement, and time travel to data lakes. It sits on top of your existing data lake (e.g., S3, ADLS) and stores data in Parquet files while maintaining a transaction log. The transaction log, stored as JSON files in the _delta_log directory, records every change (insert, update, delete) and ensures consistency across concurrent reads and writes. The Delta table is the logical view that users query; it is defined by a set of Parquet files and the log. Key features include upserts (merge), deletes, compaction, and schema evolution. Delta Lake also supports both batch and streaming workloads, allowing real‑time analytics on the same data. By providing a single source of truth and strong consistency, Delta Lake simplifies data engineering pipelines and reduces data quality issues.