How would you design a system to capture changes using Change Data Capture (CDC) from a source database?

To capture changes using CDC, first identify the source database and the type of changes needed (insert, update, delete). Choose a CDC method: log-based (most efficient and non-intrusive) or trigger-based (simpler but can add overhead). For log-based CDC, use the database’s transaction log (e.g., MySQL binlog, PostgreSQL WAL, Oracle redo logs). Tools like Debezium, AWS DMS, or Kafka Connect can tail these logs and emit change events. The pipeline typically includes:

Key decisions: choose a broker that matches latency and throughput needs; use schema registry for evolving schemas; secure the pipeline with encryption and IAM roles; monitor lag and error rates. This architecture scales horizontally, provides near real-time replication, and supports downstream analytics or microservices that react to data changes.