If a day's data is loaded twice, how do you detect it and fix it?
💡 Model Answer
Duplicate loads are usually caught by comparing source metadata with target metadata. I’d store a daily checksum or hash of the source file or database snapshot in a metadata table. When a new load arrives, I compute its hash and compare it to the previous day's hash; if they match, a duplicate has occurred. I’d also maintain a load log with timestamps and row counts. A sudden increase in row count for a specific date indicates duplication. To fix it, I’d delete the duplicate rows from the target using a delete statement keyed on the load timestamp or a unique surrogate key. If the duplicate load already committed, I’d use a rollback or a delete‑then‑re‑load strategy. For a larger reprocessing, I’d schedule a back‑fill job that re‑extracts the data for the affected period, applies transformations, and writes to the warehouse using idempotent operations (MERGE). After reloading, I’d run data quality checks to confirm the counts are correct.
This answer was generated by AI for study purposes. Use it as a starting point — personalize it with your own experience.
🎤 Get questions like this answered in real-time
Assisting AI listens to your interview, captures questions live, and gives you instant AI-powered answers on a discreet on-screen overlay.
Get Assisting AI — Starts at ₹500