Did you implement data validation and quality checks?

Situation: In my previous role at a fintech company, the data lake was receiving raw customer records from multiple sources, and we frequently encountered missing values, inconsistent formats, and duplicate entries. Task: I was tasked with designing and implementing a robust data validation and quality check framework to ensure that downstream analytics and reporting were based on clean, reliable data. Action: I leveraged AWS Glue to orchestrate ETL jobs and integrated Great Expectations to create expectation suites that validated schema conformity, null‑value thresholds, range checks, and referential integrity. I scheduled nightly jobs that ran these checks, logged failures to CloudWatch, and set up SNS alerts for critical failures. I also built a lightweight dashboard in Tableau to visualize data quality metrics over time. Result: The implementation reduced data quality incidents by 35%, cut downstream processing errors by 25%, and gave stakeholders confidence in the data, enabling faster decision‑making and reducing manual data cleansing effort.