How do you manage schema changes in time-series data? For example, if a pipeline suddenly receives five new columns, how would you handle that?

For time-series pipelines, I rely on a schema registry to track the current schema and its evolution. When new columns appear, I first validate the incoming data against the registry; if the schema is incompatible, I trigger a schema update. I then update the ingestion job to map the new columns to the appropriate fields, adding them to the target database or data lake. To avoid breaking downstream consumers, I keep the old schema version active and expose both versions via a versioned API or a transformation layer that can drop or rename columns as needed. I also update any downstream dashboards or queries to handle the new fields, using optional columns or default values. Finally, I document the change, update the data catalog, and run regression tests to ensure that the pipeline still processes data correctly. This approach balances flexibility with stability in a dynamic time-series environment.