Large-Scale Data Processing & Feature Engineering Pipeline

Engineered a robust data pipeline for handling large-scale datasets.

1. Processed TB-scale datasets using Pandas, PySpark, and Dask
2. Built automated ETL pipelines with validation and anomaly detection
3. Implemented data quality checks and schema validation
4. Optimized memory usage and processing time by >40%