Tom

Designed and developed scalable data engineering pipelines for high-volume workloads using Apache Spark, Kafka, and Airflow. Built batch and streaming, event-driven processing workflows with reliable execution, fault tolerance, and consistent downstream data availability. Implemented reusable Airflow DAGs with dependency management, scheduling, retries, and operational controls. Containerized data services and workloads with Docker and Kubernetes, and automated platform infrastructure with Terraform. Added production monitoring and operational visibility using Prometheus, and supported data access patterns across Redshift, PostgreSQL, MySQL, and Redis.

Developed scalable data pipelines to ingest and transform application, event, and behavioral data using Kafka, Apache Spark, Hive, and Presto. Built batch processing workflows across distributed datasets, implemented reusable transformation logic, and supported reliable data movement to analytics and engineering consumers. Supported Spark-based processing jobs and large-scale querying with Hive and Presto, and assisted with troubleshooting for Kafka-based ingestion and event-driven workflows.

Developed scalable data pipelines for ingesting and transforming application, event, and behavioral data. Built batch-processing workflows using Apache Spark, Hive, and Presto, creating reusable transformation logic and efficient data movement patterns for analytics and engineering consumers. Supported Kafka-based ingestion and event-driven data workflows, troubleshooting failures and data-flow issues while maintaining reliable end-to-end movement of data through pipeline components. Collaborated with software and data teams to integrate pipelines with application systems and deliver dependable datasets while meeting performance and operational standards.