Azure Data Engineer at Ascension St. John Health System

Designed, built, and maintained scalable data pipelines to ingest and transform large volumes of data from transactional systems, APIs, and external feeds. Utilized technologies including HBase, Pig, Spark, Scala, Azure Data Factory, Jenkins, Docker, Kubernetes, and Snowflake to architect robust data solutions. Developed Spark SQL applications in Databricks and integrated APIs with Kafka for real-time processing. Automated infrastructure provisioning with Terraform and implemented CI/CD pipelines. Delivered optimization and performance tuning for Snowflake data warehouse and Azure Synapse integrations to improve efficiency and reduce costs.

AWS Data Engineer at Devon Energy

Developed and maintained data models and warehouses supporting analytics and ML initiatives using AWS cloud services. Created ETL jobs and Lambda functions scheduled via CloudWatch and integrated streaming data pipelines with Kafka and Apache Spark. Leveraged AWS Glue, S3, and Lambda for data processing automation. Enhanced data extraction using PySpark data frame conversions and visualized insights with Elasticsearch and Kibana. Led CI/CD deployments using Jenkins within Kubernetes and Docker environments. Integrated Control-M for job orchestration and ensured data consistency and security across large-scale platforms.

GCP Data Engineer at Five Star Business Finance Limited

Implemented data quality monitoring, governance, and large-scale ETL/ELT pipelines in Google Cloud Platform using Dataflow, Dataproc, BigQuery, and Cloud Composer. Designed cloud infrastructure including VPC setups with secure subnetting and cluster provisioning in Databricks. Developed Python RESTful APIs for mobile applications and orchestrated workflows across Pub/Sub and Cloud Functions. Utilized Sqoop for data ingestion and managed Airflow DAGs for pipeline schedules. Played a key role in performance tuning and cost optimization of cloud environments, integrating multiple databases and seamless data migration processes.

Data Engineer at MRF

Managed and optimized data infrastructure including Hadoop and Spark clusters to ensure high performance and availability. Developed multi-threaded Java ingestion jobs and Sqoop tasks to facilitate large-scale data transfer from FTP servers and warehouses. Configured Hive tables with partitioning and bucketing for efficient querying. Implemented Kafka with Spark Streaming for real-time processing and led migration projects from on-premises SQL Server to Azure Synapse Analytics. Enhanced workflow automation by developing Apache Airflow DAGs and participated in CI/CD processes using Jenkins and Docker.

Azure Data Engineer at Ascension St. John Health System

Designing, building, and maintaining scalable data pipelines to ingest, transform, and store large volumes of data from transactional systems, third-party APIs, and external data feeds. Implemented Spark/PySpark in Databricks, orchestrated ETL/ELT workflows with Apache Airflow, and staged API and Kafka data into Snowflake. Developed schema-less data models in MongoDB and Capella, and built CI/CD pipelines with Jenkins on Kubernetes.

Data Engineer at Devon Energy

Developed and maintained data models and data warehouses to support reporting, analytics, and machine learning initiatives. Built AWS data pipelines (Redshift, Glue), created Lambda functions and triggered them with CloudWatch events (SQS, EventBridge, SNS), and processed data with Kafka and Apache Spark for real-time analytics. Implemented Elasticsearch/Kibana dashboards for quick insights.

Data Engineer at Five Star Business Finance Limited

Implemented data quality checks, data governance, and security controls. Built ETL/ELT workflows using Dataflow and BigQuery on Google Cloud Platform; provisioned Databricks clusters for batch and streaming processing; migrated datasets to MongoDB; developed REST APIs with Django/Flask; integrated Cloud Composer with Pub/Sub and Dataproc for orchestration; contributed to CI/CD pipelines with Jenkins and Docker.