Available to hire
Sr Cloud Data Engineer with around 7 years of experience designing and developing Azure/GCP cloud data engineering solutions across financial services, monitoring/observability platforms, manufacturing/IoT, and analytics. Experienced in ETL/ELT, batch and streaming pipelines, scalable data processing, and building analytics-ready data platforms.
Hands-on with Azure Databricks, Azure Data Factory, Azure Synapse, BigQuery, Delta Lake, and PySpark/Python/SQL. Strong in data modeling (medallion architecture), orchestration/CI-CD (Airflow, ADF, Azure DevOps, Terraform), monitoring/logging, data quality/reconciliation, and secure access management (RBAC, Managed Identities, Key Vault, IAM).
Skills
Experience Level
Expert
Expert
Expert
Expert
Intermediate
Intermediate
Intermediate
Beginner
Beginner
Beginner
Work Experience
Senior Data Engineer at USAA
August 1, 2024 - PresentDesigned and developed enterprise-scale Azure data engineering solutions supporting insurance operations, banking analytics, customer reporting, risk analysis, and operational intelligence. Built scalable batch and near real-time ETL/ELT pipelines using PySpark, Python, SQL, and Azure Databricks; implemented ingestion from REST APIs, Azure SQL Database, flat files, and external vendor systems via Azure Data Factory. Developed a centralized data lake using Medallion Architecture (Bronze/Silver/Gold) with Delta Lake and ADLS Gen2 to deliver analytics-ready datasets. Built near real-time and streaming pipelines using Event Hubs, Stream Analytics, and Databricks Structured Streaming for customer activity/event processing. Created curated datasets and dimensional models in Azure Synapse for compliance reporting and KPI dashboards. Implemented data quality and reconciliation checks (schema validation, null/duplicate handling, consistency checks) and optimized Spark performance (partitioning,
Enterprise Data Engineer at Dynatrace
July 1, 2023 - August 1, 2024Worked on cloud-based data engineering and analytics solutions for telemetry monitoring, operational reporting, compliance analytics, and application observability. Built batch and near real-time ETL/ELT pipelines using PySpark, SQL, and Azure Databricks; implemented Medallion Architecture with Delta Lake and ADLS Gen2. Created ingestion workflows in Azure Data Factory to load data from REST APIs, SQL databases, monitoring platforms, and cloud storage. Processed and transformed telemetry datasets to power operational dashboards and monitoring analytics. Optimized Spark performance using partitioning and query tuning. Integrated Intune and Azure Monitor data into Log Analytics with KQL for compliance/anomaly use cases. Implemented data quality checks for schema validation, duplicates, and reconciliation; developed curated datasets and dimensional models for KPI reporting. Built CI/CD pipelines with Azure DevOps, Git, Terraform, and ARM templates; configured RBAC/Managed Identities/Servi
Data Engineer at Cummins Inc
March 1, 2022 - June 1, 2023Designed and developed scalable batch and near real-time GCP data pipelines supporting manufacturing operations, IoT equipment monitoring, supply chain analytics, and operational reporting. Built ETL/ELT workflows using PySpark, Python, SQL, Google Cloud Dataflow, and distributed Spark processing for IoT telemetry, machine logs, and production events. Implemented streaming/batch pipelines with Dataflow, Pub/Sub, and Dataproc. Designed cloud-native data lake and analytics layer with GCS and BigQuery. Developed ingestion frameworks for REST APIs, ERP systems, relational databases, and flat files into BigQuery/GCS. Created curated datasets, reporting tables, and dimensional models in BigQuery for dashboards and KPI reporting. Implemented data quality validation and reconciliation (schema checks, null/duplicate handling, consistency verification). Optimized BigQuery partitioning/clustering and query execution for cost and performance. Orchestrated scheduled pipelines using Cloud Composer (
Data Engineer at Brio Technologies
June 1, 2019 - January 1, 2022Junior Data Engineer supporting cloud migration, ETL development, and analytics initiatives on GCP. Assisted in developing batch ETL/ELT pipelines using Python, SQL, and PySpark for structured/semi-structured business data. Supported processing workflows using Dataflow and Dataproc for reporting and analytics. Built SQL transformations and stored procedures for cleansing and enrichment; loaded data into BigQuery and GCS for downstream analytics. Assisted ingestion from relational databases, flat files, REST APIs, and cloud storage into GCP. Supported data validation and reconciliation (schema checks, duplicates, null validation, consistency checks). Worked with Cloud Composer (Airflow) for scheduling/orchestration, monitored job execution, and helped troubleshoot workflow failures. Participated in migration of legacy ETL and reporting systems into cloud-native platforms. Used Git for version control and supported CI/CD using Cloud Build. Prepared documentation/runbooks and supported in
Education
Bachelor in Computer Science and Engineering at Jawaharlal Nehru Technological University
September 1, 2015 - May 1, 2019Bachelors in Computer Science and Engineering at Jawaharlal Nehru Technological University
September 1, 2015 - May 1, 2019Bachelors in Computer Science and Engineering at Jawaharlal Nehru Technological University
September 1, 2015 - May 31, 2019Bachelors in Computer Science and Engineering at Jawaharlal Nehru Technological University
September 1, 2015 - May 1, 2019Bachelors in Computer Science and Engineering at Jawaharlal Nehru Technological University
September 1, 2015 - May 1, 2019Qualifications
Microsoft Certified: Azure Administrator Associate
January 11, 2030 - August 19, 2026Microsoft Certified: Azure Administrator Associate
January 11, 2030 - August 19, 2026Microsoft Certified: Azure Administrator Associate
January 11, 2030 - August 19, 2026Microsoft Certified: Azure Administrator Associate
January 11, 2030 - August 19, 2026Industry Experience
Financial Services, Manufacturing, Software & Internet, Computers & Electronics, Professional Services
Skills
Experience Level
Expert
Expert
Expert
Expert
Intermediate
Intermediate
Intermediate
Beginner
Beginner
Beginner
Hire a Data Engineer
We have the best data engineer experts on Twine. Hire a data engineer in Milford today.