Amit Almagor

Engaged through Lemon.io as an hourly contract engineer on data platform projects for venture-backed startup clients, focusing on data infrastructure for fast-scaling SaaS and platform companies. Separately engaged with a global insurance/financial services client building Azure-based analytics infrastructure: designed and implemented data ingestion and transformation pipelines on Azure Data Lake Storage Gen2, Azure Data Factory (ADF), and Azure Synapse Analytics. Built distributed Spark workloads on Synapse Spark pools for large-scale batch processing of structured and semi-structured datasets, with schema-on-read patterns over ADLS. Implemented ETL/ELT pipelines, data modeling, and partitioning strategies in ADLS to support downstream analytics and reporting workloads.

Designed, operated, and optimized large-scale data pipelines on Google Cloud (BigQuery, Dataflow, Datastream, Dataproc), supporting both real-time streaming and batch analytics workloads processing millions of records daily. Led a 14-person technical team covering BigQuery, Dataproc, Dataflow, Data Fusion, and Composer, driving faster resolution times and improved customer satisfaction. Built and maintained streaming data pipelines with Dataflow and Datastream for low-latency analytics and large-scale ETL, processing structured and semi-structured data at scale. Supported Dataform-based analytics workflows for enterprise customers, ensuring data pipeline reliability, correctness, and maintainability across production environments. Contributed to hybrid and multi-cloud architectures spanning GCP, AWS, and Azure, improving scalability and reducing vendor lock-in for distributed data services. Collaborated with Google engineers to identify and resolve global-level product bugs affecting e

Designed and automated Python-based ETL pipelines integrating with Workday LMS, cutting manual enrollment workload by 70% and improving data accuracy. Built and deployed data pipelines on GCP (BigQuery, DOMO) to process penetration test results, accelerating security reporting cycles from weeks to days. Partnered with engineering and security teams to migrate on-premises databases to BigQuery and Azure SQL, reducing infrastructure cost and improving scalability. Mentored 20+ employees in Tableau and SQL best practices, increasing data self-service adoption and team-wide data literacy.