I'm a senior data and backend engineer with 10+ years of experience building large-scale data platforms, distributed systems, and cloud-native pipelines across AWS and GCP. I enjoy turning complex data problems into robust, scalable solutions and love collaborating across teams to ship impact. I'm an IDF 8200 alumni based in Warsaw, and I work with global teams to deliver value through data, ML, and modern cloud architectures.

Ivan Zolotaryov

I'm a senior data and backend engineer with 10+ years of experience building large-scale data platforms, distributed systems, and cloud-native pipelines across AWS and GCP. I enjoy turning complex data problems into robust, scalable solutions and love collaborating across teams to ship impact. I'm an IDF 8200 alumni based in Warsaw, and I work with global teams to deliver value through data, ML, and modern cloud architectures.

Available to hire

I’m a senior data and backend engineer with 10+ years of experience building large-scale data platforms, distributed systems, and cloud-native pipelines across AWS and GCP. I enjoy turning complex data problems into robust, scalable solutions and love collaborating across teams to ship impact.

I’m an IDF 8200 alumni based in Warsaw, and I work with global teams to deliver value through data, ML, and modern cloud architectures.

See more

Experience Level

Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert
Intermediate
Intermediate
See more

Language

English
Fluent
Hebrew (modern)
Fluent
Russian
Fluent

Work Experience

Senior Software Engineer at Yahoo! Media
August 1, 2025 - September 24, 2025
Built and maintained Content Analysis Platform (CAP) processing 10,000+ news articles daily with HTML parsing and metadata extraction; engineered a real-time Apache Flink pipeline using GenAI/OpenAI for news classification; resolved region-specific parsing issues restoring content visibility; architected a cloud-native data pipeline for Wikidata entity resolution on AWS; directed CAP migration and migrated Java features to Python.
Senior Software Engineer at Yahoo! Ad Tech
January 31, 2024 - September 24, 2025
Corrected a duplicate-ad issue in the carousel boosting ad relevance and impressions; coordinated cross-region teams to deliver Brand Safety feature; developed monitoring agents to maintain stability and validate ad update events; optimized Hive ETL processes and new logic for Brand Safety.
Data Engineer at FinityX
January 31, 2022 - September 24, 2025
Architected GCP infrastructure with Airflow DAGs for scalable Kubernetes pod spawning and AI model training; deployed real-time Pub/Sub streaming for trading predictions; built Neo4j graph database from Wikidata; designed Spark-based ML prep pipeline on GCP; implemented Java HFT client via FIX protocol; created distributed data platform on Kubernetes with Scala (Monix) and Kafka with OpenTelemetry, processing 1M+ daily US stock market records.
Data Engineer at IDF Intelligence Corps (Unit 8200)
May 1, 2020 - September 24, 2025
Led enterprise-scale Hadoop cloud migration; improved Hive job performance by 80% using Spark Streaming and Spring Boot Reactive; enhanced Solr search with custom components and improved TF‑IDF ranking; redesigned Hadoop ETL workflows to reduce runtime by 50%; authored open-source SolrAdminToolkit for cloud administration.
Senior Software Engineer at Yahoo! Media
August 1, 2025 - September 24, 2025
Built and maintained Content Analysis Platform (CAP) processing 10,000+ news articles daily via HTML parsing, metadata extraction (entities, language, dates), and ML enrichment. Engineered a real-time Apache Flink pipeline leveraging GenAI/OpenAI APIs for Yahoo news classification. Restored 100% content visibility for Swedish financial news on Apple Stocks and led cloud-native Wikidata entity-resolution pipeline on AWS. Drove CAP migration, authored architecture docs, and ported CAP features from Java to Python, validating 10K+ tests with sub-50ms latency.
Senior Software Engineer at Yahoo! Ad Tech
January 1, 2024 - September 24, 2025
Resolved duplicate-ad issue in carousel, boosting ad relevance and impressions by 1.5%. Coordinated cross-regional teams to deliver Brand Safety feature for compliance and brand protection. Developed monitoring agents to ensure system stability and validated ad update events, preventing data pipeline failures. Programmed and optimized Apache Hive ETL processes, modifying data processing workflows and adding new logic for Brand Safety functionality.
Data Engineer at FinityX
January 1, 2022 - September 24, 2025
Architected GCP infrastructure with Airflow DAGs for scalable Kubernetes pod spawning and AI model training (20+ models). Deployed real-time Pub/Sub streaming for trading predictions and market data, handling 1000+ buy/sell commands. Constructed Neo4j graph database from Wikidata to map company relationships for AI training and financial analytics. Designed ML platform for data science with Apache Spark on GCP to prepare market data from 6 sources for quantitative model training. Implemented Java-based HFT client using FIX Trading Protocol and IBKR API, executing 4–5 trades per second. Created distributed data platform on Kubernetes using Scala (Monix) and Apache Kafka with OpenTelemetry monitoring, processing 1M+ daily US stock market records in 1-minute windows.
Data Engineer at IDF Intelligence Corps (Unit 8200)
May 1, 2020 - September 24, 2025
Led enterprise-scale Hadoop cloud migration, transferring 15 ETL pipelines and maintaining end-to-end stability. Improved Apache Hive job performance by 80% through redesign with Spark Streaming and Spring Boot Reactive. Enhanced Solr search by creating custom components, reindexing clusters, and tuning Lucene scoring and TF‑IDF. Redesigned Hadoop ETL workflows, removing redundant Hive writes with custom UDFs and reducing runtime by 50%. Authored the open-source SolrAdminToolkit for cloud administration, adopted by multiple teams.

Education

Software Developers Course at IDF Intelligence Corps (Unit 8200) - Mamas Israel
January 11, 2030 - January 1, 2016
High School Diploma in Computer Science and Physics at AMIT Technology & Science High School, Israel
January 11, 2030 - January 1, 2015
Software Developers Course at IDF Intelligence Corps (Unit 8200) – Mamas Israel
January 11, 2030 - January 1, 2016
High School Diploma (Computer Science & Physics) at AMIT Technology & Science High School
January 11, 2030 - January 1, 2015

Qualifications

Software Developers Course
January 11, 2030 - January 1, 2016

Industry Experience

Software & Internet, Media & Entertainment, Professional Services, Financial Services