Full-time role, Fully remote, Fluent and good English accent similar to native speakers

Design and build robust, scalable ETL/ELT pipelines to process structured and unstructured data from various sources in both real-time and batch modes. Architect and optimize data lakes and warehouses on AWS, GCP, or Azure for efficiency, reliability, and cost-effectiveness. Maintain data models, transformation workflows, and orchestration systems such as Airflow, Prefect, and Dagster. Collaborate with cross-functional teams to design data-driven products, machine learning feature stores, and analytical datasets. Responsibilities include leading data governance, quality assurance, lineage tracking, and observability, as well as developing monitoring and alerting solutions using modern tools like Datadog and Prometheus. Build and manage real-time streaming systems and internal/external APIs. Mentor junior engineers and contribute to technical design and best practices. Research and integrate new technologies to enhance the Lightsteps data platform. Candidates should have over 6 years of experience in data engineering or analytics infrastructure, strong Python or Scala skills, and cloud-native environment experience (preferably AWS). Expertise in data modeling, warehousing, orchestration, distributed computing, and CI/CD practices is desired. Familiarity with containerization, NoSQL systems, streaming data platforms, and analytics tools is important. Skills or interest in MLOps, infrastructure-as-code, and GenAI/LLM-based systems are a plus.

Budget range:
Up to $25 per hour

About the client
Co-Founder