Freelance and remote Data engineers jobs

Build and maintain scalable data pipelines and reusable datasets in a Databricks lakehouse environment to support reporting, visualization, machine learning and data-driven decision-making across finance, sales, supply chain, marketing and other business functions. Key responsibilities include: Develop and maintain scalable data pipelines and data products. Ingest, validate, clean and transform structured and unstructured data using Python, PySpark and SQL. Build data models and reporting datasets for Tableau and other analytics tools. Integrate existing data warehouses and reporting systems with modern cloud analytics platforms. Improve pipeline reliability, data quality and processing performance. Implement automated testing, monitoring, documentation and CI/CD practices. Collaborate with analysts, architects, developers and business stakeholders to deliver effective data solutions. Provide technical guidance, conduct code reviews and mentor other data engineers. Requirements At least 7 years of data engineering experience, including developing and supporting production data pipelines. Strong SQL skills and knowledge of data modelling, database design and data warehousing. Hands-on experience with Databricks, Apache Spark and cloud data platforms. Strong proficiency in Python and PySpark. Experience developing ETL/ELT pipelines and integrating data from multiple source systems. Experience creating reliable datasets for Tableau or comparable business intelligence tools. Familiarity with source control, automated testing and CI/CD practices. Ability to troubleshoot complex data issues and optimise pipeline performance. Strong communication and collaboration skills, with experience working with technical teams and business stakeholders. Preferred Experience and Qualifications Experience with Spotfire, Oracle, Informatica, R or data virtualisation. Databricks certification. Read more

We are seeking an experienced Data Engineer to design, develop, and support enterprise data solutions. The role will focus on data engineering, modeling, integration, quality, warehousing, governance, analytics, business intelligence, and data science initiatives. The successful candidate will work with modern Microsoft Azure, Informatica, Snowflake, and related data technologies to deliver scalable and reliable data platforms. They will also collaborate with technical teams, business stakeholders, and senior leadership to translate complex data and technical concepts into clear, actionable insights. This role requires a proactive, adaptable, and strategic professional who can identify opportunities for improvement, make informed decisions, manage competing priorities, and contribute to a positive and inclusive work environment. Requirements Bachelor’s degree in Computer Science, Data Science, Software Engineering, Information Systems, or a related quantitative field. At least 8 years of experience in data engineering or a closely related data-focused discipline. Experience with data modeling, ETL/ELT, data integration, master data or entity management, data quality, data warehousing, analytics, business intelligence, data science, and data governance. Proficiency in programming and query languages, including C, Python, SQL, and Java. Experience with Microsoft data technologies, including SQL Server, Synapse, Cosmos DB, HDInsight, Azure Data Factory, Azure Databricks, Power BI, and Microsoft Fabric. Experience with Informatica Cloud technologies, such as Cloud Integration Hub, Data Integration Hub, Cloud Data Governance and Catalog, Master Data Management, and Data Quality. Experience with Snowflake and other leading industry data platforms. Strong written and verbal communication skills, including the ability to communicate effectively with senior leadership and both technical and non-technical audiences. Ability to simplify and explain complex technical information in a clear, engaging, and memorable manner. Strong critical-thinking, creative problem-solving, research, analytical, and decision-making skills. Ability to identify trends in complex data, assess short- and long-term impacts, advise decision-makers, and influence stakeholders through effective persuasion and negotiation. Proactive and innovative approach, with a focus on continuous improvement, efficiency, strategic thinking, and delivering a strong client experience. Ability to adapt to changing circumstances, manage multiple workstreams, respond to unexpected issues, and balance short- and long-term priorities. Demonstrated respect for diverse backgrounds, perspectives, and working styles, with strong interpersonal awareness and collaboration skills. Positive, resilient, and professional approach to challenging situations, conflict resolution, and difficult conversations. Commitment to knowledge sharing, stakeholder engagement, and building a strong professional reputation. Additional Information Location: Remote role supporting Chicago, Illinois; local candidates are highly preferred. Employment Type: W2 only. Work Authorization: Any authorization accepted except H1B and CPT. Engagement Restrictions: C2C, 1099, and third-party profiles are not accepted. Read more

Cruz Street is seeking a hands-on Data Engineer for a contract-to-hire role supporting client data solutions and the development of DashProphet 2.0, its AI-enabled SaaS analytics product. The role begins at approximately 20 hours per week, with a potential path to full-time employment as the company grows. The successful candidate will build production-grade data pipelines across AWS and Azure, develop analytics-ready data models, support embedded and multi-tenant analytics, and prepare data foundations for AI and data science applications. Key responsibilities include: Building integrations with REST APIs, SFTP, databases, and files, including authentication, pagination, rate limiting, and incremental loading. Developing S3-based lake and lakehouse architectures using raw, staged, and curated data zones, Parquet or Iceberg, and the Glue Data Catalog. Creating ETL and ELT pipelines in Python and PySpark using AWS Glue and Azure Data Factory. Developing dimensional data models, stored procedures, and optimized queries in Amazon Redshift and Azure Synapse, with ad hoc access through Athena. Building QuickSight datasets and dashboards, including SPICE and row-level security for multi-tenant environments. Orchestrating workflows with Step Functions, EventBridge, Lambda, and related monitoring, alerting, and data quality controls. Preparing data pipelines and features for AI applications using services such as Amazon Bedrock, SageMaker, embeddings, and vector stores. Contributing to infrastructure-as-code and CI/CD using tools such as CDK, CloudFormation, Terraform, GitHub Actions, and CodePipeline. Troubleshooting production issues and conducting root cause analysis. Participating in client requirements sessions, estimating work, and producing clear technical documentation. The role will support client solutions across multiple industries, embedded analytics, the DashProphet 2.0 product, Cruz Street’s Context Warehouse and QuickSight Hub offerings, AI and data science initiatives, and reusable data engineering accelerators. Requirements Required Qualifications At least five years of AWS data engineering experience. At least five years of Python data engineering experience. At least five years of experience designing and building advanced data solutions, including APIs, SFTP, AWS Glue, Azure Data Factory, data lakes, lakehouses, warehouses, and relational databases. Strong SQL skills, including analytical data modeling and query optimization. At least one year of experience planning, building, deploying, and maintaining agentic AI applications. Working knowledge of Git, CI/CD, and collaborative software development. Strong written and verbal communication skills. Comfort working directly with clients and participating in requirements discussions. Ability to work on a 1099 contract basis for approximately 20 hours per week. Ability to commute to Vero Beach, Florida for in-person collaboration. Preferred Qualifications Five or more years of Azure data engineering experience. AWS or Azure certifications in solutions architecture, DevOps, data engineering, or AI and machine learning. Experience with Bedrock, SageMaker, OpenSearch vector search, Azure OpenAI, Azure AI Search, RAG, embeddings, and vector databases. Experience building multi-tenant SaaS data platforms or embedded analytics solutions. Experience with Apache Iceberg, Lake Formation, open table formats, and data governance. Consulting or client-facing delivery experience. Experience working with regulated data, such as HIPAA-protected information, and applying data security best practices. Additional Information Compensation and Benefits Contract rate starting at $75 per hour, negotiable based on experience and fit. Approximately 20 expected hours per week at the start. Potential full-time conversion with a starting salary of $115,000 plus healthcare, paid time off, and a 401(k). Flexible schedule. Professional development assistance. Support for AWS certifications and training. Work Arrangement Hybrid remote position based in Vero Beach, Florida 32960. In-person collaboration at Cruz Street’s headquarters, with the remainder of work performed remotely and on a deliverable basis. Application Information Applicants should submit a resume and a short description of an end-to-end data pipeline they have built, including the data sources, services used, and outcome. Applicants should also provide AWS or Azure certification IDs or Credly links. Applicants may be asked whether they have built and deployed an agentic AI application in production and to briefly describe that application and an end-to-end data pipeline. About Cruz Street Cruz Street is a data engineering, analytics, business intelligence, AI, data science, and enterprise software company headquartered in Vero Beach, Florida. Founded in New York in 2018, the company is an AWS QuickSight Service Delivery Partner serving clients across industries including media and advertising, healthcare and life sciences, SaaS, financial services, retail, and professional organizations. Cruz Street is an equal opportunity employer and welcomes applicants of all backgrounds. The company does not discriminate on the basis of any protected characteristic. Read more

This is a remote Data Engineer position within the Data & AI practice, focused on building modern data platforms for clients using Microsoft Azure, Microsoft Fabric, and Azure Databricks. The role involves designing, developing, and maintaining scalable data infrastructure that enables reliable access to analytics-ready data and supports business intelligence, reporting, and AI applications. The successful candidate will work with architects, data scientists, analysts, engineers, and client stakeholders to ingest, clean, transform, govern, and deliver trusted data. Design, build, and deploy scalable batch and streaming data pipelines from diverse data sources. Ingest and transform structured, semi-structured, and unstructured data from databases, APIs, files, and event streams. Develop analytics-ready datasets using Microsoft Fabric, Azure Data Factory, Azure Databricks, ADLS Gen2, OneLake, and Delta Lake. Apply Medallion architecture patterns across Bronze, Silver, and Gold data layers. Build and consume APIs and reusable data services for reporting, applications, and downstream analytics. Implement data quality checks, validation rules, governance standards, security controls, and monitoring. Monitor, troubleshoot, and optimize data pipelines and systems for performance, reliability, accuracy, and security. Collaborate with data scientists, analysts, architects, engineers, and client stakeholders to translate data requirements into technical solutions. Document data flows, architectures, models, transformation logic, and technical processes. Participate in code reviews and agile delivery activities, including sprint planning, stand-ups, and estimation. Support data activation for operational systems and emerging Data Agent and Copilot-style use cases. Requirements 1–3 years of professional experience in data engineering or a related hands-on technical role; relevant internships and academic projects may be considered for entry-level candidates. Strong proficiency in Python and SQL, with experience writing production-quality code. Hands-on experience with PySpark and an understanding of distributed data processing. Experience or exposure to Microsoft Azure data technologies, including Azure Data Factory, Microsoft Fabric, ADLS Gen2, and/or Azure Databricks. Knowledge of Delta Lake and Unity Catalog is advantageous. Understanding of ETL/ELT processes and the end-to-end data lifecycle from ingestion through activation. Knowledge of dimensional, normalized, and star-schema data modeling and data warehousing principles. Experience building or consuming APIs for data integration. Familiarity with Git and basic CI/CD practices; Azure DevOps experience is a plus. Understanding of data security, access control, governance, Microsoft Purview, and role-based access principles. Strong analytical, troubleshooting, and problem-solving skills, with close attention to data quality and accuracy. Ability to communicate effectively with technical colleagues and client stakeholders. Relevant certifications, such as Microsoft Azure Data Engineer Associate, are desirable but not required. Additional Information This is a remote position. Read more

We are seeking an experienced Data Engineer to design, develop, and support modern data solutions across data engineering, integration, warehousing, analytics, business intelligence, data science, governance, data quality, and master data management. The role will involve working with Microsoft Azure, Informatica, Snowflake, and other leading data technologies to build scalable data pipelines, models, platforms, and analytics solutions. The successful candidate will collaborate with technical teams, business stakeholders, and senior leadership to translate complex data and technical concepts into clear, actionable insights. The Data Engineer will be expected to apply critical and creative thinking, identify trends and opportunities in complex data, recommend solutions, and support both short-term priorities and long-term business objectives. The role also requires a proactive, adaptable, strategic, and continuously improvement-focused approach. Requirements Bachelor’s degree in computer science, data science, software engineering, information systems, or a related quantitative field. At least 8 years of overall data engineering experience. Strong experience with data modeling, ETL/ELT, data integration, master data and entity management, data quality, data warehousing, analytics, business intelligence, data science, and data governance. Proficiency in programming and database languages such as C, Python, SQL, and Java. Experience with Microsoft technologies, including SQL Server, Synapse, Cosmos DB, HDInsight, Azure Data Factory, Azure Databricks, Power BI, and Microsoft Fabric. Experience with Informatica Cloud technologies, including CIH, DIH, CDGC, Master Data Management, and Data Quality. Experience with Snowflake and other leading industry data platforms. Excellent written and verbal communication skills, including the ability to communicate effectively with senior leadership and diverse technical and non-technical audiences. Ability to simplify and explain complex technical information in a clear, concise, and engaging manner. Strong critical-thinking, analytical, research, problem-solving, persuasion, and negotiation skills. Ability to recognize trends, evaluate business implications, make timely decisions, and advise key stakeholders. Proactive and innovative approach, with a focus on identifying solutions, improving processes, increasing efficiency, and contributing ideas for continuous improvement. Ability to manage multiple priorities, adapt to changing circumstances, and balance short-term needs with long-term objectives. Demonstrated respect for diverse perspectives, emotional intelligence, collaboration, humility, and professionalism. Ability to remain positive, resilient, composed, and constructive when handling conflict or challenging situations. Commitment to knowledge sharing, stakeholder engagement, and building a strong professional reputation. Additional Information Location: Remote role based in or near Chicago, Illinois; local candidates are highly preferred. Employment Type: W2 only. Work Authorization: Any authorization accepted, except H-1B and CPT. Engagement Restrictions: C2C, 1099, and third-party profiles are not accepted. Read more

Design, build, and continuously improve the client’s online platform. Research, recommend, and implement technology solutions in line with industry best practices and standards. Take ownership of the resilience, reliability, and availability of assigned products. Contribute effectively as a collaborative and productive member of the team. Requirements Senior Data Engineer with strong expertise in knowledge graph technologies and semantic data engineering. Knowledge Graphs and Knowledge Graph Architecture Neo4j and Cypher RDF, RDF triples, and triple stores SPARQL Ontology Engineering and OWL Semantic Modeling and Graph Data Modeling Semantic Web and Linked Data Employment Details and Benefits Fixed-term employment contract Work-life balance Flexible working arrangements What GeekSoft Consulting Offers International and collaborative consulting culture Clear career development opportunities Comprehensive Dutch benefits package focused on well-being, mobility, and career growth Read more

We are seeking a Data Engineer to design, develop, and support data engineering solutions across data modeling, ETL/ELT, data integration, master data and entity management, data quality, data warehousing, analytics, business intelligence, data science, and data governance. The role involves working with modern Microsoft Azure, Informatica, Snowflake, and other industry-leading data technologies. The successful candidate will translate complex technical information for technical and non-technical audiences, identify insights from complex data, advise decision-makers, and contribute to strategic business and client outcomes. The Data Engineer will demonstrate initiative, innovation, adaptability, sound judgment, and a commitment to continuous improvement. The role also requires effective collaboration with stakeholders, clear communication with senior leadership, and the ability to build trust through respectful, positive, and resilient interactions. Requirements Bachelor’s degree in computer science, data science, software engineering, information systems, or a related quantitative field. At least 8 years of overall data engineering experience. Experience with data modeling, ETL/ELT, data integration, master data management, entity management, data quality, data warehousing, predictive and prescriptive analytics, business intelligence, data science, and data governance. Proficiency in C, Python, SQL, and Java. Experience with Microsoft technologies, including SQL Server in IaaS and PaaS environments, Azure Synapse, Cosmos DB, HDInsight, Azure Data Factory, Azure Databricks, Power BI, and Microsoft Fabric. Experience with Informatica cloud technologies, including CIH, DIH, CDGC, Master Data Management, and Data Quality. Experience with Snowflake and other relevant industry-leading data platforms. Strong written and verbal communication skills, including the ability to communicate credibly and confidently with senior leadership. Ability to simplify and explain complex technical information to diverse technical and non-technical audiences. Strong critical-thinking, creative problem-solving, research, analytical, decision-making, persuasion, and negotiation skills. Ability to recognize trends in complex data and assess both short- and long-term business implications. Proactive, strategic, innovative, and solutions-oriented approach to work. Ability to manage multiple priorities, adapt to changing circumstances, and balance short- and long-term objectives. Demonstrated respect for diverse backgrounds, perspectives, and working styles. Positive, resilient, and professional approach to collaboration, conflict resolution, and challenging situations. Commitment to knowledge sharing, continuous improvement, stakeholder engagement, and developing a strong professional presence. Additional Information Location: Remote; Chicago, Illinois. Employment type: W2 only. Work authorization: Any authorization accepted; H1B and CPT candidates are not eligible. Location preference: Local Chicago-area candidates highly preferred. Recruiting restrictions: No C2C, 1099, or third-party profiles accepted. Read more

We are seeking a Senior Azure Data Engineer with Databricks expertise to design, develop, optimize, and maintain scalable cloud-based data solutions for analytics and enterprise data platforms. The role will involve developing and optimizing SQL Server and Azure SQL databases, writing advanced T-SQL queries and database objects, implementing indexing and performance-tuning strategies, and ensuring database security, integrity, and reliability. The successful candidate will work with Azure SQL Database, Databricks, Apache Spark, Delta Lake, Spark SQL, PySpark, and SQL to support large-scale data processing, transformations, analytics, and hybrid cloud data architectures. Responsibilities include designing relational and dimensional data models, implementing Slowly Changing Dimensions (SCD), supporting ETL/ELT processes, and collaborating with data engineering and business intelligence teams. The role also requires partnering with application developers, data engineers, BI teams, and business stakeholders; mentoring junior developers; participating in code reviews; enforcing database standards; and troubleshooting production issues through root cause analysis. Requirements Minimum 8 years of experience as a Database Developer, Data Engineer, or in a similar role. Strong expertise in SQL Server and Azure SQL. Advanced proficiency in T-SQL, including query optimization and performance tuning. Hands-on experience with Databricks and Apache Spark. Experience with Delta Lake, Spark SQL, PySpark, and SQL-based data processing. Strong understanding of data warehousing, dimensional modeling, and Slowly Changing Dimensions. Experience designing and supporting ETL/ELT processes. Experience working in Azure cloud environments and hybrid data architectures. Knowledge of database security, data integrity, reliability, indexing, and performance optimization. Strong analytical, problem-solving, collaboration, communication, and leadership skills. Education and Certifications Degree in Engineering, Computer Science, or a related discipline, such as BE, ME, BTech, MTech, BSc, or MSc. Technical certifications across relevant technologies are desirable. Location and Work Arrangement Chicago, Illinois. Hybrid work arrangement requiring three days per week in the office. Candidates must be able to relocate to Chicago and work from the first day. The client interview may be conducted virtually. Read more

The organization is seeking a contract Data Engineer to design, develop, optimize, and maintain data pipelines, analytics solutions, and dashboards on cloud-based data platforms. The role will support program needs by leveraging Azure Synapse Analytics, Azure Data Factory, Power BI, and integrations with Salesforce and/or Curam/Oracle. Key Responsibilities Analyze source systems and develop scalable ETL/ELT pipelines using Azure Synapse, Azure Data Factory, and related Azure services. Build and manage Salesforce data ingestion workflows, including API integrations, schema alignment, and performance tuning. Develop and maintain data models, transformations, and semantic layers for Power BI reporting and analytics. Optimize Synapse pipelines, SQL queries, and data storage for performance, reliability, and cost efficiency. Troubleshoot pipeline failures, Salesforce API issues, and Power BI dataset or dataflow refresh problems. Requirements At least 5 years of hands-on experience configuring, designing, developing, and testing Power BI dashboards. At least 5 years of experience supporting Power BI datasets, dataflows, and DirectQuery/Import models. At least 3 years of experience working with structured and semi-structured data formats, including JSON, CSV, and Parquet. At least 3 years of experience designing and optimizing scalable data pipelines using platforms such as Azure Synapse, Snowflake, or Databricks. At least 5 years of advanced SQL experience, including query writing, ETL/ELT, performance tuning, and optimization. At least 3 years of proficiency in Python or Scala for data processing. At least 5 years of hands-on experience developing efficient PL/SQL procedures. At least 2 years of experience with SQL, T-SQL, programming, scripting, and data query optimization. At least 2 years of experience with Azure cloud services, including Synapse, Data Lake, Data Factory, Azure SQL, and monitoring tools. At least 2 years of experience with Azure Data Lake Storage, Delta Lake, data modeling, and data warehousing concepts such as star and snowflake schemas. Technology Environment Key technologies include Azure Synapse Analytics, Azure Data Factory, Azure Data Lake Storage, Delta Lake, Azure SQL, Power BI, Salesforce, Curam/Oracle, Snowflake, Databricks, SQL, T-SQL, PL/SQL, Python, and Scala. Read more

We are seeking a Senior Data Engineer to lead a greenfield and future-state data platform initiative within a regulated healthcare and medical scheme administration environment. The successful candidate will own delivery end to end, from discovery and assessment through to implementation. The role requires an autonomous technical lead who can work across complex legacy data environments while supporting the transition to modern cloud-based data platforms. The primary focus will be on reducing operational workload and improving business efficiency through: Reporting rationalisation and standardisation Process and data quality automation Self-service analytics enablement Development of reusable data assets Improved data governance, metadata and cataloguing Identification and delivery of measurable operational efficiencies Key responsibilities include: Assessing and documenting the current reporting and data landscape Identifying manual, repetitive and inefficient processes Quantifying workload and prioritising initiatives based on business value Reviewing, consolidating and standardising reporting assets Defining consistent data and metric definitions Designing and implementing robust ETL and data engineering solutions Automating validation, reconciliation, quality assurance and audit controls Developing reusable datasets and governed Power BI assets Establishing data catalogue and metadata standards Exploring Generative AI and LLM applications for data and report discovery Engaging with business and technical stakeholders to shape requirements and delivery plans Tracking KPIs related to reduced manual effort, released capacity and operational improvement Owning planning, prioritisation and delivery across assigned workstreams Requirements The ideal candidate will have strong hands-on experience in the following areas: Oracle Database Advanced SQL development and optimisation SAP BusinessObjects SAS Power BI and self-service analytics ETL and data engineering Data profiling, quality assessment and automation Data modelling Snowflake and AWS Metadata management and data cataloguing Generative AI and LLM applications The successful candidate should also: Be an experienced Senior Data Engineer with strong technical depth Be able to work independently with minimal supervision Have experience managing delivery from discovery through implementation Be comfortable working with complex, legacy or undocumented data environments Demonstrate strong data engineering, automation, data quality and governance skills Have the ability to quickly understand complex data estates and identify improvement opportunities Be commercially minded and focused on reducing operational effort Be confident engaging with senior business and technical stakeholders Be comfortable working with sensitive healthcare data in a regulated, POPIA-conscious environment Be outcome-focused, with the ability to measure delivery through efficiency gains and released capacity Contract and Location Position: Senior Data Engineer Contract: Up to 18 months, structured in six-month phases Start date: 1 October 2026 Work model: Hybrid Location: Waterfall, Midrand, Johannesburg Ideal Candidate Mindset This role is suited to a self-directed professional who can enter a complex environment, understand the current state, identify the highest-value opportunities, develop a delivery plan and execute against measurable outcomes. The successful candidate will focus not only on building data solutions, but also on using data engineering, automation and modern analytics to simplify the reporting estate, reduce manual work and create sustainable business capacity. Read more

We are seeking a Senior Data Engineer to lead a greenfield/future-state data platform initiative within a regulated healthcare and medical scheme administration environment. The role involves end-to-end ownership of delivery, from discovery and current-state assessment through to implementation. The successful candidate will work autonomously across a complex legacy data estate while supporting the transition to modern cloud-based data platforms. The primary objective is to reduce operational workload and improve business efficiency through reporting rationalisation, process automation, self-service analytics, data quality automation, reusable data assets, and improved governance. Key Responsibilities Assess and baseline the current reporting and data environment. Analyse complex, undocumented, and legacy data estates. Identify manual, repetitive, and inefficient processes, and quantify operational effort. Prioritise initiatives based on measurable business value and effort reduction. Rationalise, consolidate, standardise, and retire redundant reports. Establish consistent data and metric definitions. Design and implement robust ETL and data engineering solutions. Automate manual processes, data validation, reconciliation, and quality controls. Implement appropriate audit trails and operational controls. Develop reusable datasets and governed Power BI data products. Enable appropriate self-service analytics while maintaining governance standards. Establish and maintain a centralised reporting and data catalogue. Define metadata standards and improve the discoverability of data and reporting assets. Explore the use of Generative AI and LLMs for natural-language data and report discovery. Engage with business and technical stakeholders to shape requirements, scope, and delivery decisions. Communicate progress, risks, outcomes, and recommendations effectively. Define and track KPIs relating to reduced manual effort, released capacity, and operational improvements. Own planning, prioritisation, and delivery across assigned workstreams. Success Measures Success will be measured by business outcomes, operational efficiency, reduced manual workload, improved data reliability, and released organisational capacity rather than utilisation alone. Requirements Technical Requirements Strong hands-on experience with Oracle Database. Advanced SQL development and optimisation skills. Experience with SAP BusinessObjects and SAS. Experience with Power BI and governed self-service analytics. Strong knowledge of ETL, data engineering, and process automation. Experience with data profiling, quality assessment, validation, and reconciliation. Experience with data modelling and reusable data assets. Experience with Snowflake and AWS cloud technologies. Knowledge of metadata management and data cataloguing. Awareness of Generative AI and LLM applications in data and analytics. Experience and Capabilities Significant experience as a Senior Data Engineer or equivalent technical lead. Ability to work independently with minimal supervision. Proven experience owning delivery from discovery through implementation. Ability to quickly understand complex, legacy, and undocumented data environments. Strong commercial and analytical mindset, with a focus on reducing operational effort. Ability to identify opportunities for automation, simplification, and efficiency improvement. Strong stakeholder engagement and communication skills. Ability to work effectively with senior business and technical stakeholders. Experience working in regulated environments and handling POPIA-sensitive healthcare data. Ideal Candidate Mindset The ideal candidate is self-sufficient, proactive, and comfortable taking ownership without detailed direction. They should be able to assess a complex environment, identify the highest-value opportunities, develop a practical delivery plan, and execute against measurable outcomes. The focus is on using data engineering, automation, governance, and modern analytics to simplify reporting, reduce manual work, and create sustainable business capacity. Contract and Location Position: Senior Data Engineer Contract: Up to 18 months, structured in six-month phases Start date: 1 October 2026 Work model: Hybrid Location: Waterfall, Midrand, Johannesburg Read more

Help design, build, and continuously improve the client’s online platform and data solutions. Research, recommend, and implement technology solutions in line with industry best practices and standards. Design and maintain reliable, scalable data pipelines and processing solutions. Contribute to the resilience, availability, performance, and security of products and data platforms. Collaborate with cross-functional teams to understand data requirements and deliver effective solutions. Monitor, troubleshoot, and optimize data pipelines to ensure performance and reliability. Contribute actively and productively as a member of the team. Requirements Experience designing, developing, and maintaining complex data pipelines using Azure Data Factory and Azure Databricks. Proficiency in Azure Databricks and PySpark for large-scale data processing. Strong Python programming skills, including experience developing ETL and data transformation solutions. Experience implementing CI/CD pipelines with Azure DevOps for automated testing, integration, and deployment. Familiarity with SQL for querying and managing large datasets. Knowledge of data security, compliance, and cloud data engineering best practices. Strong problem-solving skills and the ability to work independently or collaboratively. Excellent communication and documentation skills. Willingness to stay current with trends and developments in Azure services and cloud data engineering. Benefits Fixed-term employment contract. Work-life balance. Flexible working arrangements. What GeekSoft Consulting Offers An international and collaborative consulting culture. Clear career development opportunities. A comprehensive Dutch benefits package focused on well-being, mobility, and career growth. Read more

DataSeers is seeking an experienced ETL Developer or Data Engineer with strong HPCC Systems and Enterprise Control Language (ECL) expertise to build, maintain, and optimize large-scale financial data pipelines. The role focuses on ingesting, normalizing, transforming, linking, enriching, and analyzing complex data from banks, credit unions, fintechs, payment processors, core banking systems, and other financial platforms. You will develop ECL-based workflows that convert varied source data into standardized, reliable datasets for use across DataSeers products. Develop, maintain, and optimize ETL workflows using HPCC Systems and ECL. Build datasets, records, transforms, joins, rollups, deduplication processes, and validation logic. Map client source formats to DataSeers canonical data structures. Normalize customer, account, business, signer, beneficial-owner, transaction, and counterparty data. Develop entity-linking, relationship, matching, and large-scale data-processing logic. Ingest structured and semi-structured financial data from multiple sources. Investigate data-quality problems, including duplicate records, incorrect joins, missing relationships, malformed files, and inconsistent identifiers. Build reusable ECL components and optimize HPCC jobs for performance and scalability. Develop data validation, reconciliation, and completeness checks. Support batch and near-real-time processing using technologies such as MySQL, Elasticsearch, Kafka, APIs, and related ecosystem components. Collaborate with customer implementation teams to understand source data and integration requirements. Participate in code reviews, testing, deployment validation, and production troubleshooting. Requirements At least 3 years of experience in ETL, data engineering, or large-scale data processing. Hands-on experience with HPCC Systems and strong ECL development skills. Knowledge of ECL datasets, records, transforms, joins, normalize, denormalize, rollups, deduplication, and distributed processing. Strong SQL skills and an understanding of data structures, relationships, and data modeling. Experience troubleshooting complex joins, data relationships, and large datasets. Understanding of data quality, reconciliation, referential integrity, and exception handling. Experience using Git or an equivalent source-control system. Strong analytical, debugging, and problem-solving skills. Preferred Qualifications Experience in banking, fintech, payments, or financial services. Experience with ACH, wires, cards, RTP, FedNow, checks, or other payment data. Experience with HPCC Thor and Roxie, including large-scale ECL job optimization. Experience with Elasticsearch, MySQL, Kafka, or event-driven processing. Knowledge of AML, fraud, KYC/KYB, or transaction monitoring. Experience with Python, JavaScript, Node.js, or Java. These skills are beneficial but secondary to HPCC and ECL expertise. Success in the Role Within the first several months, the successful candidate should be able to understand new client datasets and their relationships, develop or modify HPCC and ECL ingestion and transformation workflows, identify data-quality issues, correctly link related entities, and deliver validated datasets for downstream DataSeers applications. Read more

Software Resources is seeking a Senior Data Engineer for a major corporation in Glendale, California. This role supports enterprise data engineering and data platform modernization initiatives, with responsibility for designing, building, optimizing, and operating scalable data solutions. The Senior Data Engineer will own and support the Core Data platform on Databricks, developing reliable batch and streaming Spark pipelines, implementing platform governance, and providing solution architecture across AWS, Kubernetes, and Airflow. The role involves close collaboration with engineers, architects, product managers, and business stakeholders. Manage Databricks governance, including Unity Catalog, access controls, lineage, data discovery, and privacy tools. Design, develop, test, deploy, and maintain batch and streaming data pipelines using PySpark, Scala, SQL, Python, and/or Kotlin. Gather stakeholder requirements and translate them into scalable data platform solutions. Use Databricks APIs, SDKs, and CLI tools to automate updates, audit platform activity, diagnose errors, and troubleshoot integrations. Explain Spark architecture and pipeline behavior, identify root causes, and recommend solutions. Provide solution architecture across AWS, Databricks, Kubernetes, and Airflow/MWAA. Build and maintain Kubernetes containers and containerized utilities supporting data platform services. Troubleshoot networking, connectivity, and cross-platform integration issues. Administer the data platform by managing access, monitoring resource utilization and platform health, controlling costs, and evaluating stakeholder requests. Implement infrastructure automation, CI/CD practices, and deployment standards using Terraform and Git-based workflows. Develop data quality checks, monitoring, logging, and documentation to support pipeline reliability and governance. Collaborate with cross-functional engineering and product teams and participate in Agile/Scrum ceremonies. Optimize data processing environments and scalable ETL/ELT pipelines integrating structured and unstructured data sources. Requirements At least 5 years of experience developing and operating large-scale data pipelines and data engineering solutions. Strong hands-on experience with Databricks and Apache Spark, including batch and streaming pipelines. Advanced Python development skills and proficiency in at least one additional language or tool, such as Scala, Kotlin, or SQL-based pipeline tooling. Experience with Terraform and Infrastructure as Code. Strong understanding of Spark architecture, including executors, stages, partitioning, shuffles, and performance tuning. Proficiency with Databricks APIs, SDKs, and CLI tools for automation, governance, auditing, and troubleshooting. Advanced SQL skills, including query and performance tuning. Production experience with Airflow, preferably AWS Managed Workflows for Apache Airflow (MWAA). Experience managing Databricks governance, including Unity Catalog, ACLs, lineage, and access provisioning. Experience designing and optimizing scalable ETL/ELT pipelines. Experience with AWS services, including compute, storage, networking, and IAM. Proficiency with Docker and Kubernetes for containerized data platform services. Working knowledge of networking concepts and cross-platform connectivity troubleshooting. Familiarity with Snowflake or comparable data platform technologies. Experience implementing CI/CD, DevOps, data quality, monitoring, and logging practices. Ability to participate in architecture and system design discussions. Strong analytical, problem-solving, communication, and stakeholder-management skills. Familiarity with Agile and Scrum methodologies. Bachelor’s degree in Computer Science, Information Systems, or a related field, or equivalent work experience. A master’s degree is a plus. Additional Information Location: Glendale, CA Work Arrangement: Hybrid; two days onsite per week, with candidates able to select their onsite days. Duration: 12+ months Pay Rate: $90–$93 per hour, depending on experience. Required Technologies: Databricks, Python, and Terraform. Preferred Technology: AWS. About Software Resources Software Resources is a staffing partner with more than 30 years of experience specializing in technology, creative and marketing, finance, and accounting placements. The company connects skilled professionals with organizations nationwide. Available benefits may include medical, dental, and vision coverage; a 401(k) with company match; short-term disability; and life insurance with AD&D coverage. Read more

Client: Fountain Forward Location: Bogota, DC, co Contract: Freelance object Object] Read more

Client: Fountain Forward Location: Lima, Peru, IS, pk Contract: Freelance object Object] Read more

The VISION – CWIS FACS Conversion Initiative requires the extraction of data from the Google Cloud Platform (GCP) BigQuery environment and the delivery of that data into enterprise SQL data warehouses. This effort is critical to ensuring continuity and modernization for all applications that currently depend on FACS. A dedicated ETL development team is being formed from the existing resource pool, and we are seeking a highly skilled Lead ETL Developer to provide leadership, training, architecture guidance, and technical oversight throughout this conversion effort. The Lead ETL Developer will design, build, test, and deploy data ingestion and migration pipelines that source historical and real-time data from a variety of disparate systems, including flat files and relational databases. The developer will transform and standardize incoming data into enterprise data models and ensure efficient, accurate loading into data warehouses, data lakes, and analytical databases. This role is responsible for ensuring that transactional and analytical data models include required data elements and meet quality, reliability, and performance expectations. The Lead ETL Developer will mentor and guide a team of ETL developers, establishing best practices and consistent development patterns throughout the project lifecycle. Responsibilities Lead development of ETL and data ingestion processes supporting the migration from GCP BigQuery to SQL-based data warehouse environments. Architect and implement pipelines using modern ETL tools and cloud-native data processing frameworks. Train and mentor a team of 4+ ETL developers on best practices, coding standards, and data model design. Collaborate with business and technical stakeholders to validate data requirements and ensure alignment with reporting and analytical needs. Develop robust testing strategies to ensure data integrity, completeness, and accuracy throughout the migration process. Optimize pipelines for scalability, performance, and maintainability. Validate transactional and analytical data models to ensure they contain the necessary elements for downstream applications and BI solutions. Prepare and deliver clear communication, presentations, and technical documentation suitable for both technical and non-technical audiences. Requirements Minimum 7 years of hands-on ETL development experience. Expertise with modern ETL and reporting/BI tools. Strong understanding of enterprise data models and best practices in ETL pipeline design. Proven experience as a Technical Lead guiding teams of 4 or more developers. Demonstrated experience with: Google Cloud Dataform BigQuery Medallion Architecture Exceptional communication and presentation skills, capable of engaging diverse audiences. Read more

Mid-Level ETL / Data Integration Developer - Informatica Remote / Hybrid Secret Clearance Required _If you are passionate about data integration, ETL development, and mission-critical technology and want to help modernize transportation and logistics systems supporting the Department of Defense, we would love to talk._ D&G Support Services is seeking a Mid-Level ETL / Data Integration Developer to support IGC software modernization, data integration, and cloud-migration activities. In this role, you will develop and sustain Informatica-based ETL workflows, complex SQL, source-to-target mappings, data-quality controls, and enterprise interfaces supporting global transportation and logistics operations. This position offers the opportunity to deliver hands-on ETL and data-integration solutions within an Agile team while supporting secure cloud modernization and release activities. Requirements What We Are Looking For _Required_ Four or more years of professional experience in ETL development, data integration, database development, software development, or a related technical field. Hands-on experience with Informatica PowerCenter or another Informatica enterprise data-integration platform, including mappings, transformations, workflows, sessions, parameter files, and reusable components. Strong SQL skills, including complex joins, subqueries, views, stored procedures, functions, indexing, query optimization, and troubleshooting. Experience developing source-to-target mappings and implementing data-transformation and business rules. Experience with relational databases such as Oracle, PostgreSQL, MySQL, Amazon Aurora/RDS, SQL Server, or comparable technologies. Experience with data validation, reconciliation, defect resolution, data-quality controls, and ETL performance optimization. Experience using Git and working within established GitLab-based development and deployment processes. Experience developing or consuming RESTful APIs, web services, files, or enterprise-system interfaces. Working knowledge of Agile development, automated testing, secure software delivery, DevSecOps, and CI/CD practices. Bachelor's degree in Computer Science, Cybersecurity, Data Science, Software Engineering, Information Technology, Information Systems, or a related software-engineering discipline; or one current certification from the following: Certified Cloud Security Professional (CCSP), CyberSec First Responder (CFR), Federal IT Security Professional–Manager (FITSP-M), GIAC Security Essentials Certification (GSEC), CompTIA Security+, or Systems Security Certified Practitioner (SSCP). Ability to satisfy applicable DoD workforce qualification and certification requirements associated with assigned duties and system privileges. _Preferred_ Experience with Oracle and Oracle PL/SQL in enterprise data-processing environments. Experience with Informatica Intelligent Cloud Services (IICS) or Informatica Cloud Data Integration. Experience modernizing legacy ETL workflows or migrating data workloads to AWS-based environments. Familiarity with Amazon Aurora PostgreSQL or Aurora MySQL, Amazon RDS, and other approved AWS data services. Familiarity with Apache Superset or another business-intelligence platform from a data-preparation and integration perspective. Familiarity with containerized environments, infrastructure-as-code, and automated cloud deployment practices. Experience supporting DoD, USTRANSCOM, transportation, logistics, or other mission-critical data environments. Clearance and Eligibility Active Secret security clearance or the ability to obtain and maintain one, as required by the contract. This role is restricted to U.S. person(s) (U.S. citizens, permanent residents, and other protected individuals under 8 U.S.C. 1324b(a)(3)). Location Remote / Hybrid Work may be performed remotely or at a contractor facility when authorized. Personnel must be available for required on-site meetings, testing, demonstrations, release activities, classified access, or other assigned mission needs at Scott Air Force Base, Illinois. Company Information At D&G Solutions, people come first; our teammates, customers, and the communities we serve. We are one of the fastest growing companies in the Washington, DC area and proud to be recognized nationally as a Best Place to Work. As an 8(a), Service Disabled Veteran Owned, and Women Owned Small Business, we bring ownership, accountability, and care to everything we do. We also deeply believe in giving back and proudly support organizations such as Homes for Our Troops, SemperK9, and Wreaths Across America. Our teams are welcoming, collaborative, and deeply committed to mission success. We believe great work happens when people feel supported, respected, and trusted, and we work hard to create that environment every day. What We Offer Competitive salary Generous paid time off Benefits 401(k) 401(k) matching Employee assistance program Flexible spending account Health insurance Health savings account Life insurance Paid time off Parental leave Professional development assistance Referral program Retirement plan Vision insurance Final Note If you want to work for a company that cares about its people, contribute to a mission that truly matters, and be part of a team where you are valued for who you are as well as what you do, we would love to hear from you. Read more

Oats Overnight is on a mission to build a generational CPG brand by leveraging speed, storytelling, and strategy. We are looking for a Senior Data Engineer to join our Data team as our first dedicated Data Engineering hire. This role involves building and evolving the data platform that powers analytics, AI, internal tooling, and operational workflows. You will work closely with the Director of Analytics to set the technical direction for Data and ensure high-quality data is available throughout the organization. Requirements Experience: 5+ years of experience in Data Engineering, Analytics Engineering, Software Engineering with a data focus, or a related technical role. Strong proficiency in Python and SQL, with experience in writing production-quality code and building ETL/ELT pipelines. Experience with modern data tooling, including Airflow and/or dbt, and a cloud data warehouse such as Snowflake, BigQuery, or Redshift. Experience building data pipelines and integrations across APIs, databases, and third-party systems. Strong understanding of data modeling and warehouse design. Skills: Experience using coding agents and AI-assisted development tools. Ability to handle multiple priorities and navigate in a fast-moving environment. A relentless drive to get things done. Flexible and quick learner of new processes and technologies. Strong interpersonal skills to partner and influence at different levels. Nice to Have: Experience as an early data hire or building significant parts of a data platform from the ground up. Deep production experience with Snowflake, dbt, Airflow, Looker, and/or Hex. Experience building AI/LLM data infrastructure, internal APIs, or backend services. Experience with business analytics in various domains like marketing, finance, or operations. Compensation & Work Authorization The base salary range for this position is $160,000-$180,000 USD, depending on experience, qualifications, and location. Candidates must be authorized to work in the United States without current or future employer-sponsored visa sponsorship. Why You Might Like Working Here This is going to be fun. Casual, friendly culture with serious work-happiness balance. Every employee has equity in the company. High growth potential with opportunities to expand. Free oatmeal. Why You Might Not Like Working Here High expectations and performance bar. Fast-paced working environment due to rapid growth. Core Values Ownership: We are all owners and show accountability in everything we do. Rational: We focus on making high-quality decisions consistently. Iterative: We constantly improve and adapt when situations change. Curiosity: We question and learn continuously. Physical Demands Prolonged periods of sitting at a desk and working on a computer. Must be able to perform essential job functions consistent with ADA and other standards. Maintain regular, punctual attendance consistent with ADA and other standards. The Perks Full Benefits; Medical, Dental, Vision, and 401k. Parental Leave. 4 Weeks of paid time off. Weekly Pay. Stock options available for every employee. Free, delicious oatmeal. Diversity and Equal Opportunity Oats Overnight embraces diversity and encourages equal opportunity for all applicants and employees, regardless of race, color, religion, gender, sexual orientation, national origin, age, disability, marital status, or any other status. We comply with applicable state and local laws governing nondiscrimination. Read more

Project Name: Data Management Platform Integration Services The Talend Integration Platform Analyst provides development, operational support, and continuous improvement services for the Data Management Platform's enterprise integration environment. The role is responsible for designing, developing, monitoring, troubleshooting, maintaining, and enhancing Talend-based integrations that support critical business applications, data exchanges, analytics, and reporting solutions. The analyst works closely with business stakeholders, data engineers, solution architects, platform administrators, and application teams to ensure reliable, secure, and scalable integration services. The position supports ongoing platform operations while contributing new integration development and modernization initiatives across the Government of Alberta Data Management Platform. This is a hands-on technical role for someone who combines strong Talend and SQL development skills with a disciplined, service-oriented approach to platform operations including job scheduling, monitoring, error handling, and incident resolution; while collaborating with data engineers, platform administrators, ministry stakeholders, and vendor support partners. Requirements Demonstrated hands-on experience developing data integration solutions with Talend (Data Integration / Data Fabric / Talend Studio). Design, develop, test, and deploy Talend integration jobs, services, APIs, and reusable components. Translate business and data requirements into technical designs, data mappings, transformation logic, and reconciliation processes. Develop ETL/ELT solutions to integrate data across databases, files, APIs, cloud platforms, and enterprise applications. Build reusable, parameterized, and environment-aware integrations for development, test, and production environments. Implement scalable batch and near real-time integration solutions. Participate in solution design, estimation, and release planning activities. Deploy, schedule, orchestrate, and monitor Talend jobs, services, and platform components. Investigate and resolve integration issues, production incidents, data load failures, and performance bottlenecks. Perform root cause analysis and implement corrective actions to prevent recurring issues. Monitor platform health, job execution, throughput, and failures and implement and maintain logging, monitoring, alerting, and automated recovery processes. Support production releases, platform upgrades, patching, and environment maintenance. Work with infrastructure teams, vendors, and business stakeholders to resolve platform and integration issues. Utilize AI and automation tools to streamline data engineering workflows, including pipeline development, testing, monitoring, and documentation. Leverage AI-assisted tools for code generation, optimization, and review to improve development efficiency and code quality. Other duties as needed. Additional Information Duration: 2026-10-12 to 2027-03-31 (6 Months possible extension) Work Location: 9942 - 108 ST, Edmonton, Alberta, CAN, T5K 2J5 Read more

This is a remote position available to U.S. citizens residing in the United States. Our client, a growing technology services organization supporting federal government programs, is seeking an experienced Databricks Data Engineer to support a federal technology initiative. Candidates must currently hold an ACTIVE Secret security clearance or higher to be considered for this role. The ideal candidate will bring strong hands-on data engineering experience within Databricks, including the development of production-grade batch and streaming pipelines, PySpark and SQL transformations, data modeling, ingestion frameworks, and modern data architecture practices. Responsibilities Design, build, and maintain batch and streaming data pipelines using PySpark, SQL, Databricks Workflows, and Delta Live Tables. Implement Medallion Architecture across bronze, silver, and gold data layers to support data quality, transformation, and consumption. Develop scalable ingestion frameworks for structured, semi-structured, and unstructured data. Integrate data from files, databases, APIs, and streaming sources such as Kafka, Kinesis, and Databricks Auto Loader. Design dimensional and domain-specific data models supporting analytics and downstream applications. Build and consume APIs for integration with downstream systems. Optimize Spark workloads for performance and cost through partitioning, caching, cluster sizing, and related techniques. Develop and implement data quality checks, validation processes, and pipeline monitoring. Maintain documentation covering data flows, lineage, architecture, and integration points. Collaborate with engineering, platform, architecture, and federal program stakeholders throughout the development lifecycle. Requirements Active Secret security clearance or higher is required. 5+ years of professional data engineering experience. 2+ years of hands-on Databricks experience. Strong production-level experience with PySpark and SQL. Experience developing both batch and streaming data pipelines. Strong understanding of data modeling and modern data architecture principles. Proficiency with Python. Experience working within Git-based development and CI/CD environments. Familiarity with Databricks Unity Catalog, including catalogs, schemas, and permissions from a data engineering perspective. Experience integrating data from multiple source types, including databases, APIs, files, and streaming platforms. Preferred Qualifications Experience with Databricks Delta Live Tables. Experience with Databricks Workflows. Experience with Kafka, Kinesis, or similar streaming technologies. Experience working within federal, regulated, or security-sensitive environments. Bachelor's degree in Computer Science, Engineering, Information Technology, or a related discipline. Additional Information Security Clearance: Active Secret or higher REQUIRED Read more

We’re looking for an Expert Data Engineer to join a high-performing global technology environment and play a key role in building secure, scalable, and intelligent data integration solutions. You will be responsible for designing pipelines, platforms, and integrations that make critical enterprise data available, trusted, discoverable, and ready to be consumed in real time. This role requires excitement about technologies like Kafka, AWS, Python, Java, and real-time streaming, while also tackling complex data challenges. Requirements Essential Skills: Hands-on experience with Kafka and event streaming platforms for real-time data movement. Proven experience with API integration patterns, webhooks, and event/webhook ingestion. Strong proficiency in Python for data engineering, ingestion pipelines, and automation. Strong proficiency in Java for stream processing or connector development. Solid competence with enterprise databases and query languages, including performance tuning and query optimization for OLTP/operational workloads. Experience with NoSQL/document stores such as Amazon DynamoDB, MongoDB. Experience in data modeling to design schemas and standardized data representations. Experience with schema registries and contract-first designs (Avro, Protobuf) to manage producer/consumer compatibility. Strong understanding and practice of data quality techniques and tooling to ensure trusted data. Knowledge of metadata management and cataloging to support discoverability and lineage. Familiarity with ETL/ELT patterns and best practices for performant, reliable data pipelines. Experience with observability for streaming, including metrics, tracing, and logging on AWS (CloudWatch, OpenTelemetry, Prometheus/Grafana). Advantageous Skills: Awareness of frontend frameworks (e.g., Angular) to better understand downstream consumers. Experience operating container platforms and orchestration (Kubernetes/EKS) for scalable stream processing on AWS. Familiarity with enterprise systems like SAP and working with their integration interfaces. Experience with big data ecosystems (e.g., EMR, S3, Hadoop) and distributed storage/processing on AWS. Working knowledge of AWS analytics/data platform services (Glue, Athena, Kinesis, Redshift, Lake Formation, MSK). Knowledge of message delivery semantics, partitioning strategies, and capacity planning for high-throughput pipelines on AWS. Qualifications/Experience: Extensive hands-on experience (typically 6+ years) in data engineering, integration, or streaming roles with demonstrable production experience. Proven track record building and operating streaming platforms (Kafka/MSK) and API-based integrations, with strong Python and Java skills and experience with enterprise databases and query languages. Strong analytical thinking, curiosity about data, attention to detail, structured problem-solving, and ownership — able to drive topics to completion. Preferred certifications: Confluent Certified Developer for Apache Kafka, Microsoft Streaming Eventhubs, AWS Certified Data Engineer. Benefits Cutting edge global IT system landscape and processes. Flexible working of 1960 hours in a 12-month period. High work-life balance. Remote/on-site work location flexibility. Highly motivating, energetic, and fast-paced working environment. Modern, state-of-the-art offices. Dynamic global team collaboration. Application of the Agile Working Model methodology. Read more

We are looking for an experienced and hands-on Data Engineer to design, develop, and maintain scalable data pipelines and analytics solutions. The successful candidate will have strong programming capabilities in Python, advanced SQL expertise, and practical experience working with AWS-based data engineering services. The role involves integrating data from multiple structured and unstructured sources, transforming and organizing large datasets, implementing data-quality controls, and preparing reliable data layers for analytics, reporting, and business intelligence solutions. Key Responsibilities · Design, develop, test, and maintain scalable batch and event-driven data pipelines. · Build end-to-end ETL/ELT workflows using Python, SQL, and AWS data services. · Develop reusable Python components for data ingestion, transformation, validation, reconciliation, and automation. · Write and optimize complex SQL queries, stored procedures, views, functions, and data-transformation logic. · Integrate data from databases, APIs, enterprise applications, flat files, and cloud-based file systems. · Work with file-based datasets in formats such as CSV, JSON, XML, Parquet, and Excel. · Query and analyze data stored in databases. · Work with relational databases hosted on Amazon RDS, Azure Synapse, including Microsoft SQL Server or PostgreSQL. · Implement incremental data loads, change-data-capture patterns, error-handling mechanisms, restartability, and audit controls. · Perform data validation, reconciliation, profiling, cleansing, and quality monitoring. · Apply appropriate security controls, including IAM-based access, encryption, logging, and secure handling of sensitive data. · Collaborate with Power BI or Tableau developers to deliver analytics-ready datasets and semantic models. Requirements Experience: 6-8 Years Work Mode: Hybrid Type: Contract to Hire Notice Period: Immediate Joiners Mandatory Technical Skills Programming · Strong hands-on programming experience in Python. · Good understanding of object-oriented programming, modular development, exception handling, logging, and reusable coding practices. · Prior exposure to another programming language, preferably Java, is desirable. · Experience using Python libraries for data processing and database integration, such as Pandas, PySpark, Boto3, SQLAlchemy, or equivalent libraries. SQL and Databases · Strong proficiency in writing and optimizing advanced SQL queries. · Sound knowledge of relational database concepts, normalization, denormalization, indexing, and database performance. · Hands-on experience with Complex joins and subqueries, Window and analytical functions, Stored procedures, functions, and views, Query-performance optimization & Dimensional and relational data modelling. AWS Data Engineering · Hands-on experience with several of the following services: Amazon S3 AWS Glue AWS Lambda AWS Step Functions Amazon Redshift Amazon RDS AWS IAM Preferred Skills · Hands-on experience or working knowledge of Microsoft Azure data services, including: Azure Data Factory Azure Synapse Analytics Microsoft Fabric Azure Data Lake Storage Azure Functions · Exposure to business intelligence and reporting tools, particularly Microsoft Power BI or Tableau. Additional Information Do you love a career where you Experience, Grow & Contribute at the same time, while earning at least 10% above the market? If so, we are excited to have bumped onto you. Learn how we are redefining the meaning of work, and be a part of the team raved by Clients, Job-seekers, and Employees. Read more

Dhaka Technologies Limited Company (DTLC) is hiring an ETL Data Engineer on behalf of the District of Columbia Office of the Chief Technology Officer (OCTO). The Enterprise Data team at OCTO requires an ETL data engineer to support data operations for its Cloud Data Exchange. The engineer uses native Azure tools to perform ETL, data loading, and data transformation tasks, and supports the Enterprise Data team with data curation, processing, and transformation. This is a senior technical position that requires subject matter expertise, a proven track record of hands-on design and code work within large and complex systems, and the ability to convey technical and functional concepts to executive-level business users and technical experts. Requirements 15+ years developing extract, transform, and load (ETL) processes, including end-to-end pipelines with data loading from multiple sources 15+ years gathering and documenting requirements for data extraction, transformation, and load processes 15+ years of understanding of data warehousing, data lake, business intelligence, and information management concepts and standards 15+ years advising internal and external customers on appropriate tools and systems for complex data processing challenges 11+ years of knowledge and use of SQL for relational databases 11+ years with various data formats, including database specific (Oracle, SQL, Postgres, DB2), text formats (CSV, XML), and binary formats (Parquet, AVRO) 7+ years contributing to enterprise data governance standards by ensuring accuracy, consistency, security, and reliability 5+ years with Microsoft Azure tools such as Azure Data Factory, Synapse, SQL Database, Data Lake Storage Gen2, and Blob Storage 5+ years with data integration and data pipeline tools such as Informatica PowerCenter, Apache NiFi, Apache Airflow, and FME 3+ years of strong oral and written communication skills 3+ years providing excellent customer service to external partners 3+ years working independently or as part of a larger team Highly Desired 3+ years with visualization and reporting software such as MicroStrategy, Tableau, and Esri ArcGIS 3+ years performing data functions with Databricks Education Bachelor's degree in Information Technology or a related field, or equivalent experience Work Requirements This contract is open to W2 or 1099 candidates only. Corp-to-Corp (C2C) candidates will not be considered. The final interview will be conducted in person. Read more

Dhaka Technologies Limited Company is hiring on behalf of our client for a Senior Data Engineer / Data Architect with deep Databricks and insurance domain expertise. This is an onsite contract engagement in Charlotte, NC, and requires a minimum of 12 years of professional experience in data engineering and data architecture. The selected consultant will design, develop, and implement scalable enterprise data solutions on Databricks, working closely with business stakeholders, data architects, engineers, analysts, and technology teams to build modern data platforms and analytics capabilities. Key Responsibilities Design and develop scalable data architecture and data engineering solutions using Databricks. Build and maintain robust ETL and ELT data pipelines using Databricks, Apache Spark, Python, and SQL. Design data lakes, lakehouse architectures, data warehouses, and enterprise data platforms. Develop high performance batch and streaming data pipelines. Implement data ingestion from multiple internal and external sources. Develop data transformation, cleansing, validation, and integration processes. Design scalable and reusable data models for analytics and reporting. Work with Delta Lake, Delta Live Tables, Unity Catalog, and Databricks Workflows where applicable. Optimize Spark jobs, SQL queries, pipelines, and data processing workloads. Implement data quality, data governance, security, lineage, and access control processes. Collaborate with data scientists, BI teams, business analysts, product owners, and application teams. Translate business requirements into technical data architecture and engineering solutions. Participate in architecture reviews and establish data engineering best practices. Troubleshoot production data issues and provide root cause analysis. Mentor junior and mid-level data engineers. Support cloud migration and modernization initiatives. Requirements Insurance Domain Experience Insurance domain experience is mandatory. A strong working knowledge of Property and Casualty (P&C) insurance data is highly preferred, and experience in Life, Health, or other insurance lines will also be considered. The consultant should have hands-on experience with insurance data domains such as policy, policyholder, customer, claims, premium, billing, underwriting, rating, coverage, loss, agent and broker, payments, risk, product, quote, and policy administration. Experience integrating data from policy administration, claims, billing, underwriting, and other core insurance applications is a strong plus. Required Technical Skills Databricks Apache Spark and PySpark Python SQL Data engineering and data architecture ETL and ELT design and development Data lake and lakehouse architecture Delta Lake Data modeling REST APIs and data integration Git and CI/CD Cloud Experience Strong experience with at least one major cloud platform is required: Microsoft Azure, Amazon Web Services, or Google Cloud Platform. Azure Databricks experience is highly preferred. Experience with Azure Data Factory, Azure Data Lake Storage, AWS S3, AWS Glue, Snowflake, Kafka, and Airflow is a plus. Databricks Experience Databricks Workspace Apache Spark and PySpark Delta Lake and Delta Live Tables Unity Catalog Databricks Workflows Databricks SQL Cluster configuration, optimization, and performance tuning Data governance and data security within Databricks CI/CD for Databricks Qualifications Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field. Minimum of 12 years of professional experience in data engineering, data architecture, or related roles. Strong hands-on experience with Databricks and Apache Spark. Strong Python and SQL development experience. Proven experience designing enterprise scale data platforms. Strong understanding of data modeling and data integration. Experience working in Agile and Scrum environments. Strong communication and stakeholder management skills. Insurance industry experience is mandatory. Preferred Qualifications Databricks certification. Cloud platform certification. Experience with enterprise insurance platforms. Experience with Property and Casualty insurance data. Experience with data governance and master data management. Experience with real-time and streaming data. Experience with cloud migration and legacy modernization. Experience leading data architecture initiatives. Screening Note Databricks and Insurance domain experience are mandatory. Candidates with 12+ years of experience only. Additional Information Job Title: Senior Data Engineer / Data Architect - Databricks Work Arrangement: Onsite Employment Type: Contract Engagement Type: W2/1099 Pay rate- 40-50/hr Read more

Dhaka Technologies Limited Company is hiring on behalf of our client for a Senior Data Engineer / Data Architect with deep Databricks and insurance domain expertise. This role is an onsite contract engagement in Charlotte, NC, requiring a minimum of 12 years of professional experience in data engineering and data architecture. The selected consultant will design, develop, and implement scalable enterprise data solutions on Databricks, collaborating with business stakeholders, data architects, engineers, analysts, and technology teams to create modern data platforms and analytics capabilities. Requirements Screening Note Databricks and insurance domain experience are mandatory. Candidates must have 12+ years of experience. Key Responsibilities Design and develop scalable data architecture and data engineering solutions using Databricks. Build and maintain robust ETL and ELT data pipelines using Databricks, Apache Spark, Python, and SQL. Design data lakes, lakehouse architectures, data warehouses, and enterprise data platforms. Develop high performance batch and streaming data pipelines. Implement data ingestion from multiple internal and external sources. Develop data transformation, cleansing, validation, and integration processes. Design scalable and reusable data models for analytics and reporting. Work with Delta Lake, Delta Live Tables, Unity Catalog, and Databricks Workflows where applicable. Optimize Spark jobs, SQL queries, pipelines, and data processing workloads. Implement data quality, data governance, security, lineage, and access control processes. Collaborate with data scientists, BI teams, business analysts, product owners, and application teams. Translate business requirements into technical data architecture and engineering solutions. Participate in architecture reviews and establish data engineering best practices. Troubleshoot production data issues and provide root cause analysis. Mentor junior and mid-level data engineers. Support cloud migration and modernization initiatives. Insurance Domain Experience Strong working knowledge of Property and Casualty (P& C) insurance data is highly preferred, along with other insurance lines. Hands-on experience with various insurance data domains such as policy, claims, billing, and underwriting is required. Required Technical Skills Databricks Apache Spark and PySpark Python SQL Data engineering and data architecture ETL and ELT design and development Data lake and lakehouse architecture Delta Lake Data modeling REST APIs and data integration Git and CI/CD Cloud Experience Strong experience with at least one major cloud platform is required: Microsoft Azure, Amazon Web Services, or Google Cloud Platform. Azure Databricks experience is highly preferred. Databricks Experience Databricks Workspace Delta Lake and Delta Live Tables Unity Catalog Databricks Workflows Databricks SQL Cluster configuration, optimization, and performance tuning Data governance and data security within Databricks CI/CD for Databricks Qualifications Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field. Minimum of 12 years of professional experience in data engineering, data architecture, or related roles. Strong hands-on experience with Databricks and Apache Spark. Strong Python and SQL development experience. Proven experience designing enterprise-scale data platforms. Strong understanding of data modeling and data integration. Experience working in Agile and Scrum environments. Strong communication and stakeholder management skills. Insurance industry experience is mandatory. Preferred Qualifications Databricks certification. Cloud platform certification. Experience with enterprise insurance platforms. Experience with Property and Casualty insurance data. Experience with data governance and master data management. Experience with real-time and streaming data. Experience with cloud migration and legacy modernization. Experience leading data architecture initiatives. Additional Information Work Arrangement: Onsite Employment Type: Contract Engagement Type: W2/1099 Final Interview: Pay rate: $40-50/hr Read more

We are seeking a DataStage/ETL Developer with 5+ years experience to design, develop, and support data integration solutions in an Agile environment. The ideal candidate will have strong DataStage, Unix/Shell scripting, SQL, and database experience, along with excellent analytical and troubleshooting skills. Responsibilities Develop and enhance ETL solutions using DataStage 11.x. Analyze and resolve data, application, and production issues through root-cause analysis. Develop and maintain DataStage ETL jobs and Unix/Shell scripts. Identify opportunities to automate processes and improve application stability. Participate in Agile requirements, solution design, development, testing, and deployment. Support production applications and resolve defects. Collaborate with business stakeholders and technical teams to deliver effective solutions. Contribute to improvements in development tools, processes, and delivery practices. Qualifications Bachelor’s degree in Computer Science, Engineering, or equivalent experience. 5+ years of DataStage/ETL development experience, preferably DataStage 11.7+. Strong Unix/Shell scripting and SQL skills. Experience with DB2, Sybase, and/or Oracle. Experience with Control-M, Jira, and Bitbucket. Strong analytical, troubleshooting, and development skills. Experience with Agile and SDLC methodologies. Strong communication, teamwork, and problem-solving skills. Location McLean, VA Date Posted September 16, 2026 Read more