Pavani Chowdhary
Hi, I’m a results-driven Data Engineer with 5+ years of experience designing and implementing scalable ETL pipelines, automating data ingestion workflows, and integrating Databricks and Microsoft CRM for enterprise…
About
Hi, I’m a results-driven Data Engineer with 5+ years of experience designing and implementing scalable ETL pipelines, automating data ingestion workflows, and integrating Databricks and Microsoft CRM for enterprise data transformation and analytics. I thrive in fast-paced, data-driven environments and enjoy turning complex datasets into actionable insights that empower forecasting and decision making.
I’m proficient in Python, SQL, Apache Airflow, and Databricks, with strong foundations in data governance, schema design, data security, and CI/CD automation for data pipelines. I’m excited about contributing to Focus KPI, Inc. by leveraging my experience in ETL automation, enterprise data integration, and analytics to drive forecasting accuracy and business impact.
- 1 year's experience
- Joined November 2025
Industry experience
- Software & Internet
- Professional Services
Work experience
Led design and automation of streaming data pipelines (Azure Functions, Apache Airflow) for IoT data, enabling real-time operational insights. Implemented CI/CD for data engineering workflows using GitHub Actions and orchestrated ETL processes with Azure Data Factory. Performed data wrangling with Python (Pandas, NumPy) across multiple sources and built automated Power BI dashboards for KPI tracking. Deployed machine learning models on Azure ML Studio and exposed real-time predictions via REST APIs. Built cost-management pipelines and budget alerts on Azure, improving resource allocation. Enforced data security with Azure Key Vault and encryption protocols, and implemented logging/monitoring for real-time failure detection. Leveraged Python, R, SQL, and Spark for data analysis and applied cloud and data governance best practices.
Optimized batch data processing with PySpark and Databricks; integrated Apache Kafka for real-time streaming and event-driven architecture. Designed data models and schemas to support fast retrieval into data lakes (Azure Data Lake Storage). Automated data transformations and loading via Talend, ensuring seamless integration between NoSQL and relational databases (MongoDB, SQL Server). Orchestrated data flows with Apache NiFi and Airflow; built and tuned data warehouses on Google BigQuery and Amazon Redshift with partitioning and clustering; implemented disaster recovery planning and high availability strategies.
Assisted in developing and maintaining data pipelines to ingest, transform, and integrate data from multiple sources. Performed data analysis and reporting to support decision-making; documented workflows and best practices to standardize processes. Conducted data cleansing and transformation with custom Python scripts, improving data quality and reducing manual effort. Monitored performance, troubleshooted issues with senior engineers, and supported tool deployment and onboarding.
Education
Hire Pavani Chowdhary today
Post a job and invite Pavani, or contact them directly. No hidden commission.






