Skip to content
BackNov 5
Siddhesh M.

Siddhesh M.

Principal Data Engineer (Python, Spark, AWS, Kubernetes, Airflow)

Results-driven Data Engineer with over 12 years of experience in designing, building, and deploying scalable data processing pipelines using Python, Apache Spark, and SQL across diverse domains including E-commerce, Banking, and Healthcare. Proven expertise in developing end-to-end data solutions that transform raw data into actionable insights and business value. Core Competencies Data Engineering & ETL Development: Skilled in architecting and implementing high-performance data pipelines for batch and streaming data using Apache Spark, Hadoop, Kafka, and Airflow. Experienced in data modeling, transformation, and orchestration to support analytics and reporting needs. Cloud & Infrastructure: Proficient in developing and deploying data-driven microservices within containerized environments using Docker and Kubernetes. Hands-on experience in AWS (EC2, S3, EMR, RDS) and Google Cloud Platform (BigQuery, Dataflow, Cloud SQL) for cloud-native data engineering. Agile & DevOps Practices: Adept at applying Agile methodologies (Scrum, Kanban) and DevOps workflows (Gitflow, CI/CD pipelines) to accelerate software delivery and ensure high-quality deployments. Business Intelligence & Visualization: Expertise in developing interactive dashboards and analytical reports using Tableau, MicroStrategy, and Pentaho, enabling business teams to make data-driven decisions with confidence. Database & Storage Systems: Strong working knowledge of SQL and NoSQL databases including MySQL, Oracle, MS SQL Server, MongoDB, Redis, and Exasol, with experience in query optimization and performance tuning. Technical Skills Programming Languages: Python, SQL Big Data Ecosystem: Hadoop, Spark, Hive, HBase, Kafka, Apache Airflow Databases: MySQL, Oracle, MS SQL, MongoDB, Redis, Exasol Cloud Platforms: AWS (EC2, EMR, RDS, S3), GCP (BigQuery, Dataflow, Cloud SQL) Containerization & Orchestration: Docker, Kubernetes Visualization Tools: Tableau, MicroStrategy, Pentaho Version Control & Collaboration: Git, GitHub, GitLab, Jira, Trello Methodologies: Agile, Scrum, CI/CD Highlights Designed and deployed scalable, fault-tolerant data pipelines from scratch using Python, Spark, and Airflow. Integrated real-time and batch data workflows to improve data availability and reporting accuracy. Led multiple cloud migration and data modernization projects for large-scale enterprise systems. Delivered self-service BI dashboards that reduced manual reporting effort and accelerated business insights.

🇮🇳Navsari, India$40/hr100% JSS4.52 (29)Since Aug 2010

Earnings and projects

MRR (estimate)
≈$0

An estimate from an earlier read

World#70,446 / 412k
🇮🇳Country#10,649 / 45.9k
Revenue, last 30 days
$0
Total earnings
$55k

Billed in all: $57k

World#70,446 / 412k
🇮🇳Country#10,649 / 45.9k
Per project
$1.8k

$0 in the last 6 months

Projects, last 6 months
1
0% fixed100% hourly
World#128,092 / 412k
🇮🇳Country#14,379 / 45.9k
Projects in all
31

Across 45 contracts

18% fixed82% hourly
World#94,778 / 412k
🇮🇳Country#14,533 / 45.9k

Earnings, last 30 days

$57.3kNothing earned in the last 30 daysBilled in all, before Upwork's fee · drag across the chart to see what was earned in any stretch
$56k$57k$58k$59kOct 8Oct 23Nov 7

No data since Nov 5, 2026.

Skills