Siddhesh M.
Principal Data Engineer (Python, Spark, AWS, Kubernetes, Airflow)
Results-driven Data Engineer with over 12 years of experience in designing, building, and deploying scalable data processing pipelines using Python, Apache Spark, and SQL across diverse domains including E-commerce, Banking, and Healthcare. Proven expertise in developing end-to-end data solutions that transform raw data into actionable insights and business value. Core Competencies Data Engineering & ETL Development: Skilled in architecting and implementing high-performance data pipelines for batch and streaming data using Apache Spark, Hadoop, Kafka, and Airflow. Experienced in data modeling, transformation, and orchestration to support analytics and reporting needs. Cloud & Infrastructure: Proficient in developing and deploying data-driven microservices within containerized environments using Docker and Kubernetes. Hands-on experience in AWS (EC2, S3, EMR, RDS) and Google Cloud Platform (BigQuery, Dataflow, Cloud SQL) for cloud-native data engineering. Agile & DevOps Practices: Adept at applying Agile methodologies (Scrum, Kanban) and DevOps workflows (Gitflow, CI/CD pipelines) to accelerate software delivery and ensure high-quality deployments. Business Intelligence & Visualization: Expertise in developing interactive dashboards and analytical reports using Tableau, MicroStrategy, and Pentaho, enabling business teams to make data-driven decisions with confidence. Database & Storage Systems: Strong working knowledge of SQL and NoSQL databases including MySQL, Oracle, MS SQL Server, MongoDB, Redis, and Exasol, with experience in query optimization and performance tuning. Technical Skills Programming Languages: Python, SQL Big Data Ecosystem: Hadoop, Spark, Hive, HBase, Kafka, Apache Airflow Databases: MySQL, Oracle, MS SQL, MongoDB, Redis, Exasol Cloud Platforms: AWS (EC2, EMR, RDS, S3), GCP (BigQuery, Dataflow, Cloud SQL) Containerization & Orchestration: Docker, Kubernetes Visualization Tools: Tableau, MicroStrategy, Pentaho Version Control & Collaboration: Git, GitHub, GitLab, Jira, Trello Methodologies: Agile, Scrum, CI/CD Highlights Designed and deployed scalable, fault-tolerant data pipelines from scratch using Python, Spark, and Airflow. Integrated real-time and batch data workflows to improve data availability and reporting accuracy. Led multiple cloud migration and data modernization projects for large-scale enterprise systems. Delivered self-service BI dashboards that reduced manual reporting effort and accelerated business insights.
Earnings and projects
An estimate from an earlier read
Billed in all: $57,306$57k
$0 in the last 6 months
Across 45 contracts
Earnings, last 30 days
No data since Nov 5, 2026.