Narayan Gopal M.

I build data pipelines that land messy data in a lakehouse people can actually query, on AWS (Glue, Athena, Redshift, Lambda) and Microsoft Fabric. 13 years in software, the last few in data engineering, 100% Job Success. AWS Data Engineer and Microsoft Fabric Data Engineer certified. Recent work: - Team lead on a private cloud RAG platform (432 hours, 5 stars). Self-hosted on Hetzner with OpenCloud for files, AI agents, GPU inference on RunPod, and Keycloak plus OpenBao securing the whole stack, so private users get RAG without their data leaving their own cloud. - Email attachments into a Fabric Lakehouse in 1 to 5 minutes. Logic Apps on a shared mailbox, an Azure Function with Managed Identity writing to OneLake, an Eventstream trigger, and delta-rs upserts into Bronze instead of slow PySpark. CSV, XLSX and XLS normalised into one schema. - ServiceNow incident analytics on Fabric. Bronze and Silver medallion layers, plus an ML model in a Fabric notebook that flags tickets which should be escalated. - RDS and external sources into one AWS warehouse with zero downtime, using incremental timestamp loads and a reconciliation pass instead of a hard cutover. - Running Fabric in production: F64 and smaller capacities, separate Dev and Prod workspaces, Capacity Metrics monitoring and my own pipeline error tracking with auto recovery. What I work with: AWS Glue, Athena, Redshift, Lambda, Step Functions, S3, Lake Formation. Microsoft Fabric (Lakehouse, Warehouse, Data Factory, Eventstream, Mirroring) and Azure Data Factory. Python, PySpark, SQL, dbt, Airflow. Terraform and AWS CDK. RAG and LLM apps with LangChain, LangGraph and vector databases. Self-hosted stacks with Keycloak and OpenBao. Also from earlier years: Java, Go, Kafka, Flink, Elasticsearch, Postgres, MongoDB, DynamoDB, Redis, Docker. Certifications: AWS Data Engineer Associate, Microsoft Fabric Data Engineer Associate, Microsoft Azure AI. I am based in Kathmandu and overlap 4 hours with US Eastern time. Send me what is broken or slow in your pipeline and I will tell you what I would check first.