Hamed D.

Top Rated Plus Data Engineer with 100% Job Success Score and 20+ completed projects. I specialize in LLM integration, large-scale web scraping, and ETL pipeline development for international clients across e-commerce, retail, and data-intensive industries. Currently building a production AI data pipeline that integrates OpenAI GPT-4 into a live ETL workflow for automated product classification and metadata enrichment across thousands of records daily. The system uses Scrapy and Playwright for large-scale web crawling across 50+ domains with anti-bot bypass via Zyte and Diffbot APIs, and includes an embedding-based similarity search layer using sentence-transformers and FAISS on AWS S3. Previously delivered an ML-powered data matching system that deduplicated 100,000+ product records across multiple retail datasets with no standardized identifiers available. The ETL pipeline ran on Google Cloud Platform with MySQL and fed structured data directly into a consumer-facing web application. Skills and services: LLM integration and OpenAI API development with structured outputs, data enrichment, and automated classification Web scraping and data extraction at scale using Scrapy, Playwright, Zyte API, and Diffbot ETL and ELT pipeline development on AWS and GCP from raw ingestion to clean structured output Embedding generation and vector search systems using FAISS, sentence-transformers, and hybrid BM25 search Python automation and data processing to replace manual workflows entirely Machine learning for product matching, deduplication, and data classification REST API integration, JSON data handling, and MySQL database management Data quality assurance across extraction, transformation, and storage layers Every engagement starts with a clear scope and defined deliverables. I communicate proactively, flag issues early, and own the full project lifecycle from kickoff through delivery and documentation. Send me a message and let's discuss your project.