Eduardo G.

Hi, I am a Data Engineer with expertise in APIs, Airbyte, Kakfa, DBT and NIFI. I have built end-to-end pipelines from ingestion to visualization. I am proficient with Python, Unix and SQL. I already completed projects using AWS, GCP and Azure. Some of the Projects that I completed and the tools and technologies that I used: 1. Extract reporting metrics from 20 to 30 ad agencies using airbyte custom conector. Load the data to GCP staging tables and perform transformation using DBT. Finally, use Metabase for visualization. 2. Created a webhook using FASTAPI that will receive data from a calendar subscription application. The data was then processed by NIFI. It will then be processed by different processors such as Kakfa before sending to the final destination which is a SQL database for further processing by another team. 3. Created 20-30 different Airbyte connectors that pulls data from US Department of Transformation website to extract Vehicle violation and inspections information. Used GCP for storage and for data warehouse, DBT for transformation and Looker for visualization 4. Created a Python FASTAPI application that provides data from GBQ to the caller of the API. The data is about company information, insurance history, inspection history as well as violation data. The data will be used to populate PDFs that will be issued to trucking company and other transportation company. 5. Created GCP cloud function that perform transformation on the data thats already present in GBQ for normalization, formatting and data cleansing purposes. These are just some of the projects that I completed the past few months. I am a passionate data engineer and i love trying new tools and technologies.