Data Engineer

• Established Data Analytics Platform foundation using Airbyte, Google BigQuery, Apache Airflow, DBT, and Looker;
• Configured Airbyte for streamlined data ingestion.
• Leveraged Google BigQuery as the central data warehouse;
• Integrated Apache Airflow to automate and manage data workflows;
• Utilized DBT to create and maintain a streamlined data modeling and transformation process;
• Integrated Looker for data visualization;
• Supported troubleshooting and technical issues.

Joined the data team at its inception, working on building the data platform from scratch. This included selecting the technology stack, setting the foundations, and applying best practices to deliver a platform that drives data-driven initiatives and supports business operations.

• Established Google BigQuery as the core data warehouse, optimizing data storage and retrieval.
• Designed and deployed an ELT pipelines using Airbyte on Kubernetes, facilitating seamless data extraction from sources like PostgreSQL and MongoDB, and leveraging CDC for efficient updates.
• Utilized Airflow for orchestrating jobs and dbt/Dataform for data transformations, ensuring smooth workflow automation.
• Automated CI/CD processes with GitHub Actions and Google Cloud Build, enhancing deployment efficiency.
• Implemented comprehensive data quality checks to ensure data integrity and governance.
• Leveraged Looker for business intelligence and reporting, delivering actionable insights to drive decision- making.

• Contributed to the development of a Machine Learning and Data Analytics Platform using the Google Cloud Ecosystem, including BigQuery, Apache Airflow, Pub/Sub, and Google Analytics;
• Utilized BigQuery for efficient data storage and analysis;
• Integrated Apache Airflow to automate and manage data workflows;
• Implemented Pub/Sub for reliable messaging and real-time data streaming;
• Incorporated Google Analytics to gather insights from user interactions;
• Worked on optimizing platform performance and scalability.

• Contributed to the development of a Big Data Platform within the Cloudera Ecosystem, incorporating Apache Spark, Hadoop, Apache Kafka, Apache Flume, Apache Hive, and Impala;
• Designed robust and scalable data ingestion solutions and pipelines;
• Formulated and automated preprocessing and ingestion of data batches and streams.

• Led the development of an adaptable video processing and analysis platform, enabling seamless integration of numerous Artificial Intelligence and Computer Vision algorithms for efficient real-time video analysis;

• Spearheaded the design and upkeep of the platform's pipeline, leveraging significant big data tools including Apache Storm, gRPC, RabbitMQ, Redis, and Kubernetes;

• Contributed to the creation of a comprehensive Data warehouse aggregating data from diverse sources;

• Conducted data analysis by executing intricate SQL Ad-Hoc Queries in response to business requirements from multiple origins;

• Developed SQL procedures to automate ETL tasks and workflows;

• Managed the complete data warehouse lifecycle, ensuring its continued functionality and relevance;