Made by Team Funland
Give us your data, we will give you a warehouse.
This repository contains the final team project of the North Coders Data Engineering Bootcamp, showcasing a full-stack ETL (Extract, Transform, Load) pipeline designed for real-world data engineering practice. – Data ingestion from PostgreSQL into AWS S3 data lakes. – Transformation into star-schema format using pandas and awswrangler. – Deployment managed with Infrastructure-as-Code (Terraform). – Automated testing and deployment with CI/CD pipelines via GitHub Actions and Makefile. – Monitoring, logging, and alerts integrated via AWS CloudWatch and SNS. – Business dashboards and insights delivered through Tableau. – Ensure good security and engineering practises.
The Team
Shayan Shaterpoori
Physics graduate looking to enter the field of data…
engineering and solve business problems with technological and analytical skills.
Hari Sapkota
As someone from medical research, a brief exposure to…
programming languages such as Python and R sparked a great enthusiasm in me to use emerging technologies to utilise data and generate valuable insights for businesses. Such curiosity has guided my interest to become a data engineer and explore avenues for data extraction and analysis to enhance data science and machine learning.
Sarah Larkin
My professional experience is based in supply chain,…
logistics, and customer service. My most recent role was in the food industry as a customer relationship manager. Over the years I have developed an interest in how technology can be used to improve processes and assist business decisions. This has led me towards learning technical skills through the Northcoders Data Engineering bootcamp and onto a career in tech.
Elisa Quku
My name is Elisa Quku. I am a former Maths Teacher now…
trained to become a Data Engineer. I joined Northcoders to learn how to work with data and build real-world tech projects. I am very excited to start a new career where I can keep learning and apply my skills in a new way.
Adel
A creative person with an artistic flair, enjoys coding and…
exploring new technologies at coding meetups, blending art with technology.
Tech Stack
We used Python, Terraform, Makefile, Github workflows, AWS. Allows to deploy Serverless python functions, deployed with infrastructure as code.
Challenges Faced
Planning the project at the beginning (what combination of step functions and lambdas we should use), and finding python logic to extract newly updated data. However, as time went on, and we explored these problems, we overcame them.
Funland
Made by Team Funland Give us your data, we will give you a warehouse. This repository contains the final team project of the North Coders Data Engineering Bootcamp, showcasing a full-stack ETL (Extract, Transform, Load) pipeline designed for real-world data engineering practice. – Data ingestion from PostgreSQL into AWS S3 data lakes. – Transformation into…
Culinary
Made by Team Culinary Team Get it DONE This group project was carried out during the final phase of the 13-week Northcoders Data Engineering Bootcamp. The aim was to design and build a reliable data platform that extracts data from a PostgreSQL operational database (Totesys), transforms it into a denormalised form star schema, and loads…
Atkinson
Made by Team Atkinson We put the terra in terraform We worked on a etl data pipeline that extracts data from a database and loads it into a data warehouse. The Team AL-Amin Al-Amin, 27, previously worked as a data technician… training to be a data engineer Myles Myles, Accounting turned data engineer Deep Deep…
Ainsdale Beach
Made by Team Ainsdale Beach High Tides! High Tech! The Ainsdale Beach ETL project is an Extract Transform Load pipeline, using Python, hosted in AWS. The primary function of the pipeline is to ingest and transform data from an OLTP optimised database into an OLAP optimised data warehouse. The main objectives of this project are…
Student Projects – Wicklow Final Project
A project made by Team Wicklow – Seamless, Scalable, and Smart – The Future of Data Automation.
Student Projects – Tote-ally Awesome Final Project
A project created by Team Tote-ally Awesome – Database Solutions for the Modern Tote Lover
Student Projects – Team Ketts Lough Final Project
A project made by Team Ketts Lough – ETL Data Platform for ToteSys to AWS Data Warehouse
Student Projects – Team Girley project phase
A project made by Team Girley – From Raw to Refined
Student Projects – team-01-data-squid Final Project
A project created by team-01-data-squid – For ink-redible insights, dive into data with Data Squid
Student Projects – Sperrins Syndicate Final Project
A project created by Team Sperrins Syndicate – Bound by the peaks, driven by purpose.
Student Projects – Mourne Final Project
A project made by Team Mourne – Gaslight Gatekeep Girlboss
Student Projects – Lullymore-west Final Project
A project created by Team Lullymore-west – “This should only take 30 minutes”