Skip to content

Student Projects – Tote-ally Awesome Final Project

Made by Tote-ally Awesome

Database Solutions for the Modern Tote Lover

Our project extracts data from a tote bag store’s database, transforms it into a structured format, and stores it in a data warehouse. A dashboard accesses the data warehouse to allow insights to be gleamed from sales data. The project infrastructure is built on AWS and provisioned using Terraform an IoC language. Lambda functions are used in AWS running python code. We have used pg8000 to integrate postgreSQL into our code. Our project pipeline takes data from a postgreSQL database and puts it into an ‘ingestion’ S3 bucket. From here, pandas transforms the data into a star schema format and uploads it to a ‘processed’ S3 bucket. A third lambda function uploads it to our data warehouse hosted in RDS. Power BI is then used to create a dashboard for the business to analyse their sales.

The Team

Joe Christian

Joe Christian

I am a multidisciplinary engineer, Manchester Hackspace

director and visual production technician with a passion for technology. Meeting innovative minds at Hackspace has inspired me to pivot my career deeper into tech. I recently completed a full-time data engineering course and I am keen to build on my knowledge in creating quality data stacks to drive business insights. My previous career in the engineering industry saw several of my projects win prestigious awards, and I will apply my considered approach to functional design in a new industry.

Mark Bradley

Mark Bradley

With a strong background in Mathematics and problem solving

with a MSci in Mathematical Physics, I am driven to pursue a career in data engineering. Having worked as a Management Accountant for several years I am looking forward to a new career and new challenges.

Tasneem Aiar

Tasneem Aiar

A Chemistry graduate with a strong passion for data

engineering, sparked by a Python-driven project during my degree. Currently a trainee Data Engineer at Northcoders, gaining hands-on experience in Test-Driven Development (TDD), pair programming, and Git version control. With over 10 years of administrative experience, I bring a robust background in data management, process optimisation, and technical troubleshooting. I thrive in problem-solving environments and am excited to contribute to building scalable, efficient data systems. My adaptability and keen interest in data-driven solutions drive my approach to engineering challenges.

Erin Monaghan

Erin Monaghan

My journey started with statistical coding and analysing

3D-mapped crystal structures in bioinformatics, where I discovered my natural affinity for problem-solving and working with complex datasets. This sparked my interest in data engineering as I realised the power of data to not only inform scientific discovery but also drive innovation in real-world applications. I’m motivated by the opportunity to make a tangible impact, whether that’s by improving business operations, advancing scientific research, or developing new systems. Experience of business start-up and rapidly growing social media profile demonstrate ability to self-start and problem solve in real world situations.

Ben Morgan

Ben Morgan

Having come from a technical recruitment background, I’ve

always been curious about how I could fit into the technical industry myself. After several years of building relationships with technical candidates, hiring managers and HR teams, I thought what better way to extend this further with entering the tech space myself. I’ve seen first-hand the potential the wide world of open data, data science, machine learning has on my future and the world of tech. I decided to enrol in the Northcoders course to turn this into a reality.

Tech Stack

Tech Stack for this group

We used: Python (3.10) AWS (S3, Lambda, Step Functions, CloudWatch, Secrets Manager, RDS) Terraform (Infrastructure as Code) PostgreSQL GitHub Actions (CI/CD) Python was used because it is a universal and well supported language with lots of modules and plugins available. AWS was chosen as it is a major player in the cloud computing industry. Terraform is open source and well integrated with AWS, so was an ideal candidate for our Infrastructure as Code provisioning. PostgreSQL is another open source language well suited to handling large databases. GitHub was used for our repository as it is a very popular and well supported option.

Challenges Faced

We found it challenging to merge lots of languages and systems together to create one coherent piece of software for our end user. Deciding what information each function should pass to the next required many iterations of our design.