George Adye
London
Summary
I am a London based data engineer who has 7+ years industry experience creating, maintaining and improving data pipelines while using agile practises. Besides data, I also have experience with ML and as a backend engineer.
Overview
7
7
years of professional experience
2032
2032
years of post-secondary education
Work history
Data Engineer
Ki Insurance
London, City of London
04.2024 - Current
- Worked on a greenfield project to develop a novel financial solution for an external client. This involved integrating with the wider businesses data pipelines and processes, using Prefect, DBT and Bigquery. The data access layer was written in Kotlin within a Spring application, so I deepened my backend development expertise here as well.
- Worked in a data-expertise team that acted as a consultancy for data issues / new features across the wider business. I'd embed myself in a new team while building the data-solution, before moving across to the next internal-client. Demanded rapid problem-solving and adaptability while working in unfamiliar business domains.
Data Engineer
Sainsbury's
London, Holborn
11.2021 - 04.2024
- Engineered a scalable API designed to accommodate up to 20k requests per second by customers using the smartshop app. To ensure low latency, the infrastructure involved a high-performance AWS ECS cluster configured with a load balancer, load testing was run during development, and the asynch application used cacheing. We used experimentation with canary deployment to assess improvements to the API across key metrics - if there were statistically significant differences, we'd release.
- Developed the architecture behind a ELT pipeline which classifies products into health categories. This was orchestrated by airflow, which runs python scripts executing snowflake queries written using DBT - data was then written to tables designed using the data vault model.
Senior Statistical Methodologist
Office for National Statistics
London, Pimlico
08.2018 - 11.2021
- Promoted to senior late 2020.
- Took the lead on creating an ETL pipeline for the purpose of data linkage. This involved pulling massive amounts of data from AWS S3 into an AWS EMR cluster configured to run Pyspark jobs, before loading into AWS RDS. Dashboards were built on top of this to monitor algorithm performance.
Education
Master of Science - Social Policy
University of Bath
Bath 04.2001 - 10.2017
Bachelor of Arts - Philosophy
University of Birmingham
BirminghamSkills
- Python
- Kotlin
- AWS, GCP
- SQL with DBT, BQ, Snowflake
- Airflow, Prefect
- Terraform
- Pyspark, Sklearn
- CI/CD
- Knowledge of statistics and ML methodology
Timeline
Data Engineer
Ki Insurance
04.2024 - Current
Data Engineer
Sainsbury's
11.2021 - 04.2024
Senior Statistical Methodologist
Office for National Statistics
08.2018 - 11.2021
Master of Science - Social Policy
University of Bath
04.2001 - 10.2017
Bachelor of Arts - Philosophy
University of Birmingham