Sujay Chandra
London,United Kingdom
Summary
Accomplished data engineer with expertise in designing and implementing robust ETL and ELT processes, optimising data pipelines, and ensuring high-quality data integration across cloud platforms such as Microsoft Azure and AWS. Proficient in programming languages including Python, PySpark, SQL, and R, with a strong focus on performance optimisation and real-time processing. Skilled in utilising tools like Delta Lake, DBT, Airflow, and Tableau to enhance data warehousing solutions. Demonstrated ability to leverage Azure Data Factory and AWS Glue for seamless API integration and schema evolution. Committed to driving cost-effective solutions while maintaining stringent data validation standards.
Overview
13
13
years of professional experience
2
2
years of post-secondary education
1
1
Certification
Work history
Senior Data Engineer
UK Power Networks
London
2025.04 - 2026.03
- Designed and implemented scalable Azure data pipelines using Azure Data Factory, Databricks, PySpark, and Delta Lake for enterprise data ingestion and transformation.
- Built reusable ingestion frameworks for multiple source systems, including REST APIs, Salesforce, Oracle P6, and batch-based inputs.
- Developed Bronze to Silver ETL pipelines using config-driven specifications to support dynamic schema mapping, column standardisation, and reusable processing logic.
- Implemented incremental and full-load ingestion strategies using pagination, watermarking, and API optimisation to process large-volume datasets efficiently.
- Designed and optimised Delta Lake tables with merge and upsert logic to improve data consistency, deduplication, and query performance.
- Re-engineered legacy SQL pipelines into PySpark workflows to improve maintainability, scalability, and cost efficiency.
- Supported near real-time processing using structured streaming and checkpointing strategies.
- Worked closely with architects, analysts, and platform engineers to deliver scalable technical solutions aligned to business requirements.
- Contributed to data governance and security through row-level security and Unity Catalog-based access controls.
- Followed CI/CD best practices using Azure DevOps, pull requests, and code versioning processes.
Senior Data Engineer
E.ON Next
Nottingham
2024.11 - 2025.03
- Assisted in developing and optimising Databricks and PySpark pipelines for supporting customer lifetime value models for forecasting revenue and cost of goods sold across energy services.
- Integrated data from AWS Athena, S3, and Google Sheets into a unified Delta Lake architecture to streamline data processes.
- Built bronze, silver, and gold layers to facilitate scalable analytics, ensuring data quality and reporting accuracy.
Lead Data Engineer
Self-Employed
London
2024.09 - 2024.11
- Designed and delivered AWS-based data engineering solutions for batch and real-time analytics use cases.
- Architected scalable data lake solutions on Amazon S3 for durable and efficient data storage.
- Built ETL and ELT pipelines using AWS Glue, PySpark, and Python for large-scale data processing.
- Automated schema discovery and metadata management using Glue Crawlers and cataloging solutions.
- Developed streaming and event-driven workflows using Kinesis, Lambda, SQS, and SNS.
- Implemented orchestration and monitoring using AWS Step Functions and Apache Airflow.
- Delivered data warehousing solutions using Amazon Redshift and Athena to support analytics and BI reporting.
- Built data transformation models using DBT to improve analytics readiness and consistency.
- Established security and governance controls using IAM, KMS encryption, and VPC configurations.
- Set up CI/CD pipelines using Jenkins, Terraform, and GitHub Actions.
- Mentored junior engineers and supported best practices in Agile delivery environments.
Senior Data Engineer
EDF (UK)
London
2023.01 - 2024.08
- Designed and delivered cloud-native data engineering solutions on Microsoft Azure for enterprise-scale ingestion, transformation, and analytics.
- Built automated ingestion pipelines using Azure Data Factory across on-premises and cloud source systems.
- Developed ETL and ELT workflows using Azure Databricks, PySpark, SQL, and DBT for large-scale data transformation.
- Implemented scalable data storage solutions using ADLS and Azure Blob Storage for structured and unstructured data.
- Designed and optimised analytics solutions using Azure Synapse Analytics to support reporting and business intelligence.
- Enabled near real-time streaming using Azure Event Hubs and Stream Analytics.
- Built data quality frameworks using Python and Great Expectations for profiling, validation, and monitoring.
- Automated orchestration using Logic Apps and Synapse Pipelines for reliable workflows.
- Established secure data access controls using Azure Active Directory and governance best practices.
- Set up CI/CD pipelines using Azure DevOps and ARM templates for automated deployments.
- Improved Azure SQL and Synapse workload performance through tuning and optimisation.
- Monitored and optimised cloud resource usage for cost-effective platform performance.
Business Data Analyst
Harness-Design & Simulation Aid
Bangalore
2018.11 - 2020.12
- Translated business problems into data and reporting requirements to support decision-making.
- Designed dashboards and reports in Power BI to improve reporting visibility and operational insight.
- Generated business insights to support marketing objectives, profitability improvement, and planning.
- Conducted business analysis including SWOT assessment and planning against commercial targets.
- Worked with internal stakeholders and external vendors to deliver tailored reporting and analytics support.
Business Development Analyst
Blue Phoenix Consulting Partners
Bangalore
2017.12 - 2018.11
- Supported consulting and talent solutions through market analysis, client engagement, and proposal development.
- Contributed to business growth by identifying client needs and shaping solution-oriented offerings.
Data Analyst
Ek Saath Technologies Pvt Ltd
Bengaluru
2017.02 - 2017.12
- Built data pipelines for data acquisition, cleansing, and processing to support reporting and analytics.
- Developed data models and documented data flows to improve maintainability and data understanding.
- Used data visualisation techniques to extract actionable insights from complex datasets.
- Worked with large datasets to support strategic decision-making and client-facing analysis.
Product Marketing Analyst
Digific Enerpax
2014.11 - 2016.12
- Supported sales, engineering, and project teams through analysis, proposal creation, and customer presentations.
- Developed marketing materials and documentation to improve product awareness and engagement.
Test Engineer
Winon Solutions
2013.01 - 2014.11
- Created test cases and test scripts to support software quality and release readiness.
- Worked with development teams in Agile environments to improve functionality and testing processes.
- Participated in peer reviews and process improvements to strengthen product quality.
Education
Master of Science (MS) - Data Analysis and Business Intelligence
University of Leicester
Leicester 2021.01 - 2022.01
Master of Science (MSc) - Electrical, Electronics and Communications Engineering
University of South Wales
Wales 2011.01 - 2012.01
Bachelor of Engineering (BE) - Telecommunications Engineering
Visvesvaraya Technological University
BengaluruSkills
- Data Engineering: ETL, ELT, Data Pipelines, Data Warehousing, Data Modelling, Data Integration, API Integration, Data Quality, Data Validation, Incremental Loads, Full Loads, Schema Evolution, Performance Optimization, Cost Optimization, Real-Time Processing, Structured Streaming, Batch Processing
- Programming Languages: Python, PySpark, SQL, R
- Cloud Platforms: Microsoft Azure, AWS, Databricks
- Azure: Azure Data Factory, Azure Databricks, ADLS Gen2, Azure Synapse Analytics, Azure Blob Storage, Azure Event Hubs, Stream Analytics, Logic Apps, Cosmos DB, Azure DevOps, Azure Active Directory
- AWS: Amazon S3, AWS Glue, AWS Athena, AWS EMR, Amazon Redshift, AWS Lambda, Amazon Kinesis, AWS Step Functions, DynamoDB, CloudWatch, IAM, KMS, VPC
- Tools and Technologies: Delta Lake, DBT, Airflow, Great Expectations, Tableau, Power BI, Looker, Jira, Terraform, Jenkins, GitHub Actions
- Experience with etl tools
Websites
Certification
- Power BI Data Modeling with DAX
- SQL
ADDITIONAL INFORMATION
- Location: London, United Kingdom
- Eligible to work in the UK
- LinkedIn: linkedin.com/in/sujaymc
Timeline
Senior Data Engineer
UK Power Networks
2025.04 - 2026.03
Senior Data Engineer
E.ON Next
2024.11 - 2025.03
Lead Data Engineer
Self-Employed
2024.09 - 2024.11
Senior Data Engineer
EDF (UK)
2023.01 - 2024.08
Master of Science (MS) - Data Analysis and Business Intelligence
University of Leicester
2021.01 - 2022.01
Business Data Analyst
Harness-Design & Simulation Aid
2018.11 - 2020.12
Business Development Analyst
Blue Phoenix Consulting Partners
2017.12 - 2018.11
Data Analyst
Ek Saath Technologies Pvt Ltd
2017.02 - 2017.12
Product Marketing Analyst
Digific Enerpax
2014.11 - 2016.12
Test Engineer
Winon Solutions
2013.01 - 2014.11
Master of Science (MSc) - Electrical, Electronics and Communications Engineering
University of South Wales
2011.01 - 2012.01
Bachelor of Engineering (BE) - Telecommunications Engineering
Visvesvaraya Technological University