Venkata Karri
Aylesbury,Bucks
Summary
Executive Summary
Microsoft Certified Azure Data Engineer with 15+ years of experience in SDLC, including 5+ years in Azure Data Engineering, Lakehouse architecture, and cloud data platforms in various domains like
Insurance, Banking, Healthcare, Pharma, and Retail.
Strong hands-on expertise in Azure Databricks, Delta Lake, PySpark, Azure Data Factory, Azure Synapse, ADLS, SQL, and Python, building ETL/ELT pipelines and Medallion (Bronze/Silver/Gold) architectures for batch and near real-time processing.
Skilled in integrating data from relational, NoSQL, APIs, and streaming sources using Spark, Hadoop, Cloudera, and Snowflake.
Experienced in data governance, CDC, data quality, performance tuning, and cost optimisation in regulated environments.
Proficient in Python, PySpark, Scala, SQL, and CI/CD automation using Azure DevOps and Airflow orchestration.
Strong stakeholder collaboration skills delivering BI, analytics, and AI/ML-ready data solutions in Agile environments.
Overview
16
16
years of professional experience
1
1
Certification
Work history
Azure Data Engineer
Domestic and General Insurance
Wimbledon, UK
2024.03 - Current
- The project involved designing and implementing a cloud-based Azure Data Platform to consolidate data from policy administration, claims management, customer, agent, branch, and third-party systems into a centralised Lakehouse architecture. The platform enabled enterprise reporting, customer analytics, claims analysis, operational performance monitoring, customer segmentation, business decision-making through trusted, governed, and analytics-ready datasets.
- Data sources included SQL Server databases, REST APIs, JSON and CSV files, and external partner systems. The solution leveraged Azure Data Factory, Azure Databricks, Azure Data Lake Storage Gen2, Delta Lake, and Power BI to support scalable ETL/ELT processing, data quality, governance, and reporting requirements across the business.
- Domestic & General is a leading provider of appliance care and protection plans, serving millions of customers across the UK, Europe, the US, and Australia. The organisation provides insurance, warranty, repair, and maintenance services for household appliances and consumer electronics including boilers, washing machines, refrigerators, televisions, and kitchen appliances.
- Project: Enterprise Insurance Data Platform & Customer Analytics Lakehouse
Azure Data Engineer
AVIVA Insurance
London
2021.06 - 2023.12
- The solution integrated data from multiple enterprise sources including Azure SQL Database, REST APIs, JSON files, and CSV files into a scalable Medallion (Bronze, Silver, Gold) architecture. The platform enabled business users to analyse customer behaviour, claims trends, policy performance, and operational KPIs through Power BI dashboards.
- AVIVA is a retail insurance provider offering policy and claims management services similar to leading insurers. The project involved designing and implementing an end-to-end Azure-based Lakehouse platform to support claims analytics, customer segmentation, policy analysis, and business intelligence reporting.
- Key Responsibilities
- Designed and implemented an enterprise-scale Lakehouse architecture using Azure Data Lake Storage Gen2, Azure Databricks, Delta Lake, and Azure Data Factory.
- Developed scalable ETL/ELT pipelines to ingest data from Azure SQL Database, REST APIs, JSON files, and CSV files into Azure Data Lake Storage.
- Built Medallion Architecture (Bronze, Silver, Gold) data layers to support data standardisation, cleansing, transformation, and business reporting requirements.
- Implemented incremental data loading frameworks using High Water Mark (HWM) techniques to support efficient processing of claims, agent, and branch data.
- Developed Databricks notebooks using Python, PySpark, and Spark SQL to perform data cleansing, validation, enrichment, aggregation, and business rule implementation.
- Created and maintained Delta Lake tables supporting ACID-compliant, scalable, and high-performance data processing workloads.
- Designed data models and curated Gold layer datasets for customer analytics, claims reporting, customer segmentation, and operational dashboards.
- Automated orchestration of data pipelines using Azure Data Factory and Databricks Workflows, supporting end-to-end scheduling and monitoring.
- Integrated enterprise security controls using Azure Active Directory, Azure Key Vault, RBAC, and Azure Purview governance frameworks.
- Implemented monitoring, alerting, and operational support using Azure Monitor and Azure cost management best practices.
- Managed source control, release management, and CI/CD deployment processes using Azure DevOps and Git repositories.
- Worked within Agile/Scrum delivery teams, participating in sprint planning, backlog refinement, estimation, code reviews, and stakeholder demonstrations.
ETL Consultant
Brakes Group
Ashford, UK
2019.01 - 2021.06
- Environment: IBM Infosphere 8X(px, server), Quality Stage, Oracle, PL/SQL, SQL SERVER, UNIX Shell Scripting, CVS Version control tool, Unix Shell Scripting and HP Quality Center 9.2.
Sr ETL Developer
Astellas Pharmacy Europe Ltd
Surrey
2017.01 - 2018.12
- Environment: IBM Infosphere 8X(px, server), Oracle, PL/SQL, SQL SERVER 2005, UNIX Shell Scripting, CVS Version control tool, Unix Shell Scripting and HP Quality Center 9.2, TEAMS, VEEVA SYSTEMS, CEGEDIM.
Sr ETL Developer
Scottish Widows Investment Partnership
Edinburgh
2015.02 - 2016.08
- Environment: IBM Infosphere 8X(px, server), Quality Stage, Datastage 7.X, Oracle, PL/SQL, SQL SERVER 2005, UNIX Shell Scripting, CVS Version control tool, Unix Shell Scripting and HP Quality Center 9.2.
ETL Developer
Lexmark
Lexington, Kentucky
2014.01 - 2014.08
- Environment: DataStage 8.X/7.X Oracle, DB2, UNIX Shell Scripting
ETL Lead/Developer
Aetna
St Louis Park, MN
2013.06 - 2013.11
- Environment: DataStage 7X, oracle, DB2, UNIX shell scripting, CVS Version control tool
ETL Developer
Wells Fargo CCG BIDE PROJECT
Minneapolis
2011.07 - 2012.12
- Environment: DataStage 7.X, Oracle, DB2, UNIX Shell Scripting, CVS Version control tool.
ETL Developer
Centrica
Plymoth, MN
2010.07 - 2011.06
- Environment: DataStage PX 7.X, Oracle 10G, DB2 and UNIX Shell Scripting
Education
Bachelor of Technology - Computer Science & Engineering
JNT University
IndiaSkills
- Azure Data Platform
- Azure Databricks, Azure Data Factory (ADF), Azure Data Lake Storage (ADLS Gen2), Azure SQL Database, Azure Synapse Analytics, Microsoft Fabric, Azure Key Vault, Azure DevOps
- Data Engineering
- Python, PySpark, SQL, Spark SQL, Delta Lake, Delta Live Tables, ETL/ELT Pipeline Design, Lakehouse Architecture, Medallion Architecture, Data Ingestion (Batch & Streaming), API Integration
- Data Modelling & Governance
- Dimensional Modelling, Star Schema, Data Quality, Data Governance
- DevOps & Tools
- CI/CD (Azure DevOps, Git), Databricks Workflows, Terraform (basic), Apache Airflow (legacy)
- ETL Tools
- Datastage Informatica
- Reporting Tools
- Power BI
Certification
Microsoft Certified Azure Data Engineer
Timeline
Azure Data Engineer
Domestic and General Insurance
2024.03 - Current
Azure Data Engineer
AVIVA Insurance
2021.06 - 2023.12
ETL Consultant
Brakes Group
2019.01 - 2021.06
Sr ETL Developer
Astellas Pharmacy Europe Ltd
2017.01 - 2018.12
Sr ETL Developer
Scottish Widows Investment Partnership
2015.02 - 2016.08
ETL Developer
Lexmark
2014.01 - 2014.08
ETL Lead/Developer
Aetna
2013.06 - 2013.11
ETL Developer
Wells Fargo CCG BIDE PROJECT
2011.07 - 2012.12
ETL Developer
Centrica
2010.07 - 2011.06
Bachelor of Technology - Computer Science & Engineering
JNT University