Summary
Overview
Work history
Skills
Certification
Timeline
Generic
Sanjay Vaishnav

Sanjay Vaishnav

Bracknell, UK

Summary

Seasoned Techno-Functional Data Engineer with over 10 years of expertise in designing and implementing data integration solutions on both cloud and on-premise. Known for a strategic approach to optimizing data pipelines and delivering actionable insights to drive business decisions. Skilled in implementing automation, improving data processes, and ensuring alignment between data strategies and organizational objectives. Proven track record of leading teams, mentoring developers, and fostering collaboration to ensure timely, efficient, and high-quality project delivery in fast-paced environments. Experienced in facilitating cross-functional business teams with the required data, providing support for building reports, and empowering stakeholders to make data-driven decisions.

Overview

12
12
years of professional experience
1
1
Certification

Work history

Project Lead

Evelyn Partners
01.2022 - Current
  • Developed and managed complex ETL pipelines in Azure Data Factory, ingesting structured and unstructured data from multiple sources into ADLS for analytics.
  • Integrated Azure Databricks with ADF for Spark-based transformations, processing large-scale data sets into structured formats for downstream applications.
  • Designed scalable storage solutions in ADLS Gen2 and orchestrated data movement between layers with ADF for efficient access and processing.
  • Implemented CI/CD pipelines via Azure DevOps to automate ADF and Databricks deployment, ensuring version control and efficient release management.
  • Automated ETL workflows with ADF triggers, monitored performance, and set up error handling to ensure reliable data availability.
  • Leveraged advanced SQL techniques in ADF and Databricks for data modeling and query optimization, improving data retrieval times in SQL Server.

Senior Project Engineer

Cisco Systems
06.2017 - 12.2021
  • Developed data transformation projects using Hadoop and Apache Spark, contributing to the implementation of new MT and PV tables for booking data integration.
  • Conducted training sessions to enhance team skills in data management tools and methodologies.
  • Ensured 100% code compliance and data accuracy through regression testing and performance optimization of Spark jobs.
  • Automated large-scale data transfers with Unix Shell scripts, streamlining processes between Hadoop and Informatica using Tidal.
  • Optimized Hive queries, integrated new data pipelines, and addressed downstream queries for efficient data processing.
  • Guided team members on Hadoop and Spark best practices, resolving technical challenges and achieving targets in Tread NXT.

Project Engineer

Hanes Brand
03.2013 - 05.2017
  • Designed and implemented ETL processes using Informatica PowerCenter 10.2 and SSIS, ensuring efficient data extraction, transformation, and loading
  • Integrated data from retail systems like POS and inventory management, maintaining data accuracy and implementing quality checks
  • Optimized ETL performance to reduce processing times, enhancing overall efficiency
  • Developed comprehensive reports and dashboards using Power BI, providing critical insights into sales and inventory
  • Led the migration of ETL processes to advanced platforms, developing and maintaining retail-specific data models, and driving Agile projects to successful completion.

Skills


  • SQL
  • ADF
  • Azure Databricks
  • SSIS
  • Pyspark
  • Informatica PC
  • Python
  • Power BI
  • GitHub
  • Jira
  • Confluence


Certification

  • Microsoft Certified: DP - 203 - Azure Data Engineer Associate (ADF/ADB/AS)
  • Certified Informatica Developer
  • Agile – Wipro internal
  • ACE – Architect Certification – In-Progress

Timeline

Project Lead

Evelyn Partners
01.2022 - Current

Senior Project Engineer

Cisco Systems
06.2017 - 12.2021

Project Engineer

Hanes Brand
03.2013 - 05.2017
  • Microsoft Certified: DP - 203 - Azure Data Engineer Associate (ADF/ADB/AS)
  • Certified Informatica Developer
  • Agile – Wipro internal
  • ACE – Architect Certification – In-Progress

Similar Profiles

Create profile
Sanjay Vaishnav