Mangesh Yadav
Ilford,United Kingdom
Summary
Results-driven Cloud Engineer with over 10 years of experience in software industry, including 6+ years of hands-on experience in Google Cloud Platform (GCP) and Big Data technologies. Expertise in cloud storage, data processing, automation, and infrastructure management using Terraform. Proven track record of successful data migration projects from on-prem Hadoop to GCP Big Query, leveraging ETL frameworks and CI/CD pipelines.
Overview
11
11
years of professional experience
2
2
years of post-secondary education
1
1
Certification
Work History
Senior data engineer
Tata Consultancy Services
London, United Kingdom
02.2021 - Current
- Enabled seamless data ingestion from on-prem Hadoop to GCP using FTH, BIP, and DTF frameworks, populating Raw, Archive, Staging, and Consumption layers in Big Query.
- Converted legacy Spark/Java/Scala Hadoop code to GCP-compatible formats.
- Ingested FinTech finance data from AWS S3 to GCS using Confluent Kafka, Pub/Sub, Cloud Run, and Cloud Functions.
- Automated data workflows using Cloud Composer (Airflow), DBT, and Cloud Functions for Big Query loading, Cloud SQL via BCP.
- Used Jenkins for CI/CD, Spinnaker and TWS for scheduling; mentored team on GCP tools and query performance.
- Developed Infrastructure as Code (IaC) solutions using Terraform for automated cloud infrastructure provisioning and management.
- Converted complex Teradata SQL (~3000+ LOC) to optimized Big Query scripts.
Senior software engineer
Zensar Technology Ltd
Pune, India
10.2017 - 01.2021
- Migrated complex Teradata SQL (~3000+ LOC) to optimized Big Query queries; supported team in performance tuning.
- Built GCP pipelines using Composer, Airflow, and Infoworks to ingest data from on-prem to GCS and Big Query.
- Designed historical/snapshot tables in Hive and Big Query with delta load strategies.
- Guided team on GCP adoption and best practices for scalable data engineering.
Cloud Data Engineer
Blazeclan Technologies Pvt Ltd
Pune, India
04.2017 - 09.2017
- Migrated multi-source website data to AWS Redshift via S3 using Sqoop and Spark for transformation.
- Used Hive, Sqoop, and Spark for data processing; worked with AWS (S3, EC2, EMR) and tools like GIT, JIRA.
Software developer
CruncherSoft Technologies Pvt Ltd
Pune, India
12.2013 - 04.2017
- Developed ERP modules for employee profiles, attendance, and leave management from scratch.
- Gathered client requirements, created design models, and performed end-to-end testing of ERP system features.
Education
Master of Engineering - Computer
Sinhgad College of Engineering
Pune 07.2011 - 12.2013
Skills
- Cloud Platforms: GCP – Cloud Storage, Dataproc, Big Query, Cloud Composer with Apache Airflow, Cloud Run, Cloud Functions, Monitoring
- Infrastructure as Code (IaC): Terraform
- Big Data Hadoop Ecosystem: HDFS, Hive, Sqoop, Oozie, YARN, Spark
- DevOps & CI/CD: Jenkins, GitLab CI/CD, Spinnaker, Spinnaker, Docker
- ETL & Data Processing: BIP Framework, Apache Spark, Scala
Certification
Ø Google Cloud Associate Cloud Engineer (ACE)
Ø Google Cloud Professional Data Engineer (PDE)
Ø Google Cloud Professional Cloud Architect (PCA)
Ø Google Cloud - Build Infrastructure with Terraform on Google Cloud skill badge.
Timeline
Senior data engineer
Tata Consultancy Services
02.2021 - Current
Senior software engineer
Zensar Technology Ltd
10.2017 - 01.2021
Cloud Data Engineer
Blazeclan Technologies Pvt Ltd
04.2017 - 09.2017
Software developer
CruncherSoft Technologies Pvt Ltd
12.2013 - 04.2017
Master of Engineering - Computer
Sinhgad College of Engineering
07.2011 - 12.2013
Similar Profiles
Vrunda ChindhadeVrunda Chindhade
HR Manager at Tata Consultancy Services, TCS, Tata Consultancy Services LimitedHR Manager at Tata Consultancy Services, TCS, Tata Consultancy Services Limited
Sreya BasuSreya Basu
Procare Analyst at Tata Steel(Tata Consultancy Services)Procare Analyst at Tata Steel(Tata Consultancy Services)
Arun VArun V
Program Manager at Tata consultancy services | Financial Ombudsman ServicesProgram Manager at Tata consultancy services | Financial Ombudsman Services
Naimish AgnihotriNaimish Agnihotri
Assistant Delivery Manager at Tata Consultancy Services, Global Shared ServicesAssistant Delivery Manager at Tata Consultancy Services, Global Shared Services
Sanura GamalathgeSanura Gamalathge
Housekeeper at THE OCEAN COLOMBOHousekeeper at THE OCEAN COLOMBO