Manjunath YG
Reading, UK
Overview
17
17
years of professional experience
2006
2006
years of post-secondary education
Work history
Azure Data Engineer
BP
03.2024 - 12.2024
- Analyze, design and build Modern data solutions using Azure PaaS service to support visualization of data
- Understand current Production state of application and determine the impact of new implementation on existing business processes
- Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in Azure Databricks
- Created Pipelines in ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform and load data from different sources like Azure SQL, Blob storage, Azure SQL Data warehouse, write-back tool and backwards
- Developed Spark applications using Pyspark and Spark-SQL for data extraction, transformation and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage patterns
- Responsible for estimating the cluster size, monitoring and troubleshooting of the Spark Databricks cluster
- Developed JSON Scripts for deploying the Pipeline in Azure Data Factory (ADF) that process the data using the SQL Activity
- Integrated the source file storage in Azure Blobs and extracted data from Blobs to Snowflake
- Extracted source data and mapped them into downstream systems and performed ETL transformation to connect PowerBI dashboard
Data DevOps Engineer
Tradition UK Ltd
02.2021 - 01.2024
- Created pipelines in ADF using linked services/Datasets/Pipeline/ to Extract, Transform and Load data from different sources like Azure SQL, Blob storage, Azure SQL Data Warehouse
- Experience in Spark applications using Scala and Spark SQL for Data Extraction, transformation, and aggregation from multiple formats for analyzing& transforming the data to uncover insights into the customer usage patterns
- Involved in creating JSON scripts for deploying the pipeline in Azure Data Factory (ADF) that process the data using the SQL Activity
- Undertake data analysis and collaborated with down-stream, analytics team to shape the data according to their requirement
- Worked in technical standards, procedures, and guidelines used to govern data models as well as ETL development projects within
Platform Engineer
Tradition UK Ltd
11.2018 - 01.2021
- Experience of building and delivering robust highly available solutions
- Built and maintained data pipelines using AWS Glue, transforming raw data into structured formats for further analysis
- Created pipelines in ADF using linked services/Datasets/Pipeline/ to Extract, Transform and Load data from different sources like Azure SQL, Blob storage, Azure SQL Data Warehouse
- Multi system administration and Software configuration management, using various automation tools to oversee the end-to-end deployment process
- Implemented a CI/CD/CT pipeline using Cloud Agnostic Tools like Jenkins, Docker and GitHub, SVN, Nexus, Maven and Security testing code quality coverage using SonarQube and HP Fortify (Security Testing)
- Designed, developed and deployed applications on to several cloud-based solutions using configuration Management tools Ansible
- Good Working knowledge of build scripts and automated solutions using various scripting languages like Groovy, Shell and Python
Wintel Engineer
Tradition UK Ltd
01.2015 - 11.2018
- Experience of building and delivering robust highly available solutions
- Running packaged applications on all operating systems like Windows 10 21H2, performing tasks and ensuring security and stability
- Patch deployments to end user devices and keep the systems up to date
- Monitor all devices and send report to Security team regarding vulnerabilities
App-V Consultant
British Telecommunications
03.2012 - 10.2014
- Migrate legacy applications from XP, Vista, Windows 7 to BT's new Windows 8 desktop platform for delivery using SCCM 2012
- Taking applications through the application phase using tools such as App-V 4.6 sp1- App-V 5.0, Wise, InstEdit & VB scripting for deployment via Radia, SCCM 2007 and SCCM 2012
- Sequencing the applications using App-V 5.0 tool and customizing the applications as per client requirement
Senior Systems Executive
Cognizant Technology Solutions
10.2007 - 08.2010
Education
Bachelor of Engineering -
Acharya Institute of Technology
BangaloreSkills
- Data pipeline construction
- PYTHON,
- Azure Data Lake, ADF, Azure Databricks,SQL
- Azure Blob Storage, JSON, CSV, Parquet
- Amazon EC2, S3, Auto Scaling, Cloud Watch, Lambda, Glue
- Apache Airflow, Apache Pyspark, Kafka, Docker
- Snowflake, GitHub, Jenkins,
- Technical communication
- Data Visualisation, PowerBI
- Cloud computing services
Timeline
Azure Data Engineer
BP
03.2024 - 12.2024
Data DevOps Engineer
Tradition UK Ltd
02.2021 - 01.2024
Platform Engineer
Tradition UK Ltd
11.2018 - 01.2021
Wintel Engineer
Tradition UK Ltd
01.2015 - 11.2018
App-V Consultant
British Telecommunications
03.2012 - 10.2014
Senior Systems Executive
Cognizant Technology Solutions
10.2007 - 08.2010
Bachelor of Engineering -
Acharya Institute of Technology
Similar Profiles
JORGE MAGANA-GARCIA JR.JORGE MAGANA-GARCIA JR.
Lead Facility Technician at BPLead Facility Technician at BP
Umesh BarapatreUmesh Barapatre
Head of Collaboration Platforms & Intelligent Workplace at bpHead of Collaboration Platforms & Intelligent Workplace at bp
Toyia GoodwinToyia Goodwin
Price Coordinator Scheduler at BPPrice Coordinator Scheduler at BP
Gurpartap SinghGurpartap Singh
Manager at BPManager at BP
Ben MalkinBen Malkin
Delivery Driver & Kitchen Assistant at Bengal LoungeDelivery Driver & Kitchen Assistant at Bengal Lounge