Mani Prakash
London,United Kingdom
Summary
Enthusiastic Data Engineer eager to contribute to team success through hard work, attention to detail and excellent organizational skills. Practiced at cleansing and organizing data into new, more functional formats to drive increased efficiency. Excellent reputation for resolving problems and improving customer satisfaction.
Overview
12
12
years of professional experience
Work history
Lead Data Engineer VMO2 UK
Tata Consultancy Services
Reading, United Kingdom
12.2021 - Current
- Assisted in designing, developing, prototyping, operating, and implementing data pipelines using technologies such as Python, SQL, DBT, AWS Services, Airflow, and reporting tools in collaboration with business and IT stakeholders.
- Led data architecture-level cloud migration from On-prem Hadoop to AWS seamlessly for Netpulse Program to Redshift in VMO2 Account using Databricks (Batch) & Amazon Flink (Streaming).
- Developed an event-driven architecture using AWS Lambda to trigger file decryption upon upload to an S3 bucket, ensuring real-time data processing.
- Implemented secure data handling practices by retrieving encryption keys from AWS Secrets Manager for decrypting sensitive files.
- Orchestrated the data processing workflow using Airflow, scheduling tasks and dependencies to ensure efficient and reliable execution.
- Leveraged Databricks to perform complex data transformations, including joining and lookup operations, ensuring data integrity and consistency.
- Utilized DBT for structured data modeling and loading processed data into staging tables, enhancing data accessibility and usability.
- Demonstrated familiarity with cloud-based data engineering and storage technologies such as AWS or GCP, and orchestration tools such as Airflow or Composer.
- Exhibited experience in writing production-level code and deploying code to production via version controlling tools such as GitHub/GitLab merge/pull requests, including familiarity with best practices.
- Utilized Terraform as Infrastructure as Code for creating tables/objects.
Data Engineer - Vodafone Hungary
Tata Consultancy Services India
Budapest, Hungary
12.2019 - 11.2021
- Responsible for Designing and implementing Data Movement As A Platform features at GCP in collaboration with business and IT stakeholders.
- Strong understanding of data warehousing methodologies, ETL
processing and dimensional data modelling on OLAP and OLTP
systems. Ingested data from multiple sources as Batch & Realtime
using Kafka Connect. - Reduced development efforts by ~60% with Kafka Connect
implementation for Faster & Reliable File movement - Strong software engineering skills including familiarity with Python,
expertise in data manipulation using SQL and experience with
version control systems - Worked on Migrating HDFS(On Premise) to GCS. Created CDF Pipeline Template to automate developing of 1000's of CDF Pipelines which reduced manual workload by 70% monthly.
- Created CI/CD Model to automate deployment using Jira, BitBucket, JFrog & Bamboo.
Big Data Lead - the Travelers Insurance Company
Tata Consultancy Services India
Chennai, Tamilnadu
06.2016 - 07.2019
- Ingested data from multiple sources using a combination of in-house API using Spark Scala to create Hive tables for BI tools like Qlikview.
Utilized SOLR Collections for indexing Claims and improved ingestion and processing speed by 87% - Led a team of two full-time employee and two contractors.
- Developed Hive SQL Queries and created Business Intelligence
dashboards using Tableau & Qlikview. - Responsible for Reviewing code and providing feedback relative to best practices, performance improvements etc & Troubleshooting production support issues post-deployment and addressed with solutions on time.
- Excellent problem-solving skills with strong communication and
collaboration skills.
Software Analyst
Ford Motor India Company
Chennai, Tamilnadu
11.2011 - 04.2016
- Led the Offload from Mainframe DB2 to Hive, Sqoop and HDFS resulting in an annual cost savings of $90,000 per TB every year.
Developed Hive Scripts to process data from Multiple geographies to Tableau. - Designed and implemented a real-time data pipeline to process semi-structured data by integrating 150 million raw records from 30+ data sources using Sqoop and Hive on Spark and stored processed data in Data lake.
- Worked with Data Engineers to identify right open source tools to deliver product features by performing research, POC/Pilot.
Education
Bachelor of Engineering - Electronics And Communications Engineering
Anna University
Coimbatore05-2011
Skills
- AWS EMR, Glue, Kinesis, Redshift
- Redis
- Data Modelling & Data Governance
- Spark Scala, Pyspark
- GCP, Dataflow, Big query
- Airflow
- Terraform
Timeline
Lead Data Engineer VMO2 UK
Tata Consultancy Services
12.2021 - Current
Data Engineer - Vodafone Hungary
Tata Consultancy Services India
12.2019 - 11.2021
Big Data Lead - the Travelers Insurance Company
Tata Consultancy Services India
06.2016 - 07.2019
Software Analyst
Ford Motor India Company
11.2011 - 04.2016
Bachelor of Engineering - Electronics And Communications Engineering
Anna University