ARUP KUMAR NEOGY

• Project Overview: Azure based strategic enterprise data platform
• Client: Marks and Spencer
• Team Size: 20

Responsibility:

• Establish and enforce best practices in ETL pipeline development, ensuring the delivery of high-quality, optimised, and compliant code
• Design and develop scalable ETL pipelines using PySpark and SQL in Databricks on ADLS
• Write and optimize complex business logic to meet business requirements
• Collaborate with architects and data modellers to implement scalable and efficient ETL processes
• Engage closely with business stakeholders and architects to align technical solutions with business needs and objectives
• Conduct in-depth analysis of the existing ('as-is') data landscape, along with identifying gaps, inefficiencies, and opportunities for optimization to move data into our strategic azure based data platform
• Continuously identify scopes of automation by building frameworks and applications to proactive alert and mitigate issues

• Project Overview: Azure based strategic enterprise data platform
• Client: Marks and Spencer
• Team Size: 10

Responsibility:

• Collaborate with the Data Engineering Tech Lead and data modelers to implement optimized data models and ETL processes
• Built the master data dimensions and transactional core datasets from scratch by analyzing multiple source systems, ensuring data consistency, integrity, and seamless integration
• Developed a common ingestion framework to efficiently process and load various file categories into ADLS, ensuring scalability and maintainability

• Project Overview: Self-service enterprise analytics on big data (Cloudera Hadoop).
• Client: Marks and Spencer
• Team Size: 10

Responsibility:

• Worked in devops mode for Big Data platform application built over Cloudera Hadoop
• Migrated data from legacy DB with help of sqoop as well as file systems with help of flume/nifi
• Ingest files and perform various automations with help of python scripts
• Self-service enterprise analytics on big data (Cloudera Hadoop)

• Project Overview: Java based application to calculate asset risk
• Client: Citi Bank
• Team Size: 5

Responsibility:

• Writing business logic by using POJO and IBM JRULES
• Maintaining code versions by using SVN
• Identifying Bugs and resolve them as quickly as possible
• Java based application to calculate asset risk