Summary
Overview
Work History
Education
Skills
Personal Information
Timeline
Generic

Nidhin Krishnan

Bathgate,WLN

Summary

Seasoned Data Engineer with a proven track record at NatWest Group, adept in IBM InfoSphere Datastage and CDC, showcasing expertise in data integration and migration. Excelled in agile environments, enhancing data security and system efficiency. Demonstrates strong analytical skills and a collaborative approach, significantly contributing to near real-time transaction processing and push notification systems.

Overview

20
20
years of professional experience

Work History

Data Engineer

12.2022 - 11.2024
  • Confluent Kafka Connectors
  • Technology: AWS, Confluent Kafka Connect, Kubernetes, Terraform, Confluent Kafka Connectors, Kubernetes, Terraform, GitLab, Grafana, Control Center, Ubuntu Linux, WSL, Confluence, Jira, ServiceNow
  • Responsibilities:
  • Developing Kafka connectors using Confluent (CFK)
  • Completed PoC and set-up for migrating CDC Target data to AWS on Confluent Kafka
  • Deployed Confluent Kafka Connectors using Terraform
  • Set up and managed Docker containers, running Kafka Connector applications, for deployment into Kubernetes Pods
  • Developed and tested SQL Server/PostgreSQL, MongoDB Source Connectors and Snowflake, MongoDB Sink Connectors successfully and moved to production
  • Deployed Grafana dashboards for connector monitoring
  • Design Documentation created on Confluence
  • Attend daily scrum call and managed the sprint planning & Backlog refinement

Data Engineer

NatWest Group
05.2021 - 11.2024
  • IBM CDC Developer and Admin
  • National Westminster Bank, commonly known as NatWest is a major retail and commercial bank in the United Kingdom
  • Project majorly based on Near Real time transactions for Debit, Credit cards and Push Notifications
  • Technology: IBM CDC 11.4.3, Redhat Linux, Tivoli, Splunk, Confluent Kafka, Cloudera Kafka, Bitbucket GitOps, GitLab, SMM, Control Center, Grafana, Confluence, Jira ServiceNow
  • Responsibilities:
  • Installing/Upgrade IBM CDC for DB2 and Oracle on Linux server in Production and Non-Production environment
  • Creating Target datastores, Mapping tables, user access control
  • Enabling LDAP on IBM CDC access server
  • TLS encryption for target and source datastore for security
  • Creation of Autosys Jobs and testing different scripts
  • Involved in Target Migration Project and CDC Migration to New info scale cluster
  • Successfully installed IBM CDC oracle source and Target engines for Oracle use cases
  • Deployed and tested CHCCLP script on CDC server for Dashboard monitoring
  • Installed Splunk and configure CDC log path for real time monitoring
  • Completed PoC and set-up for migrating CDC Target data onto Confluent Kafka in AWS and onto Confluent Kafka on-perm for Payments
  • Design Documentation created on Confluence
  • Attend daily scrum call and managed sprint planning & Backlog refinement.

Agile BI Developer

National Grid PLC
08.2018 - 12.2019
  • Project: Asset Management Transformation Programme (Richmond) – National Grid
  • Technology: IBM InfoSphere Datastage 11.7 (Parallel Edition), IBM CDC 11.4, IBM AIX, IBM Data Architect, IGC, Oracle 12c, SQL, SQL Developer, Python, UNIX Shell Scripting, Putty, SVN, Tableau, Servicenow, Ellipse 8.0 (Enterprise Asset Management), Fieldreach, VSTS, SAFe (Scaled Agile Framework)
  • Responsibilities:
  • Attended Programme Increment meetings and subsequent sprint planning
  • Written User stories and its acceptance criteria based on requirements and business value
  • Attending Daily Stand-up in front of Scrum Board and updated the progress
  • Created an Atomic warehouse model for Data Lake
  • Prepared the mapping document for BO reporting from Atomic warehouse model
  • Prepared HLD document for GT Data Model into Data Lake
  • Prepared Detailed Application Design document for ETL
  • Designed and created schemas, tables, views and index view
  • Designed the ETL framework
  • Agile ETL Development and Testing
  • Creating complex views and writing sql query
  • Created the deployment guide
  • Live Implementation

Agile BI Developer

National Grid
04.2018 - 08.2018
  • Project: Data Lake support –, Technology: IBM InfoSphere Datastage 11.5/11.7 (Parallel Edition), IBM CDC 11.4, IBM AIX, Oracle 12c, SQL, Putty, SVN, Servicenow, Ellipse 8.0 (Enterprise Asset Management), Fieldreach
  • Responsibilities:
  • Upgrading the IBM InfoSphere suite from 11.5 to 11.7
  • Data Lake Big table maintenance, enhancements and deployments
  • Data Lake Production support

Agile BI Developer

National Grid
10.2017 - 04.2018
  • Project: GT Portfolio Planning –, Technology: IBM InfoSphere Datastage 11.5 (Parallel Edition), IBM AIX, UNIX Shell Scripting, Oracle 12c, SFTP, Putty, SVN, JIRA, Tableau, Servicenow
  • Responsibilities:
  • Agile ETL Development and Testing
  • Modeled the tactical reporting table and its mapping
  • Designed and created schemas, tables, views and index view
  • Designed the ETL jobs
  • Creating complex views and writing sql query
  • Created the Technical and functional design doc
  • Created the deployment guide
  • Live Implementation

Agile BI Developer

National Grid
08.2017 - 04.2018
  • Project: GT Process Safety Reporting –, Technology: IBM InfoSphere Datastage 11.5 (Parallel Edition), IBM CDC, IBM AIX, UNIX Shell Scripting, Oracle 12c, SQL, SFTP, Putty, SVN, JIRA, Tableau, Servicenow
  • Responsibilities:
  • Agile Data Lake Development and Testing
  • Modeled the Operational History and Big Tables for the Data Lake
  • Designed and created schemas, tables, views and index view
  • Designed the ETL framework
  • Creating complex views and writing sql query
  • Created the Technical and functional design doc
  • Created the deployment guide
  • Live Implementation
  • For LBG
  • DataStage SME

ETL
12.2015 - 02.2016

ADC Insurance
09.2015 - 01.2016
  • (AriesMI) – Lloyds Banking Group
  • Technology: IBM InfoSphere Datastage 9.1 (Parallel Edition), XML Stage, Tivoli, IBM AIX, Mainframe, UNIX Shell Scripting, SQL Server 2014, Connect Direct, ClearCase, XMLSPY, ALM, BMC Remedy
  • Responsibilities:
  • Impact Assessment, Estimation, High Level Design(MLPD), Low Level Design(DPAD), Development and Test support, Live Implementation and Runs
  • Performance analysis and tuning of the ETL jobs
  • Writing sql query for analysis
  • Responsible for the technical delivery of the project
  • Onshore-Offshore coordination
  • Operational Level Agreement
  • SOE
  • Datastage SME-, Project: BASELL III – Lloyds Banking Group
  • Technology: IBM InfoSphere Datastage 9.1 (Parallel Edition), Teradata 14, SQL Server 2014, Tivoli, IBM AIX, Mainframe, UNIX Shell Scripting, Connect Direct, BMC Remedy, ClearCase, Rational Software Architect
  • Responsibilities:
  • Impact Assessment, Estimation, Low Level Design Review(DPAD), Development and Test support, Live Implementation and Executions
  • Performance analysis and tuning of the ETL jobs
  • End-to-End Reconciliation
  • Operational Level Agreement
  • Responsible for the technical delivery of the project
  • Onshore-Offshore coordination
  • Warranty support
  • For Ricoh

ETL Consultant

British Telecom
09.2014 - 06.2015
  • Project: Eu Commissions – Ricoh Europe
  • Technology: Ascential Datastage 7.5.2 (Parallel Edition), IBM AIX, PL/SQL, SQL Oracle 10g, SQL Server, IBM Rational ClearQuest
  • Responsibilities:
  • Impact Assessment, Estimation, Solution Design, Data Analysis, ETL Development, Maintenance and Test support, Live Implementation and Executions
  • Designed and created schemas, tables, views and indexes
  • Creating complex views and writing sql query
  • Performance tuning of the ETL jobs and SQL
  • Responsible for the technical delivery of the project
  • Producing Deployment guide

MDM– Solution Architect

Wipro Technologies/LBG
02.2014 - 07.2014
  • Project: ICT&D – City of Edinburgh Council
  • Technology: IBM InfoSphere Master Data Management V11, IBM WebSphere MQ, CloverETL, Solaris, Windows Server 2008 R2, SQL Server
  • Responsibilities:
  • E2ED Design for MDM infrastructure part
  • MDM Component design
  • Scheduling doc for MDM
  • Built the Staging Area hub design
  • For

SME – Datawarehouse Consultant

IBM
09.2013 - 12.2013
  • Project: RCB-TOM – Lloyds Banking Group
  • Technology:, InfoSphere Datastage 9.1 (Parallel Edition), XMLSPY, Tivoli, IBM AIX, Mainframe, UNIX Shell Scripting, Teradata 14, SQL Server, ClearCase
  • Responsibilities:
  • Requirement Gathering, Impact Assessment, Estimation, High Level Design(MLPD), Data Analysis, Mapping, Low Level Design Review(DPAD), ETL Development and Test support
  • Responsible for the technical delivery of the project

SME – Datawarehouse Consultant

LeMans – Lloyds
12.2012 - 09.2013
  • Banking Group
  • Technology: InfoSphere Datastage 8.1 (Parallel Edition), Tivoli, IBM AIX, Mainframe, UNIX Shell Scripting, Teradata 14, Connect Direct, HP SM7, ClearCase
  • Responsibilities:
  • Requirement Gathering, Impact Assessment, Estimation, High Level Design(MLPD), Data Analysis, Mapping, Low Level Design Review(DPAD), ETL Development and Test support, Live Implementation and Runs
  • Creating complex views and writing sql query
  • Responsible for the technical delivery of the project
  • Warranty support

SME – Datawarehouse Consultant

Lloyds Bank
09.2011 - 09.2013
  • TSB Bank
  • Technology: InfoSphere Datastage 7.5.3/8.1 (Parallel Edition), Tivoli, IBM AIX, Mainframe, UNIX Shell Scripting, Teradata 13/14, Connect Direct, HP SM7, ClearCase
  • Responsibilities:
  • Requirement Gathering, Impact Assessment, Estimation, High Level Design(MLPD), Low Level Design Review(DPAD), Data Analysis, ETL Development and Test support, Live Implementation and Runs
  • Creating complex views and writing sql query
  • Responsible for the technical delivery of the project
  • Warranty support

SME – Datawarehouse Consultant

01.2012 - 04.2013
  • Project: SIRA Unsecured – Lloyds Banking Group
  • Technology: InfoSphere Datastage 8.1 (Parallel Edition), Tivoli, IBM AIX, Mainframe, UNIX Shell Scripting, Teradata 13, Connect Direct, HP SM7, ClearCase
  • Responsibilities:
  • Requirement Gathering, Impact Assessment, Estimation, High Level Design(MLPD), Low Level Design Review(DPAD),Data Analysis, ETL Development and Test support, Live Implementation and Runs
  • Responsible for the technical delivery of the project
  • Warranty support

SME – Datawarehouse Consultant

01.2012 - 05.2012
  • Project: FATCA – Lloyds Banking Group
  • Technology: InfoSphere Datastage 8.1/9.1 (Parallel Edition), Tivoli, DB2, IBM AIX, Mainframe, UNIX Shell Scripting, Connect Direct
  • Responsibilities:
  • Requirement Gathering, Impact Assessment, Estimation, High Level Design(MLPD), Mapping, POC
  • Responsible for the technical delivery of the project

Tech Lead

Datawarehouse
02.2010 - 08.2011
  • Project: Integration Programme - (LloydsTSB and HBOS Merger)
  • Technology: InfoSphere Datastage 7.5.3/8.1 (Parallel Edition),XML,Tivoli, IBM AIX, Mainframe, UNIX Shell Scripting, Teradata, Oracle, DB2, SQL, Connect Direct, HP SM7, ClearCase
  • Responsibilities:
  • Requirement Gathering, Impact Assessment, Estimation, High Level Design(MLPD), Low Level Design(DPAD), ETL Development and Test support, Live Implementation and Runs
  • Performance tuning of the ETL jobs and SQL
  • Creating complex views and writing sql query
  • Responsible for the technical delivery of the project
  • Worked on AML[Anti Money Laundering], Charter [Complaints Handling System], HR Integration, HMRC, Payroll & Bonus

ETL Analyst

Datawarehouse
12.2007 - 01.2010
  • Starlight Vision-Plus Credit Card Programme – LloydsTSB
  • Technology: InfoSphere Datastage 7.5.3 (Parallel Edition), Tivoli, IBM AIX, Mainframe, ERWIN UNIX Shell Scripting, Teradata, SQL, Connect Direct, C++, HP SM7, ClearCase, FS-LDM
  • Responsibilities:
  • Customization of FSLDM for Credit Cards
  • Requirement Gathering, Impact Assessment, Estimation for Change requirement, Low Level Design(DPAD), ETL Development and Test support, Live Implementation and Runs
  • Creating complex views and writing sql query
  • Responsible for the technical delivery of the project
  • Warranty support
  • Implementing Retail, Commercial Banking, Credit Card, Marketing, Brokerage and Insurance in a single integrated model and ties together information about financial services products, accounts, customers, channels, campaigns and events (transactions)

Programmer

ASM TECHNOLOGIES LIMITED
05.2007 - 11.2007
  • Sapphire Integration – Applied Materials, )
  • Technology: SQL, PL/SQL, Oracle 9i/10g, MKS Toolkit 6.2, Remedy, C++
  • Responsibilities:
  • Designed and created schemas, tables and indexes
  • Creating complex views and writing sql query
  • Design and Writing Stored Procedures/Triggers
  • Improving SQL Query Performance
  • Prepared the Installation and user guide documents
  • Implementation of the project in live

Programmer

Fidelity Business Service India Pvt Ltd
09.2006 - 05.2007
  • Project: PMP/CEDX – Fidelity, Technology: SQL, PL/SQL, Oracle 9i/10g, TOAD 8.5.1, SQL
  • Loader, UNIX Shell Scripts, Sun Solaris, ASP.NET, Peregrine, Erwin
  • Responsibilities:
  • Design and Writing Stored Procedures/Triggers, Control Files and Shell Scripts
  • Improving SQL Query Performance
  • Analyzing and fixing ST defects
  • Prepared the Deployment document
  • Implementation of the project in live

Programmer

MPHASIS BFL LIMITED, Reuters
London
05.2006 - 09.2006
  • Design and Development of ETL jobs
  • Analyzing and fixing ST defects
  • Support and Maintenance of ETL Environment
  • Improving SQL Query Performance
  • Prepared the Deployment and user guide documents
  • Involved in implementation of the project in live

Programmer

Integration – JPMC
New York
06.2005 - 04.2006
  • SQL, PL/SQL, Oracle 9i, TOAD, SQL
  • Loader, UNIX Shell Scripts, Sun Solaris
  • Responsibilities:
  • Requirement Analysis, Design, Writing Stored Procedures/Triggers and Control Files
  • Writing UNIX Shell Scripts
  • Analyzed and fixed ST/SIT defects
  • Improving Query Performance by rewriting the problematic SQL query
  • Prepared the Deployment document
  • Involved in implementation of the project in live

Associate Programmer

MWDW – VISY
Melbourne
09.2004 - 05.2005
  • Installation of Oracle Warehouse Builder 9.2, Oracle 9i Data base server and created customized database
  • Created Schema and granted roles, profiles
  • Imported schema objects and data from dump file
  • Created tablespaces, tables and Materialized View
  • Writing Stored Procedures and Triggers to process the data
  • Analyzing tables, created indexes
  • Moving Data from OLTP system to Staging and from Staging to ODS through PL/SQL scripts
  • Using DBMS Package created Job Scheduling to refresh ODS
  • Prepared the Deployment document
  • Qualifications:

Education

M Sc - Computer Science

Kannur University
Kannur, Kerala, India
01.2003

B Sc - Physics

Kannur University
Kannur, Kerala, India
01.2001

Skills

  • C
  • C
  • ASPNET
  • JAVA
  • J2EE
  • SQL
  • PL/SQL
  • UNIX Shell Scripting
  • UNIX(IBM-AIX,SUN OS,HP)
  • Redhat Linux
  • MS Windows XP/NT/2003/2000/9X
  • RDBMS Oracle8i/9i/10g/11g/12c,19c
  • Teradata 13/14
  • MS SQL Server 80/120
  • DB2
  • PostgreSql
  • TOAD
  • PL/SQL Developer
  • SQL Developer
  • Loader
  • MKS Toolkit 62
  • Erwin
  • Mainframe Endevor
  • ISPF
  • TSO
  • Tivoli Scheduler
  • Connect Direct
  • SFTP
  • Microsoft Office
  • Microsoft Visio
  • IBM Rational Software Architect
  • Splunk
  • Bitbucket GitOps
  • VSS
  • SVN
  • ClearCase
  • SharePoint
  • IBM Data Architect
  • Ascential Data Stage 70 (Server)/752(PX)/753(PX)/IBM InfoSphere Datastage 81/91/115/117 (Parallel Edition)
  • Informatica 63/90
  • IBM InfoSphere Information Analyser
  • IBM InfoSphere Metadata Workbench
  • IBM InfoSphere Master Data Management V11
  • IGC
  • Talend v53
  • Spark
  • Scala
  • Python
  • AWS
  • Confluent Kafka Connectors
  • Kubernetes
  • Terraform Confluent Kafka
  • Cloudera Kafka
  • SMM
  • Control Center
  • Grafana
  • GitLab
  • Ubuntu Linux
  • WSL
  • IBM IIDR CDC 1143
  • Cognos 72
  • Tableau
  • VSTS
  • HP Quality Center
  • ALM
  • IBM Rational ClearQuest
  • JIRA
  • HP SM7
  • I2MP
  • BMC REMEDY IT Service Management
  • Servicenow
  • Banking and Finance
  • Energy & Utilities
  • Retail
  • Semi-conductor and Manufacturing
  • Agile(Scrum)
  • SAFe (Scaled Agile Framework)
  • Waterfall
  • ETL
  • Data Warehousing & Data Lake
  • Data Integration
  • Data Migration
  • Data Ingestion

Personal Information

Available: Immediate

Timeline

Data Engineer

12.2022 - 11.2024

Data Engineer

NatWest Group
05.2021 - 11.2024

Agile BI Developer

National Grid PLC
08.2018 - 12.2019

Agile BI Developer

National Grid
04.2018 - 08.2018

Agile BI Developer

National Grid
10.2017 - 04.2018

Agile BI Developer

National Grid
08.2017 - 04.2018

ETL
12.2015 - 02.2016

ADC Insurance
09.2015 - 01.2016

ETL Consultant

British Telecom
09.2014 - 06.2015

MDM– Solution Architect

Wipro Technologies/LBG
02.2014 - 07.2014

SME – Datawarehouse Consultant

IBM
09.2013 - 12.2013

SME – Datawarehouse Consultant

LeMans – Lloyds
12.2012 - 09.2013

SME – Datawarehouse Consultant

01.2012 - 04.2013

SME – Datawarehouse Consultant

01.2012 - 05.2012

SME – Datawarehouse Consultant

Lloyds Bank
09.2011 - 09.2013

Tech Lead

Datawarehouse
02.2010 - 08.2011

ETL Analyst

Datawarehouse
12.2007 - 01.2010

Programmer

ASM TECHNOLOGIES LIMITED
05.2007 - 11.2007

Programmer

Fidelity Business Service India Pvt Ltd
09.2006 - 05.2007

Programmer

MPHASIS BFL LIMITED, Reuters
05.2006 - 09.2006

Programmer

Integration – JPMC
06.2005 - 04.2006

Associate Programmer

MWDW – VISY
09.2004 - 05.2005

M Sc - Computer Science

Kannur University

B Sc - Physics

Kannur University

Similar Profiles

CREATE PROFILE
Nidhin Krishnan