KAILASH BALACHANDIRAN

Passionate Data Engineer with experience in building and optimizing data pipelines, ETL workflows, and real-time analytics. Skilled in SQL, Python, PySpark, and Azure services like Data Factory, Databricks, and Microsoft Fabric to process and manage big data efficiently. Experienced in Power BI for interactive dashboards and automated reporting. Focused on solving complex data challenges, improving performance, and delivering reliable insights for better decision-making.

AWS Certified Solutions Architect – Associate

Amazon

• Gained a deep understanding of designing and deploying scalable, highly available, and fault-tolerant systems on AWS.
• Learned best practices for implementing secure and robust applications using AWS services such as EC2, S3, RDS, Lambda, and VPC.

Python Programming

Huawei Technologies

• Gained experience with Python libraries and frameworks such as Pandas, NumPy, and Flask.
• Acquired skills in debugging, testing with unit test and pytest, and optimizing Python code for various applications.

Java Programming

Udemy

• Developed a strong foundation in Java programming, including object-oriented principles, data structures, and algorithms.
• Learned to implement robust and scalable backend systems, working with databases like MySQL and PostgreSQL, and developing RESTful APIs.

Data Analytics

Coursera

• Completed a Data Analytics certification from Coursera, gaining hands-on experience in data visualization, statistical analysis, and business intelligence tools.
• Worked on real-world datasets using SQL, Python, and Power BI, applying data cleaning, transformation, and dashboard creation techniques to derive actionable insights.

Adventure Works Data Engineering Project

• Designed and implemented ETL workflows to automate data ingestion from multiple sources into Azure Data Lake Gen2.
• Built an automated ETL pipeline to move raw data into Azure Data Lake Gen2, ensuring seamless data ingestion and processing.
• Organized data into Bronze, Silver, and Gold layers, using Parquet format for efficient storage and faster queries.
• Processed and transformed large datasets using Apache Spark & PySpark, handling data cleaning, aggregation, and performance optimization.
• Created external tables and views, improving query performance through partitioning and indexing, enabling efficient data analysis.
• Connected Synapse Analytics with Power BI, designing interactive dashboards and using DAX & Power Query to present meaningful insights.

Netflix Azure Data Engineering Project

• Implemented Autoloader for efficient incremental data ingestion and leveraged Delta Live Tables to automate data transformations.
• Orchestrated the end-to-end data pipeline using Azure Data Factory, integrating version control with GitHub for CI/CD deployment.
• Managed raw, transformed, and curated datasets in a multi-layer architecture (Bronze, Silver, Gold) to optimize data storage and retrieval using Azure Data Lake Storage Gen 2.
• Designed a data warehouse with Star Schema modeling, creating optimized queries and external tables for seamless analytics using Azure Synapse Analytics.
• Connected Synapse Analytics to Power BI, building interactive dashboards that provide real-time insights and business intelligence.

Deploying an Internet of Things (IoT) sensor data in the edge and cloud setting and able to develop the machine learning-based IoT data processing pipeline.

• Designed a data injector component by leveraging Newcastle Urban Observatory IoT data streams.
• Developed a data injector component with the following functions (Code) in Azure Lab (Edge) or the Azure Lab localhost.
• Designed a Data pre-processing operator in a Docker compose file which contains the following necessary configurations and instructions for deploying and instantiating the following set of Docker images on Azure lab (Cloud).
• Designed a data pre-processing operator with the following functions(code) in Azure Lab(Edge).
• Build a Docker file to migrate the "pre-processing data operator" source code into a Docker image and then modify the docker-compose file to run it as a container locally on the Azure lab (Edge).
• Developed a Time-series data prediction and visualization using Machine Learning code.
• Designed a PM2.5 data prediction operator with the following functions(code) in Azure Lab(Cloud) or the Azure Lab localhost.

Deployed Docker-based application hosting environment and programming and deployed cloud infrastructures using

Terraform.

• Deployed a complex web application component in Docker Environment.
• Created a web application topology, on a single Docker swarm node using the Docker compose configurations. Built my own Docker image and pushed it to the Docker Hub.
• Fully deployed and ran the complex web application stack and undertook performance benchmarking activities.
• Deployed Kubernetes using Terraform and deployed a microservice.