Pavani Akkala
Summary
Senior Data Engineer with 17+ years of experience designing and delivering scalable data platforms, specialising in both batch and real-time streaming solutions. Skilled in Kafka, KSQL, StreamSets, AWS, Python, Groovy, SQL, MongoDB, with expertise in data architecture, data warehousing and pipeline automation. AWS Certified Data Engineer – Associate, focused on building reliable, high-performance data solutions that enable business growth and data-driven decision-making.
Overview
18
18
years of professional experience
Work history
Senior Data Engineer
EDGS FSCS Reporting- Rooster Money(Regulatory)
01.2026 - Current
- Configured and validated connectivity between Rooster Data Products ,Party Master Data Product and Informatica IDMC to enable enterprise data ingestion into EDI.
- Designed Proof of concept(PoC) ingestion pipelines in IDMC to assess feasibility and establish reusable ingestion from new data products.
- Participated in solution design workshops, contributing to both tactical and strategic architecture decisions for EDGS-Rooster.
- Worked closely with data product owners to analyse schemas, metadata, and ingestion requirements for new data products.
- Contributed to project planning by providing build estimates, assessing technical effort, identifying delivery dependencies, and supporting sprint planning.
Senior Data Engineer
CDD/EBA Alerting(Regulatory)
09.2020 - 12.2025
- Architected and supported near real-time streaming solutions using CDC, Kafka, KSQL and StreamSets, delivering low-latency data processing for payment, regulatory compliance and operational alerting use cases.
- Developed Python automation using StreamSets SDK and GitLab CI/CD to standardise deployments, improve operational efficiency and reduce manual configuration effort.
- Led end-to-end delivery of data engineering solutions from requirements gathering through implementation, testing and production deployment.
- Led migration of Kafka and StreamSets workloads from on-premise infrastructure to AWS cloud environments, improving scalability, resilience and platform performance.
- Implemented monitoring, alerting and troubleshooting capabilities to improve pipeline reliability, data quality and proactive incident detection.
- Collaborated with platform engineering teams to deliver Kafka cloud migration initiatives and resolve complex production issues.
- Worked closely with business stakeholders to define requirements, produce data mapping specifications and deliver scalable data solutions supporting operational analytics.
- Produced technical documentation, sprint demonstrations and implementation artefacts while mentoring engineers on streaming technologies, automation and engineering best practices.
Data Engineer
Enterprise Data Hub (EDH)
05.2017 - 09.2020
- Led automation of enterprise data sourcing processes, replacing manual extraction with scalable, reusable data pipelines.
- Designed dimensional and analytical data models supporting enterprise reporting and analytics.
- Worked closely with business stakeholders to translate reporting and analytical requirements into scalable data engineering solutions.
- Built and optimised end-to-end ETL pipelines that improved quality, consistency and availability of data from multiple business domains.
- Mentored junior analysts and engineers in SQL, data modelling and data engineering principles, helping establish consistent engineering standards across team.
Lead Test Analyst
Enterprise Data Warehouse (EDW)
12.2015 - 04.2017
- Gathered and consolidated data from multiple sources to support analysis and solution delivery. Collaborated with stakeholders and SMEs to validate requirements and identify optimal data sources.
- Actively participated in Agile ceremonies, ensuring solutions were delivered on time, within budget, and to quality standards.
- Provided guidance to team members, liaised with developers for business improvements, and prepared test plans for project estimation.
- Executed comprehensive data validation across multiple source and target systems to ensure data accuracy, integrity, and consistency throughout ETL processes.
Senior Test Analyst / Test Analyst
Oracle India Pvt Ltd/ Tata Consultancy Services
Bangalore, India
09.2008 - 02.2015
- Developed a BDD automation framework using Cucumber and Selenium.
- Translated application requirements into clear and actionable user stories.
- Participated in Scrum ceremonies including daily stand-ups, project status updates, retrospectives, and iteration reviews.
- Maintained test environments, managed builds, and prepared test data.
- Created test plans and prepared daily status reports to ensure smooth project progress and effective communication.
Skills
- Data Engineering & Streaming : Kafka, KSQL, StreamSets, Snowflake, Informatica, IDMC, Airflow, CDC
- Cloud : AWS (S3, IAM), Cloud Security Controls
- Programming : Python, Groovy, SQL
- Databases : Oracle, SQL Server, MongoDB, Teradata
- Data Warehousing & Analytics: Dimensional Data Modelling, Data Warehouse Design, Data Governance, Agile, Data Lake Architecture, Data Products
- Engineering Practices: Gitlab CI/CD, Monitoring & Alerting, Pipeline Automation
- Soft Skills : Stakeholder management , Cross-functional Collaboration, Technical Mentoring
- Certifications : AWS Certified Data Engineer Associate (DEA- C01)
Key Achievements
- Delivered the bank's first KSQL-based event processing platform for inbound payments, working closely with the platform engineering team to implement a production-ready solution enabling real-time notifications for Inbound Payments.
- Automated deployment of streaming pipelines using Python and GitLab CI/CD, coordinating cross-team adoption of shared GitLab runners and implementing the deployment framework in production, improving deployment consistency and reducing manual effort.
- Increased platform scalability to support more than 40% growth in payment transaction volumes without performance degradation.
- Improved operational reliability by implementing proactive monitoring and alerting to detect alert data loss, reducing downtime and accelerating issue resolution.
Timeline
Senior Data Engineer
EDGS FSCS Reporting- Rooster Money(Regulatory)
01.2026 - Current
Senior Data Engineer
CDD/EBA Alerting(Regulatory)
09.2020 - 12.2025
Data Engineer
Enterprise Data Hub (EDH)
05.2017 - 09.2020
Lead Test Analyst
Enterprise Data Warehouse (EDW)
12.2015 - 04.2017
Senior Test Analyst / Test Analyst
Oracle India Pvt Ltd/ Tata Consultancy Services
09.2008 - 02.2015