App Dev & Support Engineer
Hyderabad, Telangana, India
Summary
Set to lead application development and support initiatives for Conduent, focusing on enhancing operational efficiency and system reliability.
Lead Consultant
Hyderabad, Telangana, India
Summary
Led complex data engineering initiatives and implemented scalable solutions for diverse banking clients, driving significant improvements in data processing and governance.
Highlights
Designed and implemented scalable ETL pipelines processing over 10M records daily using Azure Databricks and Azure Data Factory, while optimizing Spark configurations to reduce execution time by 35%.
Led the migration of over 100 TB of Oracle data to Azure, enhancing scalability and achieving a 30% reduction in infrastructure costs for banking clients.
Architected and deployed Delta Lake solutions, enabling ACID transactions and enhancing data reliability, alongside developing PySpark transformation frameworks that boosted processing speed by 40%.
Developed Delta Live Tables (DLT) pipelines for automated data quality checks and comprehensive lineage tracking, and automated critical workflows using Databricks Jobs, reducing manual intervention by 60%.
Implemented Unity Catalog for centralized data governance and fine-grained access control, ensuring PCI DSS compliance through robust data masking and security policies.
Provided critical support for large-scale banking applications, managing high-volume transactional data, improving SQL performance by 25%, and reducing production defects by 20% through rigorous testing and impact analysis.
Microsoft Azure.
Azure Databricks, Apache Spark, PySpark, Delta Lake, Unity Catalog, Delta Live Tables (DLT), Workflows, Jobs API, Azure Data Factory (ADF).
Azure SQL, SQL Server, Oracle.
Python, SQL, PL/SQL.
ETL/ELT, Medallion Architecture, Data Modeling, Data Governance, Performance Optimization.