Senior Data Engineer Payments & Banking

Job Summary The Senior Data Engineer will design, develop, and maintain scalable batch and real-time data pipelines that support payment processing, card transactions, billing, and banking systems. The role includes developing ETL/ELT workflows, building streaming integrations, optimizing cloud-native solutions, and supporting enterprise data platforms. This position works closely with cross-functional teams in an Agile environment to ensure data quality, security, and compliance within regulated financial systems. Key Responsibilities Design, develop, and maintain batch and real-time data pipelines supporting financial transaction systems. Build and automate ETL/ELT workflows for ingestion, transformation, aggregation, and loading of high-volume financial data. Develop and support Kafka integrations including topics, producers, consumers, and streaming applications. Develop and optimize cloud-native data solutions in Azure using Azure Data Factory (ADF), Azure Synapse, Azure Data Lake, and Event Hubs. Build and manage Snowflake enterprise data platforms including data modeling, performance tuning, clustering, and secure data sharing. Develop ETL/ELT workflows using Python and SQL for ingestion, transformation, validation, and aggregation. Translate payment and banking data requirements into functional specifications, mapping documents, and technical designs. Ensure data accuracy, integrity, security, and compliance, including PCI-DSS and financial regulatory requirements. Optimize SQL queries and perform RDBMS tuning for high-throughput transactional environments. Design and maintain enterprise microservices (security, logging, APIs) using Java/Spring Boot when needed. Monitor data pipeline performance and implement corrective and optimization actions. Collaborate with product owners, architects, QA, and compliance teams in an Agile environment. Troubleshoot and resolve complex production issues across data platforms and transaction systems. Work on-site three days per week to support collaboration and Agile ceremonies. Minimal travel as required. Required Qualifications Bachelor's degree in Computer Science, Engineering, or related technical field. Minimum 5 years of experience in Data Engineering or Data Warehouse environments. Minimum 5 years of experience designing, building, and maintaining batch and real-time data pipelines. 5+ years of experience in payments, banking, or financial services supporting transaction-based systems. Strong expertise in SQL, PL/SQL, and query optimization. Hands-on experience with any ETL/ELT tool. Minimum 5 years of experience with relational databases such as Oracle, DB2, Teradata, or SQL Server. 5+ years of experience with Kafka or streaming platforms. Strong programming skills in Java and/or Python with at least 5 years of experience. 5+ years of experience developing API integrations with cloud or enterprise systems. 5+ years of experience with Unix Shell scripting. Understanding of data modeling concepts and ETL frameworks. Minimum 5 years of experience working in Agile environments. Preferred Qualifications Experience supporting PCI-DSS compliant environments. Experience working with large structured and unstructured datasets. Experience with CI/CD and DevOps practices. Prior experience working on high-volume payment gateways or banking transaction platforms.