Find Jobs
Find Jobs Near You – Available Work in Your Location
Lead Cloud Data Engineer
Choose a Location
This role is available in multiple locations. Pick one to apply.
Job Description
- _This position will follow our hybrid work model, we expect the selected candidate to be in office 2-3 days a week(10-12 days a month ) at the following office location: St. Petersburg, FL/ Memphis, TN/ Denver, CO/Southfield, MI_
- Responsibilities:
- + Design and develop scalable ETL pipelines using AWS Glue and PySpark.
+ Integrate data from various sources into Amazon Redshift, ensuring data quality and consistency.
+ Optimize the performance of data processing jobs and Redshift queries.
+ Deploy infrastructure as code (IaC) with Terraform
+ Monitor and troubleshoot data pipelines to ensure reliability and efficiency.
+ Collaborate with data scientists, analysts, and other stakeholders to understand data requirements.
+ Ensure compliance with security policies and regulations.
+ Document data workflows and best practices.
+ Mentor and guide junior engineers on the team.
+ Continuously explore new AWS services and technologies to improve existing solutions.
- Skills:
- + Expert level experience with AWS Cloud Services is preferred, knowledge of other Cloud Providers is a plus + Experience and understanding of AWS Cloud Services components like Redshift, S3, Kafka, Glue, AirFlow, SageMaker, EMR, Athena, Lake Formation, DynamoDB, Aurora, VPC, IAM and many other Cloud Service modules.
+ Working experience in Python is preferred, Java and other languages are a plus
+ Nice to have Terraform or other Infrastructure as Code experience
+ Knowledge and experience with Analytics, Data Warehouse and Big Data concepts across both batch and streaming patterns
+ Solid understanding of networking, infrastructure, cost management, SRE concepts
+ Creating patterns for adoption from scratch and being able to document and articulate coding and architecture standards for others to follow.
+ Performance tuning and optimization of code and infrastructure
+ Creating a culture of continuous improvement, performs code reviews and makes recommendations for improvement or streamlining patterns
+ Relevant experience in various database platforms, ETL solutions/products, Infrastructure, ETL architecture and On-Prem or Cloud Services.
+ Financial Services Industry knowledge is a plus.
- Ability to:
- + Identify and understand issues, problems and opportunities for automation.
+ Good communication and organization skills that clearly convey information and ideas through a variety of media to individuals or groups in a manner that engages the audience and helps them understand and retain the message.
+ Use effective approaches for choosing a course of action or developing appropriate solutions; recommend or take action that is consistent with available facts, constraints and probable consequences.
+ Demonstrate a satisfactory level of technical and professional skill or knowledge in position-related areas; remains current with developments and trends in areas of expertise.
+ Develop and use collaborative relationships to facilitate the accomplishment of work goals.
+ Make internal and external clients and their needs a primary focus of actions; develop and sustain productive client relationships.
- Certifications
- Preferred
- , not required
- AWS Certified Solutions Architect
- AWS Certified Big Data
- AWS Certified Data Analytics
- AWS Certified Developer
- AWS Certified Machine Learning