Compare your current skills to what this opportunity needs—we'll show you what you already have and what could strengthen your application.
Job Description
Data Engineer The Data Engineer will play a critical implementation role on the Data Engineering and Data Products team and be responsible for data pipeline solutions design and development, troubleshooting, and optimization tuning on the next generation data and analytics platform being developed with leading edge big data technologies in a highly secure cloud infrastructure. The Data Engineer will serve as a liaison to platform user groups ensuring successful implementation of capabilities on the new platform. Data Engineer Responsibilities Deliver end-to-end data and analytics capabilities, including data modelling, data ingestion, data transformation in collaboration with Data and Analytics stakeholder groups Develop scalable and fault-tolerant workflows Clearly document issues, solutions, findings and recommendations to be shared internally & externally Learn and apply tools and technologies proficiently, including:
Languages:
Python, PySpark, ANSI SQL, Python ML libraries
Frameworks/Platform:
Spark, Snowflake, Airflow, Kafka Cloud Computing:
AWS Analyze client data & systems to determine whether requirements can be met Effectively use AI capabilities to build, debug, improve productivity and deliver faster. Test and validate data pipelines, transformations, datasets, reports built by team Candidate Requirements Expert experience in the following Should have: - SQL, Python, PySpark, Python ML libraries. Data and analytics technologies including SQL/NoSQL/Graph databases, ETL Knowledge of CI/CD and related tools such as Gitlab, AWS services including EMR, Glue, Athena, Batch, Lambda CloudWatch, DynamoDB, EC2, CloudFormation, IAM Snowflake and Airflow. DBT Canonical Data Models and Medallion Architecture. Solid scripting skills (e.g., bash/shell scripts, Python) Candidate should be able to gather and interpret business requirements Experience with agile delivery methodologies using Jira or similar tools Experience working with remote teams Expert experience in the following Good to have: - AI assisted coding using GitHub CoPilot, AmazonQ/Kero etc. BI Tools AWS CodeCommit Exposure to tools like Estuary(Data Integration), AtScale AWS Solutions Architect / Developer / Data Analytics Specialty certifications, Professional certification is a plus Qualification Bachelor s degree in computer science relevant field, master s degree is a plus