Data Engineer

Position OverviewWe are seeking a talented Data Engineer with strong experience in Python, AWS, and Databricks to design and build scalable data pipelines and modern data platforms. The ideal candidate will help develop and maintain data infrastructure that supports analytics, machine learning, and business intelligence initiatives. This role requires hands-on experience working with large datasets, cloud-native architectures, and distributed data processing frameworks.

Key ResponsibilitiesDesign, build, and maintain scalable data pipelines and ETL/ELT workflows using Python and cloud technologies.

Develop and optimize data solutions using AWS services and Databricks.

Build and manage data lakes and data warehouses for structured and unstructured data.

Implement data transformation and processing pipelines using Apache Spark within Databricks.

Integrate data from multiple sources including APIs, databases, and streaming systems.

Ensure data quality, governance, security, and compliance across the data platform.

Monitor pipeline performance and troubleshoot data pipeline failures or latency issues.

Collaborate with data analysts, data scientists, and business stakeholders to deliver reliable datasets.

Optimize storage and compute costs within the AWS ecosystem.