Senior Data Engineer - Azure / Databricks Job Title Senior Data Engineer - Azure /

Databricks Location:

Dallas, TX /

Remote Duration:

12+

Months Contract Experience:

6+

Years Job Description:

We are seeking a Senior Data Engineer with strong experience in designing, developing, and optimizing scalable cloud-based data platforms and ETL/ELT pipelines.

Responsibilities:

Design and develop scalable ETL/ELT data pipelines using Azure Data Factory, Databricks, PySpark and SQL. Develop data ingestion and transformation workflows for structured and semi-structured data. Build and optimize PySpark and Spark SQL jobs for large-scale distributed data processing. Develop data models, data marts, and analytical datasets for enterprise reporting and analytics. Work with Azure Data Lake Storage, Azure Synapse, Databricks and Snowflake . Implement batch and real-time data processing using Kafka or similar streaming technologies. Optimize SQL queries, Spark jobs, partitions, joins and transformations for performance. Implement data quality, validation, reconciliation and monitoring processes. Collaborate with data scientists, analysts, application developers and business stakeholders. Deploy data pipelines through CI/CD using Azure DevOps, Git and Terraform . Troubleshoot production pipeline failures and improve reliability and performance.

Required Skills:

6+ years of Data Engineering experience Strong SQL and Python/PySpark Azure Data Factory Azure Databricks Azure Data Lake / ADLS Apache Spark Data Warehousing ETL/ELT Azure Synapse or Snowflake Git / CI/CD Strong data modeling and performance tuning experience

Preferred:

Kafka Terraform Kubernetes Airflow Power BI AWS/Google Cloud Platform exposure