Azure Databricks & Synapse Data Engineer -

PySpark, Spark SQL, Delta Lake Job Details Job Title:

Azure Databricks & Synapse Data Engineer Location:

Indianapolis, Indiana Work Arrangement:

Hybrid Onsite Requirement:

One day per week onsite

Residency Requirement:

Indiana residents only; local candidates only

Number of Openings:

1

Employment Type:

Contract Estimated Start Date:

November 2, 2026

Estimated End Date:

November 30, 2027

Contract Duration:

Through November 30, 2027

Potential Extension/Renewal:

Not Provided Submission Deadline:

October 28, 2026

Working Hours:

Not Provided Hours Per Week:

40 hours

Candidate Pay Rate:

$43.35 per hour

Education:

Master's degree, bachelor's degree with relevant experience, or qualifying hands-on data engineering experience, as detailed below

Background Check:

Not Provided Position Overview The Azure Databricks & Synapse Data Engineer designs, builds, and maintains secure, scalable data solutions across Azure Databricks and Azure Synapse Analytics. This role supports data integration, pipeline development, data modeling, governance, and documentation while collaborating with cross-functional teams to deliver high-quality, reliable data products. Responsibilities include developing batch and streaming pipelines, optimizing data warehouse workloads, enforcing data governance standards, gathering requirements, reviewing solutions, and providing technical support. The position also supports operational processes, contributes to policy and procedure improvements, and may provide mentorship or supervision based on agency needs. Education Candidates must meet one of the following qualification pathways: Master's degree in data science, analytics, informatics, computer science, mathematics, engineering, or a related field Bachelor's degree in a related field plus 2+ years of relevant experience 4+ years of hands-on experience in a similar data engineering role Relevant certifications and coursework may substitute for formal education or experience requirements. Required Skills / Experience The requisition specifies 4 years of experience for the technical qualifications listed below unless otherwise indicated

Databricks Development:

Developing with Databricks notebooks in PySpark, Spark SQL, and R, including transformations, joins, aggregations, and window functions

Delta Lake:

ACID transactions, MERGE upserts, schema enforcement and evolution, time travel, and table maintenance using

OPTIMIZE, Z-order, and VACUUM Lakehouse Architecture:

Medallion (bronze/silver/gold) architecture and incremental ingestion using

Auto Loader, Structured Streaming, or Lakeflow/DLT Unity Catalog:

Workspace catalogs, schemas, permissions, lineage, external locations, and storage credentials

Performance Optimization:

Right-sizing clusters and SQL warehouses and diagnosing partitioning, shuffle, skew, and caching issues using

Spark UI Workflow Orchestration:

Scheduling and managing dependencies with Azure Data Factory (ADF) pipelines and Databricks Workflows/Jobs, including Git folders (Repos) for source control

Azure Synapse Data Warehousing:

Designing MPP warehouse tables, including hash, round-robin, and replicated distribution, columnstore indexes, and partitioning ADLS Gen2

Querying:

Querying Parquet, Delta, and CSV files using OPENROWSET, external tables, and views

Dimensional Modeling:

Building star-schema facts, dimensions, stored procedures, and views, including SCD Type 1 and Type 2

Data Loading:

Developing with COPY INTO, PolyBase, and CETAS, and building parameterized Synapse/ADF pipelines using copy activities, data flows, and triggers

Warehouse Workload Management:

Workload management, resource classes, statistics, materialized views, result-set caching, and pool scaling or pausing

Azure Storage and Security:

ADLS Gen2 folder/file-format design using Parquet/Delta, Entra ID access control, managed identities, and

Key Vault CI/CD and Source Control:

Git and CI/CD in Azure DevOps or GitHub, including branching, pull requests, and deploying Synapse artifacts and

Databricks Asset Bundles Monitoring and Troubleshooting:

Monitoring pipelines and workloads with Azure Monitor/Log Analytics, setting up alerting, and troubleshooting failed jobs

Lakehouse Integration:

Using Microsoft lakehouse patterns, including interoperability and migration paths between

Synapse and Databricks Communication:

Strong written and verbal communication skills Preferred Skills / Experience Highly Desired Databricks Certified Data Engineer Associate certification; Databricks Certified Data Engineer Professional is a plus

Microsoft Certified:

Azure Data Engineer Associate (DP-203), if already held Desired 4 years of experience with Power BI connectivity to Synapse, including Import versus DirectQuery and understanding how data model design affects dashboard performance Job Responsibilities Assist with the data technology strategy, including resolving strategic data integration issues related to data quality, stewardship, real-time and event-based integration, and developing a unified vision for data integration Provide direction and oversight for designing and implementing data solutions for integration and synthesis across multiple secure systems Assist with defining, implementing, and maintaining data management policies and data engineering design standards Provide advice, training, and technical support for various projects Participate in requirements gathering, solution reviews, and explaining technical complexities to non-technical personnel Collaborate with solution leads, project managers, data architects, data scientists, and data analysts on solution design, architecture, and implementation Extract, transform, and load data from a wide variety of data sources using various data engineering tools and methods Design, build, and maintain batch and streaming data pipelines in Azure Databricks using PySpark, Spark SQL, and Delta Lake, following a medallion lakehouse architecture Develop, load, and tune dimensional data models and SQL workloads in Azure Synapse Analytics, including dedicated and serverless SQL pools, to support reporting and dashboard needs Orchestrate and monitor data workflows using Databricks Workflows/Jobs and Synapse or Azure Data Factory pipelines, including scheduling, alerting, and failure recovery Apply data governance and security controls, including Unity Catalog permissions, role-based access, row- and column-level security, and data masking, to protect sensitive health data (PHI/PII) in accordance with HIPAA and agency policy Document and demonstrate solutions through documentation, flowcharts, layouts, diagrams, charts, code comments, and clear code Conduct systems analysis and recommend changes to operational policies and procedures Perform other duties, responsibilities, and activities as assigned Work Schedule & Work Arrangement Hybrid schedule with onsite work one day per week 40 hours per week Specific daily working hours are not provided Residency & Location Requirements Candidates must currently be located in Indiana Local candidates only Onsite work is required one day per week in Indianapolis, Indiana Contract Information Contract start date: November 2, 2026 Expected end date: November 30, 2027 Candidate pay rate: $43.35 per hour Submission deadline: October 28, 2026 Ideal Candidate The strongest candidate is likely to be an experienced Azure data engineering professional with hands-on expertise in both Azure Databricks and Azure Synapse Analytics. The core of the role is building and optimizing production-grade data pipelines, implementing medallion lakehouse architecture, developing dimensional warehouse models, and managing secure, reliable data workflows. A particularly competitive candidate would demonstrate strong PySpark, Spark SQL, Delta Lake, Unity Catalog, Synapse

SQL, ADLS

Gen2, orchestration, and CI/CD capabilities. Experience protecting sensitive health information through appropriate access controls and governance practices is also important to the responsibilities of the position. Databricks certification and an existing Azure Data Engineer Associate certification are strong differentiators, while Power BI connectivity and dashboard performance knowledge are desirable. Candidates with only general data science, reporting, or dashboard development experience may be less aligned if they lack the hands-on data engineering, pipeline development, lakehouse, and Synapse warehouse skills emphasized in the requisition.

Important:

Indiana residency is mandatory, and candidates must be willing to work onsite one day per week.