Databricks Engineer at Tredence Inc. Databricks Engineer at Tredence Inc. in Wood Dale, Illinois Posted in 1 day ago.

Type:

full-time Position Summary We are seeking an experienced Databricks Lead Engineer / Technical Lead to drive large-scale data platform modernization initiatives. The ideal candidate will possess deep expertise in Databricks Lakehouse architecture , Delta Lake , Databricks Workflows , and hands-on experience migrating enterprise data platforms from Azure Synapse and SQL Server to Databricks . This role requires strong technical leadership, client-facing communication skills, and the ability to guide cross-functional teams through complex data transformation projects. Key Responsibilities Lead the design, development, and implementation of enterprise-scale data solutions using Databricks Lakehouse Platform . Design and build scalable data pipelines utilizing Databricks Workflows , Delta Lake, and Spark-based processing frameworks. Drive end-to-end migration of data warehouses, ETL pipelines, and reporting workloads from Azure Synapse and SQL Server to Databricks. Collaborate with Data Architects, Business Stakeholders, and Delivery Teams to define migration strategies and target-state architectures. Configure and manage Unity Catalog , including catalogs, schemas, tables, governance policies, and access controls. Ensure adherence to data governance, security, and compliance requirements across the Databricks ecosystem. Provide technical leadership, solution guidance, and mentorship to offshore and onshore development teams. Act as the primary technical point of contact for client stakeholders, facilitating discussions, solution reviews, and project planning. Conduct code reviews, performance tuning, and optimization of Databricks workloads. Support deployment automation and platform standardization initiatives. Required Skills & Qualifications Technical Skills Strong hands-on experience with: Databricks Workflows Databricks Lakehouse Architecture Delta Lake Apache Spark (PySpark/Scala preferred) Proven experience migrating: Azure Synapse Analytics workloads to Databricks SQL Server ETL processes to Databricks Enterprise data warehouses and data pipelines Expertise in designing and implementing scalable ETL/ELT frameworks. Hands-on experience with Unity Catalog , including: Catalog and schema management Table creation and management Role-based access control (RBAC) Data governance implementation Strong understanding of cloud-based data engineering principles and best practices. Leadership & Stakeholder Management Experience serving as an Onshore Technical Lead for data engineering programs. Strong client-facing communication and stakeholder management skills.

Ability to work closely with:

Client Business Stakeholders Data Architects Program Managers Delivery Leads Experience leading distributed teams across onshore and offshore locations. Good to Have Skills Experience with Databricks Asset Bundles (DABs) . Familiarity with Azure DevOps and CI/CD implementation for Databricks deployments. Understanding of Infrastructure-as-Code and deployment automation practices. Knowledge of retail, restaurant, or hospitality operational data systems such as: Toast HotSchedules Crunchtime Experience with modern data governance and data quality frameworks. Preferred Experience 8+ years of overall Data Engineering experience. 4+ years of hands-on Databricks development and solution design experience. Prior experience leading cloud data migration and modernization programs. Experience working in Agile/Scrum delivery environments. Education Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.