A leading infrastructure and transportation group is seeking a hands-on Senior Data Engineer to architect and build our next-generation Enterprise Reporting & Data Warehouse (ERDW.Next) on Databricks. Acting as the hands-on technical owner, you will drive target-state design, medallion architecture (Delta Lake), and legacy migrations off SQL Server/Synapse while remaining actively in the codebase. You will partner with our Enterprise Architect, Power BI team, business users, and offshore delivery partners across finance, construction, and safety domains.

QUALIFIED CANDIDATES SHOULD HAVE EXPERIENCE WITH

Serving as a Databricks SME ERP integrations (strong plus) Python / PySpark Performance tuning and optimization Databricks maintenance, including VACUUM Data governance and best practices Semantic modeling

RESPONSIBILITIES

Architecture & Strategy:

Design the ERDW.Next lakehouse (medallion/gold layer star schema) and govern data modeling, batch/streaming, and Delta Live Tables standards.

Build & Migration:

Write production-grade PySpark, Spark SQL, and Delta Live Tables pipelines orchestrated via Databricks Workflows. Re-engineer legacy SSIS/T-SQL/Synapse logic and ingest ERP data (JD Edwards, Anaplan) using CDC and Auto Loader.

Platform Operations:

Optimize cost/performance using Photon, Liquid Clustering, and

OPTIMIZE.

Manage CI/CD pipelines via Git and Databricks Asset Bundles while maintaining strict SLAs.

Governance & AI:

Implement Unity Catalog governance (entitlements, lineage, audit) and curate Databricks Genie Agents for natural-language analytics.

Partner & Legacy Management:

Support legacy platforms during transition and maintain code quality standards across offshore partner teams. Leverage agentic AI coding tools safely to accelerate delivery.

EXPERIENCE

Education:

Bachelor's degree in Computer Science, Information Systems, Engineering, or equivalent practical experience.

Track Record:

5+ years dedicated to Databricks/Apache Spark platforms and 8+ years overall in Data Engineering/BI. Proven experience taking an enterprise lakehouse end-to-end into production.

System Experience:

Deep hands-on experience migrating legacy warehouses (SQL Server, Synapse, ADF) and sourcing data from ERPs (JD Edwards strongly preferred). Certifications (Preferred): Databricks Certified Data Engineer Professional/Data Architect; Azure DP-203 or

AZ-305. TECHNICAL SKILLS

Databricks & Spark:

Advanced PySpark, Spark SQL, Databricks Workflows, Delta Live Tables, Unity Catalog, Auto Loader, and Photon.

Data Modeling & Storage:

Star Schema, SCDs, Conformed Dimensions, Delta Lake (MERGE, Liquid Clustering, Z-ORDER, Time Travel).

Cloud & DevOps:

Azure Gen2, Key Vault, Entra ID, Git, Azure DevOps/GitHub Actions, Databricks Asset Bundles, and Infrastructure as Code (Terraform).

Legacy & Integration:

SQL/T-SQL, SSIS, SSRS, Synapse, ADF, CDC/replication tools, Direct Lake/Power BI integration, and Microsoft Fabric/OneLake.

AI & Modern Workflow:

Agentic AI coding tools, natural language query curation (Databricks Genie), and metadata-driven semantics.

SOFT SKILLS

Strong technical ownership, architectural influence, and clear communication with both executives and engineers. Excellent cross-functional collaboration with offshore partners, citizen developers, and business stakeholders. Pragmatic problem-solving, critical thinking, and time management across build and legacy support. Technology Doesn't Change the World, People Do.® Robert Half is the world's first and largest specialized talent solutions firm that connects highly qualified job seekers to opportunities at great companies. We offer contract, temporary and permanent placement solutions for finance and accounting, technology, marketing and creative, legal, and administrative and customer support roles. Robert Half works to put you in the best position to succeed. We provide access to top jobs, competitive compensation and benefits, and free online training. Stay on top of every opportunity - whenever you choose - even on the go. Download the Robert Half app and get 1-tap apply, notifications of AI-matched jobs, and much more. All applicants applying for U.S. job openings must be legally authorized to work in the United States. Benefits are available to contract/temporary professionals, including medical, vision, dental, and life and disability insurance. Hired contract/temporary professionals are also eligible to enroll in our company 401(k) plan. Visit roberthalf.gobenefits.net for more information. © 2025 Robert Half. An Equal Opportunity Employer. M/F/Disability/Veterans. By clicking "Apply Now," you're agreeing to Robert Half's Terms of Use and Privacy Notice. Create a Job Alert Email me about new Senior Data Engineer (Databricks) jobs in Morristown, NJ Related Jobs Search more jobs Browse Jobs Contract Talent Full-Time Talent Consulting Solutions Finance & Accounting Technology Marketing & Creative Legal Administrative & Customer Support Discover Insights Invoice Job Directory Salary Guide Time Reports Create a Job Alert Contact Us About Robert Half Leadership Careers With Us Locations Investor Center Press Supplier Relations Fraud Alert Government Notice Privacy Notice Terms of Use Robert Half Inc. An Equal Opportunity Employer M/F/Disability/Veterans. Do Not Sell or Share My Personal Information This website uses cookies, pixel tags, and other tracking technologies to enhance user experience, personalize content and ads, and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. If we have detected an opt-out preference signal, it will be honored unless you have changed your consent preferences manually. You may opt-out of use of certain cookies through the Do Not Sell or Share My Personal Information link. Your use of the website is subject to our Terms of Use. More information about cookies and how we share information can be found in our Privacy Notice. Do Not Sell or Share My Personal Information I understand