Principal Data Engineer

Principal Data Engineer at Agility Partners Principal Data Engineer at Agility Partners in Union City, California Posted in about 17 hours ago.

Type:

full-time

Job Description:

We're looking for a Lead Databricks Consultant to drive data platform strategy and delivery across enterprise environments. This role blends client-facing solutioning with hands-on architecture and technical leadership, helping organizations modernize data, analytics, and AI platforms using Databricks and cloud technologies. What You'll Do Lead discovery sessions to uncover business goals and define high-impact data use cases Design scalable Lakehouse architectures using Databricks (Delta Lake, Unity Catalog) Own presales efforts including solution design, proposals, and delivery roadmaps Guide engineering teams building batch and streaming pipelines with Spark, Python, and SQL Establish best practices across CI/CD, data quality, observability, and governance Deliver presentations, demos, and whiteboarding sessions to business and technical stakeholders Drive performance optimization, cost efficiency, and platform adoption What You Bring Experience leading client-facing data or analytics engagements Strong Databricks expertise (Delta Lake, Workflows/Jobs, Databricks SQL) Hands-on skills with Python, PySpark, and SQL Cloud experience (Azure, AWS, or GCP) Ability to translate business requirements into scalable technical solutions Familiarity with data governance, security, and access controls Nice to Have Experience with large-scale data platform modernization or migration efforts Background in consulting or professional services Why Join Lead high-impact data and AI initiatives Work in a fast-paced, collaborative consulting environment Exposure to modern cloud and data technologies