Lead Data Engineer with Databricks & AI

Job Summary:

We are looking for a highly experienced and client-facing Lead Engineer with strong expertise in Databricks and modern data engineering practices. The ideal candidate should have hands-on experience in building scalable data pipelines, developing Gold/Presentation layer models for reporting and analytics, and working closely with business stakeholders and clients. The candidate should also have exposure to AI-driven analytics tools such as AI/BI Genie or similar AI-enabled BI platforms.

Key Responsibilities:

Design, develop, and optimize scalable data engineering solutions using Databricks. Build and maintain Bronze, Silver, and Gold data models for reporting and analytics. Develop presentation/reporting models to support business intelligence and dashboarding requirements. Collaborate with business users, analysts, and clients to gather requirements and deliver data solutions. Lead end-to-end data engineering activities including ingestion, transformation, orchestration, and performance tuning. Work with large-scale structured and semi-structured datasets in cloud environments. Implement best practices for data governance, security, monitoring, and optimization. Support AI-enabled analytics initiatives and work with tools such as AI/BI Genie or similar platforms. Mentor junior engineers and provide technical leadership to the team. Participate in client discussions, solution design, and architecture reviews.

Required Skills:

Strong hands-on experience with Databricks. Excellent coding skills in PySpark and SQL. Expertise in data modeling, especially Gold/Presentation layer development. Experience with ETL/ELT pipeline development. Strong SQL and PySpark programming skills. Experience with cloud platforms such as AWS, Azure, or Google Cloud Platform. Knowledge of Delta Lake, Data Lakehouse architecture, and performance optimization. Experience with BI/reporting tools such as Power BI, Tableau, or Looker. Exposure to AI/BI Genie or AI-powered analytics tools. Strong communication and client-facing skills. Ability to work independently in a fast-paced environment.

Preferred Qualifications:

Experience in enterprise-level data platform implementations. Knowledge of data governance and data quality frameworks. Exposure to GenAI, AI-assisted analytics, or conversational BI tools is a plus. Databricks certifications are preferred.

Soft Skills:

Excellent communication and stakeholder management skills. Strong problem-solving and analytical abilities. Leadership and mentoring capabilities. Ability to manage multiple priorities effectively.