We are seeking a hands-on Lead Data Engineer to design, develop, and lead enterprise-scale Data & AI platforms using Azure Databricks. The ideal candidate will have strong expertise in data engineering, cloud architecture, Spark, and production-grade GenAI/LLM solutions. Key Responsibilities Design and implement Lakehouse / Medallion Architecture using Bronze, Silver, and Gold layers. Develop scalable data pipelines using Python, Scala, PySpark, SQL, and Spark . Build data ingestion frameworks using Azure Data Factory, ADLS Gen2, Kafka, and Event Hubs . Develop and optimize Databricks / Delta Lake workloads. Implement CI/CD using Azure DevOps and Terraform . Build AI-powered data quality, anomaly detection, and reconciliation solutions. Design and implement RAG, LLM, and Agentic AI solutions. Work with LangChain, LangGraph, LlamaIndex, Azure OpenAI, or Databricks Model Serving . Implement data governance using Unity Catalog, RBAC, PII protection, and Azure security controls . Lead technical design reviews, mentor engineers, and establish development standards. Required Skills 12 18 years of Data Engineering experience. Expert-level Python, PySpark, and Scala . Strong Azure Databricks, Delta Lake, Unity Catalog experience. Hands-on ADLS Gen2 and Azure Data Factory . Strong SQL and data modeling skills. Experience with Kafka / Event Hubs / Spark Structured Streaming . Strong Azure DevOps, Terraform, and CI/CD experience. 3+ years of production GenAI/LLM experience . Experience with RAG, Agentic AI, embeddings, vector search, and prompt engineering . Strong architecture, troubleshooting, and technical leadership skills.