Data Architect

Data Architect Role Location:

Milpitas Job Type:

Hybrid Needs local (Bay area, CA) 4 days on site

Job Description:

Position Overview We are seeking a hands-on Data Architect with deep technical expertise in Azure and Databricks to design and build enterprise data platforms and products. This role requires a unique blend of architectural vision and hands-on implementation skills - you'll design solutions AND write code to bring them to life. Key Responsibilities Architecture & Design Design scalable, secure, and cost-effective data architectures on Azure and Databricks Define data platform standards, patterns, and best practices Create reference architectures for data lakehouse implementations Design end-to-end data product architectures from ingestion to consumption Hands-On Implementation Build proof-of-concepts and reference implementations personally Write production-grade Spark code (PySpark, Scala, SQL) Implement complex data pipelines and transformations Configure and optimize Databricks environments and Azure services Lead by example through hands-on technical contributions Data Platform Development Design and implement data lakehouse architectures using Delta Lake Architect streaming and batch processing solutions at scale Build data governance frameworks using Unity Catalog and Azure Purview Design data quality and observability solutions Technical Leadership Provide technical guidance to data engineering teams Conduct architecture reviews and code reviews Mentor engineers on Databricks and Azure best practices Evaluate and recommend new technologies and tools Stakeholder Engagement Translate business requirements into technical architectures Present architectural designs to technical and business stakeholders Collaborate with security, infrastructure, and platform teams Drive architectural decisions and alignment across teams Required Qualifications Technical Expertise 8+ years of data engineering/architecture experience Deep hands-on expertise in Databricks (Delta Lake, Unity Catalog, Medallion architecture, cluster optimization) Extensive Azure experience (Data Lake Storage, Data Factory, Synapse Analytics, Event Hubs, Purview) Expert-level Apache Spark skills (performance tuning, optimization, architecture) Strong programming skills in PySpark, Scala, and/or Python Advanced SQL knowledge and query optimization Architecture Experience Proven track record designing and implementing large-scale data platforms Experience with data lakehouse, data mesh, or modern data stack architectures Deep understanding of data modeling techniques (dimensional, normalized, data vault) Knowledge of real-time streaming architectures (Kafka, Event Hubs, Structured Streaming) Experience with data governance, security, and compliance frameworks Technical Depth

THIS IS NOT A PURE ARCHITECTURE ROLE

- Daily coding and hands-on work required Ability to debug complex Spark jobs and optimize performance Experience with infrastructure as code (Terraform, ARM templates) Understanding of DevOps practices and CI/CD for data pipelines Knowledge of data quality, testing, and observability frameworks Preferred Qualifications Azure Solutions Architect Expert certification Databricks Certified Data Engineer Professional or Solutions Architect Experience with machine learning platforms and MLOps Background in cloud migration and modernization projects Knowledge of FinOps and cloud cost optimization Experience with multiple cloud platforms (AWS, Google Cloud Platform) Previous consulting or advisory experience

Technical Skills Summary Must Have:

Azure (comprehensive knowledge across data services) Databricks (expert-level implementation and optimization) Apache Spark (architecture, tuning, advanced optimization) PySpark and/or Scala (production-level coding) SQL (complex queries, performance tuning) Data architecture patterns and best practices What Makes This Role Unique This is NOT a traditional "ivory tower" architect role.

We need someone who:

Designs architectures in the morning and implements them in the afternoon Gets into the weeds with Spark optimization and cluster tuning Can architect a solution AND build a working prototype Debugs production issues alongside the engineering team Writes code daily, not just PowerPoint slides

Ideal Candidate Profile:

Senior engineer who's ready to take on architecture responsibilities Or architect who refuses to stop coding and building Believes the best way to validate an architecture is to implement it Values pragmatic solutions over theoretical perfection Key Differentiators 70% hands-on technical work, 30% architecture/design Direct contribution to codebase and data products Technical leadership through example, not delegation Ownership of both design and implementation Education Bachelor's degree in Computer Science, Engineering, Mathematics, or related field Master's degree or equivalent experience preferred