Lead Data Engineer

Lead Data Engineer at Horizontal Talent Lead Data Engineer at Horizontal Talent in Saint Paul, Minnesota Posted in about 4 hours ago.

Job Description:

Join our team as a Lead Data Engineer, where you'll spearhead technical execution and provide architectural guidance across diverse engineering teams. This role offers a unique opportunity to influence the standards for a modern data stack centered on Snowflake and Azure. Responsibilities Lead technical efforts for distributed teams, ensuring clarity and high standards across time zones. Collaborate with Product Owners and Solution Architects to design optimal data pipelines. Develop systems for high availability and fault tolerance to ensure platform resilience. Engineer and optimize data pipelines using Fivetran, Snowflake, and dbt for large-scale datasets. Implement metadata-driven pipelines to enhance development speed and reduce manual tasks. Apply advanced modeling techniques to create high-performance datasets for analytics and AI. Optimize Snowflake performance to minimize latency and reduce costs. Ensure platform reliability through end-to-end observability and rigorous quality checks. Manage data flows within the Azure ecosystem, leveraging Azure DevOps and related services. Skills Extensive experience in building and optimizing complex data architectures and pipelines. Expert-level proficiency in Snowflake, dbt, and Fivetran. Strong skills in Azure services, including Storage, Compute, and DevOps/CI/CD. Proven ability to engineer layered data frameworks using various modeling methodologies. Experience in guiding offshore teams and conducting technical code reviews. Preferred Skills Experience in building data foundations for Machine Learning and Generative AI use cases. Knowledge of automated data privacy/masking and advanced metadata cataloging. At Horizontal, we are committed to fostering a diverse and inclusive workplace where everyone feels valued and respected. We believe that diversity drives innovation and are dedicated to creating an environment that reflects the diverse communities we serve. By applying for this position, you acknowledge and agree that Horizontal Talent may contact you regarding your application using automated technology, including phone calls, SMS/text messages, or email, which may be delivered by our virtual AI recruiter, Alex.