Job Description Overview:

Home Depot is seeking experienced Data Engineers to support a high-priority initiative focused on replicating enterprise data from Google BigQuery into Databricks. This role will be responsible for designing, building, and optimizing large-scale data replication pipelines while ensuring performance, scalability, and reliability across the data platform.

The ideal candidate will have hands-on experience with BigQuery-to-Databricks migrations and a strong background in modern data engineering, distributed processing, and cloud-based data platforms.

Key Responsibilities

• Design, develop, and maintain data replication pipelines from BigQuery to Databricks
• Build scalable ingestion frameworks supporting large-scale enterprise datasets
• Develop and optimize Databricks notebooks using PySpark
• Implement incremental data replication and change data capture (CDC) strategies
• Optimize query performance, data movement, and processing efficiency
• Work with BigQuery authorized views and federated connections to support enterprise data access patterns
• Utilize Apache Iceberg and Unity Catalog for data governance and management
• Collaborate with architects, engineers, and business stakeholders to define migration requirements and solutions
• Productionize and monitor Databricks workflows and jobs
• Troubleshoot data quality, performance, and replication issues across environments. Skills and Requirements
• 5+ years of Data Engineering experience.
• Strong hands-on experience with Google BigQuery and Databricks.
• Proven experience building and supporting BigQuery → Databricks replication or migration pipelines.
• Strong proficiency in SQL and data modeling concepts.
• Advanced experience with PySpark and Databricks notebook development.
• Experience with large-scale data movement and distributed data processing.
• Experience implementing incremental load and replication strategies.
• Strong understanding of performance tuning and optimization techniques.
• Experience working in cloud-based data environments.

Excellent analytical, troubleshooting, and problem-solving skills.

• Experience with Apache Iceberg, especially Databricks-managed Iceberg tables.
• Experience with Unity Catalog and enterprise data governance.
• Experience with BigQuery authorized views and federated connections.
• Experience orchestrating and productionizing Databricks jobs and workflows.
• Knowledge of CI/CD pipelines for data engineering solutions.
• Experience with data lakehouse architectures.
• Prior experience supporting enterprise-scale cloud migration projects.

Retail industry experience. We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to HR@insightglobal.com.