Find Jobs
Find Jobs Near You – Available Work in Your Location
Skip to job details
IG
Insight Global
Sr Data Engineer
Career Insights for Data Engineer
See where this job fits in the broader career landscape. Knowing your career path helps you see what's possible from here.
Scorecard
Based on California data
Review key factors to help you decide if this role fits your goals. How is this calculated?
What they do
A Data Engineer designs, builds and manages the information or big data infrastructure. Develops the architecture that helps analyze and process data in the way the organization needs it. Makes sure those systems are performing smoothly.
$127,645 / year median in California
+3% projected growth
Job Description
Job Description We are looking for a Sr Data Engineer focused on Foundry, governance, release engineering, and geospatial data products. This person will be developing, engineering and improving an existing platform and pipeline ecosystem. We are seeking a Data Engineer to support the transition of data pipelines from an internal IT team to our internal team. This role will focus on both enhancing existing pipelines and improving the underlying code and data infrastructure to ensure long-term maintainability, scalability, and clarity. Key Responsibilities
To learn more about how we collect, keep, and process your private information, please review
- Refactor and enhance data pipelines to be modular, maintainable, and well-documented
- Establish and promote best practices for code architecture, version control, and code management
- Collaborate closely with a core team of data scientists, machine learning engineers, and data engineers and a broader team of cross-functional partners
- Communicate technical data to non-technical stakeholders to build trust and understanding in the methodology.
- Support the development of machine learning models currently being developed in-house We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day.
To learn more about how we collect, keep, and process your private information, please review
Insight Global's Workforce Privacy Policy:
https://insightglobal.com/workforce-privacy-policy/. Skills and Requirements Required Qualifications- Strong experience in Python-based code development
- Demonstrated expertise in code architecture, software engineering practices, and maintainable pipeline design, in a cloud-based environment
- Experience with the Palantir Foundry platform (current technology, plan to move to AWS in future, should have this experience as well)
- Familiarity with PySpark and distributed computing.
- Experience with release engineering
- Repository governance experience; audit ready and branching strategy (highly preferred)
- Traceability
- Experience working with spatial-temporal datasets and geospatial packages such as Sedona, Geopandas, and Rasterio.
- Experience transitioning data pipelines from consulting or external vendors into production environments
- Experience designing cloud-optimized geospatial datasets (GeoParquet, Parquet, Zarr) with efficient partitioning and support for large-scale spatial operations and aggregations.
- Proficiency with version control (Git) and collaborative workflows (e.g., pull requests, code reviews)
- Ability to assess and improve legacy or externally developed codebases
- Excellent communication skills Preferred Qualifications
- Experience with a GIS platform such as QGIS or ArcGIS
- Experience working with ML model outputs
- Experience designing production-grade data and ML platforms with strong emphasis on reproducibility, dataset versioning, release traceability, repository governance, and maintainable software architecture.