Senior Data Pipeline & Connectivity Engineer (Databricks)

Senior Data Pipeline & Connectivity Engineer (Databricks)

Location:

Charlotte, NC (Hybrid Preferred) | Open to Remote for

Top Talent Comp:

$140K-$180K Base + Bonus + Equity Company Overview We are a rapidly growing, private equity-backed services platform operating across dozens of acquired business units. Our environment consists of 50+ disconnected systems and ERPs , each operating independently across separate networks. We are building a modern, Databricks-first data and AI platform to unify operations, enable real-time insights, and power next-generation AI use cases. The Opportunity This is a foundational engineering role focused on solving one of the hardest problems in modern data: connecting fragmented systems into a scalable, governed data platform . You will own the design and build of data pipelines, ingestion frameworks, and connectivity patterns that bring together disparate environments into Databricks-laying the groundwork for data products, analytics, and AI. This is not a support role . This is a hands-on builder role for someone who wants to architect and implement at scale. What You ll Own ? Connectivity & Data Ingestion Design and implement robust ingestion pipelines across highly fragmented environments (multiple ERPs, isolated networks, legacy systems) Build connectivity across: ERP systems (e.g., Great Plains, SAP, Infor, custom systems) APIs, flat files, streaming sources, and third-party platforms Solve for network isolation challenges and inconsistent data access patterns ?? Databricks Pipeline Engineering Build and optimize Databricks-based pipelines using: PySpark Delta Lake Structured Streaming / batch frameworks Implement scalable ingestion patterns aligned to medallion architecture (Bronze ? Silver ? Gold) Ensure data quality, reliability, and performance across pipelines ? Platform & Architecture Contribution Partner with Data Architect / Leadership to: Define ingestion frameworks and reusable pipeline patterns Standardize data models across business units Enable scalable onboarding of newly acquired companies ? Data Product Enablement Deliver clean, reliable datasets that power: Operational reporting Power BI / visualization layers AI / ML and LLM-based use cases ?

Governance, Quality & Reliability Implement:

Data validation and monitoring Logging and observability Secure and compliant data movement What You Bring ? Must-Have Experience 5-10+ years in data engineering / pipeline engineering Deep hands-on experience with: Databricks (required) PySpark Delta Lake Proven experience building: Data pipelines across multiple disconnected systems Scalable ingestion frameworks ? Strongly Preferred Experience in complex, multi-entity environments (PE-backed, M&A, roll-ups) ERP data integration experience: Great Plains, SAP, Infor, NetSuite, etc.

Experience with:

AWS or Azure data ecosystems API integrations and event-based pipelines Data orchestration tools ? Ideal Profile Builder mindset - thrives in greenfield + messy environments Comfortable operating with incomplete data and evolving requirements Can own problems end-to-end , not just execute tickets Balances speed with scalable architecture Why This Role is Different You are not maintaining pipelines - you are building the foundation Direct impact on enterprise-wide data consolidation strategy Clear path into: Data Platform Lead Data Architecture AI / Data Product leadership