Member of Technical Staff - Copilot

Designs and builds, with guidance, the underlying frameworks (based on Spark/Databricks) that allow internal teams to process massive datasets efficiently, abstracting away the complexity of "ETL" into self-service infrastructure. Modernizes data stack by moving from batch-heavy patterns to event-driven architectures, utilizing modern streaming architecture to reduce latency for AI inference. Engineers the high-throughput telemetry systems that capture user interactions with Copilot, creating the critical data loops required for Reinforcement Learning and model evaluation. Treats the data platform as software. Defines and deploys all storage, compute, and networking resources using IaC (Bicep/Terraform) rather than manual configuration. Moves beyond simple "validation checks" to build automated governance and observability systems that detect anomalies in the data mesh before they impact downstream models. Deep-dives into query execution plans and cluster performance. Optimizes shuffle operations, partition strategies, and resource allocation to ensure our platform is as cost-efficient as it is fast. Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field These requirements include but are not limited to the following specialized security screenings: Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field OR Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 1+ year(s) experience in business analytics, data science, software development, data modeling or data engineering OR equivalent experience. Demonstrated technical understanding of massive-scale compute engines (e.g., Apache Spark, Flink, Ray, Trino, or Snowflake). Proven experience in building internal developer platforms or "Data-as-a-Service" APIs. Proven experience in streaming technologies (Kafka, Azure EventHubs, Pulsar) and stateful stream processing. Reliable experience with container orchestration (Kubernetes) for deploying data applications. Proven experience enabling AI/ML workloads (Feature Stores, Vector Databases).