Senior GenAI Governance Engineer

JOB SUMMARY

We are seeking a Senior Cloud GenAI Governance Engineer to manage enterprise-scale GenAI platform engineering with a focus on critical gateway and security layers. This role involves managing the entire architecture flow from user API requests through Model Armor to the inference endpoint and finally into telemetry and Arize AI. The primary objective is to build responsible AI infrastructure and ensure AI runtime governance. This position requires a strong understanding of generative LLMs and is not suitable for individuals with traditional predictive ML or data science backgrounds. Key Responsibilities

• Act as the primary owner of AI security guardrails, implementing and managing Model Armor for prompt filtering, data leakage prevention, request/response inspection, and enterprise-safe inference.
• Manage complex API lifecycle tracing, troubleshoot endpoint bottlenecks, analyze container orchestration failures, and trace the origin of 429 errors across ingress, gateway, load balancing, and GPU saturation.
• Utilize Arize AI for runtime monitoring, inference tracing, and model behavior analysis.
• Manage distributed inference debugging, token queueing, and upstream/downstream API flows.
• Maintain GCP/Azure landing zones, Terraform deployments, and Kubernetes clusters to support governance pipelines. Required Qualifications
• 8+ years of experience in API lifecycle tracing, rate limiting, and token throughput management.
• 8+ years of hands-on expertise with AI security tools, specifically Model Armor.
• Deep understanding of enterprise AI observability platforms, particularly Arize AI.
• Strong background in cloud networking (GCP/Azure) and Kubernetes infrastructure.
• Role is focused entirely on generative LLMs; traditional predictive ML or data science backgrounds are not a fit.
• Must be onsite at client in Charlotte, NC at least 3 days/week.