Job Description
Overview We are seeking a highly skilled
HPC/AI/ML
Cluster Engineer to support the design, deployment, and ongoing operations of large-scale HPC environments powered by Kubernetes and Slurm (Slinky). This role centers on cluster engineering, administration, and performance optimization, with emphasis on GPU-accelerated computing, advanced networking, and workload scheduling. In this role, you will work closely with our researchers, vendors, and partners to manage Slurm clusters that are used for AI/ML workloads. You will work alongside our team to support in-house, partner, and customer infrastructure Responsibilities Cluster Engineering & Deployment Participate in the design and bring-up of bare metal HPC/AI/ML
environments Integrate heterogeneous hardware platforms into cohesive scheduling environments. Develop provisioning and imaging workflows (Ansible, MAAS, cloud-init, warewulf, CI/CD pipelines) for reproducible cluster build-out. Coordinate communications between vendors, researchers, and other partners during cluster bring-up and operation. Slurm Management Configure and operate the Slurm Workload Manager. Build custom Slurm plugins and scripts (epilog/prolog, pam_slurm_adopt) to extend functionality and integrate with authentication, health checking, and monitoring. Manage federated Slurm setups across multi-site or hybrid cloud environments. System Administration & Monitoring Administer Linux HPC environments, including network configuration, storage integration, and kernel tuning for HPC workloads. Deploy and maintain observability stacks for system health, GPU metrics, and job monitoring. Automate failure detection, node health checks, and job cleanup to ensure high uptime and reliability. Manage security and access control (LDAP/SSSD, VPN, PAM, SSH session auditing). User & Stakeholder Support Assist cluster users with developing workflows that make efficient use of compute resources. Containerize HPC applications with Docker/Podman/Enroot-Pyxis and integrate GPU-aware runtimes into Slurm jobs. Automate cost accounting and cluster usage reporting. Qualifications Previous experience in HPC cluster administration and engineering, with deep knowledge of Slurm. Expert in Slurm configuration, partition design, QoS/preemption policies, and GRES GPU
scheduling. Strong background in Linux system administration, networking, and performance tuning for HPC environments. Hands-on experience with parallel file system, advanced networking (InfiniBand, RoCE, 100/200 GbE), and monitoring stacks. Proficient with automation tools (Ansible, Terraform, CI/CD pipelines) and version control. Demonstrated ability to operate GPU-accelerated clusters at scale. Previous experience managing kubernetes deployments Exceptional candidates have familiarity with common AI/ML software package dependencies and researcher workflows Job Type:
Full-time Pay:
From $140,000.00 per year Benefits:
401(k) 401(k) matching Health insurance Relocation assistance Experience:
Linux and HPC cluster system administration: 1 year (Required) Language:
English (Required) Work Location:
Hybrid remote in Hickory, NC 28602
