Observability Engineer

• Remote Harbor Freight Tools USA, Inc.
• 3.3 Calabasas, CA Job Details $87,600
• $131,500 a year 2 days ago Benefits Disability insurance Health insurance Dental insurance Paid time off Vision insurance Life insurance Paid sick time Qualifications Linux support RESTful API IT system monitoring Public Cloud Windows Incident Escalation IT infrastructure Outlier detection Incident management operations support Network switching Security Event Management (security operations) Log analysis tools Cloud networking Windows support Network routing Network monitoring tools Quantitative analysis Cloud monitoring Escalation handling Automation tools Linux administration System performance monitoring Full Job Description The Observability Engineer is responsible for designing, building, implementing, and administering the tools and pipelines that give the organization full-stack visibility into its infrastructure, applications, and business services.

This role owns the day-to-day operation of the observability platform(s) and the NOC/TOC (Network/Technical Operations Center) function to ensure stability, reliability, availability, and performance of the services the business depends on. Where traditional monitoring answered, "is it up or down," this role is expected to build observability that answers "why is it behaving this way" — correlating metrics, logs, traces, and events into actionable insight, and reducing time-to-detect and time-to-resolve across the environment. Duties and Responsibilities Provides expertise and develops observability solutions for: Network technology, connections, and cloud networking (SD-WAN, VPC/VNet constructs) Directory services (Active Directory, LDAP, Entra ID/Azure AD) Messaging and collaboration platforms (Exchange Online, Microsoft 365, Teams) eCommerce and customer-facing digital environments Remote access and VPN infrastructure Wireless, mobile, and endpoint device fleets Operating system environments (Windows, Linux) Data backup, storage, and disaster recovery systems VOIP and unified communications systems Virtualized and containerized environments (VMware) Public and hybrid cloud infrastructure (AWS, Azure, GCP) Critical business services and transaction flows Design, deploy, and maintain observability platforms across metrics, logs, traces, synthetic monitoring, and real user monitoring Own and operate the NOC/TOC function end-to-end: alert intake, triage, escalation, incident coordination, and post-incident review Develop and maintain monitoring/observability standards, naming conventions, and instrumentation guidelines across infrastructure and application teams Build and tune alerting logic to reduce noise and alert fatigue; drive toward actionable, high-signal alerting Define, track, and report on SLIs, SLOs, error budgets, and other key observability/operational KPIs Develop dashboards and visualizations that give both technical teams and business stakeholders a clear picture of service health Mentor technical staff and cross-functional teams on observability concepts, instrumentation practices, and tooling Develop, document, and continuously improve monitoring and incident response processes and runbooks Troubleshoot and resolve issues within the monitoring/observability stack itself (data gaps, false positives, agent/collector health) Evaluate, pilot, and recommend new observability tools and platforms Partner with Help Desk, Infrastructure, DevOps/SRE, and Application teams to implement monitoring, alerting, and response processes Manage the call notification, paging, and escalation system for alerts Lead capacity planning, performance trending, and fault prediction using historical observability data Integrate observability data with automation/AIOps workflows to enable auto-remediation where appropriate Support security operations by feeding relevant telemetry into

SIEM/SOC

tooling and collaborating on anomaly detection After-hours work, shift work, and on-call rotation required Scope Staff supervision and development: No Decision making: Review data and provide recommendations on alert actions, monitoring and responses.

Travel:

Up to 5%

Flex Designation:

Anywhere The anticipated salary range for this position is $87,600

• $131,500 depending on location, knowledge, skills, education and experience.

This position is also eligible for an annual discretionary bonus. In addition, we offer comprehensive and competitive benefits to Associates (and their families) such as medical, dental, vision, life insurance, short-term and long-term disability. Eligible Associates are able to enroll in our company's 401k plan. Associates will accrue paid time off up to 236 hours per year (inclusive of PTO, floating holidays, and paid holidays). Paid sick time up to 80 hours per year unless otherwise required by law. Requirements Education and Experience Education Requirements Bachelors Degree in technical discipline, Computer Science or related field, or equivalent experience. Relevant Certifications are a bonus (Vendor-specific observability certifications, CCNA, AWS/Azure Certifications). Years of Experience 2-4 years experience with network/systems management and observability tools. Skills Hands-on experience with modern observability/monitoring platforms such as Dynatrace, SolarWinds, Zabbix, Datadog, New Relic, Splunk, PRTG, or Nagios Experience with log aggregation and analysis platforms i.e. Splunk Familiarity with distributed tracing and APM tooling (Dynatrace, Jaeger, Honeycomb, AppDynamics) Deep understanding of monitoring protocols and data collection methods: SNMP (v1/v2c/v3), MIB structure, OID notation, syslog, NetFlow/sFlow/IPFIX, WMI, JMX, and RESTful API-based telemetry Solid understanding of networking fundamentals: routing, switching, TCP/IP, DNS, and network troubleshooting Strong grasp of the "three pillars of observability" (metrics, logs, traces) and how they correlate to reduce

MTTD/MTTR

Strong experience with Windows and Linux systems architecture and administration Experience monitoring cloud-native and hybrid environments across AWS, Azure, and/or GCP Proficiency in scripting/automation languages (Python, Bash, PowerShell) Experience integrating monitoring data with automation, ticketing, and AIOps/anomaly-detection workflows Demonstrated experience owning or operating within a NOC/TOC environment across its full lifecycle — detection, triage, escalation, resolution, and postmortem Ability to develop quantitative, prioritized action plans to address operational and reliability issues Strong incident management skills; familiarity with ITIL or similar frameworks a plus Excellent written and oral communication and documentation skills Ability to work after hours, shifts, and rotating on call as required Physical Requirements Corporate

• Remote
• General office environment requiring ability to: Stand, walk, sit for extended periods of time .

Speak and listen to others in person and over the phone and video conferencing. Use keyboard and read from computer screen and reports. The ability to lift up to 15 lbs. Safety Must be able to perform this job safely in accordance with standard operating procedures and good manufacturing practices, without endangering the health or safety of self or others. About Harbor Freight Tools Founded in 1977, we are a $10 billion national tool retailer with the energy, enthusiasm, and growth mindset of a Championship Team. With more than 1,600 stores across 48 states, we're continuing to grow and open new locations every month. We offer our customers more than 6,000 tools and accessories, from hand tools and generators to air and power tools, shop equipment, and automotive tools. By delivering quality and value to our customers, we've built a culture where Associates can learn, grow, and do meaningful work every day. Observability Engineer

• Remote | Harbor Freight Tools