Job Description We are seeking an experienced Network Sustainment Engineer to join our Sustainment team as a daytime remediation specialist. This role picks up where the night upgrade team leaves off: triaging failed or incomplete software upgrades, diagnosing root causes, and resolving issues so that affected devices can be returned to the upgrade schedule. The engineer works from shift handoff reports, engages directly with device teams and network operations, and owns the resolution of upgrade blockers — from console access problems and configuration anomalies to routing issues and device instability. Strong analytical and troubleshooting skills, and an ability to work cross-functionally to resolve blockers quickly, are essential.

Key Responsibilities

• Review end-of-shift handoff reports from the night upgrade team and triage all failed or incomplete upgrades by priority
• Diagnose root causes of failed upgrades across platforms including Cisco IOS-XE, IOS-XR, and Nexus — including reload failures, console access loss, routing instability, configuration drift, and code-specific defects
• Resolve console access issues: coordinate with the

NTSO/NOC

to restore out-of-band access, remediate CSM configuration gaps, or arrange physical console presence where required

• Validate and restore device state after a failed upgrade or back-out, confirming the device is in a stable, known-good configuration before rescheduling
• Engage Cisco TAC, platform SMEs, and lead engineers for issues beyond standard troubleshooting — providing detailed diagnostic data to accelerate resolution
• Identify and document patterns in failures: recurring console issues, code-specific bugs, MOP gaps, or site-specific constraints, and feed findings back to the upgrade planning team
• Update MOPs or flag MOP defects discovered through failed upgrade analysis, ensuring the night team has accurate procedures
• Coordinate device rescheduling with the upgrade operations team once a device is confirmed remediated and ready to retry
• Track all open remediation cases, maintaining clear status and estimated return-to-schedule dates for each device
• Serve as the primary point of contact between the night upgrade team, NOC, platform engineering, and network operations for all upgrade-related issuesDay-to-Day Work
• Begin each day reviewing the night team's handoff report: categorize failures by type, assess urgency, and set remediation priority
• Log into affected devices to assess current state: confirm software version, check configurations against expected post-upgrade baseline, and review system logs for failure indicators
• Diagnose the specific failure mode for each device — console loss, reload hang, routing reconvergence failure, configuration error, code bug — and determine the appropriate remediation path
• Work with the NOC and NTSO team to restore console or out-of-band access for devices that are unreachable
• Execute configuration corrections, rollback validations, or partial remediation steps as appropriate to return devices to a stable, schedulable state
• Open and drive TAC cases or escalations where platform defects or code bugs are suspected, tracking to resolution
• Update the upgrade tracking system with remediation status, root cause, and rescheduled window for each device
• Communicate remediation timelines and blockers to the upgrade program lead, flagging any systemic issues that require process or MOP changes
• Participate in regular syncs with the upgrade operations team to align on device readiness and schedule impacts Skills and Requirements
• Hands-on troubleshooting experience on production Cisco network devices via CLI (IOS-XE, IOS-XR, or Nexus)
• Experience diagnosing and resolving software upgrade failures, reload issues, or post-change network instability
• Strong understanding of console and out-of-band access methods, and experience troubleshooting access failures
• Ability to read system logs, CLI outputs, and routing tables to identify failure root cause
• Experience working with network operations and NOC teams in a high-availability environment
• Strong written communication skills — ability to produce clear, concise triage and status reports
• Proven ability to manage multiple open issues simultaneously, prioritizing by impact and urgency
• Familiarity with multi-hop IOS-XR or IOS-XE upgrade paths and associated code-specific known issues or defects
• Experience with CSM (Console Server Module) configuration and troubleshooting
• Experience engaging Cisco TAC, including opening and driving escalation cases
• Background in a software upgrade or fleet sustainment program in telecom or high-availability networking
• Experience identifying and documenting systemic failure patterns and translating them into MOP improvements
• Exposure to upgrade automation tooling and understanding of how automation failures differ from manual execution failures We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day.

We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to HR@insightglobal.com.