Staff Software Engineer, Inference

Back to jobs New Staff Software Engineer, Inference Sunnyvale, CA / Bellevue, WA Apply CoreWeave is The Essential Cloud for AI™. Built for pioneers by pioneers, CoreWeave delivers a platform of technology, tools, and teams that enables innovators to build and scale AI with confidence. Trusted by leading AI labs, startups, and global enterprises, CoreWeave combines superior infrastructure performance with deep technical expertise to accelerate breakthroughs and turn compute into capability. Founded in 2017, CoreWeave became a publicly traded company (

Nasdaq:

CRWV) in March 2025. Learn more at www.coreweave.com .

What You'll Do:

Inference Platform Team The Inference team builds and operates CoreWeave's Kubernetes-native inference platform, powering low-latency, high-throughput AI workloads at massive scale. The team is responsible for request routing, scheduling, GPU resource management, and system-wide optimizations that drive performance, efficiency, and reliability across real-time inference systems.

About the role:

As a Staff Software Engineer (IC5) on the Inference team, you will act as a technical leader driving architecture, performance, and reliability across multiple services and teams. Your day-to-day will involve leading cross-team design initiatives, optimizing inference performance (latency, throughput, and GPU utilization), and improving system reliability at scale. You will work deeply in distributed systems and Kubernetes-based infrastructure, focusing on areas like scheduling, batching, and memory optimization. This role requires hands-on technical leadership and the ability to influence engineering direction across the organization.

Who You Are:

8-12+ years of experience building and operating large-scale distributed systems or cloud platforms Proven experience leading cross-team technical initiatives impacting multiple services or organizations Strong programming skills in Go, Python, or C++ Deep expertise in Kubernetes at production scale, including orchestration, scheduling, and service design Strong understanding of distributed systems, networking, and performance optimization Experience designing and operating low-latency, high-throughput systems with strict P95/P99 latency requirements Hands-on experience with inference systems, including batching or micro-batching strategies, caching, and memory optimization Experience improving system performance using metrics-driven approaches (e.g., latency, throughput, utilization) Familiarity with mixed precision (BF16, FP8) and streaming inference workloads

Preferred:

Experience with inference frameworks such as vLLM, Triton, TensorRT-LLM, Ray Serve, or TorchServe Experience with GPU systems and performance optimization (CUDA, NCCL, RDMA, NUMA, GPU interconnects) Experience leading multi-team or org-level technical initiatives Exposure to large-scale AI/ML infrastructure or hyperscale cloud environments Wondering if you're a good fit? We believe in investing in our people, and value candidates who can bring their own diversified experiences to our teams - even if you aren't a 100% skill or experience match. Here are a few qualities we've found compatible with our team. If some of this describes you, we'd love to talk. You love to design and optimize high-performance distributed systems at scale You're curious about AI inference, GPU systems, and emerging performance techniques You're an expert in building reliable, low-latency infrastructure and driving system-wide improvements Why CoreWeave? At CoreWeave, we work hard, have fun, and move fast! We're in an exciting stage of hyper-growth that you will not want to miss out on. We're not afraid of a little chaos, and we're constantly learning. Our team cares deeply about how we build our product and how we work together, which is represented through our core values: Be Curious at Your Core Act Like an Owner Empower Employees Deliver Best-in-Class Client Experiences Achieve More Together We support and encourage an entrepreneurial outlook and independent thinking. We foster an environment that encourages collaboration and enables the development of innovative solutions to complex problems. As we get set for takeoff, the organization's growth opportunities are constantly expanding. You will be surrounded by some of the best talent in the industry, who will want to learn from you, too. Come join us! The base salary range for this role is $188,000 to $275,000. The starting salary will be determined based on job-related knowledge, skills, experience, and market location. We strive for both market alignment and internal equity when determining compensation. In addition to base salary, our total rewards package includes a discretionary bonus, equity awards, and a comprehensive benefits program (all based on eligibility). What We Offer The range we've posted represents the typical compensation range for this role. To determine actual compensation, we review the market rate for each candidate which can include a variety of factors. These include qualifications, experience, interview performance, and location. In addition to a competitive salary, we offer a variety of benefits to support your needs, including: Medical, dental, and vision insurance - 100% paid for by CoreWeave Company-paid Life Insurance Voluntary supplemental life insurance Short and long-term disability insurance Flexible Spending Account Health Savings Account Tuition Reimbursement Ability to Participate in Employee Stock Purchase Program (ESPP) Mental Wellness Benefits through Spring Health Family-Forming support provided by Carrot Paid Parental Leave Flexible, full-service childcare support with Kinside 401(k) with a generous employer match Flexible PTO Catered lunch each day in our office and data center locations A casual work environment A work culture focused on innovative disruption Our Workplace While we prioritize a hybrid work environment, remote work may be considered for candidates located more than 30 miles from an office, based on role requirements for specialized skill sets. New hires will be invited to attend onboarding at one of our hubs within their first month. Teams also gather quarterly to support collaboration. California Consumer Privacy Act - California applicants only CoreWeave is an equal opportunity employer, committed to fostering an inclusive and supportive workplace. All qualified applicants and candidates will receive consideration for employment without regard to race, color, religion, sex, disability, age, sexual orientation, gender identity, national origin, veteran status, or genetic information. As part of this commitment and consistent with the Americans with Disabilities Act (ADA) , CoreWeave will ensure that qualified applicants and candidates with disabilities are provided reasonable accommodations for the hiring process, unless such accommodation would cause an undue hardship. If reasonable accommodation is needed, please contact: careers@coreweave.com . Export Control Compliance This position requires access to export controlled information. To conform to U.S. Government export regulations applicable to that information, applicant must either be (A) a U.S. person, defined as a (i) U.S. citizen or national, (ii) U.S. lawful permanent resident (green card holder), (iii) refugee under 8 U.S.C. § 1157, or (iv) asylee under 8 U.S.C. § 1158, (B) eligible to access the export controlled information without a required export authorization, or (C) eligible and reasonably likely to obtain the required export authorization from the applicable U.S. government agency. CoreWeave may, for legitimate business reasons, decline to pursue any export licensing process. Create a Job Alert Interested in building your career at CoreWeave? Get future opportunities sent straight to your email. Create alert Apply for this job

• indicates a required field Autofill with MyGreenhouse First Name
• Last Name
• Email
• Phone Country
• Phone
• 244 results found No results found Afghanistan +93 Åland Islands +358 Albania +355 Algeria +213 American Samoa +1 Andorra +376 Angola +244 Anguilla +1 Antigua & Barbuda +1 Argentina +54 Armenia +374 Aruba +297 Ascension Island +247 Australia +61 Austria +43 Azerbaijan +994 Bahamas +1 Bahrain +973 Bangladesh +880 Barbados +1 Belarus +375 Belgium +32 Belize +501 Benin +229 Bermuda +1 Bhutan +975 Bolivia +591 Bosnia & Herzegovina +387 Botswana +267 Brazil +55 British Indian Ocean Territory +246 British Virgin Islands +1 Brunei +673 Bulgaria +359 Burkina Faso +226 Burundi +257 Cambodia +855 Cameroon +237 Canada +1 Cape Verde +238 Caribbean Netherlands +599 Cayman Islands +1 Central African Republic +236 Chad +235 Chile +56 China +86 Christmas Island +61 Cocos (Keeling) Islands +61 Colombia +57 Comoros +269 Congo - Brazzaville +242 Congo - Kinshasa +243 Cook Islands +682 Costa Rica +506 Côte d'Ivoire +225 Croatia +385 Cuba +53 Curaçao +599 Cyprus +357 Czechia +420 Denmark +45 Djibouti +253 Dominica +1 Dominican Republic +1 Ecuador +593 Egypt +20 El Salvador +503 Equatorial Guinea +240 Eritrea +291 Estonia +372 Eswatini +268 Ethiopia +251 Falkland Islands +500 Faroe Islands +298 Fiji +679 Finland +358 France +33 French Guiana +594 French Polynesia +689 Gabon +241 Gambia +220 Georgia +995 Germany +49 Ghana +233 Gibraltar +350 Greece +30 Greenland +299 Grenada +1 Guadeloupe +590 Guam +1 Guatemala +502 Guernsey +44 Guinea +224 Guinea-Bissau +245 Guyana +592 Haiti +509 Honduras +504 Hong Kong SAR China +852 Hungary +36 Iceland +354 India +91 Indonesia +62 Iran +98 Iraq +964 Ireland +353 Isle of Man +44 Israel +972 Italy +39 Jamaica +1 Japan +81 Jersey +44 Jordan +962 Kazakhstan +7 Kenya +254 Kiribati +686 Kosovo +383 Kuwait +965 Kyrgyzstan +996 Laos +856 Latvia +371 Lebanon +961 Lesotho +266 Liberia +231 Libya +218 Liechtenstein +423 Lithuania +370 Luxembourg +352 Macao SAR China +853 Madagascar +261 Malawi +265 Malaysia +60 Maldives +960 Mali +223 Malta +356 Marshall Islands +692 Martinique +596 Mauritania +222 Mauritius +230 Mayotte +262 Mexico +52 Micronesia +691 Moldova +373 Monaco +377 Mongolia +976 Montenegro +382 Montserrat +1 Morocco +212 Mozambique +258 Myanmar (Burma) +95 Namibia +264 Nauru +674 Nepal +977 Netherlands +31 New Caledonia +687 New Zealand +64 Nicaragua +505 Niger +227 Nigeria +234 Niue +683 Norfolk Island +672 North Korea +850 North Macedonia +389 Northern Mariana Islands +1 Norway +47 Oman +968 Pakistan +92 Palau +680 Palestinian Territories +970 Panama +507 Papua New Guinea +675 Paraguay +595 Peru +51 Philippines +63 Poland +48 Portugal +351 Puerto Rico +1 Qatar +974 Réunion +262 Romania +40 Russia +7 Rwanda +250 Samoa +685 San Marino +378 São Tomé & Príncipe +239 Saudi Arabia +966 Senegal +221 Serbia +381 Seychelles +248 Sierra Leone +232 Singapore +65 Sint Maarten +1 Slovakia +421 Slovenia +386 Solomon Islands +677 Somalia +252 South Africa +27 South Korea +82 South Sudan +211 Spain +34 Sri Lanka +94 St.

Barthélemy +590 St. Helena +290 St. Kitts & Nevis +1 St. Lucia +1 St. Martin +590 St. Pierre & Miquelon +508 St. Vincent & Grenadines +1 Sudan +249 Suriname +597 Svalbard & Jan Mayen +47 Sweden +46 Switzerland +41 Syria +963 Taiwan +886 Tajikistan +992 Tanzania +255 Thailand +66 Timor-Leste +670 Togo +228 Tokelau +690 Tonga +676 Trinidad & Tobago +1 Tunisia +216 Turkey +90 Turkmenistan +993 Turks & Caicos Islands +1 Tuvalu +688 U.S. Virgin Islands +1 Uganda +256 Ukraine +380 United Arab Emirates +971 United Kingdom +44 United States +1 Uruguay +598 Uzbekistan +998 Vanuatu +678 Vatican City +39 Venezuela +58 Vietnam +84 Wallis & Futuna +681 Western Sahara +212 Yemen +967 Zambia +260 Zimbabwe +263 Resume/CV

• Attach Attach Dropbox Enter manually Enter manually Accepted file types: pdf, doc, docx, txt, rtf Cover Letter Attach Attach Dropbox Enter manually Enter manually Accepted file types: pdf, doc, docx, txt, rtf LinkedIn Profile Website Where are you currently located? (city, state)
• Are you comfortable interviewing for the salary outlined in the job description ? Select... Do you have the right to work in the country you are applying to?
• Select... Current Company
• Do you now or will you in the future require sponsorship to work in the country you are applying to?
• Select... Legal Address
• Are you open to working 3 days from one of our office hubs in

NYC, NJ, CA, WA?

• Select... Are you a former CoreWeave employee?
• Select... Are you now or have you ever been employed by CoreWeave?
• Select... Please indicate whether you are a "U.S. person". U.S. person is defined as a (i) U.S. citizen or national; (ii) U.S. lawful permanent resident; (iii) refugee under 8 U.S.C. § 1157; or (iv) asylee under 8 U.S.C. § 1158.
• Select.

..

Note:

This information is required and only used by CoreWeave to ensure compliance with U.S. export control laws and regulations. Do you now or will you in the future require sponsorship to work in the United States? (e.g.

H1-B, H1B1, TN, E3, CPT, OPT/STEM

OPT, H4, J2, or other visa type)

• Select.

.. Voluntary Self-Identification For government reporting purposes, we ask candidates to respond to the below self-identification survey. Completion of the form is entirely voluntary. Whatever your decision, it will not be considered in the hiring process or thereafter. Any information that you do provide will be recorded and maintained in a confidential file. As set forth in CoreWeave's Equal Employment Opportunity policy, we do not discriminate on the basis of any protected group status under any applicable law. Gender Select... Are you Hispanic/Latino? Select... Race & Ethnicity Definitions If you believe you belong to any of the categories of protected veterans listed below, please indicate by making the appropriate selection. As a government contractor subject to the Vietnam Era Veterans Readjustment Assistance Act (VEVRAA), we request this information in order to measure the effectiveness of the outreach and positive recruitment efforts we undertake pursuant to VEVRAA. Classification of protected categories is as follows: A "disabled veteran" is one of the following: a veteran of the U.S. military, ground, naval or air service who is entitled to compensation (or who but for the receipt of military retired pay would be entitled to compensation) under laws administered by the Secretary of Veterans Affairs; or a person who was discharged or released from active duty because of a service-connected disability. A "recently separated veteran" means any veteran during the three-year period beginning on the date of such veteran's discharge or release from active duty in the U.S. military, ground, naval, or air service. An "active duty wartime or campaign badge veteran" means a veteran who served on active duty in the U.S. military, ground, naval or air service during a war, or in a campaign or expedition for which a campaign badge has been authorized under the laws administered by the Department of Defense. An "Armed forces service medal veteran" means a veteran who, while serving on active duty in the U.S. military, ground, naval or air service, participated in a United States military operation for which an Armed Forces service medal was awarded pursuant to Executive Order 12985. Veteran Status Select... Voluntary Self-Identification of Disability Form CC-305 Page 1 of 1 OMB Control Number 1250-0005 Expires 04/30/2026 Why are you being asked to complete this form? We are a federal contractor or subcontractor. The law requires us to provide equal employment opportunity to qualified people with disabilities. We have a goal of having at least 7% of our workers as people with disabilities. The law says we must measure our progress towards this goal. To do this, we must ask applicants and employees if they have a disability or have ever had one. People can become disabled, so we need to ask this question at least every five years. Completing this form is voluntary, and we hope that you will choose to do so. Your answer is confidential. No one who makes hiring decisions will see it. Your decision to complete the form and your answer will not harm you in any way. If you want to learn more about the law or this form, visit the U.S. Department of Labor's Office of Federal Contract Compliance Programs (OFCCP) website at www.dol.gov/ofccp . How do you know if you have a disability? A disability is a condition that substantially limits one or more of your "major life activities." If you have or have ever had such a condition, you are a person with a disability.