[Remote] Senior Software Engineer - AI Inference

Note: The job is a remote job and is open to candidates in USA. NVIDIA is the platform upon which every new AI‑powered application is built, and they are seeking a Senior Software Engineer – AI Inference to advance open‑source LLM serving. The role involves contributing to upstream inference engines and improving the underlying stack for high‑throughput, low‑latency inference at scale.ResponsibilitiesContribute features, fixes, and optimizations upstream to vLLM/SGLang: author PRs, participate in reviews, write benchmarks/tests, and help drive designs to completionImplement and optimize inference‑runtime capabilities: batching and scheduling policies, streaming, request lifecycle management, and KV‑cache efficiency (paging/sharding) to improve throughput and tail latencyProfile and improve hot paths across layers-from Python orchestration to C++/CUDA kernels-using data to guide optimization workImprove multi‑GPU inference performance and reliability: parallelism strategies, communication patterns, and resource utilization across NVIDIA platformsBuild and maintain performance and correctness regression tests to prevent slowdowns and ensure stable behavior across model and hardware configurationsCollaborate with model, platform, and SRE teams to translate production requirements into upstreamable solutions with strong operability and maintainabilitySkills5+ years building production software with solid systems engineering fundamentals and a track record of delivering performance or reliability improvementsExperience with LLM inference/serving stacks (e.g., vLLM, SGLang) and an understanding of the tradeoffs that drive real production performanceStrong programming skills in Python plus C++ and/or CUDA; ability to debug and optimize performance‑critical codeExperience with profiling and performance investigation (microbenchmarks, flame graphs, GPU profiling) and a measurement‑driven mindsetFamiliarity with distributed systems concepts and concurrency (queues/schedulers, multi‑process/multi‑threading, scaling across GPUs/nodes)Strong communication skills and comfort working with open‑source communities (issues, PR discussions, code review)BS/MS in Computer Science, Computer Engineering, or related field (or equivalent experience)Open‑source contributions to vLLM, SGLang, PyTorch, Triton, NCCL, Dynamo or adjacent serving/runtime projectsShipped performance work such as improved attention/KV cache efficiency, speculative decoding, scheduler improvements, quantization-aware serving, or streaming latency reductionsExperience building reproducible benchmarking and performance regression infrastructure for latency/throughputSystems performance background spanning memory bandwidth, kernel fusion, PCIe/NVLink effects, and network fabrics (e.g., InfiniBand)BenefitsEquityBenefitsCompany OverviewNVIDIA is a computing platform company operating at the intersection of graphics, HPC, and AI. It was founded in 1993, and is headquartered in Santa Clara, California, USA, with a workforce of 10001+ employees. Its website is https://www.nvidia.com.Company H1B SponsorshipNVIDIA has a track record of offering H1B sponsorships, with 448 in 2026, 1872 in 2025, 1354 in 2024, 976 in 2023, 835 in 2022, 601 in 2021, 529 in 2020. Please note that this does not guarantee sponsorship for this specific role.