[Remote] Senior Engineer II, AI Inference Engine Systems

Note: The job is a remote job and is open to candidates in USA. DigitalOcean is expanding its AI Infrastructure layer to support the next generation of AI-driven applications. They are seeking a Senior Engineer II to join their AI Inference Engine Systems team, responsible for designing, developing, and delivering high-scale data plane services that power their Inference as a Service offering.ResponsibilitiesAct as a technical leader on the team, driving the end-to-end design, development, and delivery of critical data plane components hosting large generative AI modelsArchitect and refine system design proposals for our high-scale, multi-tenant AI inference cloud ecosystem, ensuring they meet rigorous availability and resiliency standardsImplement and optimize distributed inference hosting using techniques like tensor/data parallelism, KV cache optimizations, and smart routingWork cross-functionally with Product Managers, customer-facing teams, and other engineering teams to align technical roadmaps with customer needsCoach and mentor junior engineers, fostering a culture of technical excellence and continuous improvementMaintain and operate critical, high-scale services, utilizing observability tools and defining SLOs to ensure superior platform healthSkillsStrong experience with microservices, messaging systems, databases, and infrastructure as codeHands-on experience hosting large language or multimodal models using inference engines like vLLM, SGLang, or ModularFamiliarity with distributed inference serving frameworks such as llm-d, NVIDIA Dynamo, or Ray ServeUnderstanding of GPU-level optimization and experience with interconnect technologies like NVlink, XGMI, or RoCEKnowledge of common LLM architectures and optimization techniques (e.g., continuous batching, quantization)Expert-level proficiency in GoLang or Python and familiarity with gRPCProven experience shipping customer-facing software products and running critical services in a high-scale environment similar to DigitalOceanExperience integrating and building with open-source softwareBenefitsWe provide employees with reimbursement for relevant conferences, training, and education.All employees have access to LinkedIn Learning's 10,000+ courses to support their continued growth and development.Employee Assistance ProgramLocal Employee MeetupsFlexible time off policyYou may qualify for a bonus in addition to base salary; bonus amounts are determined based on company and individual performance.Equity compensation to eligible employees, including equity grants upon hire and the option to participate in our Employee Stock Purchase Program.Company OverviewDigitalOcean provides a cloud platform to deploy, manage, and scale applications of any size. It was founded in 2012, and is headquartered in New York, New York, USA, with a workforce of 1001-5000 employees. Its website is http://www.digitalocean.com.Company H1B SponsorshipDigitalOcean has a track record of offering H1B sponsorships, with 8 in 2026, 30 in 2025, 8 in 2024, 9 in 2023, 22 in 2022, 11 in 2021, 2 in 2020. Please note that this does not guarantee sponsorship for this specific role.