[Remote] Senior AI Kernel Engineer

Note: The job is a remote job and is open to candidates in USA. Modular is on a mission to revolutionize AI infrastructure by rebuilding the AI software stack. They are seeking a Senior AI Kernel Engineer to lead the design and optimization of high-performance kernels for AI inference on GPUs and custom accelerators, collaborating closely with various teams to enhance performance.ResponsibilitiesDesign, implement, and optimize performance-critical kernels for AI inference workloads (e.g., GEMM, attention, communication, fusion)Lead kernel-level optimization efforts across single-GPU, multi-GPU, and heterogeneous hardware environmentsMake informed trade-offs between latency, throughput, memory footprint, and numerical precisionDrive adoption of new hardware features (e.g., Tensor Cores, asynchronous execution, advanced memory spaces)Analyze performance using profilers, hardware counters, and microbenchmarks; translate insights into concrete improvementsWork closely with compiler and runtime teams to influence code generation, scheduling, and kernel fusion strategiesReview and mentor other engineers on kernel design, performance tuning, and best practicesContribute to technical roadmaps and long-term performance strategy for AI inferenceSkills5+ years of experience in performance-critical systems or kernel development (or equivalent depth of expertise)Strong proficiency in C/C++ and low-level programmingExtensive hands-on experience with GPU kernel programming (CUDA, HIP, or equivalent)Deep understanding of GPU architecture, including memory hierarchies, synchronization, and execution modelsProven track record of delivering measurable performance improvements in production systemsStrong problem-solving skills and ability to work independently on complex, ambiguous performance challengesExperience with PTX, assembly-level tuning, or code generation frameworks (e.g., Triton)Experience optimizing distributed or multi-GPU inference pipelinesFamiliarity with custom AI accelerators or domain-specific hardwareUnderstanding of modern AI models (e.g., transformers, LLMs, diffusion) from a systems and performance perspectiveContributions to open-source kernel libraries, compilers, or performance toolsExperience collaborating directly with hardware or compiler teamsBenefitsPremier insurance plansUp to 5% 401k matchingFlexible paid time offStock optionsAnnual target bonusEquityTeam Building EventsRegular team onsites and local meetups in Los Altos, CA as well as different citiesTraveling 2-4 times a year is expected for all rolesCompany OverviewModular provides AI infrastructure for deployment, serving, and programming GPUs. It was founded in 2022, and is headquartered in Palo Alto, California, USA, with a workforce of 51-200 employees. Its website is https://www.modular.com.Company H1B SponsorshipModular has a track record of offering H1B sponsorships, with 3 in 2026, 10 in 2025, 6 in 2024, 8 in 2023, 4 in 2022. Please note that this does not guarantee sponsorship for this specific role.