Machine Learning Engineer, Inference
Machine Learning Engineer, Inference
Rime builds voice AI for enterprises running customer experiences at scale. Our text-to-speech models are purpose-built for high-volume conversational deployments, engineered for the pronunciation accuracy, latency, and deployment flexibility that production environments actually demand.
We started from a different premise than the rest of the field: voice AI isn’t bottlenecked by model architecture. It’s bottlenecked by data. So before we trained a single model, we built our own corpus: full-duplex, studio-quality conversational speech, recorded and annotated by PhD linguists. That’s our moat. It’s also why enterprises pick Rime when pilots need to convert into production.
We’re backed by top-tier investors including Unusual Ventures, and we’ve built a team at the intersection of product, research, and craft. Building voice models is an art. We intend to master it.
Role Overview
We’re hiring a Machine Learning Engineer to own inference for Rime’s models in production. Voice is unforgiving because every millisecond shows up in the conversation. You’ll build the systems that turn our models into the lowest-latency, highest-throughput, most reliable speech systems in the industry.
What You’ll Own
In-house real-time speech-first inference stack: model compilation, kernel optimization, batching strategy, streaming output, the path from checkpoint to first-audio-byte.
Latency systems: TTFB targets across regions, KV cache management, speculative decoding, scheduler design
Deployment flexibility: cloud, on-prem, BYOC (SageMaker, Connect), the packaging and runtime story across heterogeneous environments.
Inference for full- and half-duplex models, including streaming codec encoding and decoding
What We’re Looking For
Strong software engineering fundamentals: Rust, Python, C++/CUDA welcome, distributed systems, comfort across the stack.
Hands-on experience serving ML models at scale in production, ideally for low-latency or streaming workloads.
Deep familiarity with inference engines (vLLM, SGLang), SDKs (TensorRT, ONNX, CUDA Graphs, Triton), etc.
Working knowledge of speech synthesis and/or speech recognition techniques.
Familiarity with multiple speech representations (neural codecs, semantic tokens, mel/STFT) and how they shape inference cost.
Experience optimizing transformer or autoregressive model inference: KV caching, quantization, paged attention, speculative decoding.
Willing to roll up your sleeves on unglamorous performance work — flame graphs, NSight traces, kernel tuning, paired with the agency to build the abstractions so the team doesn’t stay stuck doing it by hand.
Bias toward shipping.
Nice to Have
CUDA kernel authoring or Triton experience.
GPU profiling and microarchitecture intuition (H100, A100, L40S, Blackwell).
Experience with parallel model training infrastructure
Multi-tenant inference scheduling and fairness.
Comfort working close to research teams and influencing model architecture choices for inference-friendliness.
Why Join Rime
Build the inference stack behind a category-defining voice AI company.
Direct collaboration with founders, including a CEO with a Stanford computational linguistics PhD who takes latency as seriously as you do.
The systems you build determine what experiences our customers can deploy.
Meaningful equity upside.
High ownership, high standards, low bureaucracy.
What We Offer
Competitive base + meaningful early-stage equity
Remote-friendly
Visa sponsorship available
Access to a proprietary, full-duplex, studio-quality conversational speech corpus
Compute and tooling to do the work
Direct influence on the future of voice AI
At Rime, we...
Are outliers
Cut through the hype to focus on the craft
Move fast with agency and freedom
Maintain a growth mindset, finding joy in the struggle
Do the right things, knowing that it'll lead to making money
If that sounds like you too, you'll be a great fit for Rime!
Apply To This Job
Rime builds voice AI for enterprises running customer experiences at scale. Our text-to-speech models are purpose-built for high-volume conversational deployments, engineered for the pronunciation accuracy, latency, and deployment flexibility that production environments actually demand.
We started from a different premise than the rest of the field: voice AI isn’t bottlenecked by model architecture. It’s bottlenecked by data. So before we trained a single model, we built our own corpus: full-duplex, studio-quality conversational speech, recorded and annotated by PhD linguists. That’s our moat. It’s also why enterprises pick Rime when pilots need to convert into production.
We’re backed by top-tier investors including Unusual Ventures, and we’ve built a team at the intersection of product, research, and craft. Building voice models is an art. We intend to master it.
Role Overview
We’re hiring a Machine Learning Engineer to own inference for Rime’s models in production. Voice is unforgiving because every millisecond shows up in the conversation. You’ll build the systems that turn our models into the lowest-latency, highest-throughput, most reliable speech systems in the industry.
What You’ll Own
In-house real-time speech-first inference stack: model compilation, kernel optimization, batching strategy, streaming output, the path from checkpoint to first-audio-byte.
Latency systems: TTFB targets across regions, KV cache management, speculative decoding, scheduler design
Deployment flexibility: cloud, on-prem, BYOC (SageMaker, Connect), the packaging and runtime story across heterogeneous environments.
Inference for full- and half-duplex models, including streaming codec encoding and decoding
What We’re Looking For
Strong software engineering fundamentals: Rust, Python, C++/CUDA welcome, distributed systems, comfort across the stack.
Hands-on experience serving ML models at scale in production, ideally for low-latency or streaming workloads.
Deep familiarity with inference engines (vLLM, SGLang), SDKs (TensorRT, ONNX, CUDA Graphs, Triton), etc.
Working knowledge of speech synthesis and/or speech recognition techniques.
Familiarity with multiple speech representations (neural codecs, semantic tokens, mel/STFT) and how they shape inference cost.
Experience optimizing transformer or autoregressive model inference: KV caching, quantization, paged attention, speculative decoding.
Willing to roll up your sleeves on unglamorous performance work — flame graphs, NSight traces, kernel tuning, paired with the agency to build the abstractions so the team doesn’t stay stuck doing it by hand.
Bias toward shipping.
Nice to Have
CUDA kernel authoring or Triton experience.
GPU profiling and microarchitecture intuition (H100, A100, L40S, Blackwell).
Experience with parallel model training infrastructure
Multi-tenant inference scheduling and fairness.
Comfort working close to research teams and influencing model architecture choices for inference-friendliness.
Why Join Rime
Build the inference stack behind a category-defining voice AI company.
Direct collaboration with founders, including a CEO with a Stanford computational linguistics PhD who takes latency as seriously as you do.
The systems you build determine what experiences our customers can deploy.
Meaningful equity upside.
High ownership, high standards, low bureaucracy.
What We Offer
Competitive base + meaningful early-stage equity
Remote-friendly
Visa sponsorship available
Access to a proprietary, full-duplex, studio-quality conversational speech corpus
Compute and tooling to do the work
Direct influence on the future of voice AI
At Rime, we...
Are outliers
Cut through the hype to focus on the craft
Move fast with agency and freedom
Maintain a growth mindset, finding joy in the struggle
Do the right things, knowing that it'll lead to making money
If that sounds like you too, you'll be a great fit for Rime!
Apply To This Job