[Remote] Senior Data Platform Engineer

Note: The job is a remote job and is open to candidates in USA. Stack AV is developing revolutionary AI and advanced autonomous systems to enhance safety and efficiency in the trucking transportation industry. The Senior Data Platform Engineer will design and operate high scale data systems, enabling engineers to efficiently run compute and data intensive workloads on Stack AV infrastructure.ResponsibilitiesDesign and operate distributed storage systems for scheduling and executing large-scale batch workloadsBuild and maintain an open source, modern data platformOptimize utilization of storage resourcesImprove reliability and fault tolerance of large-scale storage systems and data platform componentsCollaborate with teams across the company to understand workload requirements and improve platform capabilitiesContribute to platform tooling, automation, and CI/CD workflowsSkills7+ years of experience building and operating distributed storage systems or modern data platformsExperience operating streaming platforms such as Kafka or PulsarFluent in Python, and SQL, with experience writing and maintaining highly available data applications using Trino and Apache SparkKnowledge of table formats (Iceberg, Delta Lake, Hudi, Xtable)Experience operating and optimizing at least one RDBMS (Postgres, MySQL)Strong debugging and problem-solving skills in complex distributed systemsAbility to collaborate across teams and communicate technical concepts clearlyCompany OverviewStack AV operates in the transportation industry that develops advanced autonomous systems. It was founded in 2023, and is headquartered in Pittsburgh, Pennsylvania, USA, with a workforce of 51-200 employees. Its website is https://www.stackav.com.