[Remote] Senior Data Engineers

Note: The job is a remote job and is open to candidates in USA. CCC Intelligent Solutions Inc. is a leading cloud platform for the insurance economy, empowering over 35,000 businesses with innovative technology. They are seeking a Senior Data Engineer to develop large scale end-to-end data pipeline applications, automate processes, and mentor junior engineers while utilizing advanced technologies in data processing and cloud environments.ResponsibilitiesDevelop large scale end to end data pipeline applications, covering multiple data sources spread across data center and AWS cloudUse developed software applications to locate and analyze source data; create data flows to extract, profile, and store ingested data; define and build data cleansing and imputation; map to a common data model; transform to satisfy business rules and statistical computations; and validate data contentProduce software data building blocks, data models, and data flows, such as dimensional data, data feeds, dashboard reporting, and data science research and explorationProduce automated software tests of data flow components and for data content qualityAutomate orchestration and error handling for use by production operation teamsProvide technical expertise to diagnose errors from production support teamsGuide junior team members in performance tuning applications in distributed computing environmentsPerform root cause analysis on all data and processes and identify opportunities for improvementDevelop metadata-driven and fully parameterized data processing toolsMentor junior engineersSkillsMaster's degree in Computer Science, Computer Engineering, Management Information Systems or related field plus 2 years of experience in software development/data processing or analysis requiredHands-on experience with: Programming using Python & PySpark; Hadoop; HDFS, map-reduce, YARN, AWS EMR, Redshift, Terraform; Hive, HBase, parquet, ORC, Spark SQL, Sqoop, Apache Hudi; Orchestrating ETL pipelines involving data sourcing, transformations & publishing using Apache Airflow; Performance tuning applications in distributed computing environments; Designing & developing data pipeline applications with Apache Kafka; Advanced SQL for data profiling & data validation; Unix commands & scripting; performing root cause analysis on internal & external data & processes to identify opportunities for improvement; JIRA, Gitlab, Subversion; Development of metadata-driven & fully parameterized data processing tools; AWSBenefits401K MatchPaid time offAnnual Incentive Plan Performance BonusComprehensive health insuranceAdoption AssistanceTuition ReimbursementWellness ProgramsStock Purchase Plan optionsEmployee Resource GroupsCompany OverviewCCC Intelligent Solutions is a SaaS platform that offers digital and data services for the insurance and automotive industries. It was founded in 1980, and is headquartered in Chicago, Illinois, USA, with a workforce of 1001-5000 employees. Its website is http://www.cccis.com.Company H1B SponsorshipCCC Intelligent Solutions has a track record of offering H1B sponsorships, with 7 in 2026, 69 in 2025, 48 in 2024, 71 in 2023, 79 in 2022, 30 in 2021. Please note that this does not guarantee sponsorship for this specific role.