[Remote] Senior Manager, Psychometric Operations & Data Science

Note: The job is a remote job and is open to candidates in USA. The College Board is dedicated to expanding educational opportunities and evidence-based decision-making. They are seeking a Senior Manager in Psychometric Operations & Data Science to lead the operational execution and continuous improvement of psychometric data workflows for assessments, ensuring accuracy and efficiency in scoring and reporting processes.ResponsibilitiesManage recurring scoring cycles, ensuring that scoring, analysis, validation, and reporting readiness activities meet defined timelines, accuracy expectations, and QA standardsExecute and oversee psychometric data workflows related to item analysis, item calibration, scoring validation, equating support, reporting file readiness, and post-administration reviewPartner with psychometricians to operationalize technical guidance for scaling, equating, calibration, form rotation, and scoring rulesRun, review, and improve statistical and data validation checks using tools such as SQL, R, Python, or similar platformsConduct root-cause analysis when scoring irregularities, data discrepancies, or reporting issues ariseMaintain strong documentation, version control, traceability, and reproducibility standards across scoring and psychometric workflowsTranslate psychometric requirements into clear operational steps, data specifications, QA checks, and handoffs across teamsEnsure that scoring and psychometric processes are defensible, repeatable, and aligned with relevant professional expectations for educational assessmentIdentify and reduce manual, file-based, or error-prone processes across scoring, item analysis, calibration, validation, and reporting workflowsDefine business and operational requirements for improved psychometric data workflows, automated checks, dashboards, and pipeline enhancementsPartner with Technology to implement data workflow improvements, while ensuring that technical solutions meet the operational needs of R&M, Scoring, and ReportingDevelop or support scripts, queries, validation routines, and repeatable processes that improve speed, consistency, and transparencyEstablish practical standards for data inputs, outputs, file formats, naming conventions, documentation, and handoffsRepresent R&M in cross-functional conversations related to scoring data, reporting data, psychometric workflows, and assessment data operationsPromote consistent documentation, data definitions, process maps, and decision logs across psychometric and assessment data workflowsApply existing College Board data policies and responsible data practices within R&M workflowsHelp define access, retention, documentation, and data-use expectations for psychometric and operational assessment data, in partnership with appropriate governance and Technology partnersCommunicate complex operational, statistical, and data quality issues clearly to technical and non-technical audiencesMentor colleagues in reproducible data practices, workflow documentation, QA routines, and responsible use of automationContribute to a culture of accuracy, accountability, continuous improvement, and thoughtful innovationSkillsMaster's degree in Data Science, Statistics, Information Systems, Educational Measurement, Psychology, Psychometrics, Computer Science, or a related field5+ years of hands-on experience in psychometrics, educational measurement, advanced statistical analysis, or similarly rigorous quantitative environments, with strong technical fluency in SQL and at least one programming language such as R or PythonExperience supporting or managing scoring cycles, assessment data workflows, statistical analysis workflows, or reporting data processesStrong technical fluency in SQL and at least one programming language such as R or PythonExperience running data validation checks, documenting workflows, troubleshooting data discrepancies, and improving repeatability of analytic processesWorking knowledge of educational assessment concepts such as item analysis, classical test theory, item response theory, scaling, equating, calibration, scoring rules, or form developmentAbility to partner effectively with psychometricians, researchers, technology teams, scoring teams, and reporting teamsStrong attention to detail and a track record of supporting processes where accuracy, documentation, and quality control are essentialAbility to communicate clearly with both technical and non-technical stakeholdersBilingual proficiency in Spanish and EnglishAbility to travel 1–2 times a year to College Board offices or on behalf of College Board businessExperience with assessment programs, educational testing, credentialing, certification, admissions, large-scale assessment, or similar testing environmentsExperience improving or automating data workflows using R, Python, SQL, or related toolsFamiliarity with structured and semi-structured data formats, such as CSV, relational tables, JSON, or similar formatsFamiliarity with cloud-based data environments or modern data platformsComfort using AI-enabled tools to improve QA, anomaly detection, documentation, workflow automation, or analytic reviewExperience working with Technology teams to define requirements for data tools, pipelines, dashboards, or workflow improvementsBenefitsA meaningful career, a supportive team, and a comprehensive package designed to help you thriveWe’re a self-sustaining nonprofit that believes in fair and competitive compensation grounded in your qualifications, experience, impact, and the marketWe aim to make our best offer upfront, rooted in fairness, transparency, and market dataWe adjust salaries by location to ensure fairness, no matter where you liveYou’ll have open, transparent conversations about compensation, benefits, and what it’s like to work at College Board throughout your hiring processCompany OverviewCollege Board is a not-for-profit organization that clears a path for all students to own their future through the Advanced Placement Program, the SAT, Official SAT Practice on Khan Academy, BigFuture, and more. It was founded in 1899, and is headquartered in New York, New York, USA, with a workforce of 1001-5000 employees. Its website is http://www.collegeboard.org.