Experienced Big Data Engineer – Hadoop, Spark & Real-Time Data Processing Solutions
Job Description:
Join arenaflex as a Big Data Engineer
Are you ready to be part of a transformative technology team that powers digital experiences for millions of customers worldwide? At arenaflex, we're engineering the future of financial technology, and we need talented individuals like you to help us build innovative solutions that make a real impact on people's lives.
As a Big Data Engineer at arenaflex, you'll join a collaborative team of passionate technologists who are solving complex problems at scale. You'll have the opportunity to work with cutting-edge technologies including Hadoop, Spark, Python, and SQL to build robust data pipelines that drive critical business decisions. Your work will directly contribute to powering the analytics and machine learning capabilities that make arenaflex a leader in the financial services industry.
What You'll Do
In this role, you'll be responsible for designing, developing, and maintaining large-scale data processing systems that handle massive volumes of data in real-time and batch modes. You'll work as an integral member of our agile product engineering team, collaborating with data scientists, analysts, and business stakeholders to deliver innovative solutions across multiple business lines.
Software Development & Engineering
Design and implement scalable data pipelines using Hadoop, Spark (Core, SQL, and PySpark), and MapReduce technologies
Write high-quality, maintainable code with proper documentation, following established best practices and coding standards
Participate actively in code reviews, providing constructive feedback to peers and incorporating feedback into your own work
Develop and execute automated testing strategies to ensure code quality and system reliability
Troubleshoot and resolve complex issues in data processing systems, identifying root causes and implementing sustainable solutions
Automate deployment processes across development, testing, integration, and production environments
Implement auto-scaling mechanisms to handle varying workloads and ensure system performance under peak conditions
Data Architecture & Design
Design and build advanced data pipelines for both batch and real-time data processing
Develop efficient data warehouse solutions and optimize complex SQL queries (Hive/PySpark DataFrames) for large-scale data processing
Create robust data models that support business analytics and reporting requirements
Optimize join operations and data transformations to process massive datasets efficiently
Implement data quality checks and monitoring mechanisms to ensure data integrity
Apply visualization techniques and rapid prototyping to accelerate concept development and stakeholder alignment
Leadership & Team Collaboration
Serve as a core contributor on agile squads, driving user story analysis, elaboration, and technical implementation
Lead by example by embracing innovation and taking calculated risks to push boundaries
Mentor junior team members and contribute to their professional growth
Identify opportunities to adopt emerging technologies and drive continuous improvement
Collaborate with cross-functional teams to deliver integrated solutions that meet business objectives
Contribute to the broader engineering community through knowledge sharing and best practice adoption
What We're Looking For
We're seeking a skilled professional with a passion for data engineering and a track record of delivering high-quality technical solutions. The ideal candidate thrives in a fast-paced environment and enjoys tackling complex technical challenges.
Essential Qualifications
Education: Bachelor's degree in Engineering, Computer Science, Information Systems, or a related technical field. A Master's degree in Computer Applications or equivalent is a plus.
Experience: 3+ years of hands-on software development experience with Big Data technologies
Technical Expertise: Strong proficiency in MapReduce, Hive, and Spark (including Core, SQL, and PySpark)
Data Warehouse Knowledge: Solid understanding of data warehousing concepts and dimensional modeling
Programming Skills: Strong coding abilities in Core Java or Python/Scala
Distributed Systems: Expert knowledge of Hadoop and Spark architecture, including working principles and ecosystem components
SQL Proficiency: Advanced skills in writing and optimizing complex SQL queries, particularly for Hive and PySpark DataFrames
Scripting: Experience with UNIX shell scripting for automation and data processing
System Development: Demonstrated experience in the full lifecycle of application development, from analysis and design through testing and deployment
Documentation: Ability to create comprehensive technical and functional specifications
Preferred Qualifications
Experience with cloud platforms such as GCP (Google Cloud Platform) or AWS (Amazon Web Services)
Knowledge of microservices architecture and building scalable, resilient solutions
3+ years of experience designing and implementing solutions using Kafka streams or message queues
Familiarity with version control systems like GitHub or Bitbucket and CI/CD pipeline implementation
Experience with NoSQL databases such as HBase, Couchbase, or MongoDB
Strong analytical and problem-solving abilities
Excellent communication skills to effectively collaborate with technical and non-technical stakeholders
Strong project management capabilities with the ability to prioritize and deliver on time
Why Join arenaflex?
At arenaflex, we believe that our people are our greatest asset. We're committed to creating an inclusive, supportive environment where every voice matters and where you have the autonomy to make decisions about the technologies you use to solve challenging problems.
Professional Growth & Development
We invest heavily in your continued learning and career advancement. You'll have dedicated time and resources to expand your skills, learn new technologies, and stay current with industry trends. Whether you're looking to deepen your expertise in Big Data technologies, explore cloud architecture, or develop leadership skills, arenaflex provides the pathways and support you need to achieve your goals.
Work-Life Balance
We understand the importance of maintaining a healthy balance between your professional and personal life. Our flexible work arrangements, generous time-off policies, and respect for boundaries ensure you can bring your best self to work every day while still having time for what matters most outside of work.
Collaborative Culture
You'll work alongside talented engineers in an open, supportive environment where your voice is truly valued. We encourage healthy debate, creative thinking, and pushing the boundaries of what's possible. Our team culture emphasizes collaboration over competition, and we celebrate both individual achievements and team successes.
Innovation Opportunities
At arenaflex, innovation isn't just encouraged—it's expected. You'll have the freedom to experiment with new technologies, propose innovative solutions, and challenge the status quo. We support open-source contributions and believe in giving back to the broader engineering community.
Compensation & Benefits
We offer competitive compensation packages that recognize your skills, experience, and contributions. Our comprehensive benefits package includes:
Competitive Salary: $28 per hour commensurate with experience and qualifications
Health & Wellness: Comprehensive medical, dental, and vision insurance coverage
Retirement Savings: 401(k) retirement plan with generous company matching
Paid Time Off: Generous vacation, sick leave, and personal days
Professional Development: Annual learning budget for courses, certifications, and conferences
Parental Leave: Supportive leave policies for growing families
Employee Assistance Program: Confidential support for personal and professional challenges
Wellness Programs: Resources and incentives for maintaining physical and mental well-being
Employee Discounts: Access to exclusive deals and discounts on products and services
Location & Work Environment
This position is based in Phoenix, USA. You'll have access to modern office facilities designed for collaboration and productivity. While we value in-person collaboration, we also offer flexible hybrid work arrangements that allow you to balance remote work with office presence based on team needs and project requirements.
Ready to Make an Impact?
If you're excited about working with cutting-edge technologies, solving complex problems at scale, and being part of a team that's shaping the future of financial services, we want to hear from you. Join arenaflex and discover where your skills and ambitions can take your career.
At arenaflex, we believe in providing equal opportunities to all candidates. We encourage applications from individuals of all backgrounds and experiences. All offers of employment are contingent upon successful completion of a background check, in accordance with applicable laws and regulations.
Take the next step in your career and apply today. We can't wait to see the amazing things you'll accomplish as part of the arenaflex team!
Join arenaflex as a Big Data Engineer
Are you ready to be part of a transformative technology team that powers digital experiences for millions of customers worldwide? At arenaflex, we're engineering the future of financial technology, and we need talented individuals like you to help us build innovative solutions that make a real impact on people's lives.
As a Big Data Engineer at arenaflex, you'll join a collaborative team of passionate technologists who are solving complex problems at scale. You'll have the opportunity to work with cutting-edge technologies including Hadoop, Spark, Python, and SQL to build robust data pipelines that drive critical business decisions. Your work will directly contribute to powering the analytics and machine learning capabilities that make arenaflex a leader in the financial services industry.
What You'll Do
In this role, you'll be responsible for designing, developing, and maintaining large-scale data processing systems that handle massive volumes of data in real-time and batch modes. You'll work as an integral member of our agile product engineering team, collaborating with data scientists, analysts, and business stakeholders to deliver innovative solutions across multiple business lines.
Software Development & Engineering
Design and implement scalable data pipelines using Hadoop, Spark (Core, SQL, and PySpark), and MapReduce technologies
Write high-quality, maintainable code with proper documentation, following established best practices and coding standards
Participate actively in code reviews, providing constructive feedback to peers and incorporating feedback into your own work
Develop and execute automated testing strategies to ensure code quality and system reliability
Troubleshoot and resolve complex issues in data processing systems, identifying root causes and implementing sustainable solutions
Automate deployment processes across development, testing, integration, and production environments
Implement auto-scaling mechanisms to handle varying workloads and ensure system performance under peak conditions
Data Architecture & Design
Design and build advanced data pipelines for both batch and real-time data processing
Develop efficient data warehouse solutions and optimize complex SQL queries (Hive/PySpark DataFrames) for large-scale data processing
Create robust data models that support business analytics and reporting requirements
Optimize join operations and data transformations to process massive datasets efficiently
Implement data quality checks and monitoring mechanisms to ensure data integrity
Apply visualization techniques and rapid prototyping to accelerate concept development and stakeholder alignment
Leadership & Team Collaboration
Serve as a core contributor on agile squads, driving user story analysis, elaboration, and technical implementation
Lead by example by embracing innovation and taking calculated risks to push boundaries
Mentor junior team members and contribute to their professional growth
Identify opportunities to adopt emerging technologies and drive continuous improvement
Collaborate with cross-functional teams to deliver integrated solutions that meet business objectives
Contribute to the broader engineering community through knowledge sharing and best practice adoption
What We're Looking For
We're seeking a skilled professional with a passion for data engineering and a track record of delivering high-quality technical solutions. The ideal candidate thrives in a fast-paced environment and enjoys tackling complex technical challenges.
Essential Qualifications
Education: Bachelor's degree in Engineering, Computer Science, Information Systems, or a related technical field. A Master's degree in Computer Applications or equivalent is a plus.
Experience: 3+ years of hands-on software development experience with Big Data technologies
Technical Expertise: Strong proficiency in MapReduce, Hive, and Spark (including Core, SQL, and PySpark)
Data Warehouse Knowledge: Solid understanding of data warehousing concepts and dimensional modeling
Programming Skills: Strong coding abilities in Core Java or Python/Scala
Distributed Systems: Expert knowledge of Hadoop and Spark architecture, including working principles and ecosystem components
SQL Proficiency: Advanced skills in writing and optimizing complex SQL queries, particularly for Hive and PySpark DataFrames
Scripting: Experience with UNIX shell scripting for automation and data processing
System Development: Demonstrated experience in the full lifecycle of application development, from analysis and design through testing and deployment
Documentation: Ability to create comprehensive technical and functional specifications
Preferred Qualifications
Experience with cloud platforms such as GCP (Google Cloud Platform) or AWS (Amazon Web Services)
Knowledge of microservices architecture and building scalable, resilient solutions
3+ years of experience designing and implementing solutions using Kafka streams or message queues
Familiarity with version control systems like GitHub or Bitbucket and CI/CD pipeline implementation
Experience with NoSQL databases such as HBase, Couchbase, or MongoDB
Strong analytical and problem-solving abilities
Excellent communication skills to effectively collaborate with technical and non-technical stakeholders
Strong project management capabilities with the ability to prioritize and deliver on time
Why Join arenaflex?
At arenaflex, we believe that our people are our greatest asset. We're committed to creating an inclusive, supportive environment where every voice matters and where you have the autonomy to make decisions about the technologies you use to solve challenging problems.
Professional Growth & Development
We invest heavily in your continued learning and career advancement. You'll have dedicated time and resources to expand your skills, learn new technologies, and stay current with industry trends. Whether you're looking to deepen your expertise in Big Data technologies, explore cloud architecture, or develop leadership skills, arenaflex provides the pathways and support you need to achieve your goals.
Work-Life Balance
We understand the importance of maintaining a healthy balance between your professional and personal life. Our flexible work arrangements, generous time-off policies, and respect for boundaries ensure you can bring your best self to work every day while still having time for what matters most outside of work.
Collaborative Culture
You'll work alongside talented engineers in an open, supportive environment where your voice is truly valued. We encourage healthy debate, creative thinking, and pushing the boundaries of what's possible. Our team culture emphasizes collaboration over competition, and we celebrate both individual achievements and team successes.
Innovation Opportunities
At arenaflex, innovation isn't just encouraged—it's expected. You'll have the freedom to experiment with new technologies, propose innovative solutions, and challenge the status quo. We support open-source contributions and believe in giving back to the broader engineering community.
Compensation & Benefits
We offer competitive compensation packages that recognize your skills, experience, and contributions. Our comprehensive benefits package includes:
Competitive Salary: $28 per hour commensurate with experience and qualifications
Health & Wellness: Comprehensive medical, dental, and vision insurance coverage
Retirement Savings: 401(k) retirement plan with generous company matching
Paid Time Off: Generous vacation, sick leave, and personal days
Professional Development: Annual learning budget for courses, certifications, and conferences
Parental Leave: Supportive leave policies for growing families
Employee Assistance Program: Confidential support for personal and professional challenges
Wellness Programs: Resources and incentives for maintaining physical and mental well-being
Employee Discounts: Access to exclusive deals and discounts on products and services
Location & Work Environment
This position is based in Phoenix, USA. You'll have access to modern office facilities designed for collaboration and productivity. While we value in-person collaboration, we also offer flexible hybrid work arrangements that allow you to balance remote work with office presence based on team needs and project requirements.
Ready to Make an Impact?
If you're excited about working with cutting-edge technologies, solving complex problems at scale, and being part of a team that's shaping the future of financial services, we want to hear from you. Join arenaflex and discover where your skills and ambitions can take your career.
At arenaflex, we believe in providing equal opportunities to all candidates. We encourage applications from individuals of all backgrounds and experiences. All offers of employment are contingent upon successful completion of a background check, in accordance with applicable laws and regulations.
Take the next step in your career and apply today. We can't wait to see the amazing things you'll accomplish as part of the arenaflex team!