Senior Data Platform Engineer - Big Data / Hadoop

The Hartford is seeking a seasoned Senior Data Platform Engineer to join our Cloud Big Data Platform Engineering & Operations team. This role is instrumental in fostering a customer-first mindset and ensuring the stability, scalability, and reliability of our data platforms to support the evolving needs of Data & Analytics applications across the enterprise. 

As a technical lead, you will apply your deep expertise in AWS Big Data/EMR infrastructure, Infrastructure as Code (IaC), security, automation and observability to engineer, optimize, and maintain robust, scalable solutions. You will collaborate closely with data engineers to analyze requirements & challenges to recommend platform-driven solutions that maximize performance and efficiency. 

We’re looking for a passionate technologist who thrives in a dynamic, fast-paced environment and is committed to building resilient, future-ready data platforms. You will mentor and guide fellow platform and reliability engineers, promoting a culture of technical excellence, innovation, and collaboration. 

RESPONSIBILITIES 

• Administer and engineer Big Data platforms across multiple Hadoop clusters in the cloud (AWS EMR), including serverless and containerized environments, to ensure scalability, performance, and reliability. 

• Design, implement and maintain highly scalable and resilient multi-tenant Data Platforms through Infrastructure as Code (IAC) aligning with The Hartford’s engineering, security and governance principles 

• Ensure operational excellence, independently drive the triaging and service restoration of all high impact incidents to minimize the mean time to service restoration and impact to the business. Demonstrate end to end ownership 

• Apply Site Reliability Engineering (SRE) principles to design and implement robust tooling, proactive alerting, and automated response mechanisms that identify, mitigate, and resolve reliability risks—focusing on prevention, early detection, and continuous improvement through automation. 

• Own and evolve the architecture of Platform, PaaS, and SaaS solutions to meet current and future business need-driving innovation, scalability, and operational excellence. 

• Participate in an on-call rotation, providing hands-on technical expertise during service-impacting incidents to ensure rapid diagnosis, effective resolution, and continuous improvement of system reliability. 

• Evaluate, implement, and manage emerging data technologies with a focus on big data, analytics, data wrangling, business intelligence, and data visualization to drive innovation and efficiency. 

• Serve as a subject matter expert and technical lead for data platforms, tools, and application interfaces - driving root cause analysis, resolving complex technical issues, and ensuring platform reliability and performance. 

• Provide technical leadership and mentorship to junior and mid-level data engineers, fostering skill development and promoting engineering best practices. 

• Collaborate with and empower data engineers, data scientists, and business analysts by enabling self-service capabilities for data wrangling, exploration, and analysis 

• Develop training materials and deliver end-user training sessions to drive adoption, ensure effective use of data solutions, and enhance customer engagement. 

• Support the documentation, metadata management, and visualization of data assets to promote data discoverability, transparency, and self-service analytics. 

• Foster a culture of accountability and collaboration by building strong team commitment to shared priorities and strategic goals. 

QUALIFICATIONS 

• Bachelor's degree in computer science, Computer Engineering, or a closely related field (or foreign equivalent). Relevant experience may be obtained through a qualifying post-baccalaureate academic program. 

• Extensive expertise in platform administration, big data technologies, data engineering, analytics, and operations. 

• Proficiency in a broad range of tools and concepts including Hadoop, Linux, Python, SQL, Spark, Kerberos, cloud platforms, security protocols, performance tuning, machine learning algorithms, production engineering, job scheduling, and operational support 

• At least 7 years of progressive and diverse experience in IT, platform administration, database management, analytics or an equivalent combination of education and work experience 

• At least 3 years of hands-on engineering experience in developing platform solutions on AWS using Infrastructure as Code (CloudFormation, Terraform, Ansible) and CICD pipelines (Jenkins, Nexus, AWS CodeBuild, CodeDeploy or CodePipeline).  

• At least 3 years of experience providing architectural guidance and technical direction to developers, or platform administrators 

• Knowledge in data engineering, with exposure to designing and building data applications using database platforms. 

• Responsibilities should include data ingestion, data preparation, ETL processes, data aggregation, data mining, and the development of database and analytics applications 

• Experience in managing change through Change Management and Incident Management processes 

• Experience in implementing Reliability Engineering practices and Observability dashboards leveraging Splunk, Dynatrace, CloudWatch etc. will be a plus 

• Ability to learn new technologies quickly, and perform major job responsibilities proficiently within 6-12 months 

• Strong analytical ability, problem analysis techniques, and broad knowledge of alternatives technology 

• Strong communication skills, and the ability to work effectively with business and IT resources 

PREFERRED SKILLS 

• Advanced experience in primary AWS services (EMR, EKS, EC2, IAM, RDS, Route53 & S3, etc) 

• Experience in Configuration management using CloudFormation / Terraform. 

• Advanced experience with programming and/or scripting languages (Python, bash) 

• Preferred to have “AWS Solution Architect Certification”. 

• Preferred to have “Cloudera Admin Certification”. 

This role will have a Hybrid work schedule, with the expectation of working in an office (Columbus, OH, Chicago, IL, Hartford, CT or Charlotte, NC) 3 days a week (Tuesday through Thursday).

Candidates must be authorized to work in the US without company sponsorship. The company will not support the STEM OPT I-983 Training Plan endorsement for this position.