The Staff Product Support Engineer will design, optimize, and manage Hadoop and Spark applications, solve data challenges, and ensure operational stability in a 24/7 environment.
We are seeking a Staff Product Support Engineer / Product Specialist - Hadoop Operations SME who will be responsible for designing, optimizing, migrating, and scaling Hadoop and Spark-based data processing systems. This role involves hands-on experience with Hadoop and other core data operations, focusing on building resilient, high-performance distributed data systems.
You will collaborate with customer engineering teams to deliver high-throughput Hadoop, Nifi, and Spark applications and solve complex data challenges in migration, upgrades, reliability, and optimise post-migration system performance.
This role requires flexibility to work in rotational shifts, based on team coverage needs and customer demand. Candidates should be comfortable supporting operations in a 24/7 environment and be willing to adjust their working hours accordingly.
We’re looking for someone who perform:
- Design and optimise distributed Hadoop-based applications, ensuring low-latency, high-throughput performance for big data workloads.
- Troubleshooting: Provide expert-level support for data or performance issues in Hadoop, Nifi, and Spark jobs and clusters.
- Data Processing Expertise: Work extensively with large-scale data pipelines using Hadoop, Nifi, and Spark's core components.
- Performance Tuning: Conduct deep-dive performance analysis, debugging, and optimization of Nifi, Impala, and Spark jobs to reduce processing time and resource consumption.
- Cluster Management: Collaborate with DevOps and infrastructure teams to manage Nifi, Impala, and Spark clusters on platforms like Hadoop/YARN, Kubernetes, or cloud platforms (AWS EMR, GCP Dataproc, etc.).
- Migration: Provide dedicated support to ensure the stability and reliability of the new ODP Hadoop environment during and after migration. Promptly address evolving technical challenges and optimise system performance after migration to ODP.
Good to have:
- Master’s degree and Experience working with scripting languages (Scala, Python, Bash, PowerShell).
- Familiarity with virtual machine technologies and multi-node environment (50+ nodes).
- Proficient with Linux, NFS, and Windows, including application installation, scripting, and working with the command line.
- Working knowledge of application, server, and network security management concepts. Certification on any of the leading Cloud providers (AWS, Azure, GCP ) and/or Kubernetes.
- Knowledge of databases like MySQL and PostgreSQL.
- Be involved with and work on other support-related activities - Performing POC & assisting with Onboarding deployments of Acceldata & Hadoop distribution products.
Top Skills
Aws Emr
Bash
Gcp Dataproc
Hadoop
Impala
Kubernetes
Linux
MySQL
Nifi
Postgres
Powershell
Python
Scala
Spark
Similar Jobs
Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
Lead and manage software development teams, coordinate with product, design, and support, oversee daily development activities, mentor staff, integrate AI into workflows, enforce coding standards and best practices, and deliver high-quality solutions aligned with company priorities.
Top Skills:
Java,C++,Ruby,Shell,Javascript,Servicenow,Ai
Aerospace • Information Technology • Software • Cybersecurity • Design • Defense • Manufacturing
Design, build, and maintain secure CI/CD pipelines and tooling for safety-critical avionics software. Automate deployment, integration, testing, and security controls across cloud, container, and hybrid environments while supporting certification and cross-functional teams.
Top Skills:
AWSAzureCi/CdDevsecopsDockerGCPJavaKubernetesLinuxPythonWindows
Aerospace • Information Technology • Software • Cybersecurity • Design • Defense • Manufacturing
Develop and maintain Python-based DevOps tools, implement and update GitLab CI pipelines, create Bazel modules, and support migration and orchestration/packaging/testing automation for Boeing Software Factory. Collaborate with teams to integrate embedded software and CI/CD tooling across the organization.
Top Skills:
Python,Gitlab Ci,Bazel,Java,Rust,C,C++,Rtos,Ci/Cd,Embedded Systems,Agile
What you need to know about the Charlotte Tech Scene
Ranked among the hottest tech cities in 2024 by CompTIA, Charlotte is quickly cementing its place as a major U.S. tech hub. Home to more than 90,000 tech workers, the city’s ecosystem is primed for continued growth, fueled by billions in annual funding from heavyweights like Microsoft and RevTech Labs, which has created thousands of fintech jobs and made the city a go-to for tech pros looking for their next big opportunity.
Key Facts About Charlotte Tech
- Number of Tech Workers: 90,859; 6.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Lowe’s, Bank of America, TIAA, Microsoft, Honeywell
- Key Industries: Fintech, artificial intelligence, cybersecurity, cloud computing, e-commerce
- Funding Landscape: $3.1 billion in venture capital funding in 2024 (CED)
- Notable Investors: Microsoft, Google, Falfurrias Management Partners, RevTech Labs Foundation
- Research Centers and Universities: University of North Carolina at Charlotte, Northeastern University, North Carolina Research Campus
