The Staff Product Support Engineer will design, optimize, and manage Hadoop and Spark applications, solve data challenges, and ensure operational stability in a 24/7 environment.
We are seeking a Staff Product Support Engineer / Product Specialist - Hadoop Operations SME who will be responsible for designing, optimizing, migrating, and scaling Hadoop and Spark-based data processing systems. This role involves hands-on experience with Hadoop and other core data operations, focusing on building resilient, high-performance distributed data systems.
You will collaborate with customer engineering teams to deliver high-throughput Hadoop, Nifi, and Spark applications and solve complex data challenges in migration, upgrades, reliability, and optimise post-migration system performance.
This role requires flexibility to work in rotational shifts, based on team coverage needs and customer demand. Candidates should be comfortable supporting operations in a 24/7 environment and be willing to adjust their working hours accordingly.
We’re looking for someone who perform:
- Design and optimise distributed Hadoop-based applications, ensuring low-latency, high-throughput performance for big data workloads.
- Troubleshooting: Provide expert-level support for data or performance issues in Hadoop, Nifi, and Spark jobs and clusters.
- Data Processing Expertise: Work extensively with large-scale data pipelines using Hadoop, Nifi, and Spark's core components.
- Performance Tuning: Conduct deep-dive performance analysis, debugging, and optimization of Nifi, Impala, and Spark jobs to reduce processing time and resource consumption.
- Cluster Management: Collaborate with DevOps and infrastructure teams to manage Nifi, Impala, and Spark clusters on platforms like Hadoop/YARN, Kubernetes, or cloud platforms (AWS EMR, GCP Dataproc, etc.).
- Migration: Provide dedicated support to ensure the stability and reliability of the new ODP Hadoop environment during and after migration. Promptly address evolving technical challenges and optimise system performance after migration to ODP.
Good to have:
- Master’s degree and Experience working with scripting languages (Scala, Python, Bash, PowerShell).
- Familiarity with virtual machine technologies and multi-node environment (50+ nodes).
- Proficient with Linux, NFS, and Windows, including application installation, scripting, and working with the command line.
- Working knowledge of application, server, and network security management concepts. Certification on any of the leading Cloud providers (AWS, Azure, GCP ) and/or Kubernetes.
- Knowledge of databases like MySQL and PostgreSQL.
- Be involved with and work on other support-related activities - Performing POC & assisting with Onboarding deployments of Acceldata & Hadoop distribution products.
Top Skills
Aws Emr
Bash
Gcp Dataproc
Hadoop
Impala
Kubernetes
Linux
MySQL
Nifi
Postgres
Powershell
Python
Scala
Spark
Similar Jobs
.png)
Artificial Intelligence • Blockchain • Fintech • Financial Services • Cryptocurrency • NFT • Web3
You will upgrade blockchain nodes, automate processes to enhance service effectiveness, and identify investment areas for further automation.
Top Skills:
AutomationBlockchainIntegration TestingService Oriented Architecture
Healthtech • Software
The Lead Analytics Engineer will design and implement data models, standardize semantic layers, enhance data quality, and mentor other engineers while ensuring compliance with healthcare standards.
Top Skills:
AWSDbtEmrGlueIcebergParquetS3SparkSQL
Artificial Intelligence • eCommerce • Information Technology • Internet of Things • Automation
This role involves selling Azure services, managing the full sales lifecycle, and cultivating client relationships while meeting sales goals.
Top Skills:
AzureSalesforce
What you need to know about the Charlotte Tech Scene
Ranked among the hottest tech cities in 2024 by CompTIA, Charlotte is quickly cementing its place as a major U.S. tech hub. Home to more than 90,000 tech workers, the city’s ecosystem is primed for continued growth, fueled by billions in annual funding from heavyweights like Microsoft and RevTech Labs, which has created thousands of fintech jobs and made the city a go-to for tech pros looking for their next big opportunity.
Key Facts About Charlotte Tech
- Number of Tech Workers: 90,859; 6.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Lowe’s, Bank of America, TIAA, Microsoft, Honeywell
- Key Industries: Fintech, artificial intelligence, cybersecurity, cloud computing, e-commerce
- Funding Landscape: $3.1 billion in venture capital funding in 2024 (CED)
- Notable Investors: Microsoft, Google, Falfurrias Management Partners, RevTech Labs Foundation
- Research Centers and Universities: University of North Carolina at Charlotte, Northeastern University, North Carolina Research Campus
