Earth Is Our Runway
Shield AI Logo

Shield AI

Sr. Staff Platform/Data Reliability Engineer, Databricks (R5537)

Posted One Month Ago
Remote
Hiring Remotely in USA
180K-270K Annually
Senior level
Remote
Hiring Remotely in USA
180K-270K Annually
Senior level
Lead operational reliability and platform enablement for Databricks: build monitoring, CI/CD, deployment standards, compute and job policies, observability, runbooks, and governance to support secure, cost-aware, production data workloads across regulated environments. Mentor engineers and align platform with cloud/infrastructure and compliance requirements.
The summary above was generated by AI
Shield AI is a venture-backed defense-tech company with the mission of protecting service members and civilians with intelligent systems. Its products include Hivemind autonomy software, V-BAT and X-BAT aircraft, and Aechelon simulation and synthetic reality technologies. With offices and facilities across the U.S., Europe, the Middle East, and Asia-Pacific, Shield AI’s technology actively supports operations worldwide. For more information, visit www.shield.ai. Follow Shield AI on LinkedIn, X, Instagram, and YouTube. 

Job Description:

The Sr. Staff Platform / Data Reliability Engineer is responsible for making Databricks a reliable, secure, scalable, and operationally mature platform for enterprise use. This role focuses on the platform itself rather than domain-specific business modeling, ensuring the lakehouse can support multiple domains, regulated data handling, and a growing set of production workloads without becoming fragile, expensive, or hard to govern. 

This role partners closely with the Senior Staff Data Engineer, who focuses more heavily on ingestion and medallion implementation patterns, and with the existing Cloud & Infrastructure team, which owns cloud accounts, networking, and foundational infrastructure. The Platform / Data Reliability Engineer owns the Databricks operational layer that sits above that foundation: reliability, observability, deployment standards, compute and job policies, and platform enablement patterns for internal users.  

What you'll do:

  • Own operational excellence for the Databricks platform, including monitoring, alerting, observability, incident response support, and production runbook patterns for data jobs and platform services. 
  • Define and maintain CI/CD and promotion standards for Databricks assets, including workflows, jobs, notebooks, code packages, infrastructure configuration, and environment promotion from dev to prod. 
  • Design and maintain platform standards for job orchestration, cluster and compute policies, service principal usage, environment isolation, and production execution reliability.  
  • Establish reusable operational templates and enablement patterns for new domains onboarding to Databricks, including logging conventions, job tagging, metadata capture, and support handoff expectations.  
  • Partner with the Senior Data Engineer to ensure ingestion and medallion patterns are implemented in a way that is observable, recoverable, cost-aware, and secure in production. 
  • Work with the cloud/infrastructure team to align Databricks configuration and usage patterns with broader enterprise cloud standards, especially where commercial and future government-hosted environments are involved.  
  • Help enforce technical controls for data segregation, access boundaries, and operational compliance in a highly regulated environment.  
  • Track and improve platform health metrics such as job success rates, incident trends, data pipeline reliability, cost efficiency, and environment drift. 
  • Document platform standards, operational expectations, and support models so the Databricks platform can scale beyond a small founding team.  
  • Mentor internal engineers who are growing into platform responsibilities, helping expand Databricks operational knowledge within the team.  

Required qualifications:

  • 12+ years of relevant experience in data platform engineering, platform operations, site reliability engineering, or modern cloud data infrastructure.  
  • Hands-on experience with Databricks or a closely related cloud data platform in production environments. 
  • Experience designing or operating CI/CD, environment promotion, version control, and deployment automation for data platforms and pipelines.  
  • Strong understanding of platform operations concepts such as observability, monitoring, alerting, incident management, and reliability engineering.  
  • Experience with compute policy design, workload isolation, service principals, and secure production execution patterns on cloud data platforms.  
  • Ability to work effectively in a regulated or security-sensitive environment with strong expectations around access control, auditability, and operational discipline.  
  • Strong collaboration skills and comfort partnering with cloud/infrastructure, security, data engineering, and analytics stakeholders.  

Preferred qualifications:

  • Databricks certification and/or strong demonstrated expertise with Delta Lake, Unity Catalog, Workflows, and Databricks Asset Bundles.  
  • Experience with infrastructure-as-code and platform automation in enterprise environments.  
  • Experience supporting commercial and government or otherwise segregated environments with different compliance and access requirements.  
  • Experience in defense, aerospace, federal, or another regulated industry.  

#LI-KE1
#LE

Full-time regular employee offer package:
Pay within range listed + Bonus + Benefits + Equity
 
Temporary employee offer package:
Pay within range listed above + temporary benefits package (applicable after 60 days of employment)
 
Salary compensation is influenced by a wide array of factors including but not limited to skill set, level of experience, licenses and certifications, and specific work location. All offers are contingent on a cleared background and possible reference check. Military fellows and part-time employees are not eligible for benefits. Please speak to your talent acquisition representative for more information.
 
###
 
Shield AI is proud to be an equal opportunity workplace and is an affirmative action employer. We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, marital status, disability, gender identity or Veteran status. If you have a disability or special need that requires accommodation, please let us know. 

Similar Jobs at Shield AI

32 Minutes Ago
In-Office or Remote
United States
88K-130K Annually
Mid level
88K-130K Annually
Mid level
Aerospace • Artificial Intelligence • Machine Learning • Robotics • Software • Defense Technology
Designs and integrates aircraft mechanical systems and components, including installation layouts, subsystem routing, CAD assemblies, engineering drawings, BOMs, and product releases. Applies GD&T, tolerance analysis, and structural calculations while supporting prototype builds, production integration, and issue resolution. Collaborates with avionics, manufacturing, systems engineering, stress, quality, supply chain, and test teams to deliver manufacturable, maintainable, and reliable aerospace hardware.
Top Skills: 3DexperienceAsme Y14.5CadCatiaCreoEwisFeaGd&TNxPlmSolidworksTeamcenterWindchill
Yesterday
In-Office or Remote
United States
190K-290K Annually
Expert/Leader
190K-290K Annually
Expert/Leader
Aerospace • Artificial Intelligence • Machine Learning • Robotics • Software
Lead multi-year strategic planning and annual operating cycles, evaluate portfolio tradeoffs, build financial models for capital allocation, and translate strategy into cross-functional execution plans with KPIs. Conduct market sizing, competitive analysis, scenario modeling, and track performance through operating cadences to identify bottlenecks and recommend solutions.
Top Skills: AechelonHivemindV-BatX-Bat
4 Days Ago
Remote
USA
200K-300K Annually
Junior
200K-300K Annually
Junior
Aerospace • Artificial Intelligence • Machine Learning • Robotics • Software
Develop state estimation and navigation algorithms for autonomous systems operating in GPS-denied environments. Build production-grade C++ software for embedded Linux robotic platforms, create modeling and simulation tools, and implement comprehensive testing and validation. Collaborate on roadmap planning and agile execution while improving benchmarking and performance-analysis pipelines. The role requires professional C++ expertise, state estimation algorithm experience, sensor-fusion familiarity, and the ability to deploy reliable autonomy software.
Top Skills: Automated TestingBarometersC++CamerasCeresCi PipelinesEkfEmbedded LinuxGpsGraph-Based OptimizationGtsamImuLaser AltimetersMagnetometersMatlabOpenvinsParticle FiltersPythonSlamUkfVio

What you need to know about the Charlotte Tech Scene

Ranked among the hottest tech cities in 2024 by CompTIA, Charlotte is quickly cementing its place as a major U.S. tech hub. Home to more than 90,000 tech workers, the city’s ecosystem is primed for continued growth, fueled by billions in annual funding from heavyweights like Microsoft and RevTech Labs, which has created thousands of fintech jobs and made the city a go-to for tech pros looking for their next big opportunity.

Key Facts About Charlotte Tech

  • Number of Tech Workers: 90,859; 6.5% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Lowe’s, Bank of America, TIAA, Microsoft, Honeywell
  • Key Industries: Fintech, artificial intelligence, cybersecurity, cloud computing, e-commerce
  • Funding Landscape: $3.1 billion in venture capital funding in 2024 (CED)
  • Notable Investors: Microsoft, Google, Falfurrias Management Partners, RevTech Labs Foundation
  • Research Centers and Universities: University of North Carolina at Charlotte, Northeastern University, North Carolina Research Campus

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account