Design, orchestrate, and optimize large-scale LLM pre-training across 1,000+ GPUs. Implement 3D parallelism, manage GPU clusters (SLURM/Kubernetes), optimize InfiniBand/RDMA networking and memory, and automate checkpointing and failure recovery for long training runs.
We are seeking a highly skilled LLM Pre-training & Distributed Systems Engineer. This role is essential for orchestrating large-scale machine learning training runs and optimizing distributed infrastructure. The ideal candidate will have a deep understanding of GPU clusters and extensive experience in system engineering to ensure efficient and reliable training processes.
Responsibilities:
- Orchestrate distributed training runs across 1,000+ GPUs using PyTorch, DeepSpeed, or Megatron-LM.
- Optimize networking (InfiniBand/RDMA) and memory management to prevent out-of-memory errors.
- Automate checkpointing and failure recovery during month-long training runs.
Required Skills:
- Deep expertise in 3D parallelism (Data, Tensor, Pipeline).
- Experience managing SLURM or Kubernetes-based GPU clusters.
- Strong systems engineering background (C++, CUDA, Python).
Similar Jobs
.png)
eCommerce • Fintech • Hardware • Payments • Software • Financial Services
Lead GTM strategy and operations for Global Account Management to improve retention, reduce churn, and drive expansion. Build coverage models, book construction, KPIs, dashboards, and operational cadence. Automate workflows and AI-driven experiments across CRM and GTM tools. Partner cross-functionally with Data Science, Product, Engineering, Finance, and AM to translate strategy into scalable programs, technical requirements, and measurable impact.
Top Skills:
APIsClayCRMLookerN8NSalesforceSnowflakeSQLZapier
eCommerce • Fintech • Hardware • Payments • Software • Financial Services
Lead vision-heavy product design for Square Point of Sale: craft visual and interaction designs, build prototypes, collaborate with product and engineering, integrate AI-driven experiences, and drive projects from concept through shipped product while raising design standards and systems thinking.
eCommerce • Fintech • Hardware • Payments • Software • Financial Services
Lead and build Square's global commercial operating system, owning Deal Desk, pricing operations, strategic deal support, and commercial governance. Drive pricing execution, approval workflows, AI-enabled automation, reporting, and cross-functional alignment to increase deal velocity, consistency, and profitability while managing a global team.
What you need to know about the Charlotte Tech Scene
Ranked among the hottest tech cities in 2024 by CompTIA, Charlotte is quickly cementing its place as a major U.S. tech hub. Home to more than 90,000 tech workers, the city’s ecosystem is primed for continued growth, fueled by billions in annual funding from heavyweights like Microsoft and RevTech Labs, which has created thousands of fintech jobs and made the city a go-to for tech pros looking for their next big opportunity.
Key Facts About Charlotte Tech
- Number of Tech Workers: 90,859; 6.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Lowe’s, Bank of America, TIAA, Microsoft, Honeywell
- Key Industries: Fintech, artificial intelligence, cybersecurity, cloud computing, e-commerce
- Funding Landscape: $3.1 billion in venture capital funding in 2024 (CED)
- Notable Investors: Microsoft, Google, Falfurrias Management Partners, RevTech Labs Foundation
- Research Centers and Universities: University of North Carolina at Charlotte, Northeastern University, North Carolina Research Campus
