Lead the design and development of CentML's deployment infrastructure for ML training and inference across multiple cloud platforms, focusing on GPU clusters.
About Us
We believe AI will fundamentally transform how people live and work. CentML's mission is to massively reduce the cost of developing and deploying ML models so we can enable anyone to harness the power of AI and everyone to benefit from its potential.
Our founding team is made up of experts in AI, compilers, and ML hardware and has led efforts at companies like Amazon, Google, Microsoft Research, Nvidia, Intel, Qualcomm, and IBM. Our co-founder and CEO, Gennady Pekhimenko, is a world-renowned expert in ML systems who holds multiple academic and industry research awards from Google, Amazon, Facebook, and VMware.
Position Overview:
We are seeking a highly motivated and skilled senior infrastructure engineer to join our team in a key role focused on designing, developing, and maintaining the CentML platform that offers a cost effective infrastructure for serving and training large scale machine learning models. As an infrastructure engineer, you will be responsible for laying out the design of a deployment infrastructure for ML training and inference jobs over GPU clusters that spans across multiple cloud service providers like AWS, GCP, Azure, Coreweave, and OCI. You should also be responsible for leading a team of engineers and building a scalable, performant, and reliable platform, enabling our customers to seamlessly access and utilize a comprehensive suite of ML services that we offer.
Responsibilities
- Design and lead the development of the deployment infrastructure of the CentML platform. The deployment infrastructure manages the hardware resources necessary to deploy the ML training and inference applications.
- Implementing GPU cluster scheduling solutions for large scale ML training and inference workloads to efficiently utilize the hardware resources in the GPU cluster.
- Communicate with our product teams and define new features and goals for improving the CentML platform.
Qualifications
- 4+ years of experience working with containerized deployment systems (e.g, kubernetes, openshift, terraform etc.).
- A big plus if you have contributed to kubernetes and have expertise in container runtime technologies like docker engine, containerd, or CRI-O
- Experience with deploying and managing cloud infrastructure on AWS, GCP, Azure
- Past experience in building GPU clusters for large scale ML training and inference is desirable.
- Knowledge in GPU architecture and Nvidia GPU virtualization technologies is highly desirable.
- Strong coding skills in languages like Python, Java, Go, and/or C/C++.
Benefits & Perks
- An open and inclusive work environment
- Employee stock options
- Best-in-class medical and dental benefits
- Parental Leave top-up
- Professional development budget
- Flexible vacation time to promote a healthy work-life blend
We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, disability, and any other protected ground of discrimination under applicable human rights legislation.
CentML strives to respect the dignity and independence of people with disabilities and is committed to giving them the same opportunity to succeed as all other employees.
Inclusiveness is core to our culture at CentML, and we strive to ensure you get the most from your interview experience. CentML makes reasonable accommodations for applicants with disabilities. If a reasonable accommodation is needed to participate in the job application or interview process, please reach out to the Talent team.
Top Skills
AWS
Azure
C
C++
GCP
Go
Gpu
Java
Kubernetes
Openshift
Python
Terraform
Similar Jobs
Artificial Intelligence • Cloud • Consumer Web • Productivity • Software • App development • Data Privacy
The Senior Infrastructure Software Engineer builds and optimizes scalable systems, manages massive data efficiently, and collaborates across teams to innovate solutions.
Top Skills:
C/C++GoJavaPython
Fintech • Financial Services
Join Ramp as a Senior Software Engineer focusing on database and infrastructure. Lead projects, mentor teammates, and solve complex engineering challenges.
Top Skills:
AWSAzureCeleryEcsElasticache RedisElbGCPKafkaKubernetesMySQLPostgresS3Terraform
Artificial Intelligence • Cloud • Consumer Web • Productivity • Software • App development • Data Privacy
Lead design and implementation of AI-powered developer tools that improve engineering productivity, reduce manual work, and enhance code quality at scale.
Top Skills:
AIC/C++GoJavaMlPython
What you need to know about the Charlotte Tech Scene
Ranked among the hottest tech cities in 2024 by CompTIA, Charlotte is quickly cementing its place as a major U.S. tech hub. Home to more than 90,000 tech workers, the city’s ecosystem is primed for continued growth, fueled by billions in annual funding from heavyweights like Microsoft and RevTech Labs, which has created thousands of fintech jobs and made the city a go-to for tech pros looking for their next big opportunity.
Key Facts About Charlotte Tech
- Number of Tech Workers: 90,859; 6.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Lowe’s, Bank of America, TIAA, Microsoft, Honeywell
- Key Industries: Fintech, artificial intelligence, cybersecurity, cloud computing, e-commerce
- Funding Landscape: $3.1 billion in venture capital funding in 2024 (CED)
- Notable Investors: Microsoft, Google, Falfurrias Management Partners, RevTech Labs Foundation
- Research Centers and Universities: University of North Carolina at Charlotte, Northeastern University, North Carolina Research Campus
