Build and operate production machine learning training systems on Amazon SageMaker using AWS Trainium. The role focuses on optimizing PyTorch code for Trainium hardware, understanding NeuronCore compilation, diagnosing device-level issues, tuning distributed training for cost and throughput, and delivering production pipelines. The engineer will collaborate directly with client and internal teams in a forward-deployed engineering environment.
Robots & Pencils is an AWS Partner building production AI systems for enterprise clients who need real engineering, not proofs of concept that never ship. We work forward-deployed, embedded directly with client teams, solving the problems that are too new or too specialized for a typical vendor relationship to handle.
The Role
We're looking for an engineer who can operate and train models on Amazon SageMaker running on AWS Trainium, AWS's custom silicon built specifically for large-scale model training. This isn't a role where you call an API and wait. You'll be walking up the stack: understanding what a training request actually looks like at the Trainium hardware and compiler level, then carrying that understanding all the way up through PyTorch training code and into a production SageMaker pipeline.
PyTorch is the backbone of this work. If you know the framework deeply and you're comfortable reasoning about how your code actually behaves on custom accelerator hardware rather than treating it as a black box, this role is built around that skill set specifically.
What You'll Do
- Train and operate models on Amazon SageMaker with AWS Trainium as the underlying compute
- Write and optimize PyTorch training code with a real understanding of how it compiles and executes on Trainium (NeuronCore architecture, compiler behavior, memory and throughput tradeoffs)
- Diagnose training run issues that show up specifically because of the hardware, not just the model, distinguishing a data or code problem from a compiler or device-level one
- Translate a request for "a Trainium job" into an actual working, cost-aware training pipeline, end to end
- Tune distributed training runs for throughput and cost on SageMaker's training infrastructure
- Work directly with client and internal engineering teams to scope and deliver real production training workloads, not experiments that stay in a notebook
What You'll Bring
- Strong, hands-on PyTorch experience, ideally including distributed or multi-device training
- Production experience with Amazon SageMaker for training and/or inference
- Comfort working close to the hardware layer: you understand device-specific compilation and can debug issues that are actually about the accelerator, not just the model
- AWS Trainium or Inferentia (Neuron SDK) experience is a strong plus; if you don't have it yet but have deep PyTorch and a track record of picking up new hardware targets fast, we want to talk to you
- Solid Python fundamentals and comfort operating in a client-facing, production engineering environment
Similar Jobs
Cloud • Real Estate • Software • PropTech
Manage strategic commercial accounts by developing growth and upsell strategies, maintaining client relationships, driving retention, monitoring KPIs, resolving issues, and coordinating with Sales, Operations, Product, and Implementation teams. The role involves understanding client portfolios, ensuring projects meet budget and service expectations, gathering feedback, documenting account activity, and identifying opportunities to expand revenue. Travel is required up to 25% of the time.
Top Skills:
Communication SoftwareCrm SoftwareData Analytics SoftwareProject Management PlatformsSpreadsheet Software
Big Data • Fintech • Information Technology • Business Intelligence • Financial Services • Cybersecurity • Big Data Analytics
Leads strategic sales for communications service providers and text message aggregators, generating new business and managing the full sales cycle from prospecting through closing. Develops territory and account plans, builds executive relationships, negotiates deals, maintains forecast accuracy, and grows pipeline. Collaborates with marketing, pre-sales, subject matter experts, and account teams to deliver customer-focused solutions and share market insights. The role is remote with occasional attendance at work-related events.
Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
Staff Software Engineer responsible for operating production Kubernetes across hybrid and multi-cloud environments, building auto-remediation and observability systems, developing Infrastructure-as-Code and GitOps pipelines, defining SLOs and alerting, supporting incident response, and reducing operational toil. The role also includes mentoring SRE engineers, improving reliability practices, and contributing to cloud, data center, and on-call operations.
Top Skills:
AksAWSAzureBashCi/CdCloudFormationEc2EksGCPGitopsGkeGoInfrastructure As CodeKubernetesLinuxMachine LearningPythonRdsService MeshTerraform
What you need to know about the Charlotte Tech Scene
Ranked among the hottest tech cities in 2024 by CompTIA, Charlotte is quickly cementing its place as a major U.S. tech hub. Home to more than 90,000 tech workers, the city’s ecosystem is primed for continued growth, fueled by billions in annual funding from heavyweights like Microsoft and RevTech Labs, which has created thousands of fintech jobs and made the city a go-to for tech pros looking for their next big opportunity.
Key Facts About Charlotte Tech
- Number of Tech Workers: 90,859; 6.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Lowe’s, Bank of America, TIAA, Microsoft, Honeywell
- Key Industries: Fintech, artificial intelligence, cybersecurity, cloud computing, e-commerce
- Funding Landscape: $3.1 billion in venture capital funding in 2024 (CED)
- Notable Investors: Microsoft, Google, Falfurrias Management Partners, RevTech Labs Foundation
- Research Centers and Universities: University of North Carolina at Charlotte, Northeastern University, North Carolina Research Campus
