Lead delivery of LLM/AI-enabled systems: define LLM selection, prompt engineering, RAG/vector DB and knowledgebase design, AI agents, guardrails, evaluation frameworks, observability, latency and cost management. Manage offshore engineering team, run sprints, unblock delivery, and report status to leadership.
Key Responsibilities
- Own delivery of LLM API integration and SDK patterns used across applications.
- Set organizational guidance on which LLM to use for what use case and drive delivery of multi-LLM scenarios.
- Define standards for advanced prompt engineering and context window management.
- Own delivery of RAG systems, including vector database selection/topology and knowledgebase design.
- Drive delivery of AI agent and multi-agent systems and tool-use/MCP integration patterns.
- Own guardrails delivery (safety, compliance, PII handling in prompts/outputs) — critical in a financial-services context.
- Define evaluation frameworks and real-time eval strategy; set standards for AI testing in CI/CD.
- Own latency profiling, AI observability, and cost tracking/management for LLM-backed systems.
- Run day-to-day delivery of the offshore development team: sprint commitments, code/design review, real-time unblocking, and hands-on work on critical-path AI features.
- Report delivery status, risks, and blockers to engineering leadership.
Must-Have Qualifications
- 6+ years in software engineering, with 2+ years as a tech lead owning end-to-end delivery of LLM/AI-powered systems (not a pure design/review architect role).
- Proven track record of shipping AI-powered features on committed timelines, including hands-on troubleshooting under delivery pressure.
- Strong, hands-on Python skills at an architectural/systems level.
- Proven experience architecting LLM API integrations and SDK-level abstractions across multiple providers.
- Demonstrated judgment on model selection (cost, latency, capability trade-offs) across use cases.
- Deep expertise in prompt engineering and context window management at scale.
- Proven design experience with RAG systems, including vector database architecture and knowledgebase design.
- Experience architecting AI agents/multi-agent systems and tool-use patterns (MCP or equivalent).
- Strong understanding of guardrails design — content safety, PII protection, compliance controls for AI outputs.
- Experience defining evaluation frameworks and integrating AI testing into CI/CD.
- Proven ability to design for latency, observability, and cost management of AI systems in production.
- Financial-services or regulated-industry experience strongly preferred given compliance/guardrail stakes.
- Strong stakeholder communication; able to directly manage day-to-day delivery of an offshore team (standups, unblocking, sprint accountability).
Nice-to-Have Qualifications
- Direct experience with specific frameworks (LangChain, LlamaIndex, Semantic Kernel, or equivalent).
- Experience with AWS Bedrock or comparable managed LLM platforms.
- Contributions to or deep familiarity with MCP (Model Context Protocol) implementations.
- Experience building internal LLM gateways.
- Familiarity with responsible-AI/model-risk-management frameworks used in financial services.
Compensation, Benefits and Duration
Minimum Compensation: USD 56,000
Maximum Compensation: USD 196,000
Compensation is based on actual experience and qualifications of the candidate. The above is a reasonable and a good faith estimate for the role.
Medical, vision, and dental benefits, 401k retirement plan, variable pay/incentives, paid time off, and paid holidays are available for full-time employees.
This position is not available for independent contractors
No applications will be considered if received more than 120 days after the date of this post
Similar Jobs
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
Promote Pfizer vaccines within an assigned territory through live and virtual customer engagement, drive sales and formulary access, develop territory call plans, collaborate cross-functionally, use analytics and digital tools, and complete compliant administrative tasks.
Cloud • Fintech • Software • Business Intelligence • Consulting • Financial Services
As an Audit Manager, you'll manage audit engagements, ensure compliance with regulations, lead teams, and assist clients with accounting standards, while enhancing relationships and driving operational excellence.
Top Skills:
Fasb Accounting StandardsGaapExcel
Cloud • Fintech • Software • Business Intelligence • Consulting • Financial Services
The role involves auditing financial statements, assessing risks, testing internal controls, and training incoming staff. It requires collaboration with clients and adherence to auditing standards.
Top Skills:
AicpaGaapGaasPcaobSox
What you need to know about the Charlotte Tech Scene
Ranked among the hottest tech cities in 2024 by CompTIA, Charlotte is quickly cementing its place as a major U.S. tech hub. Home to more than 90,000 tech workers, the city’s ecosystem is primed for continued growth, fueled by billions in annual funding from heavyweights like Microsoft and RevTech Labs, which has created thousands of fintech jobs and made the city a go-to for tech pros looking for their next big opportunity.
Key Facts About Charlotte Tech
- Number of Tech Workers: 90,859; 6.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Lowe’s, Bank of America, TIAA, Microsoft, Honeywell
- Key Industries: Fintech, artificial intelligence, cybersecurity, cloud computing, e-commerce
- Funding Landscape: $3.1 billion in venture capital funding in 2024 (CED)
- Notable Investors: Microsoft, Google, Falfurrias Management Partners, RevTech Labs Foundation
- Research Centers and Universities: University of North Carolina at Charlotte, Northeastern University, North Carolina Research Campus
