Senior AI Engineer

The Role

In this role, you will work alongside engineers on the AI Platform team to design, build, and
maintain the platform that powers our customer-facing AI tools. You will contribute to systems
for prompt management, model evaluation, and knowledge base maintenance, while also
helping integrate AI models into production systems and supporting the reliability and
performance of the platform.

This position will report to the Engineering Manager of the AI Platform team, which sits within
the broader AI Products division of the Engineering org. The AI Platform team is responsible for
building and maintaining the internal platform used to manage prompts, run evaluations, and
maintain knowledge bases that power our customer-facing AI tools. This is a remote role.

Responsibilities

• Build and maintain APIs that integrate generative AI models into production systems,
  with a focus on scalability and low latency.
• Collaborate with cross-functional teams to identify use cases for generative AI, define
  requirements, and contribute to impactful solutions.
• Implement and optimize workflows for model inference using large-scale pre-trained
  models.
• Contribute to generative AI model pipelines, supporting monitoring and ongoing
  performance.
• Conduct experiments to evaluate generative AI models against real-world performance
  metrics such as fluency, coherence, and factual accuracy.
• Stay current with advancements in generative AI research and help integrate new
  techniques into the organization’s capabilities.
• Support the reliability, scalability, and security of generative AI systems in deployment
• Document processes, architectures, and technical decisions to facilitate collaboration
  and future improvements.

Qualifications

• 4+ years of experience in ML/AI engineering, with experience contributing to generative
  AI products.
• Experience with LLM design patterns, including Retrieval-Augmented Generation (RAG),
  as well as familiarity with broader search and retrieval systems such as vector search,
  semantic search, and document indexing.
• Strong knowledge of large language model (LLM) concepts including prompt
  engineering, AI feature evaluation, and working with vector databases.
• Experience working with Vertex AI or similar managed AI platforms.
• Hands-on experience with LLM evaluation frameworks, including dataset management,
  recall testing, and LLM-as-judge patterns.
• Familiarity with experiment tracking tools such as Weights & Biases.
• Working knowledge of MLOps practices: deployment, model monitoring, and CI/CD
  workflows.
• Strong problem-solving, system design, and technology decision-making skills.
• Some exposure to application development, such as contributing to or building simple
  software applications.
• Experience with Node.js and/or TypeScript, particularly in building or maintaining API
  services.
• Experience designing and versioning REST APIs.
• Working knowledge of PostgreSQL, including schema design, migrations, and JSONB
  usage.

Bonus Qualifications

• Familiarity with the GCP ecosystem.