Software Engineer - AI Workflows

The Role:

At Flume Health, we’re leveraging AI to reimagine how healthcare data is mapped, validated, and integrated faster, more reliably, and at scale. We’re looking for a Software Engineer focused on AI workflows to help build and orchestrate pipelines that use foundation models like OpenAI’s or Google’s large language models (LLMs) to generate and iteratively refine data mappings.

You won’t be training models from scratch but you will be at the forefront of building robust workflows that interface with LLM APIs, run validation checks on outputs, and implement automated loops for refinement. If you enjoy piecing together modular AI processes, designing data-driven feedback loops, and working in a fast-paced, compliance-aware environment, this role is for you.

What You’ll Do:

• Architect AI workflows that interpret user-provided specifications and generate data mappings using LLM APIs (e.g., OpenAI, Vertex AI), refining outputs based on validation feedback.
  
• Implement intelligent retry and refinement loops to improve mapping accuracy using techniques like prompt chaining, structured validation, and contextual injection.
• Develop prompt engineering strategies with support for domain-specific augmentation (e.g., via vector search or RAG).
  
• Orchestrate pipelines with tools like Python and Airflow, integrating structured QA steps to monitor output quality and flag failures.
  
• Collaborate across Data, Platform, and Product teams to embed these workflows into Flume’s core systems, ensuring performance, traceability, and compliance.
  
• Stay up to date on the evolving LLM ecosystem understanding tradeoffs in cost, latency, and capability across vendors.

What You’ll Need:

• 3+ years of software engineering experience, ideally in data, ML, or AI-driven product development.
  
• Demonstrated experience building or orchestrating workflows that integrate with LLM APIs (e.g., OpenAI, Anthropic, Google).
  
• Proficiency with Python and orchestration tools (e.g., Airflow, Dagster, Prefect).
  
• Familiarity with prompt engineering concepts and iterative refinement techniques (e.g., looped retries, RAG, validation-based reruns).
  
• Comfortable working in cloud-native environments (GCP or AWS) using Docker and Kubernetes.
  
• Strong communication skills able to translate business needs into repeatable, explainable, and traceable AI workflows.

Technologies We Use:

• LLM Tools & APIs: OpenAI, Vertex AI, DSPy, LangChain (where appropriate)
  
• Workflow & Orchestration: Python, Airflow, custom feedback/retry frameworks
  
• Cloud & Infra: GCP (BigQuery, Pub/Sub), Kubernetes, Docker, Terraform
  
• Healthcare Standards: EDI, FHIR, HIPAA-compliant pipelines
  
• Monitoring & Versioning: Logging/metrics pipelines, prompt version control

Nice to Haves:

• Experience with vector databases or semantic search tools (e.g., Pinecone, Weaviate, Qdrant).
  
• Exposure to regulated data environments, such as healthcare or finance.
  
• Understanding of LLM orchestration frameworks (LangChain, DSPy, HuggingFace Accelerate).
  
• Interest in evaluating the effectiveness of LLM outputs using custom metrics or domain-specific rules.
  
• Contributions to open-source AI tooling or AI workflow libraries.

Special Note:

• Candidates will be required to travel for in person team meetings once a quarter.

Salary Range:

• $150K to $180K