Data Engineer

Duties/Responsibilities

• Build, maintain, and improve batch and incremental data ingestion pipelines to meet agreed freshness needs, including API pagination and rate limits, retries, schema changes, deletes, and replay or backfill without duplicating data.  

• Use approved AI-assisted development tools for coding, testing, documentation, and troubleshooting; protect sensitive data and credentials, and validate generated output through testing, engineering judgment, and peer review.  

• Evaluate supported connectors, source-provided shares, file delivery, and custom extraction against required coverage, recovery, security, supportability, and operating cost; favor simpler supported approaches where they meet the need.  

• Develop SQL and Python solutions that collect, validate, transform, and publish data for downstream consumption.  

• Use Snowflake and dbt to build reliable transformations, incremental models, and reusable datasets that reconcile cross-system identifiers, preserve relevant history, and implement agreed transformation logic. Implement data product standards for technical metadata, schemas, data quality checks, lineage, and freshness monitoring.  

• Partner with application and Integration Platform teams on ingestion and governed read access; coordinate source-system writes and operational actions with the responsible integration or application owners.  

• Build automated unit, integration, and data-quality tests covering completeness, freshness, uniqueness, consistency, referential integrity, and reconciliation with source systems; investigate discrepancies before publishing data.  

• Implement approved access controls with Security, IAM, and data owners, including least-privilege roles, service identities, classification, masking, and row- or column-level controls; test both permitted and denied access.  

• Build version-controlled deployment processes for code, database changes, and access grants across development, quality assurance, and production environments, including safe cutover and rollback.  

• Monitor pipeline health, freshness, performance, and failures; troubleshoot root causes, recover failed loads, maintain technical documentation and runbooks, and communicate incidents and downstream impacts.  

• Optimize Snowflake workloads, queries, transformations, and storage patterns for performance, reliability, and cost discipline.  

• Partner with Analytics to reuse existing data models and implement agreed transformation logic, keeping shared datasets aligned with Analytics-owned semantic models.  

• Participate in backlog refinement, estimation, code review, and iterative delivery within a Kanban engineering team; break work into testable increments and communicate dependencies and delivery risks.  

• Proactively identify and recommend improvements to data architecture, ingestion and transformation patterns, and engineering practices; implement agreed changes to improve reliability, scalability, maintainability, and cost efficiency.  

Education and Experience

• Bachelor’s degree in computer science, information systems, engineering, mathematics, or a related field, or equivalent experience.  

• 3 or more years of experience in data engineering, software engineering, analytics engineering, or a related technical role.  

• Professional experience writing production-quality SQL and Python.  

• Experience independently building, deploying, and supporting production data pipelines, with hands-on orchestration experience covering scheduling, dependency management, monitoring, and failed-run recovery.  

• Hands-on experience with a cloud data warehouse and a transformation framework; Snowflake and dbt experience is strongly preferred.  

• Understanding of relational and dimensional data modeling, keys, grain, ELT/ETL patterns, APIs, source-system integration, schema evolution, and historical-change handling.  

• Experience with software engineering practices including source control, code review, automated testing, and CI/CD.  

• Experience using AI-assisted development tools effectively, with critical review of generated code and tests.  

• Understanding of data quality, metadata, lineage, role-based access, privacy, and secure handling of enterprise data, including secrets and service credentials.  

• Ability to investigate data issues, communicate findings clearly, and work through ambiguity with teammates and stakeholders.  

• Ability to collaborate effectively with engineers, analysts, source-system owners, security teams, and business stakeholders to implement and support reliable data solutions.  

Preferred

• Experience with Azure cloud storage, containerized or serverless workloads, and infrastructure as code, such as Terraform.  

• Experience ingesting data from enterprise applications such as Salesforce or NetSuite using REST or GraphQL APIs.  

• Familiarity with pipeline migration, observability, data catalogs, lineage tooling, or data quality platforms.  

• Experience building datasets that support BI reporting or governed application access through APIs.  

• Experience reconciling customer, product, order, or other business identifiers across systems, including reference data, entity matching, and historical change tracking.  

• Experience supporting production pipelines or datasets with freshness monitoring, versioning, access controls, and support procedures.  

• Curiosity about emerging data platform technologies and a practical approach to adopting them.  

Physical Requirements

 

Ability to safely and successfully perform the essential job functions consistent with the ADA, FMLA, and other federal, state, and local standards, including meeting qualitative and/or quantitative productivity standards. 

• Ability to maintain regular, punctual attendance consistent with the ADA, FMLA, and other federal, state, and local standards. 

• Primarily office and computer-based work with standard engineering and collaboration expectations for an enterprise technology role.