Lead end-to-end data engineering on GCP: design scalable data platforms and architectures (lake/lakehouse/warehouse), build and optimize BigQuery/Spark pipelines, ensure data quality/security, enable ML workflows with Data Science, mentor engineers, and own delivery and performance.
We are seeking a Senior Data Engineer (Lead) to drive and own end-to-end data engineering initiatives. This role will lead all data engineering efforts, working closely with Data Science and Analytics teams to design scalable data platforms, enable advanced analytics, and support machine learning use cases.
The ideal candidate will bring deep expertise in cloud data engineering (GCP), strong data modeling capabilities, and proven experience in leading enterprise-grade data solutions.
Responsibilities- Lead and manage end-to-end Data Engineering delivery across projects and initiatives
- Act as the primary technical owner for data pipelines, architecture, and platform design
- Mentor and guide a team of data engineers, ensuring best practices and coding standards
- Design, build, and optimize scalable data pipelines on GCP
- Define and implement modern data architectures (data lake, lakehouse, warehouse)
- Ensure high performance, reliability, and data quality across pipelines
- Partner closely with Data Science teams to enable ML/AI workflows
- Translate business and modeling requirements into optimized data structures
- Support feature engineering, model training, and deployment pipelines
- Design logical and physical data models for analytics and ML use cases
- Implement dimensional modeling (Star/Snowflake schemas) and data vault where applicable
- Optimize datasets for performance, scalability, and usability
- Build and manage solutions using GCP services such as: BigQuery, Cloud Composer (Airflow), Cloud Storage, Dataproc
- Ensure security, governance, and cost optimization on GCP
Required Qualification and Skills:
- 8+ years of experience in Data Engineering, with leadership experience
- Strong expertise in GCP ecosystem and services
- Proficiency in SQL, Python, and/or Scala
- Hands-on experience with ETL frameworks and distributed processing
- Solid experience in Dimensional modeling, Data warehousing concepts, Data structures for ML/analytics
- Experience with Apache Spark and Real-time and batch processing frameworks
- Experience working with cross-functional teams (Data Science, Analytics, Business)
- Proven ability to lead, mentor, and drive delivery
- Strong ownership mindset with leadership capabilities
- Excellent problem-solving and architectural thinking
- Ability to operate in a fast-paced, collaborative environment
Preferred Qualifications:
- Experience in ML data pipelines / feature stores
- Knowledge of data governance, lineage, and quality frameworks
- Exposure to healthcare/payor domain (nice to have)
- Certifications in GCP (Professional Data Engineer)
Similar Jobs
Artificial Intelligence • Fintech • Machine Learning • Social Impact • Software
Lead lifecycle strategy for Upstart's Personal and Small Dollar Loans, manage a team of Lifecycle Marketers, design and optimize customer journeys using segmentation, experimentation, personalization and AI, partner cross-functionally, forecast revenue, analyze funnel performance, and improve lifecycle operations and orchestration at scale.
Top Skills:
Ai-Enabled Marketing ToolsAi-Powered DecisioningCustomer Data Platforms (Cdp)Enterprise Marketing Automation PlatformsExperimentation Frameworks
Artificial Intelligence • Fintech • Machine Learning • Social Impact • Software
Manage HMDA and CRA data integrity, perform validation and quality control, coordinate annual LAR and CRA submissions, resolve data discrepancies with cross-functional teams, maintain documentation and controls, support regulatory exams and audits, monitor regulatory changes, and prepare management reporting on data quality and submission readiness.
Top Skills:
CraData Reporting SystemsHmda LarLoan Origination Systems
Artificial Intelligence • Fintech • Machine Learning • Social Impact • Software
Lead and grow one or more engineering teams building bank-critical platform systems (payments, funding, reporting, integrations). Translate regulatory, product, and operational needs into technical plans, drive architecture and execution for reliable, auditable infrastructure, manage cross-functional dependencies, build controls and change-management processes, and hire and develop engineering talent.
Top Skills:
Cloud PlatformsDepositsDistributed SystemsFinancial IntegrationsFunding SystemsLedgeringPaymentsReporting
What you need to know about the Charlotte Tech Scene
Ranked among the hottest tech cities in 2024 by CompTIA, Charlotte is quickly cementing its place as a major U.S. tech hub. Home to more than 90,000 tech workers, the city’s ecosystem is primed for continued growth, fueled by billions in annual funding from heavyweights like Microsoft and RevTech Labs, which has created thousands of fintech jobs and made the city a go-to for tech pros looking for their next big opportunity.
Key Facts About Charlotte Tech
- Number of Tech Workers: 90,859; 6.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Lowe’s, Bank of America, TIAA, Microsoft, Honeywell
- Key Industries: Fintech, artificial intelligence, cybersecurity, cloud computing, e-commerce
- Funding Landscape: $3.1 billion in venture capital funding in 2024 (CED)
- Notable Investors: Microsoft, Google, Falfurrias Management Partners, RevTech Labs Foundation
- Research Centers and Universities: University of North Carolina at Charlotte, Northeastern University, North Carolina Research Campus
