Designs, develops, and maintains scalable ETL pipelines, databases, and cloud data infrastructure. Integrates data from APIs, third-party services, and on-premises or cloud databases. Develops automated processing workflows, optimizes data warehouses, improves data quality, supports analytics users, and performs testing and production deployment activities. Collaborates with data scientists, analysts, stakeholders, and vendors to deliver reliable, accessible data solutions.
Role Overview
The Data Engineer, Specialist is responsible for designing, developing, and maintaining scalable data pipelines and infrastructure to support analytics and business intelligence initiatives. This role involves building robust ETL (Extract, Transform, Load) processes, managing databases and optimizing cloud-based data platforms to ensure efficient and seamless integration of data from multiple sources.
Responsibilities- Develop and maintain scalable ETL (Extract, Transform, Load) processes to efficiently extract data from diverse sources, transform it as required and load it into data warehouses or analytical systems.
- Design and optimize database architectures and data pipelines to ensure high performance, availability and security while supporting structured and unstructured data.
- Integrate data from multiple sources, including APIs, third-party services and on-prem/cloud databases to create unified and consistent datasets for analytics and reporting.
- Collaborate with data scientists, analysts and business stakeholders to understand their data needs, develop data solutions and enable self-service analytics.
- Develop automated workflows and data processing scripts using Python, Spark, SQL, or other relevant technologies to streamline data ingestion and transformation.
- Optimize data storage and retrieval strategies in cloud-based data warehouses such as AWS Redshift, Google Big Query, or Azure Synapse, ensuring scalability and cost-efficiency.
- Maintain and improve data quality by implementing validation frameworks, anomaly detection mechanisms and data cleansing processes.
- Thoroughly tests code to ensure accuracy and alignment with its intended purpose. Reviews the final product with end users to confirm clarity and understanding, providing data analysis guidance as needed.
- Offers tool and data support to business users and team members, ensuring seamless functionality and accessibility.
- Conducts regression testing for new software releases, identifying issues and collaborating with vendors to resolve them and successfully deploy the software into production.
- Minimum of three years data engineering, programming, database administration, or data management experience.
- Undergraduate degree or equivalent combination of training and experience.
- Strong proficiency in SQL, Python for data manipulation, automation and pipeline development.
- Experience working with big data processing frameworks such as Apache Spark, Hadoop, or Kafka.
- Hands-on experience with cloud-based data platforms such as AWS (Glue, S3, Lambda), Data Bricks(Lakehouse, Worklows), or Azure (Synapse, Data Factory).
Special Factors
Sponsorship
Vanguard is not offering visa sponsorship for this position.About Vanguard
At Vanguard, we don't just have a mission—we're on a mission.
To work for the long-term financial wellbeing of our clients. To lead through product and services that transform our clients' lives. To learn and develop our skills as individuals and as a team. From Malvern to Melbourne, our mission drives us forward and inspires us to be our best.
How We Work
Vanguard has implemented a hybrid working model for the majority of our crew members, designed to capture the benefits of enhanced flexibility while enabling in-person learning, collaboration, and connection. We believe our mission-driven and highly collaborative culture is a critical enabler to support long-term client outcomes and enrich the employee experience.
Vanguard Charlotte, North Carolina, USA Office
Two North Falls Plaza, Charlotte, NC, United States, 28217
Similar Jobs
Fintech
Builds and maintains scalable ETL processes, database systems, data products, and real-time or offline analytics tools. Integrates large-scale data, ensures pipeline reliability and data integrity, translates business requirements into maintainable code, and supports analytical solutions such as Tableau dashboards. Partners with stakeholders, troubleshoots production issues, conducts testing and releases, provides data guidance, and trains users on tools and data sources.
Top Skills:
AWSPysparkPythonSQLTableau
Cloud • Information Technology • Security • Software • Cybersecurity
Partner with sales to design and present data security solutions for SLED customers. Lead technical evaluations, gather requirements, build test plans, configure proof-of-concepts, and guide customers to successful outcomes while collaborating across GTM teams.
Top Skills:
Cloud-NativeData ProtectionSaaSWeb TechnologiesZero Trust ExchangeZscaler
Energy • Renewable Energy
Leads end-to-end data science and machine learning initiatives, from problem discovery and proof of concept through production deployment and continuous improvement. Develops statistical, optimization, simulation, and machine learning solutions using strong software engineering, MLOps, and Responsible AI practices. Partners with stakeholders and multidisciplinary teams, provides technical direction, evaluates feasibility and risks, communicates analytical recommendations, and remains hands-on in solution development.
Top Skills:
SparkDatabricksMachine Learning LibrariesMlflowMlopsPysparkPythonResponsible AiSQL
What you need to know about the Charlotte Tech Scene
Ranked among the hottest tech cities in 2024 by CompTIA, Charlotte is quickly cementing its place as a major U.S. tech hub. Home to more than 90,000 tech workers, the city’s ecosystem is primed for continued growth, fueled by billions in annual funding from heavyweights like Microsoft and RevTech Labs, which has created thousands of fintech jobs and made the city a go-to for tech pros looking for their next big opportunity.
Key Facts About Charlotte Tech
- Number of Tech Workers: 90,859; 6.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Lowe’s, Bank of America, TIAA, Microsoft, Honeywell
- Key Industries: Fintech, artificial intelligence, cybersecurity, cloud computing, e-commerce
- Funding Landscape: $3.1 billion in venture capital funding in 2024 (CED)
- Notable Investors: Microsoft, Google, Falfurrias Management Partners, RevTech Labs Foundation
- Research Centers and Universities: University of North Carolina at Charlotte, Northeastern University, North Carolina Research Campus
