Designs and maintains scalable data pipelines, warehouses, lakes, integration solutions, and analytics platforms. Develops AWS and Snowflake data solutions using Python, SQL, dbt, ETL/ELT, and Data Vault modeling. Supports replication, APIs, governance, metadata, lineage, and data quality across enterprise environments. Troubleshoots complex data issues and provides technical leadership while collaborating with engineering, architecture, analytics, and business teams.
Job Posting Title
Principal Data EngineerJob Description
*Applicants must be authorized to work for ANY employer in the U.S. We are unable to sponsor or take over sponsorship of an employment Visa at this time.*
Location: Charlotte, NC/Detroit, MI
Work Arrangement: Onsite
Job Type: Full-Time
We are seeking an experienced Principal Data Engineer to design, develop, and maintain scalable data solutions supporting enterprise data engineering and analytics initiatives. This role will work across data engineering, data warehousing, data integration, and cloud technologies to build reliable and high-performing data platforms.
The ideal candidate has strong hands-on experience with AWS, Snowflake, Python, SQL, ETL/ELT, and modern data engineering practices, along with the ability to work across complex enterprise data environments.
Key Responsibilities- Design, develop, and optimize scalable data pipelines and data integration solutions.
- Develop and maintain data architectures supporting enterprise data warehouses, data lakes, and analytics platforms.
- Build and optimize ETL/ELT processes using modern data engineering tools and technologies.
- Develop solutions using AWS services including S3, Lambda, and DynamoDB.
- Design and implement data solutions within Snowflake and other cloud-based data environments.
- Develop and maintain data models, including Data Vault modeling methodologies.
- Write and optimize complex SQL and Python code for data processing and integration.
- Work with dbt to develop, transform, test, and manage data workflows.
- Support data replication and integration using tools such as Qlik Replicate.
- Work with enterprise data platforms including IBM InfoSphere DataStage and CP4D.
- Develop and integrate APIs to support enterprise data and application needs.
- Establish and maintain data quality, governance, metadata management, and data lineage processes.
- Collaborate with engineering, architecture, analytics, and business teams to translate requirements into scalable data solutions.
- Troubleshoot performance, data quality, and integration issues across complex data environments.
- Provide technical leadership and guidance on data engineering architecture and best practices.
- Strong experience in data engineering and enterprise data environments.
- Hands-on experience with AWS, particularly S3, Lambda, and/or DynamoDB.
- Strong experience with Snowflake and cloud data warehousing.
- Advanced Python and SQL development skills.
- Experience developing ETL/ELT and data integration solutions.
- Experience with data warehousing and data modeling, including Data Vault.
- Experience with dbt or similar modern data transformation frameworks.
- Experience with data quality, governance, metadata management, and data lineage.
- Strong understanding of relational and non-relational databases.
- Experience with enterprise data integration platforms and tools.
- Ability to work independently while providing technical leadership to other engineers.
- Experience with Qlik Replicate.
- Experience with IBM InfoSphere DataStage.
- Experience with IBM CP4D.
- Experience developing and integrating APIs.
- Experience with NoSQL databases.
- Experience working with Git and DevOps practices.
- Experience in large-scale enterprise environments.
- Strong communication and cross-functional collaboration skills.
Work Environment
This is an onsite position that requires the ability to work from the client site on a regular basis.
Kaleidoscope, an Infosys Company, is an equal opportunity employer, and all qualified applicants will receive consideration without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, protected veteran status, spouse of protected veteran, or disability.
Similar Jobs
.png)
Artificial Intelligence • Cloud • Payments • Software • Business Intelligence • Generative AI • Automation
Define and govern enterprise-scale data architecture across batch, streaming, warehouse, lakehouse, transactional, and AI use cases. Establish standards for data quality, lineage, access, cataloging, governance, observability, and SLAs. Architect AI-enabled workflows, resolve complex architecture issues, influence roadmaps, and mentor engineers through hands-on technical leadership. The role requires 15+ years of software, data engineering, or architecture experience and expertise in large-scale data platforms and modeling.
Top Skills:
AIBatch ProcessingBigQueryData CatalogsData WarehousesDbtFeature StoresGCPLakehousesOlapOltpStreaming ArchitecturesVector Stores
Fintech • News + Entertainment • Software • Database • Financial Services
Lead the architecture and development of scalable AWS-based data ingestion, transformation, and orchestration pipelines. Build reliable data infrastructure using Python, SQL, Airflow, Lambda, ECS, SQS, and Terraform. Establish data modeling, quality, lineage, monitoring, testing, and observability practices while partnering with analysts, scientists, and backend engineers. Mentor senior engineers, guide technical decisions, and provide hands-on leadership for complex data platform initiatives.
Top Skills:
Amazon EcsAmazon KinesisAmazon RedshiftAmazon S3Amazon SqsApache AirflowApache FlinkAWSAws GlueAws LambdaBeautifulsoupCi/CdDatabricksDockerGreat ExpectationsKafkaMonte CarloMwaaPythonScrapySnowflakeSQLTerraform
Digital Media • Fintech • Information Technology • Machine Learning • Financial Services • Cybersecurity • Automation
Serves as the senior technical authority for enterprise browser data protection and security. Owns architecture, roadmap, standards, lifecycle, browser-based DLP controls, integrations, automation, and operational support. Leads complex troubleshooting, major incident response, root-cause analysis, runbook development, and continuous improvement. Partners across cybersecurity, identity, endpoint, network, cloud, risk, compliance, and vendor teams while mentoring engineers and influencing enterprise security design.
Top Skills:
Active DirectoryAzure DevopsCasbChrome Enterprise PremiumCisco Secure AccessCloud PlatformsCrowdstrikeCyberarkDigital GuardianDnsEntra IdForcepoint DlpGitGitIslandJAMFMecm/SccmMenlo SecurityMicrosoft Defender XdrMicrosoft Edge For BusinessMicrosoft GraphMicrosoft IntuneMicrosoft Purview DlpMicrosoft SentinelNetskope DlpOktaPalo AltoPalo Alto Prisma BrowserPing IdentityPowershellPythonRest ApisSaseSplunkSwgSymantec DlpTaniumTerraformTls/HttpsVpnZscalerZtna
What you need to know about the Charlotte Tech Scene
Ranked among the hottest tech cities in 2024 by CompTIA, Charlotte is quickly cementing its place as a major U.S. tech hub. Home to more than 90,000 tech workers, the city’s ecosystem is primed for continued growth, fueled by billions in annual funding from heavyweights like Microsoft and RevTech Labs, which has created thousands of fintech jobs and made the city a go-to for tech pros looking for their next big opportunity.
Key Facts About Charlotte Tech
- Number of Tech Workers: 90,859; 6.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Lowe’s, Bank of America, TIAA, Microsoft, Honeywell
- Key Industries: Fintech, artificial intelligence, cybersecurity, cloud computing, e-commerce
- Funding Landscape: $3.1 billion in venture capital funding in 2024 (CED)
- Notable Investors: Microsoft, Google, Falfurrias Management Partners, RevTech Labs Foundation
- Research Centers and Universities: University of North Carolina at Charlotte, Northeastern University, North Carolina Research Campus
