Sr. Data Engineer

Description
This role is based remotely but if you live within a 50-mile radius of an office [Atlanta, Austin, Detroit, Warren, or Mountain View], you are expected to report to that location three times a week, at minimum.
The Role
As a Data Engineer, you will build industrialized data assets and optimize data pipelines in support of Business Intelligence and Advance Analytic objectives. You will work closely with our forward-thinking Data Scientists, BI Developers, System Architects and Data Architects to deliver value to our vision for the future. Are you ready to join a future facing team?
What You'll Do

• Communicate and maintain Master Data, Metadata, Data Management Repositories, Logical Data Models, Data Standards
• Create and maintain optimal data pipeline architecture
• Assemble large, complex data sets that meet functional / non-functional business requirements
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build industrialized analytic datasets and delivery mechanisms that utilize the data pipeline to deliver actionable insights into customer acquisition, operational efficiency and other key business performance metrics
• Work with business partners on data-related technical issues and develop requirements to support their data infrastructure needs
• Ability to create highly consistent and accurate analytic datasets suitable for business intelligence and data scientist team members

Your Skills & Abilities (Required Qualifications)

• 5 + years' experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.
• Bachelor's or Master's degree in Computer Science, Software Engineering, or a related field or equivalent experience
• Strong and hands-on experience in databricks data engineering and management.
• Experience with setting up CI/CD environment in Azure and Databricks leveraging Azure Devops, ARM templates, GitHub actions and DBX Asset bundles and Terraform etc.
• Strong understanding of Gen AI tools and proven ability to integrate AI tools such as Microsoft co-pilot, data bricks genie with daily tasks in Databricks.
• Experience with big data frameworks and tools like Apache Hadoop, Apache Spark, or Apache Kafka for processing and analyzing large datasets.
• Strong understanding and ability to provide mentorship in the areas of data ETL processes and tools for designing and managing data pipelines.
• Experience in designing streaming data pipelines using FiveTran, Azure Event Hubs, Auto Loader and delta lake in Azure Databricks.
• Hands on experience with data serialization formats like JSON, Parquet, YML and XML.
• Background in designing data solutions that are highly automated and provide consistent and accurate outcomes.
• Hands on experience in orchestrating automated workflows or batch jobs on Azure Databricks platform.
• Understanding of data governance principles, data privacy regulations, and experience with implementing security measures for data protection.
• Ability to work effectively in cross-functional teams, collaborate with data scientists, analysts, and stakeholders to deliver data solutions.
• At least 3 years of hands-on experience with Big Data Tools: Hadoop, Spark, Kafka, etc.
• Ability to identify tasks which require automation and automate them
• A demonstrable understanding of networking/distributed computing environment concepts
• Ability to multi-task and stay organized in a dynamic work environment

Additional Description
What Can Give You a Competitive Advantage (Preferred Qualifications)

• Data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
• AWS cloud services: EC2, EMR, RDS, Redshift

This job is not eligible for relocation benefits. Any relocation costs would be the responsibility of the selected candidate.
Compensation:

• The expected base compensation for this role is: $94,800 - $159,600. Actual base compensation within the identified range will vary based on factors relevant to the position.
• Bonus Potential: An incentive pay program offers payouts based on company performance, job level, and individual performance.
• Benefits : GM offers a variety of health and wellbeing benefit programs. Benefit options include medical, dental, vision, Health Savings Account, Flexible Spending Accounts, retirement savings plan, sickness and accident benefits, life insurance, paid vacation & holidays, tuition assistance programs, employee assistance program, GM vehicle discounts and more.

#LI-CC1
About GM
Our vision is a world with Zero Crashes, Zero Emissions and Zero Congestion and we embrace the responsibility to lead the change that will make our world better, safer and more equitable for all.
Why Join Us
We believe we all must make a choice every day - individually and collectively - to drive meaningful change through our words, our deeds and our culture. Every day, we want every employee to feel they belong to one General Motors team.
Total Rewards | Benefits Overview
From day one, we're looking out for your well-being-at work and at home-so you can focus on realizing your ambitions. Learn how GM supports a rewarding career that rewards you personally by visiting Total Rewards resources.
Non-Discrimination and Equal Employment Opportunities (U.S.)
General Motors is committed to being a workplace that is not only free of unlawful discrimination, but one that genuinely fosters inclusion and belonging. We strongly believe that providing an inclusive workplace creates an environment in which our employees can thrive and develop better products for our customers.
All employment decisions are made on a non-discriminatory basis without regard to sex, race, color, national origin, citizenship status, religion, age, disability, pregnancy or maternity status, sexual orientation, gender identity, status as a veteran or protected veteran, or any other similarly protected status in accordance with federal, state and local laws.
We encourage interested candidates to review the key responsibilities and qualifications for each role and apply for any positions that match their skills and capabilities. Applicants in the recruitment process may be required, where applicable, to successfully complete a role-related assessment(s) and/or a pre-employment screening prior to beginning employment. To learn more, visit How we Hire.
Accommodations
General Motors offers opportunities to all job seekers including individuals with disabilities. If you need a reasonable accommodation to assist with your job search or application for employment, email us [email protected] or call us at 800-865-7580. In your email, please include a description of the specific accommodation you are requesting as well as the job title and requisition number of the position for which you are applying.