Observability Engineer

Comcast Advertising is driving the TV advertising industry forward, from delivering ads to linear and digital audiences to pioneering the tech that makes it possible. We help brands connect with their audiences on every screen using advanced data, technology, and premium video content. Our media sales division helps local, regional, and national brands reach potential customers through multiscreen TV advertising. Our ad tech division FreeWheel provides comprehensive adtech that makes it easier to buy and sell premium video advertising across all screens, data types, and sales channels.
Job Summary
The Observability team builds the platform that all the Comcast Advertising engineers use to investigate the health of their services at scale. The team's scope includes Logging, Distributed Tracing, Error Reporting, and Profiling. You will support a team of 5-10 engineers focused on building the next generation of logging & tracing systems at Comcast Advertising, as part of the broader Observability team. In addition to helping define the roadmap for the next 3-5 years, you will be interacting with many other managers and their teams at Comcast Advertising, who rely on the Observability platform to deliver stable and scalable services to our customers.
Job Description
Preferred Qualifications

• 3-5 years of experience as Software Development Engineer, Site Reliability Engineer (SRE), or similar role.
• Experience defining SLIs/SLOs
• Strong proficiency in implementing and maintaining observability tools, such as, Elastic ELK Stack (Elasticsearch, Logstash, Kibana), and APM (Application Performance Monitoring) tools.
• Experience with Open Telemetry
• Experience with data transformations and standardization
• Solid experience with instrumentation practices, including metrics, logging, and distributed tracing
• Hands-on experience with cloud platforms (AWS, GCP) and containerization technologies (Docker, Kubernetes)
• Proficiency in scripting and automation using languages such as Python, Bash, or PowerShell
• Excellent problem-solving skills with the ability to analyze complex issues and provide efficient solutions
• Strong communication skills and the ability to collaborate effectively across teams
• Understanding of Agile and DevOps principles and their application in observability and monitoring contexts

Core Responsibilities

• Develop and implement observability solutions to gain insights into application and services for metrics, logs, traces, and other relevant data.
• Collaborate with development, operations, and other teams to instrument applications
• Design and implement monitoring solutions using industry-standard tools and practices to detect, analyze, and mitigate incidents and anomalies.• Create and manage dashboards, alerts, and visualization tools to provide real-time visibility into system behavior and performance.• Perform in-depth analysis of system behavior and trends to identify areas for improvement, optimization, and increased efficiency.• Troubleshoot complex issues by analyzing data from various sources to quickly diagnose and resolve incidents, minimizing downtime
• Continuously evaluate and recommend improvements to observability processes, tools, and practices to align with industry best practices
• Contribute to the development of automation scripts and tools to enhance observability and incident response
• Collaborate with development teams to improve application design for better observability, including implementing distributed tracing and structured logging
• Stay updated with emerging trends, technologies, and methodologies in observability, monitoring, and performance analysis.

Employees at all levels are expected to:

• Understand our Operating Principles; make them the guidelines for how you do your job.
• Own the customer experience - think and act in ways that put our customers first, give them seamless digital options at every touchpoint, and make them promoters of our products and services.
• Know your stuff - be enthusiastic learners, users and advocates of our game-changing technology, products and services, especially our digital tools and experiences.
• Win as a team - make big things happen by working together and being open to new ideas.
• Be an active part of the Net Promoter System - a way of working that brings more employee and customer feedback into the company - by joining huddles, making call backs and helping us elevate opportunities to do better for our customers.
• Drive results and growth.
• Respect and promote inclusion & diversity.
• Do what's right for each other, our customers, investors and our communities.

Disclaimer:

• This information has been designed to indicate the general nature and level of work performed by employees in this role. It is not designed to contain or be interpreted as a comprehensive inventory of all duties, responsibilities and qualifications.

Comcast is proud to be an equal opportunity workplace. We will consider all qualified applicants for employment without regard to race, color, religion, age, sex, sexual orientation, gender identity, national origin, disability, veteran status, genetic information, or any other basis protected by applicable law. Comcast will consider for employment applicants with arrest or conviction records in accordance with the requirements of applicable law, including the San Francisco Fair Chance Ordinance, the Los Angeles Fair Chance Initiative for Hiring Ordinance, the Los Angeles County Fair Chance Ordinance for Employers and the California Fair Chance Act. Please note that federal state, or local laws and regulations may restrict or prohibit Comcast from hiring individuals convicted of certain crimes. Additionally, an applicant's criminal history may have a direct, adverse, and negative relationship on the job duties of this position, which may result in the withdrawal of a conditional offer of employment.
Skills:
Customer Experience (CX); Detailed Design; Building Architecture; Design
Salary:
National Pay Range: $72,075.53 USD-$168,927.03 USD
Comcast intends to offer the selected candidate base pay within this range, dependent on job-related, non-discriminatory factors such as experience. The application window is 30 days from the date job is posted, unless the number of applicants requires it to close sooner or later.
The application window is 30 days from the date job is posted, unless the number of applicants requires it to close sooner or later.
Base pay is one part of the Total Rewards that Comcast provides to compensate and recognize employees for their work. Most sales positions are eligible for a Commission under the terms of an applicable plan, while most non-sales positions are eligible for a Bonus. Additionally, Comcast provides best-in-class Benefits to eligible employees. We believe that benefits should connect you to the support you need when it matters most, and should help you care for those who matter most. That's why we provide an array of options, expert guidance and always-on tools, that are personalized to meet the needs of your reality - to help support you physically, financially and emotionally through the big milestones and in your everyday life. Please visit the compensation and benefits summary on our careers site for more details.
Education
Bachelor's Degree
While possessing the stated degree is preferred, Comcast also may consider applicants who hold some combination of coursework and experience, or who have extensive related professional experience.
Relevant Work Experience
7-10 Years