Observability Engineer

Get to Know Us:
It's fun to work in a company where people truly believe in what they're doing!
At BlackLine, we're committed to bringing passion and customer focus to the business of enterprise applications.
Since being founded in 2001, BlackLine has become a leading provider of cloud software that automates and controls the entire financial close process. Our vision is to modernize the finance and accounting function to enable greater operational effectiveness and agility, and we are committed to delivering innovative solutions and services to empower accounting and finance leaders around the world to achieve Modern Finance.
Being a best-in-class SaaS Company, we understand that bringing in new ideas and innovative technology is mission critical. At BlackLine we are always working with new, cutting edge technology that encourages our teams to learn something new and expand their creativity and technical skillset that will accelerate their careers.
Work, Play and Grow at BlackLine!
Make Your Mark:
The Observability Engineer - NOC, you will build and maintain tools, applications and services supporting the BlackLine SaaS application and internal teams. A successful candidate must possess solid critical thinking skills and have experience supporting large server farms, 24x7 High Availability mission-critical traffic-intensive web infrastructures and be familiar with container technologies. Large SaaS experience is very desirable.
You'll Get To:

• Ensure 99.99%+ availability of the services and infrastructure that spans across multiple global datacentres in private and public clouds.

• Troubleshoot BL container platforms and supporting automation in a highly available, high traffic environment.

• Monitor and maintain health, performance, and security of all infrastructure components.

• Build systems and perform necessary tasks to deliver against committed project timelines. Desire to automate everything

• Solve real-life problems in a bleeding-edge, high-performance, and high-traffic environment. Maintain documentation and operational knowledge base.

• Triaging first level events and incidents.

• Adhere to the change management and other established processes and procedures.

• Respond to and troubleshoot incidents (Incident Management). Conduct root cause analyses.

• Evaluate and analyse systems, performance, issues and metrics in order to provide recommendations for continuous improvements.

• Adhere to SLA compliance as defined.

• Participate in a scheduled 24/7 on-call rotation for second tier support escalations.
• Should be willing to work 3 days from office.

What You'll Bring:

• 3 - 6 years industry experience

• 3+ years supporting Unix and/or Linux (Ubuntu, CentOS, Redhat) and/or Windows

• 3+ years supporting a SaaS/Hosting type critical revenue-generating environment.

• 2+ years working with development and continuous integration related tooling (Jenkins, BitBucket, GitHub)

• 2+ years working with tools like New Relic, Jira, Foglight.

• 1+ years of experience using container platforms and tooling (Kubernetes, Docker, Rancher, Helm, Anthos, Istio, GKE, AKS, etc...)

• Experience in hybrid cloud and/or multi-cloud environments (GCP (primary), Azure, AWS, VMWARE)

• Understanding of software development processes and methodologies.

• Experience with scripting and/or systems programming languages (Bash, PowerShell, Python, Golang, C#).

• Hands-on problem-solving skills, technical leadership and mentoring qualities.

• Strong written and oral communication skills.

• Ability to participate in On-Call rotation

• A minimum of two years of experience in a 24x7 operations organization, deploying and operating complex cloud infrastructure at scale

• 3 days hybrid mandatory.

We're Even More Excited If You Have:

• Significant experience with open source platforms and technologies.

• Track record of architecting, developing, implementing robust, distributed online solutions.

• Familiarity with Configuration Management tools (Puppet, Chef, Salt, Ansible).

• Experience in mixed Windows/Unix/Linux environments.

Thrive at BlackLine Because You Are Joining:

• A technology-based company with a sense of adventure and a vision for the future. Every door at BlackLine is open. Just bring your brains, your problem-solving skills, and be part of a winning team at the world's most trusted name in Finance Automation!
• A culture that is kind, open, and accepting. It's a place where people can embrace what makes them unique, and the mix of cultural backgrounds and varying interests cultivates diverse thought and perspectives.
• A culture where BlackLiner's continued growth and learning is empowered. BlackLine offers a wide variety of professional development seminars and inclusive affinity groups to celebrate and support our diversity.

BlackLine is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to sex, gender identity or expression, race, ethnicity, age, religious creed, national origin, physical or mental disability, ancestry, color, marital status, sexual orientation, military or veteran status, status as a victim of domestic violence, sexual assault or stalking, medical condition, genetic information, or any other protected class or category recognized by applicable equal employment opportunity or other similar laws.
BlackLine recognizes that the ways we work and the workplace itself have shifted. We innovate in a workplace that optimizes a combination of virtual and in-person interactions to maximize collaboration and nurture our culture. Candidates who live within a reasonable commute to one of our offices will work in the office at least 2 days a week.