Develop and maintain high-volume Kafka data ingestion pipelines in Python using Producer/Consumer APIs and Kafka Connect (Debezium/JDBC). Configure topics, brokers, schema registry, converters, and connectors; implement CDC, custom producer/consumer logic and transformations; optimize broker/producer/consumer/connect/stream settings; deploy multiple source/sink connectors to production and integrate Kafka with messaging systems like RabbitMQ, Redis, and AWS SQS.
The summary above was generated by AI
This is a remote position.
Please go through the entire job post thoroughly before pressing Apply. Post pressing Apply, you shall reach the assessment page that must be attempted.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Busigence is a Decision Intelligence Company. We create decision intelligence products for real people by combining data, technology, business, and behaviour enabling strengthened decisions.
Kafka Developer
Team: Engineering
Location: Remote
Relevant Exp: 0-3 Years
Background: Been there-Done that
Compensation: Above industry standards
Requirements This is an immediate requirement. We shall have an accelerated interview process for fast closure - you would required to be proactive and responsive
Remote position (work-from-anywhere)
Immediate joiners must apply
Kafka Experienced (<5 years)
Competitive compensation
************
| MUST HAVE |
************
https://kafka.apache.org/documentation/
1. Developed (in Python - not Java) data ingestion pipelines using Kafka Producer-Consumer APIs? Refer Kafka documentation Chapter 2.1 & 2.2.
2.Implemented data ingestion pipelines using Kafka Connect (Debezium and/or JDBC)?
Refer Kafka documentation Chapter 2.1 & 2.4 and 8.
(It means built connectors to extract data from sources, applying minor transformations at topics level, and load data into destinations. Sources can be databases, streams, rest/soap APIs and destinations can be databases, flat files)
3.Configured event streams for data platforms ?
Refer Kafka documentation Chapter 3
===========================================
We are offering one of the most challenging & exciting work on distributed events streams. You shall be working on sophisticated platforms, products and applications
===========================================
We are looking for developer with real passion for data ingestion pipelines. This is a specialist and individual contributor role. Product development experience preferably at a startup or a lean team is desired
ROLE
We are looking for engineers with real passion for data pipelines with actual hands-on experience developing data application on Kafka. You would be required to work with our data science team on development of several data applications.
Mandatory
1. Must have hands-on experience working on Kafka connect using schema registry in a very high volume environment
2. Must have worked with JDBC connectors and APIs
3. Must have worked on Kafka topics, Kafka brokers, zookeepers, and Kafka Control center
4. Must have worked on on AvroConverters, JsonConverters, and StringConverters
5. Must have worked on Debezium source/sink connectors with CDC implementation
6. Must have worked on producer API custom logics
7. Must have worked on consumer API complex transformations
8. Must have worked on setting optimal configuration for broker, topics, producer, consumer, connect, stream and admin
9. Must have deployed atleast ten source/sink connectors in production
10. Must have worked on distributed computing, parallel processing and large-scale data management
11. Must have integrated Kafka with RabbitMQ/ Reddis/ AWS SQS
Preferred
1. Good to have worked on Admin API, Connect API and Stream API
2. Good to have worked on development of data ingestion platform
3. Good to have worked on vinaigrette KSQL, KStream
4. Good to have worked on confluent connectors
5. Good to have build connectors from scratch using Java
Benefits How to Apply
You should apply online by clicking "Apply Now".
For more information, visit http://www.busigence.com
Products: http://busigence.com/offering
Careers: http://careers.busigence.com
Research: http://research.busigence.com
Jobs: http://careers.busigence.com
Scaling established startup innovating & disrupting various domains through artificial intelligence. We bring those people onboard who are dedicated to deliver wisdom to humanity by solving the world’s most pressing problems differently thereby significantly impacting thousands of souls, everyday.
We are a deep rooted organization with success story having worked with folks from top tier background (IIT, NSIT, DCE, BITS, IIITs, NITs, IIMs, ISI etc.) maintaining an awesome culture with a common vision to build great data products. In past we have served fifty five customers and presently developing Enterprise AI products. More details at http://busigence.com/offering
We work extensively & intensely on big data, data science, machine learning, deep learning, reinforcement learning, data analytics, natural language processing, cognitive computing, and business intelligence.
We look for talent, not skills
1. You must be [super sharp]
2. You must be [extremely energetic]
3. You must be [honourably honest]
In addition to the regular stuff which every good startup offers – Lots of learning, Food, Parties, Open culture, Flexible working hours, and what not….
We offer you: [Greatest work of life]
You shall be working on our revolutionary products which are pioneer in their respective categories. This is a fact.
We try real hard to hire fun loving crazy folks who are driven by more than a paycheque. You shall be working with creamiest talent on extremely challenging problems at most happening workplace
