Krzysztof Debski
From Allegro Group
Technical enthusiast, software engineer and product owner of backbone tools developed in Allegro. He is open to all new technologies and programming languages and he has no fear to work on legacy systems and rewrite it in modern way.
Blog: http://alllegro.pl
Methodology & DevOps
PubSub++ - few tips that make your life with kafka easier.
You’ve just set up your Kafka cluster and now you are ready to process tens of thousands events per second. You decoupled your architecture and now all the communication goes via pubSub bus and you can focus only on providing business value. It would be great if that can be true. In real life you need to do a lot of tweaks to have your backbone ready to handle all the traffic you want. In the talk we will focus on the project of redesigning the core event bus of the Allegro auction platform. We will talk issues on all the levels from network to data serialization. After the presentation you will have answers to the questions: How to assure our clients that not even a single event is lost? How to respond in less than 65ms even if Kafka is rebuilding its cluster? ...And many other questions you will face.