Ruby addict, Erlang and Elixir enthusiast, and curious about all things distributed. Worked in event driven architectures for security and performance analysis systems, now working in Logstash core at Elasticsearch. I have a black labrador, like tango and martial arts and live with my girlfriend of almost 10 years in sunny Lisbon.
What is Logstash? Do you use it? What is it for? Why Ruby? How does it work? Does it fit my architecture?
Logstash is the perfect junction in an event driven architecture: it allows you to stream data from point A to point B and perform manipulation/validation/decoration inflight. The most common use case consists of pulling log data from your IT infrastructure, structuring it and feeding the information to an analytics platform, but there's SO MUCH MORE you can do, I'll show you!
I'll also talk about its present and future: the soon to be released version 1.5 brings decoupling of the core of Logstash from its >100 plugins, providing the end user with the ability to install, uninstall and upgrade plugins between Logstash releases. What about version 2.0? It will target much desired features such as resiliency, changing the configuration in runtime and clustering!
So.. Has it been all puppies, sunshine and rainbows? Of course not, I'll go through some of the problems we had (and have) to deal with, elephants in the room, mistakes we made, lessons learned and things we should(?) address but chose not to (for now).
Come and learn how to deal with your logs (or whatever data) by collecting, enriching and sending them to elasticsearch and/or pagerduty.
In this workshop, you will learn how logstash actually works and how you can use it on your apache logs to understand for example the location of your service DDoS.