ICP-0701 Apache Spark - when things go wrong | Devoxx

Apache Spark - when things go wrong


cloud Cloud & BigData

Room 4A - Metrosoft

Tuesday at 15:10 - 16:10

Are you familiar with this scenario: it’s a perfect morning, your new greenfield project (based on top of Apache Spark) runs just smoothly… and then something happens. It might be that your job drops suddenly on its initial performance or you ar struggling with an exception that ‘googled’ gave you just 3 results (2 of them in some foreign language). How to handle situations when things go wrong? This is hands-on talk, with many code examples & best practices. We will describe basic concepts like: tasks, stages, DAG schedulers. We will focus on problems of partitioning, shuffling, GC. We will see how profiles, spark-ui and solid knowledge of the API can help you with your daily Spark challenges.

Paweł Szulc Paweł Szulc

Software engineer, programmer, developer. Experienced with Java ecosystem. Currently having tons of fun working with Scala, Akka and Apache Spark. Humble apprentice of Functional Programming. Runs a blog http://www.rabbitonweb.com