Memory allocation in spark has three key contention points, this post is a break down of the three, and a description of…
Deploying applications on yarn using Apache Twill – introduction
With the introduction of yarn, hadoop had transformed from a pure map reduce computation engine (and dfs), into a general…
Spark application logging
When coding a spark application, we often want to write some application logs to trace or track our application’s progress.…
Editing log4j configuration using cloudera manager
If you are using cloudera’s distribution, you should be aware that hadoop’s configuration files locations are not static, but are…
Setting up a central logging infrastructure for hadoop and spark
logs are critical for troubleshooting, but when an application is distributed across multiple machines, things gets complicated. things gets even…
Getting more business value with Task based UI
We control the user’s interaction style with our system through the provided UI, but what style of interaction will provide…
