Editing log4j configuration using cloudera manager

If you are using cloudera's distribution, you should be aware that hadoop's configuration files locations are not static, but are generated every time they are changed and placed in a new location under /var/run/cloudera-scm-agent/process/. because of that reason, our only option for editing log4j's configuration is using cloudera manager's (CM) safety valve. From cloudera's documentation:... Continue Reading →

Setting up a central logging infrastructure for hadoop and spark

logs are critical for troubleshooting, but when an application is distributed across multiple machines, things gets complicated. things gets even more complicated when your application uses 3rd party APIs, and the answer you are looking for is hiding in one of those other systems¬†logs (which are distributed as well). you end up going through lots... Continue Reading →

Website Powered by WordPress.com.

Up ↑