Editing log4j configuration using cloudera manager

If you are using cloudera’s distribution, you should be aware that hadoop’s configuration files locations are not static, but are generated every time they are changed and placed in a new location under /var/run/cloudera-scm-agent/process/.
because of that reason, our only option for editing log4j’s configuration is using cloudera manager’s (CM) safety valve.

From cloudera’s documentation:

A “safety valve” lets you insert a text string verbatim into a configuration file, such a XML configuration files, properties files, text files, or an environment. The safety value mechanism is intended for advanced use in cases where there is a specific Hadoop configuration setting that you find is not exposed in Cloudera Manager; contact Cloudera Support if you are required to use it. Safety valve configuration generally overrides normal configuration.

when you edit a safety valve, CM merges the new content with the original files, using a diff tool.we wanted to add a new appender to the main logger,so we were also editing an existing line, and not just appending content to the file.
So what’s the problem ? just add the new appender’s name to the root logger line like so:

log4j.rootLogger=TRACE,newAppender

well, it turns out that the original file creates a local variable named ‘main.logger’ that holds all appenders for the main logger, and then sets the value for root.logger:

root.logger=${log.threshold},${main.logger}

so, to be able to add appenders, you should edit the line that sets the ‘main.logger’ variable.
a complete example for yarn’s node manager’s log4j safety valve:

main.logger=RFA,SA

log4j.appender.SA=org.apache.log4j.net.SocketAppender
log4j.appender.SA.Port=4560
log4j.appender.SA.RemoteHost=h135
log4j.appender.SA.ReconnectionDelay=10000

log4j.appender.FILE=org.apache.log4j.FileAppender
log4j.appender.FILE.File=/tmp/rm-log/log.log
log4j.appender.FILE.layout=org.apache.log4j.PatternLayout
log4j.appender.FILE.layout.conversionPattern=%m%n