If you are using cloudera’s distribution, you should be aware that hadoop’s configuration files locations are not static, but are generated every time they are changed and placed in a new location under /var/run/cloudera-scm-agent/process/.
because of that reason, our only option for editing log4j’s configuration is using cloudera manager’s (CM) safety valve.
From cloudera’s documentation:
A “safety valve” lets you insert a text string verbatim into a configuration file, such a XML configuration files, properties files, text files, or an environment. The safety value mechanism is intended for advanced use in cases where there is a specific Hadoop configuration setting that you find is not exposed in Cloudera Manager; contact Cloudera Support if you are required to use it. Safety valve configuration generally overrides normal configuration.
when you edit a safety valve, CM merges the new content with the original files, using a diff tool.we wanted to add a new appender to the main logger,so we were also editing an existing line, and not just appending content to the file.
So what’s the problem ? just add the new appender’s name to the root logger line like so:
log4j.rootLogger=TRACE,newAppender
well, it turns out that the original file creates a local variable named ‘main.logger’ that holds all appenders for the main logger, and then sets the value for root.logger:
root.logger=${log.threshold},${main.logger}
so, to be able to add appenders, you should edit the line that sets the ‘main.logger’ variable.
a complete example for yarn’s node manager’s log4j safety valve:
main.logger=RFA,SA log4j.appender.SA=org.apache.log4j.net.SocketAppender log4j.appender.SA.Port=4560 log4j.appender.SA.RemoteHost=h135 log4j.appender.SA.ReconnectionDelay=10000 log4j.appender.FILE=org.apache.log4j.FileAppender log4j.appender.FILE.File=/tmp/rm-log/log.log log4j.appender.FILE.layout=org.apache.log4j.PatternLayout log4j.appender.FILE.layout.conversionPattern=%m%n
