Adding system environment properties to log4j filename

Hi, Happy Friday!

The holiday is just around corner!  I feel like posting something before the holiday is really here. And it happens I do have something worth posting. 🙂

This week we found that in a cluster environment, log4j may encounter an issue with writing log entries to FileAppender. The problem is essentially that when multiple nodes from the same cluster are trying to write to the same log file, the competing threads can cause some of the events to be dropped without being written to the file. The losing rate depends on how heavy the logging events.

We found a way to mitigate this issue by adding a hostname to the logging filename. This way, each node will be writing to its own logging file, avoiding IO contention.

How to achieve this involves two steps. First, adding a system variable in your application startup process, such as:

System.setProperty(“HostName“, InetAddress.getLocalHost().getHostName()); 

Then reference this HostName system variable in your Log4j.xml configuration file, something like the following:

<appender name=”FileAppender” class=”org.apache.log4j.RollingFileAppender”>
 <param name=”file” value=”/tmp/logs/app_${HostName}.log” />
 <param name=”MaxFileSize” value=”1024KB” />
 <param name=”MaxBackupIndex” value=”30″ />
 <param name=”Threshold” value=”DEBUG” />
 <layout class=”org.apache.log4j.PatternLayout”>
  <param name=”ConversionPattern” value=”%5p [%d] [%t] (%F:%M:%L) – %m%n” />

The application will write logging entries to the logging file with an actual host name in its filename. Another beautiful side effect of this is that one can quickly tell where the logging event is originated from, which often provides valuable information about the host in the cluster.

Of course, if you want to add more environment variables besides the HostName, it can be done in similar fashion. Also, the variables don’t need to be in the filename, it can be in the content of the logging message.

Cheers and have a happy holiday season!

-T. Y.





Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s