log4j AsyncAppender missing class/method/line numbers???

Hi, There:

This post is about Log4j again. We keep getting three ?:?:? in our log4j logging files for the Class name:Method name and Line Number. Basically we want to have this %F:%M:%L  in the ConversionPattern but got no information back but ?:?:?.

The reason is that we use org.apache.log4j.AsyncAppender for better performance. This AsyncAppender references to org.apache.log4j.RollingFileAppender. Because AsyncAppender uses another thread when write the logging event to the FileAppender, the location information about class/method/line are lost in the FileAppender.

The simple fix to this is to set the boolean attribute LocationInfo to true, it will then preserve the calling thread information from the AsyncAppender to the next FileAppender.

<appender name=”AsyncAppender” class=”org.apache.log4j.AsyncAppender”>
   <appender-ref ref=”FileAppender” />
   <param name=”LocationInfo” value=”true”/>

The log file now have all the class/method/line number we can use during debugging.





Adding system environment properties to log4j filename

Hi, Happy Friday!

The holiday is just around corner!  I feel like posting something before the holiday is really here. And it happens I do have something worth posting. 🙂

This week we found that in a cluster environment, log4j may encounter an issue with writing log entries to FileAppender. The problem is essentially that when multiple nodes from the same cluster are trying to write to the same log file, the competing threads can cause some of the events to be dropped without being written to the file. The losing rate depends on how heavy the logging events.

We found a way to mitigate this issue by adding a hostname to the logging filename. This way, each node will be writing to its own logging file, avoiding IO contention.

How to achieve this involves two steps. First, adding a system variable in your application startup process, such as:

System.setProperty(“HostName“, InetAddress.getLocalHost().getHostName()); 

Then reference this HostName system variable in your Log4j.xml configuration file, something like the following:

<appender name=”FileAppender” class=”org.apache.log4j.RollingFileAppender”>
 <param name=”file” value=”/tmp/logs/app_${HostName}.log” />
 <param name=”MaxFileSize” value=”1024KB” />
 <param name=”MaxBackupIndex” value=”30″ />
 <param name=”Threshold” value=”DEBUG” />
 <layout class=”org.apache.log4j.PatternLayout”>
  <param name=”ConversionPattern” value=”%5p [%d] [%t] (%F:%M:%L) – %m%n” />

The application will write logging entries to the logging file with an actual host name in its filename. Another beautiful side effect of this is that one can quickly tell where the logging event is originated from, which often provides valuable information about the host in the cluster.

Of course, if you want to add more environment variables besides the HostName, it can be done in similar fashion. Also, the variables don’t need to be in the filename, it can be in the content of the logging message.

Cheers and have a happy holiday season!

-T. Y.