Glassfish v2 and syslogd – an Adventure

By | October 10, 2008

I did not post here for a long time. One of the reasons was, that I was not really able to say too much about what we were trying to build. But now “Project Kenai” has made it to the public beta. So I can say that I was working on it for the last 18 or so month;-)

We run JRuby inside Glassfish and that’s why I was interested in GF again. We have all kinds of servers running and we would like to consolidate all of the log information for easier handling. Most of the native Unix daemons don’t have a problem with that, they can all log their information via syslogd. Glassfish claims to be able to do that as well. But I would currently say, it’s more a claim than reality in the official builds:-(

We tried to let GF log via syslogd and found very quickly two problems:

  • You can kill GF (the running JVM) with the right kind of log messages:-( Off course, it took us only days  to hit that  problem , figuring  out the reason was a little bit harder. But I did it and filed issue 6291 but not to much moved on it:-(
  • If you want the access logs also via syslogd, you’re out of luck:-( That is hard-coded to to go to files.

But GF is OpenSource, right? I had already looked at the code to find the reason for the 6291 bug. Could I change it so that it would do what I wanted? It turns out that it was not too hard to get a first version working;-)

It’s probably not perfect and you can clearly see the steps I took but the two little patches below will:

  • Fix the bug, very important;-)
  • Allow to change the syslog facility to values other than LOG_DAEMON via an environment variable called GF_LOG_FACILITY. (Up to that point, I staid within the native code;-)
  • Allow for the access logs to be written to the syslogd as well. To do that, I check for a magic ‘::’ at the beginning of the ‘accesslog’ property of the ‘virtual-server’ elements in the domain.xml. That property is supposed to be a directory where GF would create the access log files. I thought, not too many directories would start with ‘::’ 😉

My first tests were successful. Now we have to decide if we want to use a patched version of GF.

The two patches are against the modules appserv-core and appserv-native.

For appserv-core, I followed the normal build instructions from the wiki and used the TSJSAS91_FCS_BRANCH tag. (I did a complete build but I don’t think that would be needed) I then transplanted the two interesting class files into a copy (appserv-rt.jar) of our official GF version for testing.

I did not get the appserv-native module via the normal CVS/maven checkout. But since I found the problematic file before with the Fisheye in the SVN tree, I was able to check that module out via SVN. The v3 build instructions help to find the right SVN tree, but we are still interested in v2 here;-) It’s native code, so if somebody wants to try that, you will need the Sun C compilers etc.

After building and replacing the libutilforsyslog.so, everything seem to be working as expected and I can enjoy my logs via the normal syslogd setup;-) Below are both, the normal server log and the access log directed to LOG_LOCAL6:

Oct 10 21:22:58 kenaidev13 SJSAS81_server.log: [ID 702911 local6.info] [#|2008-10-10T21:22:58.934-0700|INFO| \
sun-appserver9.1|javax.enterprise.system.container.web|_ThreadID=17;_ThreadName=httpSSLWorkerThread-8080-0;| \
PWC1412: WebModule[] ServletContext.log():
Oct 10 21:22:58 kenaidev13 Processing HomeController#index (for 127.0.0.1 at 2008-10-10 21:22:58) [GET]
Oct 10 21:22:58 kenaidev13   Session ID: 376348f6751afdd3c1220777a0ee15ca
Oct 10 21:22:58 kenaidev13   Parameters: {"controller"=>"home", "action"=>"index"}
Oct 10 21:22:58 kenaidev13 Rendering template within layouts/kenai.com
Oct 10 21:22:58 kenaidev13 Rendering home/index
Oct 10 21:22:58 kenaidev13 Completed in 0.34706 (2 reqs/sec) | Rendering: 0.30226 (87%) | DB: 0.00000 (0%) | 200 OK [http://localhost/]
Oct 10 21:22:58 kenaidev13 |#]
Oct 10 21:22:58 kenaidev13 [9374]: [ID 702911 local6.info] "127.0.0.1" "NULL-AUTH-USER" "10/Oct/2008:21:22:58 -0800" \
"GET / HTTP/1.0" 200 14058

If you made it all the way here, those are the two little patches:  appserv-core.patch & appserv-native.patch.  They are in no way official  or fully tested. But they show that somebody can quickly add functionality if needed;-)

Have fun & I hope I’ll write more often again;-)
— Marco

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.