Marco's Corner

May 2013
M	T	W	T	F	S	S
« Mar
	1	2	3	4	5
6	7	8	9	10	11	12
13	14	15	16	17	18	19
20	21	22	23	24	25	26
27	28	29	30	31

Solaris 11 – IPS Packages and Puppet

March 18, 2013 at 20:35 admin No comments

Our installation runs on Solaris 11 zones with many custom built IPS packages. That all works pretty well but sometimes config files need to change depending on which version of a package is installed or even if a particular package is installed at all on that zone. To make that information available to puppet, I decided to create a custom fact-set and the corresponding puppet extensions to compare versions.

The facter extension is pretty small and creates one fact for every installed package we are interested in. It looks like:

.../puppet/modules/pppt/lib/facter# cat foo_packages.rb
if File.executable?("/usr/bin/pkg")
  output = %x{/usr/bin/pkg list -Hv --no-refresh 'foo/*'}
  output.split(/$\n/).each do |str|
    if str =~ /^pkg:\/\/foo(\/foo\/[^@]+)@([^,]+)/
      p=$1
      v=$2
      Facter.add(("pkg" + p.gsub("/", "_")).to_sym) do
        setcode do
          v.to_s
        end
      end
    end
  end
end

The puppet parser function looks like:

.../puppet/modules/pppt/lib/puppet/parser/functions# cat pkg_installed.rb 
#
# A function to check if a IPS package is installed or not with relation
# to a given version.
# args[0] - a string of the package name, something like "foo/source/bugzilla"
# args[1] - a relation to the version, something like
#   "not" - not installed (third arg is not used)
#   "eq", "ge", "gt", "le", "lt", "ne" - relation to the version number in the third
#           argunment
# args[2] - a string specifying the version with the same restriktions as IPS 
#
module Puppet::Parser::Functions
  newfunction(:pkg_installed, :type => :rvalue) do |args|
    name = args[0]
    rel = args[1]
    vers = args[2]
    ret = nil
    done = nil

    # warning "mw pkg_installed(#{name}, #{rel}, #{vers})"
    if name && rel
      fct_name = "pkg_" + name.gsub("/", "_")
      fct_vers = lookupvar(fct_name)
      # warning "mw t1 name= #{fct_name} => #{fct_vers}"

      if rel == "not"
        ret = true if fct_vers == :undefined
        done = true
      else
        if (fct_vers == :undefined) || vers.nil?
          done = true
        end
      end
    end

    unless done
      # warning "mw t2 tests done"

      a_version = fct_vers.split('.')
      b_version = vers.split('.')

      l = a_version.length
      l = b_version.length if b_version.length > a_version.length

      0.upto(l - 1) do |i|
        # warning "mw t3 [#{i}] a_version= #{a_version[i]} <==> b_version= #{b_version[i]}"

        if a_version[i] > b_version[i]
          if [ "ge", "gt", "ne" ].include?(rel)
            ret = true
            done = true
          elsif [ "eq", "le", "lt" ].include?(rel)
            done = true
          end
          break
        elsif a_version[i] < b_version[i]
          if [ "le", "lt", "ne" ].include?(rel)
            ret = true
            done = true
          elsif [ "eq", "ge", "gt" ].include?(rel)
            done = true
          end
          break
        end
      end
    end

    unless done
      ret = true if [ "eq", "ge", "le" ].include?(rel)
    end

    ret
  end
end

That makes it pretty easy to create conditional steps either in the manifests or even in templates. For manifests, it would look like:

if pkg_installed("foo/tomcat6/password", "ge", "0.1.201301032306") {
  ...
}

In templates, the similar construct would look like:

<% if scope.function_pkg_installed([ "foo/tomcat6/password", "ge", "0.1.201301240309" ]) -%>
  ...
<% end -%>

I hope this helps somebody;-)

Uncategorized none

Science Olympiad – Div C – Robot Arm

March 15, 2013 at 21:36 admin No comments

This post is about how little differences might cause huge effects in the end. Our team was building a Robot Arm for the Science Olympiad, Division C event. After telling it where the bonus box is located and giving it the start command, it was supposed to be working completely autonomous via a per-determined script to move all elements into their respective target boxes.

The main brain of the arm was again a little Arduino Mega ADK supported by a Maestro Servo Controller to handle the RC servo commands. The motors were different RC servos, selected for the required torque. Many support elements came from the lynxmotion servo erector set or were build by hand. The Arduino sketch and all the required supporting docs are available @ my (the dad’s) github account;-)

This worked pretty well in the test environment in our living room. One early test run in November 2012 looked ok:

This was gradually improved with the extra ball claws and updated programming.

BUT on the day of the competition, not much worked as expected. The arm caught the West goal box pretty early one and pushed that around on the field which in turn pushed many of the elements away from the program expected them:-( So a little problem caused the run to end up with a score of 23 points or so:-( Not what we expected. Thanks to the event supervisors for the video. This was a closed event, so the video was all I saw.

After the competition, I tried it one last time before the arm was partially dismantled and stored. That run was again much better and I calculated a possible score above 80 points.

So what did we learn? Autonomous works ok for the time limit, but a little difference can quickly screw up everything:-(

As always, have fun exploring!

Uncategorized none

Science Olympiad – Div C – Maglev

March 5, 2013 at 20:20 admin 8 comments

The reason for the maglev track and the photo gates was actually to test vehicles for the Science Olympiad Division C – Maglev event. Unfortunately, in San Joaquin County, this was a closed event, so I don’t have a video of the actual run.

But below is a video of the winning vehicle later in our living room. The event distance was set at 70 cm, so that’s the distance in the video as well. The vehicle took 0.54 seconds for that run, but that was one of the slower ones. Test runs were as fast as 0.49 seconds for the 70 cm distance. The weight of the vehicle as run is 523 grams.

That would have been a Vehicle Score of 969 in an official run.

The official run last Saturday helped our son a lot to win the first place in this event. But Maglev is a dual-event, which has a exam/test and the build/run part which are weighted equally.

Update: Comments below asked for some close-up pictures. The vehicle does not exist any longer. But the motor mount is now part of an experimental hovercraft setup. I took some photos of that, the motor box is still the same. A box out of model-grade plywood (LxHxW: 25mm x 150mm x 67mm) with holes for the fans and two smaller holes to bring the cables forward. The fans are hold in place by those blue rubber bands in the back. One picture shows the difference between the original intake (top fan) and the cut intake to make the EDF conform to the requirements.

The box was screwed to the vehicle base (another piece of plywood) with two M4 screws, similar to the base it’s on now without the servo & steering setup;-) The battery was sitting on the front part of the base, secured by one of those removable 3M hook strips. So we could experiment with it before deciding on the best spot (balance, …).

So I hope that helps;-)

Uncategorized none

Gates and Maglev Track

November 2, 2012 at 21:34 admin 2 comments

Earlier I wrote about my DIY photogates. They work pretty well but there is always room for improvements. So I added a little 16×2 LCD display to the Arduino setup. That way, you can use the large computer-displayed stopwatch or you can rely on the little LCD. Both work simultaneously and on the same Arduino time base;-) The new sketch is here: gates2

Without the USB power from the computer, you will have to find a different power source, but the Arduino will work with very different ones. I tested four or six AA NiMH packs and a 9V wall-wart. An USB wall-wart which could take a normal A-B USB cable (not a fixed mini-B or Micro-B) would also work.

Our maglev track for the Science Olympiad (the reason for all of this;-) is also ready for first tests;-) It’s 3″ wide (the maximum) and six feet long. The rules only require a four feet long track, but that way there is a `braking room’ on both side. All the way at both ends, a pair of reversed magnets will really slow down the vehicle;-)

A couple of pictures from the current setup;-)

Have fun.

Uncategorized none

Arduino-based Photogate Timer

October 19, 2012 at 19:29 admin 3 comments

Mostly for the Science Olympiad – Division C – Maglev event, but also just to learn more about Arduino in general, I tried to build a photogate setup on the cheap;-) I expect the Maglev event’s timed part to be somewhere in between two and three seconds. So even for test runs, a hand based stop watch is not really an option. Some high schools apparently have photogate timers for their Physics education (we had them even at the middle school level in Germany around 1982), but our charter school does not. So those are basically the reasons why I set out to build something to use.

After looking at the commercially available setups and finding that they are somewhat out of what I wanted to spend, I found Liudr’s Photopage which gave me hope. He used rather small gates which would not work for me. So the first step was to find some gates which could handle >= 8cm distance. I found that the Honeywell HOA 6299 gates should be able to do that and that it should be really easy to interface them to the Arduino ports;-) I got two pairs @ newark.com for $63, not exactly cheap but cheaper than one complete commercial gate. They work very well with my Mega ADK;-) I currently use a breadboard, but it is not really needed. The 5V pin works well as power supply for the gates, and the outputs go straight to pin 2 and 3 respectively. Any 5V based Arduino should be able to handle the gates, the ADK is kind of overkill for this, but that’s what I have;-)

For the software, there are two parts. The Arduino sketch gets interrupts from the two gates and basically takes it’s internal time and sends lines over the USB/serial connection to the computer which acts as display and interim time source. The Arduino does not care what will be done with the events, it just sends one line with three parts separated by colons. The gate number ( 1 or 2), the time in milliseconds since the start of the board and the microseconds within the current millisecond. Lines look something like

1:123324:003
2:134456:345

That makes it really easy to do different things depending on what you want to do. My little sketch PhotoGates.ino is here, if somebody wants to play with similar things;-)

For the stopwatch/display on the computer (I prefer Linux;-), I Frankenstein’ed the dclock sources, to do what I want;-) That was the best source which would work well in different sizes, I updated it to show three digits for the milliseconds Right now that piece is really raw!! But it takes the lines from the USB/Serial port (Watch out, I don’t think the option to change the port actually works yet!!) and handles them correctly. When it sees a line from gate one, it stores the time values and starts a clock from null on the computer to show a moving time until a line from gate two arrives. When the line from gate two is handled, the difference between that and the save time from gate one is pushed to the display. So no matter how long the events take to get to the destination or how busy the computer is, the time should be correct in the end;-) My sources are here dclock-2.2.2-mw.tar, but they might still need some improvements. For other OSs, I don’t know but something similar should be possible as well. But I prefer Linux;-)

Right now, I have the two gates right behind each other, something like 7 cm apart. That’s just for now for easy testing;-) The fastest I could cover that distance with a pencil or a kebab stick (1/8″ dowel;-) in my hand, was 0.014 seonds;-) That’s about 18kph/11 mph. But that just my slow hand;-)

So far, I’m pretty happy with the setup. It should definitely be able to do what I wanted it for. If somebody wants to play with similar stuff of has a question about it, real comments are always welcome;-)

As always, have fun playing with things in new ways;-)

Update: The Complete Track and Timer is here;-)

Uncategorized none

Arduino and LEGO NXT is fun

October 15, 2012 at 23:34 admin No comments

OK, I knew about the Arduino platform for quite some time. But I was not sure what I would like to to do with it. But with the next Science Olympiad looming on the horizon and the need to find a cheap photo gate timing system, I had the idea of an Arduino-based system. The first thing I built was not the photo gate setup, but a little autonomous car;-)
The little car is a combination of LEGO MINDSTORMS NXT parts with a Arduino ADK board, a NXShield-M to connect the ADK to the NXT pieces and a TinkerKit Mega Sensor Shield for the LED illumination;-) The sensors are a mindsensors NXTSumoEyes IR detector as the main eyes and a standard NXT touch sensor just as backup in case the eyes do miss an obstacle.

In the picture on the left, the front shows the SumoEyes, the touch sensor with a front bumper and the TinkerKit LEDs on the top. In the picture on the right, the Arduino shield stack is nicely visible. The battery pack (6*AA) is not really visible, it’s under the NXShield-M. Since the TinkerKit uses pretty large plugs for it’s sensors and LEDs, I needed to add some extra headers as spacers between the TinkerKit shield and the NXShield-M. All the way on the top sits the ADK;-)

Two NXT motors build the drive train and a little cap nut in the back is used as third `wheel’. I did not find a nice little wheel which would work well for quick turning and the nut works pretty well on laminate (not so well on carpet;-).
This was my first Arduino project after the obligatory `led on pin 13′ tests;-) I’m surprised how quickly you can create little projects which work actually pretty well. The whole Arduino sketch is only 214 lines long;-) My little sketch is here when you’re interested. Have fun experimenting with it.

Below is a little `night time’ drive video. My phone was not too happy with the low light, so some parts are not as sharp as I would have liked. But overall it shows pretty well how good the eyes work;-) The car does not even need it’s touch sensor in that run.

As always have fun experimenting;-)

Uncategorized none

Jruby and Thread Dumps

August 7, 2012 at 22:00 admin 1 comment

OK, today I would like to share a little patch I was toying around with in the JRuby world.

First a little background. We run some Ruby on Rails application via JRuby in JavaEE web containers, either in Tomcat or in GlassFish. That’s all nice and good when everything is working. But we have some requests once in a while which take forever. So far nobody could find a good cause for that and it usually happens ones the server is up and running for a while:-( The JVM has a helper for situations like that: you can send a SIGQUIT to the running process and it will produce a nice thread dump for all threads. That works OK for Java since it gives you the source files and line numbers, so you can get a good idea where & what your different threads are doing. Trying to do that for JRuby, you get a nice dump of the JRuby internals, but no real useful information at the Ruby level:-(

A SIGQUIT thread dump for a mongrel setup (my dev environment) looks something like the piece below, for a Tomcat/GF setup, it would have many more threads but they would be very similar, at least the tops of them. Look at it and try to spot, that the sleep() on the top is a sleep() I added at the first line of one of our RoR views.

~/testapp$ ~/jruby/bin/jruby -S script/server  &
[1] 8260
mw install MwSignalHandler(USR2)
=> Booting Mongrel
=> Rails 2.3.11 application starting on http://0.0.0.0:3000
** [NewRelic] NewRelic Agent Developer Mode enabled.
** [NewRelic] To view performance information, go to http://localhost:3000/newrelic
** [NewRelic] New Relic RPM Agent 2.9.5 Initialized: pid = 8260
** [NewRelic] Agent Log found in ~/testapp/log/newrelic_agent.log
=> Call with -d to detach
=> Ctrl-C to shutdown server

~/testapp$ jobs
[1]+  Running                 ~/jruby/bin/jruby -S script/server &
~/testapp$ kill -QUIT %1
2012-08-07 20:37:00
Full thread dump Java HotSpot(TM) 64-Bit Server VM (20.1-b02 mixed mode):

"RubyThread-32: ~/testapp/vendor/bundle/jruby/1.8/gems/mongrel-1.1.5-java/lib/mongrel.rb:285" daemon prio=10 tid=0x00007f55ac099800 nid=0x20a5 in Object.wait() [0x00007f55aa7bc000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000fe1b29f8> (a org.jruby.RubyThread)
at org.jruby.RubyThread.sleep(RubyThread.java:801)
- locked <0x00000000fe1b29f8> (a org.jruby.RubyThread)
- locked <0x00000000fe1b29f8> (a org.jruby.RubyThread)
at org.jruby.RubyKernel.sleep(RubyKernel.java:824)
at org.jruby.RubyKernel$s$0$1$sleep.call(RubyKernel$s$0$1$sleep.gen:65535)
at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:630)
at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:207)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169)
at org.jruby.ast.FCallOneArgNode.interpret(FCallOneArgNode.java:36)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.ast.EnsureNode.interpret(EnsureNode.java:96)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:75)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:212)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:187)
at org.jruby.RubyClass.finvoke(RubyClass.java:576)
at org.jruby.RubyBasicObject.send(RubyBasicObject.java:2772)
at org.jruby.RubyKernel.send(RubyKernel.java:2097)
at org.jruby.RubyKernel$s$send.call(RubyKernel$s$send.gen:65535)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:342)
at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:212)
at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:221)
at org.jruby.ast.CallTwoArgBlockNode.interpret(CallTwoArgBlockNode.java:62)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)
at org.jruby.runtime.InterpretedBlock.evalBlockBody(InterpretedBlock.java:374)
at org.jruby.runtime.InterpretedBlock.yield(InterpretedBlock.java:295)
at org.jruby.runtime.InterpretedBlock.yieldSpecific(InterpretedBlock.java:229)
at org.jruby.runtime.Block.yieldSpecific(Block.java:99)
at org.jruby.ast.ZYieldNode.interpret(ZYieldNode.java:25)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.ast.EnsureNode.interpret(EnsureNode.java:96)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:75)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:212)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:187)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:322)
at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:178)
at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:187)
at org.jruby.ast.CallOneArgBlockNode.interpret(CallOneArgBlockNode.java:60)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:75)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:255)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:203)
at org.jruby.internal.runtime.methods.AliasMethod.call(AliasMethod.java:91)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:342)
at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:212)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:216)
at org.jruby.ast.FCallSpecialArgBlockPassNode.interpret(FCallSpecialArgBlockPassNode.java:40)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)
at org.jruby.runtime.InterpretedBlock.evalBlockBody(InterpretedBlock.java:374)
at org.jruby.runtime.InterpretedBlock.yield(InterpretedBlock.java:295)
at org.jruby.runtime.InterpretedBlock.yieldSpecific(InterpretedBlock.java:229)
at org.jruby.runtime.Block.yieldSpecific(Block.java:99)
at rubyjit.trace_method_execution_with_scope_78ABF05203AC3E33573FAF5CB6B4659DC346B6C3.chained_2_ensure_1$RUBY$__ensure__(~/testapp/vendor/plugins/newrelic_rpm/lib/new_relic/agent/method_tracer.rb:62)
at rubyjit.trace_method_execution_with_scope_78ABF05203AC3E33573FAF5CB6B4659DC346B6C3.__file__(~/testapp/vendor/plugins/newrelic_rpm/lib/new_relic/agent/method_tracer.rb:61)
at rubyjit.trace_method_execution_with_scope_78ABF05203AC3E33573FAF5CB6B4659DC346B6C3.__file__(~/testapp/vendor/plugins/newrelic_rpm/lib/new_relic/agent/method_tracer.rb)
at org.jruby.internal.runtime.methods.JittedMethod.call(JittedMethod.java:87)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:272)
at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:80)
at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:89)
at org.jruby.ast.CallManyArgsBlockNode.interpret(CallManyArgsBlockNode.java:59)
....

As you can see, it’s mostly AST handling in the JRuby runtime but not much useful. I actually took only the top of the dump, it’s about 1000 lines long, but I wanted to show that it sometimes has some references to JITed methods which can give you a hint about a location. But even that’s not really granular:-(

But JRuby can create nice Ruby level stack traces when it hits a `Ruby level exception’. So the runtime system has to keep all that information somewhere. After that realisation, I started looking at the OpenJDK and the JRuby sources.

I looked at OpenJDK to see if there is any possibility to combine the Java and JRuby dumps somehow. Either as an additional section or somehow intermixed. But I could not find any way to do that, the SIGQUIT handling is completely done at the C/C++ level without an easy way to extend it:-( Even chaining signal handlers for the SIGQUIT does not seem to be possible, at least not in the OpenJDK and the Oracle JDK. So I had to use a different way, I decided to use SIGUSR2 for now, but that could easily be switched in the patch.

The excursion into JRuby was to find where it kept it’s stack information and see if there is a way to print it whenever I wanted to. Most of it is handled in ThreadContext.java and I found a reasonably usable spot for my hack in RubyThread.java. So when I now send a SIGUSR2 to my patched jruby, I get a Ruby level tread dump for all Ruby threads.

For the same sleep(), that looks something like

~/testapp$ kill -USR2 %1
========== MwSignalHandler.signalAction(USR2) ==========
========== RubyThread = Thread[main,5,main]
========== ThreadContext= org.jruby.runtime.ThreadContext@4b0fc9d1
#<Class:0xea211cf>#run at ~/testapp/vendor/bundle/jruby/1.8/gems/rack-1.1.2/lib/rack/handler/mongrel.rb:37
#<Class:0x10cbd8dc>#(root) at ~/testapp/vendor/bundle/jruby/1.8/gems/rails-2.3.11/lib/commands/server.rb:110
========== RubyThread = Thread[RubyThread-1: ~/testapp/vendor/plugins/newrelic_rpm/lib/new_relic/agent/stats_engine.rb:29,5,main]
========== ThreadContext= org.jruby.runtime.ThreadContext@798aded6
NewRelic::Agent::StatsEngine#spawn_sampler_thread at ~/testapp/vendor/plugins/newrelic_rpm/lib/new_relic/agent/stats_engine.rb:31
========== RubyThread = Thread[Thread-2,5,]
========== RubyThread = Thread[Thread-3,5,]
========== RubyThread = Thread[RubyThread-12: ~/testapp/vendor/bundle/jruby/1.8/gems/mongrel-1.1.5-java/lib/mongrel.rb:268,5,main]
========== ThreadContext= org.jruby.runtime.ThreadContext@c2bf1f1
Mongrel::HttpServer#run at ~/testapp/vendor/bundle/jruby/1.8/gems/mongrel-1.1.5-java/lib/mongrel.rb:271
========== RubyThread = Thread[RubyThread-13: ~/testapp/vendor/bundle/jruby/1.8/gems/mongrel-1.1.5-java/lib/mongrel.rb:285,5,main]
========== ThreadContext= org.jruby.runtime.ThreadContext@19190b75
ActionView::Base::CompiledTemplates#_run_erb_app47views47home47index46html46erb at ~/testapp/app/views/home/index.html.erb:0
#<Class:0x136157d8>#render at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_view/renderable.rb:33
ActionView::Base#with_template at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_view/base.rb:305
ActionView::Renderable#render at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_view/renderable.rb:29
#<Class:0x136157d8>#render_with_trace_View___path__Rendering at ~/testapp/vendor/plugins/newrelic_rpm/lib/new_relic/agent/method_tracer.rb:160
ActionView::Template#render_with_trace_View___path__Rendering at ~/testapp/vendor/plugins/newrelic_rpm/lib/new_relic/agent/method_tracer.rb:159
ActionView::Template#render_template at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_view/template.rb:204
ActionView::Base#render at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_view/base.rb:264
ActionView::Base#_render_with_layout at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_view/base.rb:347
ActionView::Base#render at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_view/base.rb:261
ActionController::Base#render_for_file at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/base.rb:1251
ActionController::Base#render at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/base.rb:935
HomeController#render_with_benchmark at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/benchmarking.rb:50
#<Class:0x669b9a30>#ms at ~/testapp/vendor/bundle/jruby/1.8/gems/activesupport-2.3.11/lib/active_support/core_ext/benchmark.rb:16
Benchmark#realtime at ~/jruby/lib/ruby/1.8/benchmark.rb:307
#<Class:0x669b9a30>#ms at ~/testapp/vendor/bundle/jruby/1.8/gems/activesupport-2.3.11/lib/active_support/core_ext/benchmark.rb:16
ActionController::Benchmarking#render_with_benchmark at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/benchmarking.rb:50
ApplicationController#render at ~/testapp/app/controllers/application_controller.rb:117
ActionController::Base#default_render at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/base.rb:1327
ActionController::Base#perform_action at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/base.rb:1333
ActionController::Filters::InstanceMethods#call_filters at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/filters.rb:616
ActionController::Filters::InstanceMethods#perform_action_with_filters at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/filters.rb:609
HomeController#perform_action_with_benchmark at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/benchmarking.rb:67
#<Class:0x669b9a30>#ms at ~/testapp/vendor/bundle/jruby/1.8/gems/activesupport-2.3.11/lib/active_support/core_ext/benchmark.rb:16
Benchmark#realtime at ~/jruby/lib/ruby/1.8/benchmark.rb:307
#<Class:0x669b9a30>#ms at ~/testapp/vendor/bundle/jruby/1.8/gems/activesupport-2.3.11/lib/active_support/core_ext/benchmark.rb:16
ActionController::Benchmarking#perform_action_with_benchmark at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/benchmarking.rb:67
ActionController::Rescue#perform_action_with_rescue at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/rescue.rb:159
ActionController::Flash::InstanceMethods#perform_action_with_flash at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/flash.rb:150
HomeController#perform_action_with_newrelic_trace at ~/testapp/vendor/plugins/newrelic_rpm/lib/new_relic/agent/instrumentation/controller_instrumentation.rb:137
NewRelic::Agent::MethodTracer#trace_method_execution_with_scope at ~/testapp/vendor/plugins/newrelic_rpm/lib/new_relic/agent/method_tracer.rb:61
HomeController#perform_action_with_newrelic_trace at ~/testapp/vendor/plugins/newrelic_rpm/lib/new_relic/agent/instrumentation/controller_instrumentation.rb:121
NewRelic::Agent::MethodTracer#trace_method_execution_no_scope at ~/testapp/vendor/plugins/newrelic_rpm/lib/new_relic/agent/method_tracer.rb:37
NewRelic::Agent::Instrumentation::ControllerInstrumentation#perform_action_with_newrelic_trace at ~/testapp/vendor/plugins/newrelic_rpm/lib/new_relic/agent/instrumentation/controller_instrumentation.rb:116
ActionController::Base#process at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/base.rb:531
ActionController::Filters::InstanceMethods#process_with_filters at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/filters.rb:605
#<Class:0xdf20541>#process at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/base.rb:390
#<Class:0xdf20541>#call at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/base.rb:385
ActionController::Routing::RouteSet#call at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/routing/route_set.rb:437
ActionController::Dispatcher#dispatch at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/dispatcher.rb:86
ActionController::Dispatcher#_call at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/dispatcher.rb:120
ActionController::Dispatcher#build_middleware_stack at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/dispatcher.rb:129
ActionController::StringCoercion#call at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/string_coercion.rb:24
Rack::Head#call at ~/testapp/vendor/bundle/jruby/1.8/gems/rack-1.1.2/lib/rack/head.rb:8
Rack::MethodOverride#call at ~/testapp/vendor/bundle/jruby/1.8/gems/rack-1.1.2/lib/rack/methodoverride.rb:23
ActionController::ParamsParser#call at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/params_parser.rb:14
ActionController::Session::AbstractStore#call at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/session/abstract_store.rb:176
ActiveRecord::QueryCache#call at ~/testapp/vendor/bundle/jruby/1.8/gems/activerecord-2.3.11/lib/active_record/query_cache.rb:28
ActiveRecord::ConnectionAdapters::QueryCache#cache at ~/testapp/vendor/bundle/jruby/1.8/gems/activerecord-2.3.11/lib/active_record/connection_adapters/abstract/query_cache.rb:33
ActiveRecord::QueryCache::ClassMethods#cache at ~/testapp/vendor/bundle/jruby/1.8/gems/activerecord-2.3.11/lib/active_record/query_cache.rb:8
ActiveRecord::QueryCache#call at ~/testapp/vendor/bundle/jruby/1.8/gems/activerecord-2.3.11/lib/active_record/query_cache.rb:27
ActiveRecord::ConnectionAdapters::ConnectionManagement#call at ~/testapp/vendor/bundle/jruby/1.8/gems/activerecord-2.3.11/lib/active_record/connection_adapters/abstract/connection_pool.rb:360
ActionController::Failsafe#call at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/failsafe.rb:25
Rack::Lock#call at ~/testapp/vendor/bundle/jruby/1.8/gems/rack-1.1.2/lib/rack/lock.rb:10
Rack::Lock#call at ~/testapp/vendor/bundle/jruby/1.8/gems/rack-1.1.2/lib/rack/lock.rb:10
ActionController::Dispatcher#call at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/dispatcher.rb:113
#<Class:0x56683a8d>#run at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/reloader.rb:33
ActionController::Dispatcher#call at ~/testapp/vendor/bundle/jruby/1.8/gems/actionpack-2.3.11/lib/action_controller/dispatcher.rb:107
Rails::Rack::Static#call at ~/testapp/vendor/bundle/jruby/1.8/gems/rails-2.3.11/lib/rails/rack/static.rb:30
Rack::URLMap#call at ~/testapp/vendor/bundle/jruby/1.8/gems/rack-1.1.2/lib/rack/urlmap.rb:48
Rack::URLMap#call at ~/testapp/vendor/bundle/jruby/1.8/gems/rack-1.1.2/lib/rack/urlmap.rb:40
Rails::Rack::LogTailer#call at ~/testapp/vendor/bundle/jruby/1.8/gems/rails-2.3.11/lib/rails/rack/log_tailer.rb:16
Rack::ContentLength#call at ~/testapp/vendor/bundle/jruby/1.8/gems/rack-1.1.2/lib/rack/content_length.rb:12
Rack::Chunked#call at ~/testapp/vendor/bundle/jruby/1.8/gems/rack-1.1.2/lib/rack/chunked.rb:14
Rack::Handler::Mongrel#process at ~/testapp/vendor/bundle/jruby/1.8/gems/rack-1.1.2/lib/rack/handler/mongrel.rb:66
Mongrel::HttpServer#process_client at ~/testapp/vendor/bundle/jruby/1.8/gems/mongrel-1.1.5-java/lib/mongrel.rb:158
Mongrel::HttpServer#process_client at ~/testapp/vendor/bundle/jruby/1.8/gems/mongrel-1.1.5-java/lib/mongrel.rb:157
Mongrel::HttpServer#run at ~/testapp/vendor/bundle/jruby/1.8/gems/mongrel-1.1.5-java/lib/mongrel.rb:284

The line numbers seem to be 0 based but that’s how JRuby stores the info internally. I did not really change that. I did some checks that it seems to work as expected.

I did not do the complete github fork & patch routine yet, if I find the need to change this more in the future, I probably will do that. But for now, jruby-1.6.7.2-SigUSR2Patch.tar is my little patch. It contains the changed source and all the class files for org.jruby.RubyThread.java based on JRuby 1.6.7.2. That source file did not change since 1.6.5, so I don’t expect many changes in the next couple of iterations. Applying the changes (for tests) is pretty easy, just stick all the *.class files into the jruby.jar to replace the originals.

But as always, this is a hack, use at your own risk!! But have fun trying it out;-)

Java none

Apache Tomcat – Multi-Service Manager App

June 29, 2012 at 22:54 admin No comments

Ok, more things I don’t like about Apache Tomcat We have a web application which wants to be installed into the root context. But that means that the normal /manager webapp would either show up in it’s `directory-space’ or we would not be able to use it at all We have an Apache web server in front of the Tomcat so we could try to filter certain paths, but that creates extra interdependencies which I would like to avoid in `security critical’ areas.

The normal setup, which works for things like zapcat or Jolokia, is to create a new service in the same server, which listens on a different port and use that. Unfortunately, the default ManagerServlet class can not handle such a setup.

So I took the org.apache.catalina.manager.ManagerServlet as base and extended it so that it understands two additional request parameter service & host. If just one of them is used, the other uses a sensible default. I could not really sub-class the original class, because the changes were all over the place

So with a server.xml specifying two services “Catalina, listening on 8080″ and “Management, listening on 8081″, I can list the deployed applications on both from the “Management” service:

root@box:/tmp# curl 'http://root:root@box:8081/manager/list         
OK - Listed applications for virtual host localhost
/manager:running:0:manager
/j4p:running:0:j4p
/host-manager:running:0:host-manager
root@box:/tmp# curl 'http://root:root@box:8081/manager/list?service=Catalina
OK - Listed applications for virtual host localhost
/:running:0:ROOT
root@box:/tmp# curl 'http://root:root@box:8081/manager/list?service=Management
OK - Listed applications for virtual host localhost
/manager:running:0:manager
/j4p:running:0:j4p
/host-manager:running:0:host-manager

That works the same way for deploy & undeploy as well So now my root context application is happy and all the `management apps’ can be accessed on a different port with different access parameters

The source & class for Tomcat 6.0.35 is here, in this little mw-manager.jar. After reading the source , just drop it into your tomcat lib directory, adapt your manager-application’s web.xml to use the AllManagerServlet instead of the default ManagerServlet, and have fun

The manager application allows to control the tomcat installation. So please be careful!! While I think, the little new servlet works for me, make sure it does the things you expect!! Or use it as example for your own excursions into the Tomcat code

As always, have fun exploring

– Marco

Java none

Apache Tomcat and Logging – The 2nd

June 26, 2012 at 23:13 admin 5 comments

After I got the access logs for our Tomcat‘s to syslogd (see the entry below), I found that the Solaris syslogd can not handle the the RFC-5424 header format:-( So the info in that section was not really useful for us.

That brought up a new problem, I needed to include the process ID in the log records. So I updated my little extension jar to handle an extra %P format specifier for the SyslogAccessLogValve class.

The normal log4j 1.2 also has no real way to include the process ID in the records. So I added a little org.apache.log4j.PidPatternLayout which handles the %P in addition to the normal PatternLayout.

The new jar file with both the sources and all the classes is here;-) mw2.jar

I hope it helps some people. Have fun playing,
– Marco

Update: Here is a new version of the jar: mw3.jar which hopefully removes the StringManager dependency for Tomcat7. I still did not test it with Tomcat7!!

Java none

Apache Tomcat Access Log to Syslogd

June 12, 2012 at 21:12 admin 9 comments

OK, I searched the WWW far and wide, but I could not really find a good solution how to send the access log from Apache Tomcat to a syslogd:-( So I checked the source;-) and there were at least more than the default AccessLogValve.java implementation. There was even one for logging to a DB. But not for syslogd:-(

I don’t normally do much in Java any more, but I still understand it;-) So I started to look into what it would take to create a new Valve to do what I wanted. It did turn out to be not that hard, especially with the help of the Apache log4j SyslogAppender.java and friends;-)

This little jar contains my shiny new SyslogAccessLogValve as source & class. It was compiled against Apache Tomcat 6.0.35, but I don’t think it would be a big problem to port it to version 7. The new class lives in the same package as all the others valves. But keep in mind that it’s NOT an official apache.org source!!

Drop the jar into CATALINA_BASE/lib and configure the new valve as any other AccessLogValve:

    <Valve className="org.apache.catalina.valves.SyslogAccessLogValve"
        hostname="localhost"
        facility="local6"
        pattern="common"
        resolveHosts="false"/>

This will try to send the messages to the syslogd running on localhost and use the LOCAL6 facility. Since this class is a subclass of the orginal AccessLogValve, the pattern can contain the same elements as it’s superclass. I just used “common” for my tests so far, but the formatting is dome by the superclass;-)

And now the final output from syslogd, actually rsyslogd on my Linux system;-)

Jun 12 20:32:16 feather4 http-8080-1[10763] 0:0:0:0:0:0:0:1 - - [12/Jun/2012:20:32:16 -0700] "GET / HTTP/1.1" 200 7777
Jun 12 20:32:16 feather4 http-8080-1[10763] 0:0:0:0:0:0:0:1 - - [12/Jun/2012:20:32:16 -0700] "GET /tomcat.gif HTTP/1.1" 200 1934
Jun 12 20:32:16 feather4 http-8080-2[10763] 0:0:0:0:0:0:0:1 - - [12/Jun/2012:20:32:16 -0700] "GET /asf-logo-wide.gif HTTP/1.1" 200 5866
Jun 12 20:32:16 feather4 http-8080-2[10763] 0:0:0:0:0:0:0:1 - - [12/Jun/2012:20:32:16 -0700] "GET /tomcat-power.gif HTTP/1.1" 200 2324

Those are the four accesses when a browser hits the Apache Tomcat default home page.

I don’t know if that’s helpful for others, but since I saw a couple of places where the same question came up, it might be. In that case, have fun with it;-)

Java none

Page 1 of 8
1
2
3
4
...
8
Next

Marco's Corner

Calendar

Posts

Links

Technology

Nature

Solaris 11 – IPS Packages and Puppet

Science Olympiad – Div C – Robot Arm

Science Olympiad – Div C – Maglev

Gates and Maglev Track

Arduino-based Photogate Timer

Arduino and LEGO NXT is fun

Jruby and Thread Dumps

Apache Tomcat – Multi-Service Manager App

Apache Tomcat and Logging – The 2nd

Apache Tomcat Access Log to Syslogd

Calendar

Posts

Links

Technology

Nature

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this: