Notes to self

This is another one of those things that are impossible to remember.

For the record, the OS I'm using here is Ubuntu Linux, so the commands and paths will be different if you happen to be on Windows (keep googling, I know that a page is out there for you).

Step One is obviously to go grab the certificate from the web server. We'll use openssl, the tool, to dump the security information we need:

$ openssl s_client -connect that-server.com:443 > cert

Now open the newly created "cert" file in a text editor and remove everything but the blob that looks like this, but with different data in it:

-----BEGIN CERTIFICATE-----
MIIDIjCCAougAwIBAgIQbt4NlJn3RTPdEpf8Qqk74TANBgkqhkiG9w0BAQUFA8Bn
[... etc, etc, etc ...]
byB0lP6qDtnVOyEQp2Vx+QIJza6IQ4XIglhwMO4V8z12Hi5Fprw=
-----END CERTIFICATE-----

This cleaned up file is now the certificate that you want to import.

Step Two is to find the key-store that your Java installation uses (google will tell you that this file is called "cacerts"):

$ locate cacerts
/etc/java-1.5.0-sun/security/cacerts
/etc/java-6-openjdk/security/cacerts
/etc/java-6-sun/security/cacerts

So I obviously have more than one keystore because I have more that one version of Java installed. I use the "java-6-sun" version by default. The command

$ update-alternatives --display java

will tell you which version is your default.

Step Three is to finally install this certificate into the keystore. This command ends up doing it for me:

$ sudo keytool -importcert -keystore /etc/java-6-sun/security/cacerts -alias that-server -file cert

You will be asked for a password to access the keystore. By default, this keystore password is changeit - another thing easily forgotten.

And now your JVM should be able to create secure connections without those pesky SSL exceptions.

Someone posted this gem on the concurrency-interest mailing list: http://gceclub.sun.com.cn/java_one_online/2005/TS-3281/ts-3281.pdf

It is a PDF presentation by Hans Boehm (the GC guy) from JavaOne 2005. The presentation goes through a very simple example, pointing out flaws left and right - the number of pitfalls and the effort required to avoid them was surprisingly high.

The guys behind the Stripes web framework shoved a version 1.5 out the door today (well, yesterday - it's past midnight here).

Here's the link: http://www.stripesframework.org/

So, this is where one usually talk about why this is cool, but truth is; I only just started creating anything with Stripes today/yesterday, and that is even with the previous version 1.4.3. Still, I'de like to get my initial judgment across, because I think that this framework is undervalued and dosn't get the talking-about it deserves.

The thing about Stripes is how it combines simplicity and flexibility. You can configure and customize a ton of things - but you don't have to, because almost everything has a reasonable default. The only required configuration is that you need to tell it where you controllers, or ActionBeans, as they're called, live. From then on, Stripes is pretty much just a request dispatch mechanism and a tag library.

This brings me to my second nice observation: Stripes is all about the presentation layer, and Stripes is only about the presentation layer. It dosn't care about how you talk to a database or if you even have one. It is also isn't particulary picky about how you organise you controllers - it uses annotations to figure out who does what and when. These annotations can also be used for input validation on forms, it seems, but I have yet to try out that feature.

The requests are dispatched to your action beans, and by default the URL is based on the fully qualified class-name of that bean. However, this can be overwriten with a UrlBinding annotation on the class. The methods on the class may then be annotated as handlers of some sort. You usually annotate a DefaultHandler, and then annotate the other methods to match the names of submit buttons on forms, or stripes:link tags. To communicate state from request parameters to your ActionBean, and from your ActionBean to you JSP or other view file, you use getter and setter methods on you ActionBean - this makes them non-thread safe, but that dosn't matter because they're created on a per request basis.

At first you might run into problems getting Stripes to find the right ActionBeans for the right URLs - especially if you use the UrlBinding annotation, but fret not. Just leave your JSP files in their default place, make sure you have configured the proper packages for the ActionResolver.Packages init-param in your web.xml, and finally check that the URL that you specified in your annotation will actually be hit by the Stripes DispatcherServlet and then you should be on your way.

With this, I'm starting to like Stripes; the simplicity and reasonable default configuration let me see some fireworks in short order, and yet it allows me to configure and tweak and customize the parts I need. Simple on the surface, flexible underneath - that should be their slogan. They put it like this on their home page:

Stripes aims to provide an experience similar to owning Apple hardware, Sony TVs and luxury German cars (without the price premium); things just work, they feel right and every now and again you get that "oh, it does that too? awesome!" feeling.

And I'm thinking they're getting close to home on that one. So even though I've only been working with this framework for a few hours, I have a feeling I'll be sticking to it.

This appears to be the third installment in the saga of my PermGen problems.

For good measure, here are the previous posts:

• PermGen strikes back
  
• Good riddance, PermGen OutOfMemoryError
  

The newsflash this time is that I stumbled upon some guy Thompson's blog post about this very problem - by itself nothing new; concluding by recommending a JVM with infinite perm-space (BEA or IBM).

The interesting part, however, is stoney's comment where he links to two posts on a blog by Frank Kieviet.

The first of Franks posts is concerned with what the PermGen error is really all about, namely how these leaks in the reloading classloaders in our application servers happen. His second post is a bit more involved and shows how you can use new profiling tools in Java 6 to track down classloader leaks.

So the good news is that we now have the knowledge and tools required to kill this bug.
The bad news is that we've also learned that walking the walk will probably be a lot tougher than talking the talk.

We have to scrutinize our dependency tree, and think really, really, REALLY hard before we put a jar in our servers indorsed directory.

In fact, I think I'll try an experiment tomorrow (today, actually, after I've slept) where I strip down a Tomcat instance (JBoss is probably too intertwined with itself to really strip down) and create some web applications who's only runtime dependencies are the servlet-api and the JRE!

I should really think that classloader leaks are really hard to create with something as simple as the servlet-api, and if you use your JRE with care, then it should damn well be possible to avoid classloader leaks with that dependency as well.

Man, I'm really excited about the idea that there's actually hope for us lay-coders to actually rid ourselves of this nasty ghost-bug-thing/issue.

I'll be sure to report back when I've done some experiments with this new knowledge!

Hi, you're probably here because you are hot-deploying some Java web applications and you have problems with some PermGen: OutOfMemoryErrors. Then you somehow found a link to an article that claimed to solve this very problem.

Well, that article was wrong. So I put up a big red note, but people still link to it.

So, in an attempt to set things straight again, I present a short list of facts that I have collected since I wrote the original article:

• Bugs in the Sun garbage collector is one source of this problem.
  
• There are (were? can't be arsed to check) more than one bug, hence confusion.
  
• Your app failing to undeploy completely will leak all classes in your app, every time you redeploy - this is another source of the problem.
  
• None of these sources can be fixed with GC flags.
  
• The JRocket JVM does NOT fix this problem - it simply lets the permgen grow unbounded, until something bad happens, like swap-thrashing or the OS kills it.
  

The only solution I have found that is feasible, reliable and portable is this: reboot your servers!

P.S. Don't bother giving me links to other articles on this issue. Even if you happen to find something I haven't already read, chances are that it is simply rehashing stuff I already know.

Thanks!


Bellow this line is the original article with all it's faulty conclusions.
-----------------------------------------------------------------------------------------------------------------------
I did it!

Yes, lads & lasses! If you've been annoyed with having JBoss or Tomcat die with an OutOfMemoryError every fifth time you redeploy your beloved brainchild of a web application, then this is your lucky day! Because I found a fix! It's true! Yay! ....!!!!!11one (can you tell this has been a pain to me?)

Boring but Serious Theory and Hypothesis part:
The "PermGen" error happens, when the Java virtual machine runs out of memory in the permanent generation. Recall that Java has a generational garbage collector, with four generations: eden, young, old and permanent.

In the eden generation, objects are very short lived and garbage collection is swift and often.

The young generation consists of objects that survived the eden generation (or was pushed down to young because the eden generation was full at the time of allocation), garbage collection in the young generation is less frequent but still happens at quite regular intervals (provided that your application actually does something and allocates objects every now and then).

The old generation, well, you figured it. It contains objects that survived the young generation, or have been pushed down, and garbage collection is even less infrequent but can still happen.

And finally, the permanent generation. This is for objects that the virtual machine has decided to endorse with eternal life - which is precicely the core of the problem. Objects in the permanent generation are never garbage collected; that is, under normal circumstances when the jvm is started with normal command line parameters.

So what happens when you redeploy your web application is, that your WAR file is unpacked and its class files loaded into the jvm. And here's the thing: almost always ends up in the permanent generation... Because, seriously, who wants to garbage collect their classes?!? Well, apparently application servers do, and here's how we make that happen;

PermGen, The Fix:
The standard garbage collector can't collect in the permanent generation, but the concurrent collector can. So the first thing we need to do is to make sure that the jvm uses the concurrent garbage collector. This is done by putting this:

> -XX:+UseConcMarkSweepGC

In java's command line arguments. But this is not enough. We must also specifically tell it to collect in the permanent generation, and this is done with this command line argument:

> -XX:+CMSPermGenSweepingEnabled

Good, now the concurrent collector will take the permantent generation under its wings. But wait! Classes are special, and the jvm is reluctant to let go of them, so we must also explicitly allow classes to be unloaded:

> -XX:+CMSClassUnloadingEnabled

Now we're certain that the permanent generation will be properly cleaned. But this raises another issue: what if the jvm unloads classes that might still be needed? I imagine it can be hard for a collector to tell whether or not a class might still be needed with the amount of reflection that goes on in an application server. Therefor, we might want to tweak the amount of memory allocated for the permanent generation, and this is done with this command line parameter:

> -XX:MaxPermSize=128m

Which will set the maximum size of our permanent generation to 128 megabytes - tweak it to fit your needs.

With these parameters properly applied to the jvm that runs your application server, your chances of running into a PermGen OutOfMemoryError will be considerably lessened.

Take care!