Upgrading Oracle Glassfish Server to v. 3.1

Oracle Corporation just released Oracle Glassfish Server v. 3.1: this release includes many interesting features and I started upgrading as soon as I could allocate some time. The official Glassfish Server 3.1 Upgrade Guide can be found here.

In a very recent post, I described how you can upgrade your Glassfish instance to v. 3.0.1 using the tools provided by the application server. With the release of version 3.1 things haven't changed so much so that, if you're in a hurry, you can just follow those instructions.

Nevertheless, I'll take advantage of this news to summarize which are the possible upgrade paths for your Glassfish instances so that you can choose the one that best fits your needs:

• Upgrade tool.
• Update tool.
• pkg tool.
• Software Update Notifier.

Upgrade Tool

The upgrade tool let us perform side-by-side upgrades and it's the only tool that can be used to upgrade a Glassfish instance earlier than Glassfish v. 3.0.1 or Enterprise Server v. 3.0.

The ability to perform side-by-side upgrades is important in the cases in which you do not want to upgrade your existing installation. In the case something goes wrong during the upgrade procedure, the original instance is not modified and the administrator is free to try the upgrade procedure again or restart the old instance until further investigation is done.

The upgrade tool, installed on $GF_INST/glassfish/bin/asupgrade, can be used both in graphical or command line mode. To use this tool, both the source and the target server directories must be accessible from the system in which asupgrade is launched and the user must have read permission on the source and read and write permission on the target directory.

Before launching the tool, the Glassfish domain to be upgraded must be stopped. Also, the official documentation suggests to manually copy libraries in the server /lib directory from the source to the target server. As pointed out in Bobby's comment, older versions of this tool (such as v. 3.0.1) even tried to detect and copy user libraries from $GF_INST/glassfish/lib: the upgrade tool bundled with GlassFish 3.1 will not so that it's important to manually check this directory to manually migrate libraries form one server installation to another.

The options accepted by the tool are the following:

• -c|--console: Launches the tool in command-line mode.
• -V|--version: Dumps the tool's version number.
• -h|--help: Dumps the tool's arguments' help.
• -s|--source: Specifies the source domain directory.
• -t|--target: Specifies the target domain root-directory.
• -f|--passwordfile: Specifies a password file with Glassfish master password.

The quickest way to launch the tool is just:

asupgrade

to launch it in GUI mode, or

asupgrade -c

if you prefer using the CLI mode. In either mode, just supply the required parameters and proceed with the upgrade information.

If you need to automate the upgrade procedures of multiple domains, perhaps you'd rather use the asupgrade tool in a script:

asupgrade -c \

  -s /path/to/source/domain \

  -t /path/to/target/root/domain

At the end of the upgrade procedure, you should check the logs to verify that no error was encountered during the process. If everything is ok, you can now start your new Glassfish v. 3.1 domain.

Update Tool and pkg Tool

The update tool and the pkg tool are, respectively, a GUI and a CLI utility, which are part of the Update Center Tools. Although very similar, they do not provide the same set of features to the user. Nevertheless, as far as it concerns the upgrade procedure, they can be used interchangeably and whether you want to use one or the other is just a matter of taste and opportunity.

The biggest differences with the upgrade tool describe in the previous section is that these tools cannot be used to upgrade instances earlier than 3.0.1 and they do modify the server installation directory during the upgrade procedure. Should the procedure fail, although both the tools and the upgrade procedure are very resilient, the administrator could be left with a non-functional Glassfish installation. The biggest advantage is that it's a very quick upgrade path.

Before upgrading, as usual, Glassfish domains should be stopped.

To update tool can be launched with the:

$GF_INST/glassfish/bin/updatetool

command and its GUI will appear. On the main window, click the Available Updates control, select all items on the list and proceed with the upgrade procedure. The server will be upgraded to version 3.1 and, at the end of the procedure, the existing domains must be upgraded starting them with the --upgrade options:

asadmin start-domain your-domain --upgrade

Despite the name of the command, at the end of the upgrade the domain will be stopped. After upgrading all of your domains, you can start them normally.

Upgrading using the pkg command will be familiar to long time (Open)Solaris users. Glassfish bundles a version of the pkg command to support IPS repositories. To upgrade Glassfish to the latest version is sufficient to launch the following command:

$GF_INST/glassfish/bin/pkg image-update

After the upgrade completes correctly, each Glassfish domain should also be upgraded as explained in the update tool case.

Software Update Notifier

The Software Update Notifier is yet another Update Center tool and, as such, it enables the administrator to perform upgrades under the same conditions explained in the previous section.

The Software Update Notifier, if installed, periodically checks if updates are available and, when they are, it suggests the administrator to install them with a fancy notification balloon. When all domains are stopped, the upgrade process can be started accepting the notification in the balloon.

After the upgrade completes correctly, each Glassfish domain should also be upgraded as explained in the update tool case.

Your Glassfish instance doesn't start. Have you checked the OSGi cache?

I've seen this many times: you just upgraded your NetBeans and the bundled Glassfish and it suddenly stops responding. Or maybe you upgraded a Glassfish instance and your domain fails to start leaving you watching and endless stream of dots and no further activity:

$ asadmin start-domain your-domain
Waiting for DAS to start......................
$ vmstat 1

 kthr      memory            page            disk          faults      cpu
 r b w   swap  free  re  mf pi po fr de sr s0 s1 -- --   in   sy   cs us sy id
 0 0 0 945484 1100116 222 58 947 7 7  0  0  0 12  0  0 1423 1516 1185  3  1 96
 0 0 0 838032 1383588 7  23  0  0  0  0  0  0  4  0  0  601  757 1085  0  1 99
 0 0 0 838032 1383588 4   4  0  0  0  0  0  0  1  0  0  598  782 1071  0  0 100
 0 0 0 838032 1383588 4   4  0  0  0  0  0  0  0  0  0  565 1079 1296  1  0 99

If this is happening to you, chances are that the OSGi cache of your domain is poisoned and prevents your server from working properly. In this case, try to clean it up with ($GR is Glassfish installation root):

$ rm -r $GR/glassfish/domains/your-domain/osgi-cache

and start the server again.

Upgrading Oracle Glassfish Server to v. 3.0.1

Since version 3, Oracle Glassfish Server, formerly known as Sun Glassfish Enterprise Server, bundles an IPS-based "update tool" that makes life easier to system administrators while upgrading their Glassfish instances.

The update tool, a standalone GUI application, has been integrated into the Glassfish Admin Console since version 3.0.1, and can now be easily used to check installed components or available package updates.

The update tool can be found at $GLASSFISH_ROOT/updatetool/bin/updatetool or, starting with version 3.0.1, at $GLASSFISH_ROOT/glassfishv3/bin/updatetool.

Sometimes, though, administrators prefer using a safer upgrade path than patching a running instance, in case they need to quickly rollback to the previous instance in case something goes wrong.

As described in the official Glassfish documentation, administrators can now use the asupgrade tool to upgrade an existing Glassfish domain to a new installation path so that, if the upgraded instance does not work properly, they can fall-back to the old one.

Upgrading Glassfish using the asupgrade tool is straightforward:

• Install the Glassfish instance you're going to upgrade to. If you're using the zip distribution, just unzip it.
• Stop the domain to be upgraded.
• Launch the asupgrade tool to upgrade a Glassfish domain:

asupgrade [-c] \
  --source /old/domain/path
  --target /new/domain/root/path

The optional -c option instructs asupgrade to use the upgrade command line utility instead of the default GUI upgrade utility. This is handy when you need to upgrade instances on remote machine and you cannot, or wish not, use a GUI interface.

Please note that the --source argument points to a domain while the --target argument points to a domain root. You should, then, use something like this:

asupgrade -c \
  --source /opt/ogs-3/glassfish/domains/domain1
  --target /opt/ogs-3.0.1/glassfish/domains

Depending on your domain name, the tool may ask you whether you wish to rename an existing domain on the new server instance. That's typically the case when you're upgrading a domain called domain1 since the default Glassfish installation bundles a domain with such a name.

The tool will start the upgrade process that will be logged in the upgrade.log file for later inspection. If the upgrade process finishes correctly, you can now start the upgraded instance and check that all of your applications work as expected.

So far, I haven't experienced any major problem with the asupgrade tool and it's a very easy upgrade process that takes care of everything. Resources such as data sources, connection pools, JavaMail sessions, were all replicated correctly on the destination server. The tool even took care of copying the database drivers from one instance to another.

The only glitch I experienced is being unable to start the server without cleaning up the OSGi cache first but that's easily solved.

Glassfish 3.0.x Admin Console Not Starting: Is It Behind a Proxy?

Today I was performing yet another Glassfish v. 3.0.1 installation, one of the easiest pieces of software to set up there, on a Solaris 10 system:

• Install the software (unzipping a zip or installing a native package).
  
• Optionally create a new domain.
• Start the domain.

To my surprise, when I tried to connect to the Admin Console, the browser got stuck just after inserting my user credentials. I tried to restart the domain over and over again but I had no luck.

The domain logs (that you can find in $YOUR_DOMAIN/logs/server.log) were telling nothing interesting: the domain was starting up correctly and the admin console application was logging no errors. The last line of the logs always was:

[#|[snip]|admin console: initSessionAttributes()|#]

After a while, it appeared a line such as this:

[#|[snip]|Cannot refresh Catalog: Connection timed out: Connect|#]

This was the clue! Glassfish is able to check for updates automatically from an IPS repository: do you remember that fancy update icon in the upper left corner of the Admin Console that shows up to suggest you the available updates? This server is behind a proxy: maybe I was experiencing a glitch related to this.

A quick search indeed revealed that other users were experiencing the same problem when Glassfish sat behind a proxy server. The workarounds I tested to work are the following:

• Have Glassfish use your proxy.
• Use Glassfish updatetool to disable automatic updates (which is something I would always suggest for a production environment).
• Remove the console-updatecenter-plugin.jar.

The third suggestion comes from a thread published in the java.net Forums. Unfortunately the link is broken now and I could only read it using Google's cache.

Have Glassfish Use Your Proxy

To setup Glassfish to use a proxy you can use the following Java system properties:

• http.proxyHost
• http.proxyPort
• http.proxyUser
• http.proxyPassword

You can either use the asadmin program or the admin console to do that. Using asadmin is as simple as:

$ asadmin
asadmin> create-jvm-options "-Dhttp.proxyXXX=value"
asadmin> create-jvm-options ...

If you prefer using the Admin Console, just navigate to Enterprise Server/System Properties and use the web interface to add the values.

The values you set will be reflected in the domain.xml domain configuration file:

[...snip...]
<java-config ...>
[...snip...]
  <jvm-options>-Dhttp.XXX=value</jvm-options>
</java-config>
[...snip...]

Use Glassfish updatetool To Disable Automatic Updates

You can use the updatetool program to update Glassfish and configure the autoupdate feature:

$ $GLASSFISH_INST/bin/updatetool

The first time you launch updatetool, it will ask you to install this feature. Since you're behind a proxy, you need to setup some environment variables. If you're using an HTTP proxy, you can just set the http_proxy variable:

$ export http_proxy=http://user:password@proxy.host:port

When updatetool finished installing the required packages, you can start it again and the updatetool windows will show up. In the Preferences window you can tune the update behaviour or disable it at all.

Remove the console-updatecenter-plugin.jar

The last thing you can do is removing the guilty plugin. The Glassfish plugins are deployed in the $GLASSFISH_INST/glassfish/modules directory. You can just move the console-updatecenter-plugin.jar to console-updatecenter-plugin.jar.old and Glassfish won't use it.

"No Network Access" for the Admin Console

As pointed out in a comment, Glassfish can be configured to have "no network access" by setting the

com.sun.enterprise.tools.admingui.NO_NETWORK

property to true. As usual, this can be done adding the

-Dcom.sun.enterprise.tools.admingui.NO_NETWORK=true

to the JVM parameters list or using the asadmin tool:

$ asadmin create-jvm-options \
  -Dcom.sun.enterprise.tools.admingui.NO_NETWORK=true

Conclusion

This is a bothering bug with an easy workaround. Looking forward to Glassfish v. 3.1.