situ: smf

Showing posts with label smf. Show all posts

Sunday, 6 June 2010

Registering JIRA as a Solaris SMF-Managed Service

My company and many of its clients use Atlassian JIRA as a bug or project tracking solution. JIRA is distributed in two bundles:

As a Java EE application to deploy on a compliant application server.
As a standalone distribution bundled with Apache Tomcat.

I usually recommend, unless you already have a Java EE application server to leverage or some other requirement to take into account, to install JIRA standalone distribution. It's very easy to install, configure and deploy. I would also recommend you to deploy JIRA on Solaris.

Solaris has differential advantages when compared with other operating system. Just to name a few (not to repeat myself over and over):

ZFS and ZFS snapshots and clones.
Zones.
DTrace.
RBAC, Projects and Resource Caps.
SMF.

A Typical Deployment Scenario

My typical deployment scenario is the following:

I deploy JIRA standalone (once for each JIRA version) on a dedicated ZFS filesystem.
I snapshot and clone the corresponding filesystem whenever I need a new JIRA instance.
Every JIRA instance is configured and executed on a dedicated Solaris 10 Sparse Zone.
I define a resource cap for each zone that runs JIRA.
I configure JIRA as a Solaris SMF-Managed Service.

Most of the previous tasks have been described earlier in this blog. An overview of a JIRA installation on Solaris has been given on this post.

The purpose of this entry is describing how to configure JIRA as a Solaris SMF-Managed Service.

Configuring JIRA with Solaris SMF

Solaris 10 Service Management Facility (SMF) often praised advantages include the ability for Solaris to check the health and dependencies of your service and take a corrective action in case something goes wrong. Registering a service with Solaris SMF isn't strictly necessary but I encourage you to do so. Although JIRA is pretty easy to start and stop and has got very view dependencies (or none...) with other services, preparing a service manifest will enable you to install and deploy JIRA on Solaris in a question of minutes.

Dependencies

As stated above, JIRA's dependencies really depend on your deployment. As a minimum, JIRA global dependencies will include the two following (obvious) services:

svc:/milestone/network:default
svc:/system/filesystem/local:default

Because of its own nature, there's probably no reason to have JIRA instances running when network services are not available. The dependency with the local filesystem service is obvious, too.

Any other dependency will probably be a service instance dependency. Since JIRA, in the simplest deployment scenario, will just depend on a database, we will assume that our JIRA instance will depend on a specific Solaris PostgreSQL service:

svc:/application/database/postgresql:version_82

Our first manifest fragment will be the following:

<?xml version='1.0'?>

<!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>

<service_bundle type='manifest' name='export'>

<service_fmri value='svc:/milestone/network:default'/>

</dependency>

<service_fmri value='svc:/system/filesystem/local:default'/>

</dependency>

<service_fmri value='svc:/application/database/postgresql:version_82'/>

</dependency>

</instance>

<common_name>

<loctext xml:lang='C'>Atlassian JIRA</loctext>

</common_name>

</template>

</service>

</service_bundle>

Beware that in this example we're using PostgreSQL services as defined in Solaris 10 05/10. Please check your operating system to define correct dependencies. A missing dependency will make Solaris put the service into the offline state until the dependency can be resolved.

Execution Methods

The next thing to do is define JIRA execution methods. We could directly use JIRA startup.sh and shutdown.sh scripts but we want to avoid coupling the SMF service definition with JIRA implementation details. Rather, we will take the common approach of writing a script to declare execution methods. This approach, moreover, will let us parametrize distinct JIRA instances in the SMF manifest and thus manage multiple JIRA instances with just a SMF service manifest and one execution methods script.

A draft script is the following:

#!/sbin/sh

#ident "@(#)JIRA 4.1.1 06/05/10 SMI"

. /lib/svc/share/smf_include.sh

# SMF_FMRI is the name of the target service.

# This allows multiple instances

# to use the same script.

getproparg() {

val=`svcprop -p $1 $SMF_FMRI`

[ -n "$val" ] && echo $val

}

# ATLBIN is the JIRA installation directory

# of each JIRA instance.

ATLBIN=`getproparg atlassian/bin`

# JIRASTARTUP is JIRA startup script

JIRASTARTUP=$ATLBIN/bin/startup.sh

# JIRASHUTDOWN is JIRA shutdown script

JIRASHUTDOWN=$ATLBIN/bin/shutdown.sh

# Check if the SMF framework is correctly initialized.

if [ -z $SMF_FMRI ]; then

echo "Error: SMF framework variables are not initialized."

exit $SMF_EXIT_ERR

# check if JIRA scripts are available

if [ ! -x $JIRASTARTUP ]; then

echo "Error: JIRA startup script cannot be found."

exit $SMF_EXIT_ERR

if [ ! -x $JIRASHUTDOWN ]; then

echo "Error: JIRA shutdown script cannot be found."

exit $SMF_EXIT_ERR

case "$1" in

'start')

$JIRASTARTUP

;;

'stop')

$JIRASHUTDOWN

;;

echo "Usage: $0 {start|stop}"

exit 1

;;

esac

exit $SMF_EXIT_OK

The previous scripts just define the start and stop service methods. As far as I know there's no easy JIRA refresh method so we will skip it in our scripts.

Service Instance Parameters

As shortly explained in the previous section, SMF services provide the ability of being parametrized by virtue of an easy syntax to declare parameters in the service manifest and use them in the service method script. In the previous example we're using a property, called atlassian/bin, to declare the installation directory of each of ours JIRA instances. Let's declare it in our service manifest for a fictional JIRA 4.1.1 instance:

<?xml version='1.0'?>

<!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>

<service_bundle type='manifest' name='export'>

<service_fmri value='svc:/milestone/network:default'/>

</dependency>

<service_fmri value='svc:/system/filesystem/local:default'/>

</dependency>

<exec_method name='start' type='method' exec='/root/bin/svc/method/atlassian/jira start' timeout_seconds='60'/>

<exec_method name='stop' type='method' exec='/root/bin/svc/method/atlassian/jira stop' timeout_seconds='60'/>

<service_fmri value='svc:/application/database/postgresql:version_82'/>

</dependency>

<property_group name='atlassian' type='application'>

<propval name='bin' type='astring'

value='/opt/atlassian/atlassian-jira-enterprise-4.1.1-standalone'/>

</property_group>

</instance>

<common_name>

<loctext xml:lang='C'>Atlassian JIRA</loctext>

</common_name>

</template>

</service>

</service_bundle>

Environment Variables

JIRA needs at least the JAVA_HOME environment variable to be set for startup and shutdown scripts to work correctly. Thus we need to modify our manifest so that SMF will set it before launching the execution methods:

<?xml version='1.0'?>

<!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>

<service_bundle type='manifest' name='export'>

<service_fmri value='svc:/milestone/network:default'/>

</dependency>

<service_fmri value='svc:/system/filesystem/local:default'/>

</dependency>

<exec_method name='start' type='method' exec='/root/bin/svc/method/atlassian/jira start' timeout_seconds='60'>

<method_context>

<method_environment>

<envvar name='JAVA_HOME' value='/opt/sun/jdk/latest'/>

</method_environment>

</method_context>

</exec_method>

<exec_method name='stop' type='method' exec='/root/bin/svc/method/atlassian/jira stop' timeout_seconds='60'>

<method_context>

<method_environment>

<envvar name='JAVA_HOME' value='/opt/sun/jdk/latest'/>

</method_environment>

</method_context>

</exec_method>

<service_fmri value='svc:/application/database/postgresql:version_82'/>

</dependency>

<property_group name='atlassian' type='application'>

</property_group>

</instance>

<common_name>

<loctext xml:lang='C'>Atlassian JIRA</loctext>

</common_name>

</template>

</service>

</service_bundle>

Credentials, Projects and Resources

As explained in another post, SMF service manifest let the administrator specify advanced parameters for a service instance such as:

The credentials under which the service instance will be executed.
The Solaris project in which the service instance will be executed.
The resource pool assigned to a service instance.

In this case, being a multi instance service manifest, we will let the administrator declare such parameters in every service instance. This way, the maximum degree of service parametrization is achieved:

<?xml version='1.0'?>

<!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>

<service_bundle type='manifest' name='export'>

<service_fmri value='svc:/milestone/network:default'/>

</dependency>

<service_fmri value='svc:/system/filesystem/local:default'/>

</dependency>

<exec_method name='start' type='method' exec='/root/bin/svc/method/atlassian/jira start' timeout_seconds='60'>

<method_context>

<method_environment>

</method_environment>

</method_context>

</exec_method>

<exec_method name='stop' type='method' exec='/root/bin/svc/method/atlassian/jira stop' timeout_seconds='60'>

<method_context>

<method_environment>

</method_environment>

</method_context>

</exec_method>

<service_fmri value='svc:/application/database/postgresql:version_82'/>

</dependency>

<property_group name='method_context' type='framework'>

<propval name='project' type='astring' value=':default'/>

<propval name='resource_pool' type='astring' value=':default'/>

<propval name='working_directory' type='astring' value=':default'/>

</property_group>

<property_group name='atlassian' type='application'>

</property_group>

</instance>

<common_name>

<loctext xml:lang='C'>Atlassian JIRA</loctext>

</common_name>

</template>

</service>

</service_bundle>

Please note the slight different syntax for the method_context attributes with respect to the example in my previous post. This gives you an idea of SMF manifest flexibility. In that example, distinct execution methods were specified for every service instance. In this case, we specified the same service methods for every JIRA instance. The difference is that parametrization occurs in the service instance property groups. The same script, then, can be properly parametrized and reused for any of the instances you'll control with SMF.

Privileges Required to Open a Privileged Network Port

If you're running JIRA as a non-root user (that is considered a best practice but perhaps is questionable if you use a dedicated Solaris 10 Zone) you should be aware that a specific privilege is required for Tomcat to open a privileged network port (< 1024) such as port 80 (the default HTTP port.) If you plan to run JIRA as a non root user, then assign the net_privaddr privilege to the JIRA user as shown in the following manifest fragment:

[...snip...]
<exec_method name='start' type='method' exec='/root/bin/svc/method/atlassian/jira start' timeout_seconds='60'>
  <method_context>
   <method_credential
   user='jira'
   group='jira'
   privileges='basic,net_privaddr' />
[...snip...]

Conclusion

The following manifest and script can be used as a starting point for a working JIRA SMF service manifest. It's up to you, system administrator, to apply the modifications you need to fit it into your execution environment.

Service Manifest

<?xml version='1.0'?>

<!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>

<service_bundle type='manifest' name='export'>

<service_fmri value='svc:/milestone/network:default'/>

</dependency>

<service_fmri value='svc:/system/filesystem/local:default'/>

</dependency>

<exec_method name='start' type='method' exec='/root/bin/svc/method/atlassian/jira start' timeout_seconds='60'>

<method_context>

<method_environment>

</method_environment>

</method_context>

</exec_method>

<exec_method name='stop' type='method' exec='/root/bin/svc/method/atlassian/jira stop' timeout_seconds='60'>

<method_context>

<method_environment>

</method_environment>

</method_context>

</exec_method>

<service_fmri value='svc:/application/database/postgresql:version_82'/>

</dependency>

<property_group name='method_context' type='framework'>

</property_group>

<property_group name='atlassian' type='application'>

</property_group>

</instance>

<common_name>

<loctext xml:lang='C'>Atlassian JIRA</loctext>

</common_name>

</template>

</service>

</service_bundle>

Execution Methods Script

#!/sbin/sh

#ident "@(#)JIRA 4.1.1 06/05/10 SMI"

. /lib/svc/share/smf_include.sh

# SMF_FMRI is the name of the target service.

# This allows multiple instances

# to use the same script.

getproparg() {

val=`svcprop -p $1 $SMF_FMRI`

[ -n "$val" ] && echo $val

}

# ATLBIN is the JIRA installation directory

# of each JIRA instance.

ATLBIN=`getproparg atlassian/bin`

# JIRASTARTUP is JIRA startup script

JIRASTARTUP=$ATLBIN/bin/startup.sh

# JIRASHUTDOWN is JIRA shutdown script

JIRASHUTDOWN=$ATLBIN/bin/shutdown.sh

# Check if the SMF framework is correctly initialized.

if [ -z $SMF_FMRI ]; then

echo "Error: SMF framework variables are not initialized."

exit $SMF_EXIT_ERR

# check if JIRA scripts are available

if [ ! -x $JIRASTARTUP ]; then

echo "Error: JIRA startup script cannot be found."

exit $SMF_EXIT_ERR

if [ ! -x $JIRASHUTDOWN ]; then

echo "Error: JIRA shutdown script cannot be found."

exit $SMF_EXIT_ERR

case "$1" in

'start')

$JIRASTARTUP

;;

'stop')

$JIRASHUTDOWN

;;

echo "Usage: $0 {start|stop}"

exit 1

;;

esac

exit $SMF_EXIT_OK

Credentials and Projects for a Solaris 10 SMF-Managed Service

Introduction

Some posts ago we've learnt about Solaris 10 basic Projects and Resource Caps administration. In that article we examined Solaris 10 Projects facility and how it enables the administrator to group and organize running processes into tasks which are running in a project context. Since projects may define resource caps, processes running inside a project will be subject to such caps.

In the previous post we discovered how easy it is for an administrator to define Solaris 10 projects and assign processes to them. We also said that there was a little gotcha: how would an administrator define the credential (and/or the Solaris 10 project) under which an SMF-managed service will run? In this articles we'll explore the basic Solaris 10 SMF Service Manifest DTD to discover how to define user credentials, privileges and projects for a specific SMF service instance.

The SMF Service Manifest DTD

Solaris 10 SMF uses manifests to describe the characteristics of a managed service. The manifest DTD (Solaris 10, version 1) can be found on:

/usr/share/lib/xml/dtd/service_bundle.dtd.1

Although in some cases there's no need for an administrator to define a manifest for an SMF service (as in the case of inetd-manage services because Solaris 10 will build a manifest for you) it's not bad to quickly glance at the service DTD, at least to discover what can be done.

Method Context

The method context, defined by:

<!ELEMENT method_context

( (method_profile | method_credential)?,

method_environment? ) >

<!ATTLIST method_context

working_directory CDATA ":default"

project CDATA ":default"

resource_pool CDATA ":default" >

lets the administrator define credentials and resource managements attributes for an execution method. The method_context element defines the following three attributes:

Name	Description
working_directory	The working directory to launch the execution method from. The :default token can be used to indicate the user specified for the method (by means of the method_profile or the method_credential element.)
project	The project under which to run the current execution method. The project can be specified in either the numeric or the text form. The :default token can be used to indicate the use of the default project for the user specified (see working_directory description.)
resource_pool	The resource pool name to launch the method on. The :default token can be used to indicate the default pool for the specified project (see project description above.)

Method Credentials

The method credentials, as outlined in the previous section, can be specified with the optional method_credential subelement of the method_context element. The method_credential element is defined as follows:

<!ELEMENT method_credential EMPTY>

<!ATTLIST method_credential

user CDATA #REQUIRED

group CDATA ":default"

supp_groups CDATA ":default"

privileges CDATA ":default"

limit_privileges CDATA ":default" >

The attributes of the method_credential element are defined as follows:

Name	Description
user	The user id for the current execution method. The user id can be specified in either the numeric or the text form.
group	The group id for the current execution method. The group id can be specified in either the numeric or the text form. The :default token can de used to specify the default group for the specified user (see user attribute above.)
supp_groups	Optional supplemental groups to associate with the current execution method. A list of groups ids can be specified using a space as a separator. If absent or when the :default token is specified, initgroups(3C) will be specified.
privileges	An optional privilege set.
limit_privileges	An optional limit privilege set.

If you're wondering about privileges, please check out the official Solaris RBAC documentation.

An Example Manifest

Many Solaris 10 bundled SMF services use RBAC, hence their manifest make extensive use of such elements. You can, for example, check the default PostgreSQL service manifest:

$ svccfg export postgresql

[...snip...]

<method_context project=':default' resource_pool=':default' working_direct

ory=':default'>

<method_credential group='postgres' limit_privileges=':default' privileg

es=':default' supp_groups=':default' user='postgres'/>

</method_context>

[...snip...]

In the service manifest fragment above you can see how the postgresql service definition leverages RBAC and projects to define credential, privileges and resource pools and caps for the default PostgreSQL service.

Flexibility and Simplicity

This mechanisms is indeed simple and flexible: since Solaris 10 RBAC and Solaris 10 Projects use loosely coupled layers to define privileges and resource caps, it's easy to establish a relation between and user and a project (or a privilege set, or whatever), and then Solaris will make the rest for you. As you can see in the previous manifest fragment, there's most of the tokens are :default. The PostgreSQL processes will run under the specified user credentials (postgres) and all of the other parameters (resource pool, project, working directory, privileges, etc.) will be discovered at runtime by Solaris.

Should you need to cap PostgreSQL resources, just define a suitable project and establish it as the default project for the postgres user: you won't (almost) ever need to change the service manifest.

Inetd-Managed Services

As explained in another post, configuring an inetd-managed service on Solaris 10 is almost a one-liner. There's was a gotcha, though. Although the user specified with the legacy inetd syntax was assigned to a default project with a resource cap, I noticed that Solaris 10 wasn't honoring it. Let's try and check the manifest that Solaris 10 generated for us:

$ svccfg export svc:/network/svn/tcp

[...snip...]

<exec_method name='inetd_start' type='method' exec='/opt/csw/bin/svnserve -i -r /home/svnuser/svnrepos' timeout_seconds='0'>

<method_context>

<method_credential user='svnuser' group='svngroup'/>

</method_context>

</exec_method>

[...snip...]

There's no default project specified indeed! Let's notice that a missing project attribute does not tell Solaris 10 to use the default project for the specified user. To solve this, just dump the service definition to a file, modify it and reimport it:

$ svccfg export svc:/network/svn/tcp > svn.xml

$ [add project=':default' attribute]

# svccfg -v import svn.xml

Just check that the service manifest has been correctly import and restart the service.

Conclusion

Solaris 10 offers great and sophisticated tools to ease the life of the system administrator and enable the deployment of flexible and controlled execution environments. But beware: sophisticated doest not imply complicated in the Solaris 10 universe. On the contrary: as much as I discover and use Solaris 10 as much I admire and appreciate its clean and usable administration tools. As a minimum, I never have to modify a descriptor by hand. There always is a command line interface tool to protect me from errors and to ease my scripting experience. Just look for the corresponding *adm or *cfg command to do your job. Solaris 10 documentation and the community around OpenSolaris, moreover, are great places where to go and find the help you need.

For an alternate example that emphasizes the flexibility of the SMF service manifests, have a look at this blog post.

Enjoy.

Saturday, 17 January 2009

Setting up Subversion on Solaris 10 as an inetd service

Some posts ago I described how I set up Subversion on Solaris 10 in daemon mode and managed via SMF. It exists another way to invoke svnserve: it can be run via inetd. svnserve, when invoked with the -i|--inetd option will use stdin and stdout to communicate to a Subversion client: this is the expected behavior of an inetd-based service.

These instructions are valid for an UNIX system with an inetd implementation. Registration of this configuration as an SMF service is Solaris 10 specific.

Configuring /etc/services

The first thing to do is checking if /etc/services defines the Subversion service. IANA has reserved port 3690 for the Subversion service, so that /etc/services should contain:

svn 3690/tcp
svn 3690/udp

Please note that svn is just a service identifier which will be used during inetd configuration.

Setting up the inetd daemon

The standard UNIX inetd daemon accepts configuration on a file usually called /etc/inet/inetd.conf or /etc/inetd.conf for compatibility with BSD. In the case of Solaris 10, that file is not used anymore and inetd-based services are managed by SMF, too. There indeed is a /etc/inetd.conf file if you check your file system: it's used as a configuration file to be fed as input of the inetconv utility during the configuration of a service. inetconv can be invoked with the -i option to use an alternate file, which is what I usually do. Then, touch a new empty file or make a copy of /etc/inetd.conf which will be used later to configure the SMF service.

inetd.conf syntax

The expected syntax for /etc/inetd.conf file *or the file you'll feed inetconv with) is the following (see the man page for more details):

service-name endpoint-type protocol wait-status uid server-program server-arguments

service-name: the name specified in the /etc/services file.
endpoint-type: the type of the endpoint, which can be stream, dgram, raw, seqpacket, tli.
protocol: a valid protocol entry in the /etc/inet/protocols file. Additionally, tcp6 and udp6 may also be used. In the case of RPC services, it's a string of the type rpc/* or rpc/..., where ... is a combination of nettypes and/or netids.
wait-status: this field may assume the values wait or nowait. Basically, it instructs inetd whether to pass over the listening socket to the service process and waiting for it to finish before resuming listening.
uid: the user id under which the service will run
server-program: the pathname of the server process executable or an internal name if inetd itself will provide this service.
server-arguments: the entire command line arguments for the service process, including the service process name itself (argument 0)

Subversion entry

The syntax to use for svnserve is the following:

svn stream tcp nowait svnuser /opt/csw/bin/svnserve svnserve -i -r /home/svnuser/svnrepos

I previously set up a standard user, svnuser, under which the Subversion service will run.
In this case I'm using the Subversion package installed from Blastwave which installs into /opt/csw/bin.
The repositories served by this instance are located into directory /home/svnuser/svnrepos.

Setting up the Subversion user

Creating the user

Subversion server process will run as a regular user called svnuser. This user is a basic user created with:

# useradd -d /export/home/svnuser -m -k /etc/skel -s /bin/bash svnserve
# passwd svnuser
[...]

Assuming that home automounting under /home is configured, I then modify the user with:

# usermod -d /home/svnuser svnuser

In my case, this user was created in the LDAP directory which I manage with Sun Java System Directory Server Enterprise Edition but basically the result is equivalent to running the above mentioned commands.

Creating a default user project (Solaris 10 specific)

I also created an user project for this user, called user.svnuser, so that processes launched after svnuser login will be associated with its project. The project I created just caps the CPU associated with its processes:

# projadd -K "project.cpu-cap=(privileged,50,deny)" user.svnserve

Setting up the SMF service (Solaris 10 specific)

After setting up inetd, under Solaris 10 we have to configure the described service with inetconv. inetconv it's an utility which reads files containing inetd.conf-like records and maps them to auto-generated SMF services. If the file we've generated it's called svn.conf, we have to run the following command:

# inetconv -i svn.conf

and a new SMF service description will be generated, imported and enabled.

Conclusion

Setting up inetd based services it's even easier with Solaris 10. Moreover, the powerful SMF framework will allow the sysadmin to manage and observe every service and enjoy the advantages of SMF. The only problem not (yet) addressed by this blog post is that a nowait inetd service is fork()-ed and exec()-ed by inetd and as Solaris 10 documentation states, during the fork() system call the process and task controls are inherited. This has the effect that even if svnserve is fork()-ed and then run under the svnuser account, the project.cpu-cap entry is having no effect. To discover how to configure an execution method of an SMF-managed service to run into a Solaris 10 project, read this post.

Next steps
You can now:
Read this post about how you can set up Solaris projects or zones to control resource usage.
Read this post about setting up SSH access to Subversion repositories.
Read this post about how to set up Subversion post-commit hooks.
Read this post about filtering Subversion commit using a post-commit hook.

Wednesday, 20 August 2008

Setting up Subversion on Solaris 10

In this series of posts I'm going to describe how I set up Subversion on a Solaris 10 machine. The instructions here are valid for subsequent releases such as Solaris Express Community Edition (Nevada) and OpenSolaris (Indiana), the only difference being the fact that Subversion is bundled with the Operating System.

I decided to host some Subversion (1.4.5) repositories on a Solaris 10 (update 5) server so I downloaded Subversion packages from Blastwave and installation was done in less than 5 minutes with a simple:

# pkg-get -i subversion

Once Blastwave's installer was done, the first thing I wanted to do was registering Subversion's daemon (svnserve) in the Service Management Facility. Writing an SMF manifest is not so hard but fortunately I found a manifest and a management script in OpenSolaris website and the job was done! Great work, guys.

Next steps

You can now:

Read this post about how to set up Subversion as an inted service.
Read this post about how you can set up Solaris projects or zones to control resource usage.
Read this post about setting up SSH access to Subversion repositories.
Read this post about how to set up Subversion post-commit hooks.
Read this post about filtering Subversion commit using a post-commit hook.