Connecting to the Grid

Moore’s law, as formidable as it is in its ability to exponentially increase the processing power available on our everyday computers, is still unable to keep up with the computation demands that cutting edge scientific research poses. These demands have led the National Science Foundation and several Universities and National Labs to combine efforts to create a world wide grid of shared computational resources, known as the Open Science Grid. Access is mainly restricted to the institutions that compose it, but some attempts have been made to allow individual researchers (like myself) who don’t belong to any large user groups to access it. In this short series of posts I describe how I use it in the course of my research. I start by describing the basing installation and configuration procedure.

Installing the OSG client

The installation procedure has been made almost completely automatic, but there are still, however, a few things you must do yourself. The first step is to download pacman a user space package manager that will be responsible for determining which packages you need to install. All files will be installed in the current directory, so you should probably create one exclusively for this purpose.

You can download pacman using:

666

wget http://physics.bu.edu/pacman/sample_cache/tarballs/pacman-3.19.tar.gz

After downloading the tarball and unpacking it you should temporarily import some definitions to your shell environment.

666

source setup.sh

To install all the packages necessary all you need to do is:

666

pacman -get OSG:Client

which instructs pacman to begin the process. The complete download and installation will probably require a considerable amount of time, specially if you have a slow internet connection. When it finishes, you will have all the tools required to access the grid. You can see the output of a complete instalation in my OSG Installation Log

Setting up the keys

Before you can use the grids computational resources you need to install the DoEs X509 certificates that are used to securely identify you to the machines that you will need to use. I am assuming you have already obtained authorization to you the system and that your certificates are already available on you computer. You can learn how to download the certificates here.

The first step is to convert the p12 certificate in to voms-proxy certificate using two commands:

666
667

openssl pkcs12 -nocerts -in OSG.p12 -out userkey.pem
openssl pkcs12 -clcerts -nokeys -in OSG.p12 -out usercert.pem

After which two new files userkey.pem and usercert.pem will have been created in the current directory. You need to move them to the ~/.globus directory (create it, if it doesn’t already exist) and make sure the permissions are correct, using:

666
667

chmod 700 ~/.globus
chmod 600 ~/.globus/*

You should now have the full power of the grid accessible to you.

Firewall Configuration

For added security, it is recommended that you use a firewall in your submit machine. However, this requires some extra steps. I’m assuming you are using iptables in a linux machine, but most of what is said should be applicable to other platforms as well.

First, you must tell globus which port range to use by setting an environment variable. You can add this line to the setup.sh file in the OSG directory so it will be set every time you use the grid, or you can do it manually at the command line.

666

export GLOBUS_TCP_PORT_RANGE=9600,9700

You must also instruct Condor-G to use the same port range by adding (or uncommenting) two lines in your condor_config file located in ~/osg/condor/etc/condor_config.

LOWPORT=9600
HIGHPORT=9700

You must restart condor before these changes cat take effect. In case you are wondering why the large port range (9600-9700), the OSG recomended troubleshooting guide claims that you need 3 ports per user, but my experience has been quite different, so I prefer to use a larger range. Also, its seems you need to allow both tcp and udp traffic on those ports. The iptables rules you need to achieve this are:

-A INPUT -p tcp -m tcp --dport 9600:9700 -j ACCEPT
-A INPUT -p udp -m udp --dport 9600:9700 -j ACCEPT

A different type of firewall will probably use a different syntax so you should look up in the respective manual how to do it. You can find extensive documentation on the iptables package at the projects official website.

In the next post, I’ll exemplify how you can submit jobs to the grid using globus and condor.