Hasso-Plattner-Institut Potsdam Operating Systems and Middleware Group at HPI University of Potsdam, Germany
Operating Systems and Middleware Group at HPI

Installation of Globus 3.2

Peter Tröger

Installing Globus is a nightmare. This page describes our experiences with the installation of Globus 3.2 (stable) on an Itanium machine with Debian Woody. Some details are specific for the HPI testbed, but most things are general solutions and tricks.

Beside the Globus documentation, there are also nice documents from IBM: http://publib-b.boulder.ibm.com/Redbooks.nsf/RedpieceAbstracts/redp3697.html
http://www-106.ibm.com/developerworks/grid/library/gr-factory/

Similar troubleshooting documents are available here and here.

Parts of this document are contributed by Lars Lindner.

Prerequisites

Look at http://www-unix.globus.org/toolkit/docs/3.2/installation/install_support.html.
  • We left out the tomcat installation, anyway there is no documentation for running GT3.2 in Tomcat.
  • All tools are installed under /usr/local:
    lrwxrwxrwx 1 1001 staff 17 May 24 11:22 ant -> apache-ant-1.6.1/
    drwxr-xr-x 6 1001 1001 4096 Feb 12 14:28 apache-ant-1.6.1
    drwxrwsr-x 2 root staff 4096 Jan 1 2000 bin
    lrwxrwxrwx 1 root staff 11 Jun 1 13:38 globus -> globus3.2.0
    drwxr-sr-x 25 globus globus 4096 Jun 3 11:28 globus3.2.0
    drwxrwsr-x 2 root staff 4096 Jan 1 2000 include
    drwxrwxr-x 9 root staff 4096 May 27 16:18 j2sdk1.4.2_04
    lrwxrwxrwx 1 root staff 14 May 27 16:26 java -> j2sdk1.4.2_04/
    lrwxrwxrwx 1 root staff 21 Oct 16 2003 junit -> /usr/local/junit3.8.1
    drwxrwxr-x 5 root root 4096 Sep 4 2002 junit3.8.1
    drwxrwsr-x 4 root staff 4096 Sep 18 2003 lib
    drwxrwsr-x 2 root staff 4096 Jan 1 2000 man
    drwxrwsr-x 2 root staff 4096 Jan 1 2000 sbin
    drwxrwsr-x 3 root staff 4096 Jul 17 2003 share
    drwxrwsr-x 2 root staff 4096 Jan 1 2000 src
  • PostgreSQL was installed with apt-get.
  • There was a problem with a Debian package called star, which is some kind of alternative tar. The configure script claimed a version mismatch. We removed the package so that configure used the usual /bin/tar.
  • Before running the installation, you should create /usr/local/globus3.2.0 as root and change the ownership to globus.globus.

Source package installation

Look at http://www-unix.globus.org/toolkit/docs/3.2/installation/install_installing.html
  • Remember to execute all installation scripts as user "globus".
  • Don't ignore compiler error messages. In our case, we needed to install the additional Debian packages libgdbm-dev and libdb-dev.
  • After the execution of install-gt3 you get a message, which encourages you to execute another setup-script. Please perform this step as preparation for later authentification mechanisms. The script needs an already existing /etc/grid-security directory. After that you can execute the install-gt3-mmjfs script, again as user globus.

Core und SimpleCA configuration

Look at http://www-unix.globus.org/toolkit/docs/3.2/installation/install_config_req.html
  • We created our own script (/usr/local/globus/hpienv.sh) for the setup of all relevant environment variables:

    export ANT_HOME=/usr/local/ant
    export JAVA_HOME=/usr/local/java
    export JUNIT_HOME=/usr/local/junit
    export GLOBUS_LOCATION=/usr/local/globus
    source $GLOBUS_LOCATION/etc/templates/unix/setClassPath
    export CLASSPATH=$CLASSPATH:$JUNIT_HOME/junit.jar:${LOCALCLASSPATH}
    export PATH="${ANT_HOME}/bin:${JAVA_HOME}/bin:${PATH}";
    source ${GLOBUS_LOCATION}/etc/globus-user-env.sh

    Interested users can source this script in their ~/.bashrc file.
  • We were not able to reuse Globus 2.4 host certificates, which were generated with our our own OpenSSL CA (HPI OSM CA). Therefore all GT3.2 host certificates were recreated with the GT3-SimpleCA. Remember to call setup-globus-gram-job-manager after copying the new host certificates.
    The user certificates are generated by the HPI Active Directory. This certificates can be obtained from an internal web page, without administrative intervention, but only from Windows machines. It was no problem to add the new CA certificates to /etc/grid-security/certificates. You need the hash value of the certificate for the filename. It can be generated by executing the following command:
    openssl x509 -hash < CA_cert.pem
  • Don't forget to run $GLOBUS_LOCATION/bin/setperms.sh as root after the SimpleCA installation.
  • Create /etc/grid-security/grid-mapfile as user root. Add all users that should be able to connect to grid services.
    The subject line for a user can be obtained by calling grid-cert-info -subject. Type double-qoutes around the subject string.
  • A wrong passphrase for the CA certificate in OpenSSL can lead to a segmentation fault. This is a well-known bug in OpenSSL.
  • Test your installation as described here. Remember to call globus-start-container only from $GLOBUS_LOCATION.
  • If you cancel the execution of globus-start-container with Ctrl+C, it could happen that UHE's still run. Kill all regarding Java processes and remove the $GLOBUS_LOCATION/var/uhePortMapping file (taken from mailing-list). Users could have an uhe-* directory in their ~/.globus/ directory. It's save to delete it, if you want to get a clean environment.
    This can be the solution for a "ERROR: ping failed for a uhe so restarting it" message at container startup.
  • Our HPI certificates have an "emailAddress=xxx" entry in the subject line. The security libraries of Globus translate this to "E=xxx", so the gridmap-file has to consider this:

    troeger@str:~$ grid-cert-info -subject
    /emailAddress=peter.troeger@hpi.uni-potsdam.de/CN=Peter Troeger
    troeger@str:~$ less /etc/grid-security/grid-mapfile |grep peter.troeger
    "/E=peter.troeger@hpi.uni-potsdam.de/CN=Peter Troeger" troeger

Deploying the SimpleCA certificates to another machine

According to the Globus documentation, you only have to build the generated CA install package on the new machine:

$GLOBUS_LOCATION/sbin/gpt-build globus_simple_ca_HASH_setup-0.17.tar.gz gcc32dbg (maybe gcc64dbg, if you have a 64-Bit machine)
$GLOBUS_LOCATION/setup/globus_simple_ca_CA_Hash_setup/setup-gsi -default

  • The second step can fail with the following error:

    Error running grid-security-config. Aborting. at /usr/local/globus/setup/globus_simple_ca_5912a40f_setup/setup-gsi.pl line 152.
    Reason: The grid-security-config und grid-cert-request-config files are only available as .in templates in the TAR file. Create the missing files by yourself.

Configuring MMJFS

Look at http://www-unix.globus.org/toolkit/docs/3.2/installation/install_config_resource.html
  • If you call managed-job-globusrun, copy the factory reference from your container output. Don't use localhost as machine name, always work with the FQDN or IP-address (depends on the container output) to ensure a correct behaviour of the service lookup.
  • Our problem with the host certificates showed itself with an ASN parsing error on container side and an unspecified error on the client side. If you have similar problems, read the exceptiËon stack trace on container side, and triple-check your certificates on both sides. Prepare some SimpleCA certificates for tests, if you use your own CA.
  • Please check if you have a valid local-server-config.wsdd and local-client-config.wsdd in your $GLOBUS_LOCATION. If not. copy them from server-config.wsdd and client-config.wsdd.

Configuring GridFTP

Look at http://www-unix.globus.org/toolkit/docs/3.2/installation/install_config_gridftp.html
  • Since we do not have a configured CAS service for authentification, you should remove /etc/grid-security/gsi-authz.conf and /etc/grid-security/gsi-gaa.conf. Otherwise GridFTP tries to use CAS, which results in a client-side error 530 (No local mapping for Globus ID).

Configuring RFT

Look at http://www-unix.globus.org/toolkit/docs/3.2/installation/install_config_rft.html
  • Please ensure that your CLASSPATH contains all Globus-relevant JAR files, for example in your environmental setup script:
    ...
    source $GLOBUS_LOCATION/etc/Ãtemplates/unix/setClassPath
    export CLASSPATH=$CLASSPATH:$JUNIT_HOME/junit.jar:${LOCALCLASSPATH}
    ...
  • Don't be confused by the GAR deployment description in the Globus docs - if you used the full source installer, RFT is already installed.
  • Setting up the database is a little bit tricky for PostgreSQL newbies (like me). If you follow the Globus document, you get a database with the correct tables. You should now add a new database user, e.g. "globus":

    CREATE USER globus WITH PASSWORD 'globus'

    The problem: All tables in the ogsa database are owned by the postgres database user. You can change this afterwards with the following commands:

    ALTER TABLE proxyinfo OWNER TO globus;
    ALTER TABLE request OWNER TO globus;
    ALTER TABLE restart OWNER TO globus;
    ALTER TABLE transfer OWNER TO globus;
    SELECT * FROM pg_user;
    usename | usesysid | usecreatedb | usesuper | usecatupd | passwd | valuntil | useconfig
    ---------+----------+-------------+----------+-----------+----------+----------+-----------
    postgres | 1 | t | t | t | ******** | |
    globus | 100 | f | f | f | ******** | |
    UPDATE pg_database SET datdba=100 WHERE datname='ogsa';
    GRANT ALL ON proxyinfo,request,restart,transfer TO globus;
    GRANT ALL ON request_seq,transfer_seq TO globus;

    Next time we should create the user first (CREATE USER globus WITH PASSWORD 'globus' CREATEDB) and build the database afterwards.
  • The documented example for the transfer file is not complete, you should use transfer.xfr from your installation as template. The DN lines in the file must contain the host certificate subjects from the source and the destination machine.
  • The attribute names for the server-config.wsdd are not explained correctly, in our version we had to change password, dbusername, and the last part of connectionURL for the database name. The record name stated in the HowTo is also not correct, simply search for the term "postgres" to find the right section.

Installing MPICH-G2

Remember to re-call setup-globus-job-manager-fork after the installation of MPICH-G2.

Installing Globus2 services (MDS and GRAM)

See http://www-unix.globus.org/toolkit/docs/3.2/installation/install_config_prews.html and MDS GRIS/GIIS configuration

  • Please note that the Globus sbin directory contains prepared startup scripts for the /etc/rcXXX directories.
Use the following commands to test the installation:
  • Anonymous query to local GRIS: grid-info-search -x
  • Anonymous query to a remote GRIS: grid-info-search -x -h <FQDN>
  • Non-Anonymous query to local GRIS: grid-info-search
  • Please note that GSI needs a working reverse lookup for your machines, otherwise only anonymous queries will work.

Configuring Globus SDE Browser

See http://www.globus.org/ogsa/releases/final/docs/infosvcs/sdbconfig.html
  • The tool needs to be started from the Globus installation directory.

Installing MyProxy

Just follow the manuals: http://grid.ncsa.uiuc.edu/myproxy/adminguide.html