Notes about Grid - Part 1: Installation (060604 ver. 2.0) ***************************************************************** (a summary of my personal knowledge of the Grid computing environment) In these notes I describe the procedure I followed in order to use the Grid (http://en.wikipedia.org/wiki/Grid_computing) to submit jobs. The information I have used is taken from the web, most of all from http://grid-it.cnaf.infn.it and from its links (in particular https://edms.cern.ch/document/454439/2 and http://grid-it.cnaf.infn.it/fileadmin/users/grid-experience/grid-experience.html). Some other reference links are given below: please notice that the information they contain doesn't always describe the actual up-to-date status of the Grid computing environment. In this file the "INSTALLATION" procedure I have followed is described. The file grid-HOWTO-2.txt contains my hints about using the Grid. Lots of acronyms are involved when speaking about the Grid. I have tried to clarify their meaning in the following, when needed: this link http://grid-it.cnaf.infn.it/fileadmin/users/dictionary/dictionary.html contains some of them and can be useful too. These instructions refers to my Linux Redhat 9 computing system: I can't assure neither that these are the best steps to be followed, nor that they work with other Linux distributions, but they can be a track to be followed somehow. I'm not a Grid expert, but if you need to contact me about this (sort of) guide you can find me at bongi@fi.infn.it. Good luck and have fun! ***************************************************************** INSTALLATION ------------ The first part of the procedure is taken directly from the "Access to the grid" section of the http://grid-it.cnaf.infn.it website. I describe here how these steps have been performed in my particular case: please refer to those web pages for additional details or to solve specific problems. Before being able to use the Grid for the first time, there are 3 things a user has to do: - to obtain access to a computer (User Interface) were the Grid tools are available ("Install your UI"), - to obtain an electronic personal certificate which allows him/her to be recognized by the Grid structure ("Get your certificate"), - and to become a member of a "Virtual Organization", ("Register to a VO"). ----------------------------------------------------------------- ********* * 1 * Install your UI. ********* I have chosen to "install" a UIPnP (the "First possibility" in the site instructions): in this way it is possible to access the Grid commands via AFS directly from my computer, without having to actually install any specific software. Each user can set-up this (no root access on your machine is needed): the only requirement is a working AFS client with access to the infn.it cell. Some hints about AFS can be found here: http://hep.fi.infn.it/PAMELA/linux/afs-HOWTO.txt. After AFS is properly set-up, simply execute this shell script (both sh and csh versions are present in the same AFS directory), which sets all the required environment variables and paths that make the Grid commands available to the user: [bongi@alterra bongi]$ source /afs/infn.it/project/infngrid/ui/UIPnP/UIPnP-AFS-Client.sh removed `vomses/alice-lcg-voms.cern.ch' removed `vomses/alice-voms.cern.ch' removed `vomses/argo-voms.cnaf.infn.it' removed `vomses/atlas-lcg-voms.cern.ch' removed `vomses/atlas-voms.cern.ch' removed `vomses/bio-voms.cnaf.infn.it' removed `vomses/biomed-cclcgvomsli01.in2p3.fr' removed `vomses/cdf-voms.cnaf.infn.it' removed `vomses/cms-lcg-voms.cern.ch' removed `vomses/cms-voms.cern.ch' removed `vomses/compchem-voms.cnaf.infn.it' removed `vomses/dteam-lcg-voms.cern.ch' removed `vomses/dteam-voms.cern.ch' removed `vomses/egrid-voms.cnaf.infn.it' removed `vomses/enea-voms.cnaf.infn.it' removed `vomses/esr-mu4.matrix.sara.nl' removed `vomses/euchina-voms2.cnaf.infn.it' removed `vomses/eumed-voms2.cnaf.infn.it' removed `vomses/gridit-voms.cnaf.infn.it' removed `vomses/inaf-voms.cnaf.infn.it' removed `vomses/infngrid-voms.cnaf.infn.it' removed `vomses/ingv-voms.cnaf.infn.it' removed `vomses/lhcb-lcg-voms.cern.ch' removed `vomses/lhcb-voms.cern.ch' removed `vomses/libi-voms.cnaf.infn.it' removed `vomses/na48-na48-voms.cern.ch' removed `vomses/pamela-voms.cnaf.infn.it' removed `vomses/planck-voms.cnaf.infn.it' removed `vomses/theophys-voms.cnaf.infn.it' removed `vomses/virgo-voms.cnaf.infn.it' removed directory: `vomses' UIPnP settings... Setting PATH ... Found glibc2.3: UIPnP will use Python2.2 for glibc2.3 Setting LD_LIBRARY_PATH ... Setting LIBPATH ... Setting MANPATH ... removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' removed `/tmp/vo.tmp' Done. Welcome in UIPlugAndPlay!!! Other details can be found in /afs/infn.it/project/infngrid/ui/UIPnP/README. The script creates a $HOME/.globus directory, and an AFS-UIPnP.conf file inside it: edit the file and change the default directory for your job output files. In my case: # The default output directory is #/tmp/jobOutput # Uncomment and set the following line to change the output directory OUTPUT_STORAGE=/home/bongi/work/pamela/grid/GRIDjobOutput Some of the environment variables which are used by Grid commands and which are needed in my case had not been not set by the script, so I exported them (and added them to my ~/.bash_profile, too), like this: export MYPROXY_SERVER=myproxy.cnaf.infn.it export LCG_CATALOG_TYPE=lfc export LFC_HOST=lfcserver.cnaf.infn.it export LCG_GFAL_VO=pamela The above UIPnP README file suggests to put some lines in your ~/.bash_profile in order to execute the script automatically at your login: I prefer to avoid this (what if the network/AFS is not working? ...) and execute it only when I decide to use the Grid (I wrote an alias for sourcing the script, so I have not to remember the full AFS path). ----------------------------------------------------------------- ********* * 2 * Get your certificate. ********* Since I already had a personal certificate issued by INFN Certification Authority (CA), in my case most of the work had already been done here. In any case you can have a look at http://grid-it.cnaf.infn.it/index.php?usercert&type=1 for detailed instructions on how to get one. As regards non-INFN users, I'm not sure about how they can obtain a recognized personal certificate: http://lcg.web.cern.ch/LCG/users/registration/certificate.html and http://ca.nordugrid.org can maybe be useful in order to find an available CA. After getting a certificate, I just had to convert it from the .p12 file format (which I got by exporting it from my browser) to the .pem format accepted by Grid tools, and put it in the right directory, with this couple of commands: [bongi@alterra bongi]$ openssl pkcs12 -nocerts -in MY_CERTIFICATE_2007.p12 -out ~/.globus/userkey.pem Enter Import Password: MAC verified OK Enter PEM pass phrase: Verifying - Enter PEM pass phrase: [bongi@alterra bongi]$ openssl pkcs12 -clcerts -nokeys -in MY_CERTIFICATE_2007.p12 -out ~/.globus/usercert.pem Enter Import Password: MAC verified OK The commands ask you for the "Import Password" which had been set when you exported your certificate. The "PEM pass phrase" which you set now will have to be typed every time you want to access the Grid. The .pem files must have special attributes, which you have to set with: [bongi@alterra bongi]$ chmod 0400 ~/.globus/userkey.pem [bongi@alterra bongi]$ chmod 0600 ~/.globus/usercert.pem [bongi@alterra bongi]$ ls -l ~/.globus -rwxr-xr-x 1 bongi wizard 864 May 22 17:27 AFS-UIPnP.conf -rw------- 1 bongi wizard 2087 May 11 16:18 usercert.pem -r-------- 1 bongi wizard 1917 May 11 16:17 userkey.pem drwxr-xr-x 2 bongi wizard 4096 May 24 09:55 vomses ----------------------------------------------------------------- ********* * 3 * Register to a VO. ********* In my case, the Virtual Organization I wanted to register to (pamela) was listed in this page: http://grid-it.cnaf.infn.it/index.php?voregister&type=1. By clicking on the appropriate link it is possible to submit a request to the VO manager, which will answer with a confirmation email. ----------------------------------------------------------------- ----------------------------------------------------------------- ----------------------------------------------------------------- If you are interested in reading some hints about "USING" the Grid, please go on and read grid-HOWTO-2.txt, which should be available in the same location where you have found this file.