Procedure for Installing the Cirpass Data Reduction Pipeline
Contents
For personal use only
- Set up the links to the cirpass iraf packages. The most
convenient way to do this would be to just move the
cirpass/iraf tree directly over to your iraf directory, e.g. with:
mv -i cirpass/iraf/* iraf/
However, this will overwrite your loginuser.cl file if you
already have one. (In this case just add the appropriately
modified versions of the lines shown below to your loginuser.cl
file). Alternatively you can install to a
custom location. You will need to add the following lines
to your loginuser.cl file, changing them to reflect the actual
location of the cirpass cl code.
set cirpass= home$Cirpass/
task $cirpass.pkg = cirpass$cirpass.cl
- Define the location of the cirpass executables by setting the
environment variable CIRPBIN. Under tcsh this would be:
setenv CIRPBIN /home/_userid_/cirpass/bin/
(don't forget the trailing forward slash). This needs to be done in
any xterm / xgterm from which you start iraf. Ideally just
place this line in your login file.
- If you intend to do data pre-processing i.e. you need to make
cirpass MEF files from completely unprocessed data (and usually
only true if you're sat at the instrument), you
will need to edit the iraf variables which specify the location
of the cirpass raw and reduced data. These are defined in
cirpass.cl and are:
set cirpraw = /data1/cirpass/
set cirpred = /data2/datared/
cirpraw is the raw data, produced by the instrument. cirpred is
the location for the reduced data. You will need write
permission on this directory.
For site-wide access for every user
- Log in as the user iraf. Place the cirpass iraf package in the
relevant iraf sub-directory on your system e.g. with:
mv -i /cirpass_untar_location/cirpass/iraf/* /iraf/extern/
This will move a directory called Cirpass into the site-wide
directory. Although see NOTE, 3 points below.
- Edit extern.pkg to define the new package, e.g. from an iraf cl
prompt, as user iraf:
cd hlib$
vi extern.pkg
(hlib$ usually = /iraf/iraf/unix/hlib or something similar) and add
the following two lines before the reset helpdb entry, and changing the
cirpass environment variable to reflect the actual installed location,
issued in the move command above.
reset cirpass = /iraf/extern/Cirpass/
task $cirpass.pkg = cirpass$cirpass.cl
Note the capital C for the Cirpass directory!!
- The location of the cirpass executables is set in the
environment variable CIRPBIN. Every user wishing to use the
cirpass package will require this to be set, and so this could be
added to the site-wide variables that are set at
login. Alternatively, each user could add the following line to
e.g. their .mytcshrc:
setenv CIRPBIN /cirpass_untar_location/cirpass/bin/
(don't forget the trailing forward slash).
- NOTE: The IFUdat and Scripts subdirectories under
/cirpass_untar_location/cirpass/ are found relative to the CIRPBIN
variable - so if you decide to move the cirpass/bin directory,
cirpass/Scripts and cirpass/IFUdat should also be moved to the same
place. These are epar able locations in iraf, but things will work
with least effort if only cirpass/iraf/* is moved to /iraf/extern/ and
all other directories are left with the same structure. Alternatively
/cirpass_untar_location/cirpass could be moved to /iraf/extern/ and
then the extern.pkg file changed appropriately (reset cirpass =
/iraf/extern/cirpass/iraf/Cirpass/, CIRPBIN=/iraf/extern/cirpass/bin/)
this would keep all cirpass specific code in the same place, but would
result in compiled c code / libraries e.t.c. in your /iraf directory
(shouldn't be a problem!).
- If users will be required to do data pre-processing i.e. they
have access to
cirpass MEF files from completely unprocessed data (and usually
only true when sat at the instrument),
the iraf variables which specify the location
of the cirpass raw and reduced data will need to be
edited. These are defined in
Cirpass/cirpass.cl and are:
set cirpraw = /data1/cirpass/
set cirpred = /data2/datared/
cirpraw is the raw data, produced by the instrument. cirpred is
the location for the reduced data. You will need write
permission on this directory.
- The cirpass package runs on fits image files. Thus it is necessary
to have the iraf image type set to fits (it usually is, from a cl
prompt,
show imtype
should be fits). This can be achieved
with: reset imtype = fits
from the cl prompt, or changed permanently by uncommenting and
changing the "imh" to "fits" in the:
#set imtype = "imh"
line in the ~/iraf/login.cl file.
- Change to the cirpass installation directory and customise the
makefile for your system. The makefile should not require any
changes for a redhat linux system. All necessary modifications
should be in the top few lines. Check that the compilers are
correct (the instpath variable should not need changing) and
uncomment any lines appropriate to your operating system. A
default install does not include the code linked against
pgplot, instructions for compiling this code are below, but should
be omitted for now.
- Compile the external cfitsio package used for the fits
interface, type:
make cfitsio
This will take a little while. Now compile the cirpass package
- Type:
make
- Logout and in again if you've edited your login files.
- You should now be ready to go. Start up iraf and type cirpass to
load the cirpass package.
If pgplot and cpgplot is already on your system
- Edit the line in the cirpass makefile:
PGDIR = $(instpath)/extern/pgplot5.2
to reflect the location of pgplot on your system. Also, modify
the lines below this as instructed in the makefile.
- From cirpass/ type:
make usepgplot
- Then set up the environment variable defining the location of
pgplot, this is best done in a login file and for tcsh would be:
setenv PGPLOT_DIR /usr/local/pgplot/location/
- You may also want to set the default output device:
setenv PGPLOT_DEV /xwindow
If pgplot is not already installed
- Change to the extern directory under cirpass.
- Create the directory pgplot5.2. This is the installation
location for the compiled code. This is the default location
expected by the cirpass makefile but is configurable if you
already have pgplot compiled elsewhere (see above).
- Untar the pgplot source (gtar -zxvf pgplot522.tar.gz).This will
create a pgplot directory in which there is an install.txt
file (and install-unix.txt). This file contains installation
instructions for pgplot and you should follow these, however, a
summary is given below:
- Make the drivers file in the pgplot destination directory (or
use the cirpass
default
one)
- From the destination directory run
../pgplot/makemake ../pgplot linux g77_gcc
- From the destination directory run
make
(Had to change the X11 bit of the Loader library-flags in the
makefile to -L/usr/X11R6/lib)
- Then run
make clean
- and
make cpg
for the C binding.
- Finally set up the environment variable defining the location of
pgplot, this is best done in a login file and for tcsh would be:
setenv PGPLOT_DIR /home/_userid_/cirpass/extern/pgplot5.2/
- You may also want to set the default output device:
setenv PGPLOT_DEV /xwindow
- Linux also requires the location of the shared library to be set
explicitly:
setenv LD_LIBRARY_PATH /home/cirpass/cirpass_software/extern/pgplot5.2
- Edit the cirpass makefile (cirpass/makefile) and uncomment the relevant lines relating to pgplot. Make the cirpass pgplot code, in cirpass/ type:
make usepgplot
- The data reduction machine needs the accounts obs, data and irgroup.
- data does not need to login and its shell can be /usr/bin/nologin. irgroup
is
the privileged group that can edit / delete all cirpass
files. Typical /etc/passwd entries are:
obs:x:506:507:Observer:/home/obs:/bin/tcsh
data:x:507:508:data account:/home/data:/usr/bin/nologin
and /etc/group has:
irgroup:x:506:cirpass,ajd,raj,irp
obs:x:507:cirpass,raj,ajd,irp
data:x:508:cirpass,ajd,raj,irp
- Install the utils c programs with:
make utils
and install pgplot with make usepgplot
- The iraf script mktoday sets up the raw and reduced directories
for the cirpass data for each day. This program calls setuid C
programs to set the permissions properly. Thus the user and
group id's of these programs needs to be set and this can be
accomplished with the setdatapermissions.sh script in
cirpass/Scripts/. Once you have run make utils, change to the
cirpass Scripts directory, su to root, and then run:
./setdatapermissions.sh
- Make sure that the location of the file makedata.tcsh e.t.c is
readable by the obs account i.e. if it's
/home/cirpass/cirpass_software/Scripts/makedata.tcsh then
/home/cirpass will need to be world readable. That's not the default
for a RedHat install!
- The program mktoday can now be epared to set the raw and
reduced data directory path. By default it will use
cirpraw and cirpred (you shouldn't have to change this!),
cirpraw and cirpred are set in cirpass.cl and any changes
should be made there.
- Note that the link and the actual directory written by
mktoday now have the same path. To access the same directory on
cirpass1 and cirpass4 requires different absolute paths depending
on the computer you are on. This can confuse mktoday. The fix is
to put a soft link on cirpass4, so that the link that is written
by cirpass1 also works on cirpass4. i.e. on cirpass4:
ln -s / datastore
- The following two points no longer apply, as of the updated eparable
mktoday, however, they've been left to make it clear how
mktoday works.
- Don't do this, for info only! The maketoday source
will need editing to reflect the
location of the shell scripts it calls, the C source for
them is in cirpass/src/utils/. You will also need to make
the program remminus and put the executable in a location
searched by your path. (Although remminus should be made
by default and the path to it should now be sensible.)
- Don't do this, for info only! maketoday.tcsh and
makedata.tcsh (in Scripts/) may need to
be edited to reflect the location of the raw and reduced data
directories on the data reduction computer.
- /data1/cirpass must be owned by data:data (set this on
cirpass4 as it's nfs exported for the new system!)
/data2/datared must be owned by obs:irgroup.
- Set up samba to export the relevant directories. (More info
here. CARE
with the smb.conf file! cirpass4 has the latest version of
samba and a more minimal configuration than the one referred to
here).
Users at the IoA who type "cirpass" from iraf, load the package
from /home/cirpass/iraf/Scripts/Cirpass
and the C code
from /home/cirpass/C/
. Modifications to the software
should be made (and tested!) in those directories as the user
cirpass. The software for public release is maintained in
/home/cirpass/Release/
. Any files changed as a result of
software updates should be copied to the relevant subdirectory
below Release
and then the distribution tar ball recreated with:
cd /home/cirpass/Release
gtar -zcvf cirp_code_20030428.tar.gz cirpass
su optics
cd /home/optics/public_html/cirpass/Software/
cp /home/cirpass/Release/cirp_code_20030428.tar.gz .
rm cirp_soft.tar.gz
ln -s cirp_code_20030428.tar.gz cirp_soft.tar.gz
exit
where 20030428
is today's date. Care should be taken when
copying files over to Release/
! Only overwrite
source iraf or C files. cirpass.cl and the makefile are very
different in the Release/
directory from those used by
the site, as they are the generalised forms. DO NOT overwrite
cirpass.cl or the makefile in Release/
. If new programs
are written they should be edited into these files by hand.
Iraf was installed using the standard iraf documentation that is
available from http://iraf.noao.edu, the X11iraf
package was also installed to provide ximtool and xgterm's. The stsdas
and tables packages were also installed by following the links from
the main iraf site.
- Obtain iraf by ftp from ftp://iraf.noao.edu. You need the as.pcix.gen,
nb.rhlx.x86 and the ib.rhlx directories, found in
iraf/v211/PCIX/, installation instructions should be found here
also.
- X11Iraf is also needed to provide xgterm.
- Make individual user accounts by creating an iraf directory
(mkdir iraf) and running mkiraf in that directory
- Install the stsdas and tables packages. The packages (you need
binaries and source) can be
downloaded from ftp://ftp.stsci.edu/pub/software/ and
installation instructions found at
ftp://ftp.stsci.edu/pub/software/stsdas/INSTALL (see tar options
gotcha below and remember to change to bin.redhat before
installing the binaries). The packages
were installed to /iraf/extern/ on the cirpass machines.
- Install the fitsutil package. Available from
ftp://iraf.noao.edu in /iraf/extern/ and retrieve the
relevant source package (fitsutil.tar.Z), binary package and
readme file. fitsutil.readme contains the installation
instructions.
Gotchas!
- When creating the user account iraf, it will make things much
easier if its shell is csh.
- Iraf must own its directories if you want to be able to log in
as iraf!
- When doing the install use bin.redhat not bin.linux (if
appropriate).
- For mkpkg to work for the stsdas external package, the hlib environment
variable needs to be set to the location of the hlib directory
(do a "locate extern.pkg" and set hlib, in the shell or .cshrc e.t.c.,
to the path to the directory containing this file).
- If you find onedspec packages don't work, like splot and
dispcor, it may be that the symbolic link to the binary
directories is wrong. (This was the case for both installations I
did.) I had to edit the symbolic link bin.redhat in
/iraf/iraf/noao/ to be ../../irafbin/bin.redhat (cd
/iraf/iraf/noao ln -s ../../irafbin/bin.redhat bin.redhat) as
it was originally noao.bin.redhat.
- The installation documentation contained an error at the time of
writing: when unpacking the tar files the order of the options in the
tar command should be xvpf i.e. tar -xvpf - and not xvfp as
this will cause the extraction to fail.
- Use the new install script provided at the ftp site, not the
extracted one installed by default as it doesn't work!
- The apropos package is part of the stsdas distribution and details
for its installation can be found in the full version of the stsdas
installation documentation.
- Printing from iraf: If a printer has been set up on the system
and a default is present in loginuser.cl then printing from ximtool
should be working already. To print from an xgterm using
":.snap" requires that the printer be mentioned in the iraf
graphcap file found in iraf$dev/. Add an entry to this file for the
desired printer by copying the syntax for the appropriate printer type
or by looking at the local system iraf installation graphcap file and
copying over the appropriate parts.
Andrew J. Dean.
<ajd@ast.cam.ac.uk>
Last modified: Thu Aug 21 17:45:45 2003