


User Contributed Perl Documentation               CIRDR_STATUS(1)



NNNNAAAAMMMMEEEE
     cirdr_status - produce a summary of pipeline status

SSSSYYYYNNNNOOOOPPPPSSSSIIIISSSS
     cccciiiirrrrddddrrrr____ssssttttaaaattttuuuussss [----llll _l_i_s_t_f_i_l_e] [----nnnn] [_p_i_p_e_l_i_n_e] [...] _o_u_t_p_u_t_d_i_r

DDDDEEEESSSSCCCCRRRRIIIIPPPPTTTTIIIIOOOONNNN
     cccciiiirrrrddddrrrr____ssssttttaaaattttuuuussss determines the current processing status of a
     CIRDR data reduction pipeline for WFCAM data, from the FITS
     headers in the image and catalogue files produced by the
     pipeline.  A summary of the current processing status, and
     details for all files being processed by each named pipeline
     are produced.

     The default is to create the output in HTML format.  A
     summary table is written to the file _s_u_m_m_a_r_y._h_t_m_l in the
     directory _o_u_t_p_u_t_d_i_r (which will be created if necessary),
     containing links to individual HTML summaries for each
     pipeline, written in files named {_p_i_p_e_l_i_n_e}._h_t_m_l where
     {_p_i_p_e_l_i_n_e} is the name of the pipeline.  An alternative
     ASCII output mode can also be used (see OPTIONS, below).

     The pipeline names, supplied via the parameter _p_i_p_e_l_i_n_e, are
     mapped to the pipeline locations through the file _l_i_s_t_f_i_l_e.
     The format of this file is described in FILES, below.  If no
     pipeline names are supplied, all pipelines in the list file
     are processed.

     Under each location from the list, the script performs a
     recursive search through this directory, and all
     subdirectories, for files named with ._f_i_t, ._f_i_t_s and ._f_t_s
     extensions.  Only target files are considered by filtering
     out any catalogues and calibration frames.  The search code
     will be revised when a final directory structure for
     pipeline processing is fixed.

     The FITS headers of each target frame are interrogated to
     ascertain the current status of processing.  Catalogue files
     are opened also if they exist, and are searched for under
     the standard names used by the CIRDR software.  A single-
     line summary of the stages of processing completed (see
     STAGES, below) is printed to the detailed summary file for
     each target frame in a simple ASCII format, described under
     SUMMARY FORMAT.  Any warnings generated during this process
     are written to a section named _W_a_r_n_i_n_g_s at the end of the
     HTML output, or printed to the standard error stream if the
     output is ASCII.

     A line is written to the overall summary for each pipeline,
     detailing the stages of processing which the script deems to
     be complete.




2003-07-01          Last change: perl v5.6.0                    1






User Contributed Perl Documentation               CIRDR_STATUS(1)



OOOOPPPPTTTTIIIIOOOONNNNSSSS
     The following command-line options are supported:

     -llll _l_i_s_t_f_i_l_e
         Uses _l_i_s_t_f_i_l_e as the pipeline list file instead of the
         default of _p_i_p_e_l_i_n_e_s._l_i_s_t in the current directory.

     -nnnn  Disables HTML output.  The summary file is now written
         in ASCII format as _s_u_m_m_a_r_y._l_i_s_t, with the summary for
         pipeline {_p_i_p_e_l_i_n_e} written to {_p_i_p_e_l_i_n_e}._l_i_s_t.  The use
         of this option is not supported, and it may be removed
         in the future.

SSSSUUUUMMMMMMMMAAAARRRRYYYY FFFFOOOORRRRMMMMAAAATTTT
     The detailed summary consists of an ASCII table containing a
     column for each stage of processing.  Some stages of the
     processing are only required by a subset of data-sets
     processed.  These are handled slightly differently.

     For stages which must be passed by all data-sets, the column
     contains a blank (space) character if the stage has not been
     completed, and an `x' character if the stage has been
     completed.  For stages which are not required for all data,
     a `n' character indicates that the stage is not required.
     Otherwise, it is required and a blank and `y' character
     signify that the stage is not complete, or is complete,
     respectively.

     The following columns are present for WFCAM data, in
     approximate order of processing within the pipeline
     software:

     SSSSttttaaaaggggeeee 1111

         _D_o_n_e
             Stage 1 processing has been run, and is considered
             to be complete by the script.  This should mean that
             all four of the stages below have been completed.

         _L_i_n The image has been linearity corrected.

         _R_e_s_e_t
             The image has had reset correction (subtraction of a
             bias image) performed.

         _D_a_r_k
             Dark correction has been performed: a dark frame has
             been subtracted.

         _F_l_a_t
             Flatfield correction has been performed: the image
             has been divided by a flatfield frame.



2003-07-01          Last change: perl v5.6.0                    2






User Contributed Perl Documentation               CIRDR_STATUS(1)



     DDDDeeeeffff Defringing should be (if ` ') / has been (if `y')
         performed on the image.

     SSSSkkkkyyyy Sky subtraction has been performed.

     TTTTaaaarrrrttttuuuupppp

         _D_o_n_e
             The processing in _c_i_r__t_a_r_t_u_p._c has been run, and is
             considered to be complete by the script.  This
             should mean that all of the stages below have been
             completed.

         _R_e_s_e_t
             Correction of reset anomalies has been performed.

     WWWWCCCCSSSS

         _0   An initial rough world co-ordinate system (WCS) has
             been written to the FITS headers.  This is derived
             directly from the telescope pointing information.

         _1   A first-pass world co-ordinate system has been
             written.  This is derived from the image only using
             an APM-like algorithm to extract sources and refine
             the coordinates against an astrometric catalogue.

     CCCCaaaattttaaaalllloooogggguuuueeeessss

         _D_o_n_e
             The source catalogue for this frame has been
             produced.  This means that _i_m_c_o_r_e has been run and a
             catalogue file exists.

         _A_s_t_r_o_m
             A second-pass world co-ordinate system has been
             written.  This is derived by refining the WCS
             against an astrometric catalogue using the result of
             the source extraction in _i_m_c_o_r_e.

         _C_l_a_s_s
             The object morphological classifier has been run.

     SSSSkkkkyyyy%%%%
         Sky percentage corrections have been computed and
         written to the FITS headers.

     PPPPhhhhooootttt
         The photometry stage of the pipeline has been run.  Here
         the photometric calibration of the catalogues is
         performed.




2003-07-01          Last change: perl v5.6.0                    3






User Contributed Perl Documentation               CIRDR_STATUS(1)



     The short summary format used in the _s_u_m_m_a_r_y._h_t_m_l file
     contains one line per defined pipeline, containing one
     column for each of the stages listed below, and _C_o_m_p_l_e_t_e,
     _I_n_c_o_m_p_l_e_t_e or _N_o_t _S_t_a_r_t_e_d if the script deems the relevant
     processing to have finished, have seen started but not
     finished, or not started, respectively.  An overall column
     at the right summarises processing for this pipeline.

     The following stages are listed (see above for
     descriptions):

     +o _S_t_a_g_e_1

     +o _D_e_f_r_i_n_g_e

     +o _S_k_y_s_u_b

     +o _R_o_u_g_h_w_c_s

     +o _T_a_r_t_u_p

     +o _F_i_r_s_t_w_c_s

     +o _I_m_c_o_r_e

     +o _A_s_t_r_o_m

     +o _C_l_a_s_s_i_f_y

     +o _P_e_r_c_e_n_t

     +o _P_h_o_t_o_m

EEEEXXXXAAAAMMMMPPPPLLLLEEEESSSS
     Process pipelines `example1' and `example2', producing HTML
     output in _o_u_t_d_i_r.

       cirdr_status example1 example2 outdir

     Process all pipelines listed in the alternative list file
     /_d_i_r/_l_i_s_t placing the output in _o_u_t_d_i_r.

       cirdr_status -l /dir/list outdir


FFFFIIIILLLLEEEESSSS
     _p_i_p_e_l_i_n_e_s._l_i_s_t
         A list of pipeline names, with locations and the
         instrument used to generate the data.  The file format
         is based on the ._i_n_i format, in particular it resembles
         that used in _s_m_b._c_o_n_f(5) from the Samba package.




2003-07-01          Last change: perl v5.6.0                    4






User Contributed Perl Documentation               CIRDR_STATUS(1)



         The character `#' is a comment, and instructs the parser
         to ignore the rest of the line.  The remainder of the
         file contains sections, describing each pipeline,
         denoted by `[{pipeline}]' on a line by itself for
         pipeline {_p_i_p_e_l_i_n_e}.  The following lines up to the next
         section delimiter are keyword-value pairs, with the key
         and value separated by an `=' character, with optional
         surrounding whitespace, ie. `{key}={value}'.  The
         following keys are supported:

         _p_a_t_h
             Specifies the full path to the pipeline.  This must
             be given for each pipeline in the file.

         _i_n_s_t_r_u_m_e_n_t
             Specifies the instrument used to take the data.
             This currently does nothing, and should be specified
             as `wfcam'.  A future version of the script may
             support multiple instruments.

NNNNOOOOTTTTEEEESSSS
     This script is very specific to the draft WFCAM FITS headers
     and pipeline.

     Since there is no file-locking by the pipeline we have to
     guess that the file is being written if CFITSIO complains.
     The code is likely to give unreliable results for the
     particular file being written, especially if, for example,
     extra FITS headers were appended which caused the HDU to
     need an extra FITS block assigning.

     The recursive searching utilised here will give unreliable
     results in certain pipeline directory structures.  When a
     standard directory hierarchy for pipeline processing is
     decided, this script will be modified to use it instead.

SSSSEEEEEEEE AAAALLLLSSSSOOOO
     CIRDR pipeline documentation
     http://www.ast.cam.ac.uk/~jrl/cirdr/index.html

     _s_m_b._c_o_n_f(5)

AAAAUUUUTTTTHHHHOOOORRRR
     Jonathan Irwin (jmi@ast.cam.ac.uk)











2003-07-01          Last change: perl v5.6.0                    5




