Adding a block to an existing CWN (column-wise ntuple)

The Issue:

By way of definition, a column-wise ntuple is a new style of ntuple that has data typing, compression (essentially bit packing) where appropriate, and variable length arrays, which are very useful for describing data. The CWN is composed of variables as before, but now arranged as blocks. The documentation suggests that this makes it easy to extend the ntuple, not by adding variables, but by adding new blocks, and filling them with whatever new data you have.

It seems that adding a block to an existing column-wise ntuple would be a trivial undertaking. It is, if you follow the documents in the CERNLIB documentation and FAQs. The trouble is that all of the examples cover adding independent blocks to an existing ntuple. This is not usually what is needed. What is more often needed is to add blocks to an existing ntuple that depend on data already in the ntuple. For example calculating the energy from an number of hits or pulse height in an ntuple. For this example I extended a previously existing ntuple containing data of emulsion tracks and segments of tracks. I calculated a kink angle between two consecutive stubs on the same track. If a kink has no next stub, its kink is set to zero. I also calculated the distance of the kink from the vertex, and an index, myindex, which is just an event count.

The main problem is in reading data from an ntuple into a common block after a new "block" has been added to the ntuple. My suspicion is that it is a bug, that has to do with trying to load the "new" block from the old ntuple that is already on disk. In any case, I have found a method of loading the ntuple data, performing calculations, and storing the results in the new block of the ntuple. The recipe is fairly simple, but there are a number of combinations to try before finding this one.

The Recipe:

Define the variables and common blocks using the HUWFUN definitions

Define the variables and common blocks for the new CWN block

Initialize higz common block, and set limits (HLIMIT)

Open the existing ntuple file (HROPEN)

Read it in (HRIN)

Decide on variables to be read, either from a paw nt/print command, or using HGIVEN as in the example program. (Hint: the variable names you need to define are the bare names, no indices, e.g. UPOS(NVERTEX) is referenced simply as 'UPOS')

Get the number of entries in the existing ntuple (HGNOENT)

Read in one row of the ntuple with HGNTV, this sets up the pointers for the rest of the accesses

Define the new block with HBNAME, or HBNAMC

Loop over the events using HGNTF to load the next event into the common blocks

Use HFNTB to fill the new ntuple block with the data for each event

As a courtesy, it is nice to supply the output of an HUWFUN call, as I did at http://www.hep.umn.edu/~mualem/e872/ntuple/skeleton.f

The Example:

The full text of the Example can be found nicely formatted at: http://www.hep.umn.edu/~mualem/e872/ntuple/addblock.html or not so nicely formatted, but compilable at: http://www.hep.umn.edu/~mualem/e872/ntuple/addblock.f.
The sample ntuple can also be found at http://www.hep.umn.edu/~mualem/e872/ntuple/period_12.hbook_orig.

The output:

Original ntuple in:

 ******************************************************************
 * Ntuple ID = 1000   Entries = 27        EX04
 ******************************************************************
 * Var numb * Type * Packing *    Range     *  Block   *  Name    *
 ******************************************************************
 *      1   * I*4  *         *              * MAIN     * RUNNO
 *      2   * I*4  *         *              * MAIN     * EVTNO
 *      3   * I*4  *         *              * MAIN     * MODULE
 *      4   * I*4  *         *              * MAIN     * RANGE(2)
 *      5   * I*4  *         *              * MAIN     * NTRACK
 *      1   * I*4  *         * [0,500]      * SEGMENT  * NSEGS
 *      2   * I*4  *         *              * SEGMENT  * TRACK(NSEGS)
 *      3   * I*4  *         *              * SEGMENT  * INDEX(NSEGS)
 *      4   * R*4  *         *              * SEGMENT  * SEGUPOS(NSEGS)
 *      5   * R*4  *         *              * SEGMENT  * SEGVPOS(NSEGS)
 *      6   * R*4  *         *              * SEGMENT  * SEGZPOS(NSEGS)
 *      7   * R*4  *         *              * SEGMENT  * SEGUANG(NSEGS)
 *      8   * R*4  *         *              * SEGMENT  * SEGVANG(NSEGS)
 *      1   * I*4  *         * [0,100]      * VERTEX   * NVERTEX
 *      2   * R*4  *         *              * VERTEX   * UPOS(NVERTEX)
 *      3   * R*4  *         *              * VERTEX   * VPOS(NVERTEX)
 *      4   * R*4  *         *              * VERTEX   * ZPOS(NVERTEX)
 *      1   * C*16 *         *              * CHARACT  * MODTYP
 *      2   * C*16 *         *              * CHARACT  * SCANTYP
 ******************************************************************
 *  Block   *  Entries  * Unpacked * Packed *   Packing Factor    *
 ******************************************************************
 * MAIN     *  27       * 24       * 24     *       1.000         *
 * SEGMENT  *  27       * 14004    * Var.   *    Variable         *
 * VERTEX   *  27       * 1204     * Var.   *    Variable         *
 * CHARACT  *  27       * 32       * 32     *       1.000         *
 * Total    *    ---    * 15264    * Var.   *    Variable         *
 ******************************************************************
 * Blocks = 4            Variables = 19      Max. Columns = 3810  *
 ******************************************************************

New ntuple out: (new variables in block KINKS, with 27 entries)

 ******************************************************************
 * Ntuple ID = 1000   Entries = 27        EX04
 ******************************************************************
 * Var numb * Type * Packing *    Range     *  Block   *  Name    *
 ******************************************************************
 *      1   * I*4  *         *              * MAIN     * RUNNO
 *      2   * I*4  *         *              * MAIN     * EVTNO
 *      3   * I*4  *         *              * MAIN     * MODULE
 *      4   * I*4  *         *              * MAIN     * RANGE(2)
 *      5   * I*4  *         *              * MAIN     * NTRACK
 *      1   * I*4  *         * [0,500]      * SEGMENT  * NSEGS
 *      2   * I*4  *         *              * SEGMENT  * TRACK(NSEGS)
 *      3   * I*4  *         *              * SEGMENT  * INDEX(NSEGS)
 *      4   * R*4  *         *              * SEGMENT  * SEGUPOS(NSEGS)
 *      5   * R*4  *         *              * SEGMENT  * SEGVPOS(NSEGS)
 *      6   * R*4  *         *              * SEGMENT  * SEGZPOS(NSEGS)
 *      7   * R*4  *         *              * SEGMENT  * SEGUANG(NSEGS)
 *      8   * R*4  *         *              * SEGMENT  * SEGVANG(NSEGS)
 *      1   * I*4  *         * [0,100]      * VERTEX   * NVERTEX
 *      2   * R*4  *         *              * VERTEX   * UPOS(NVERTEX)
 *      3   * R*4  *         *              * VERTEX   * VPOS(NVERTEX)
 *      4   * R*4  *         *              * VERTEX   * ZPOS(NVERTEX)
 *      1   * C*16 *         *              * CHARACT  * MODTYP
 *      2   * C*16 *         *              * CHARACT  * SCANTYP
 *      1   * I*4  *         * [0,500]      * KINKS    * KSEGS
 *      2   * R*4  *         *              * KINKS    * KINK(KSEGS)
 *      3   * R*4  *         *              * KINKS    * KDIST(KSEGS)
 *      4   * I*4  *         *              * KINKS    * MYINDEX
 ******************************************************************
 *  Block   *  Entries  * Unpacked * Packed *   Packing Factor    *
 ******************************************************************
 * MAIN     *  27       * 24       * 24     *       1.000         *
 * SEGMENT  *  27       * 14004    * Var.   *    Variable         *
 * VERTEX   *  27       * 1204     * Var.   *    Variable         *
 * CHARACT  *  27       * 32       * 32     *       1.000         *
 * KINKS    *  27       * 4008     * Var.   *    Variable         *
 * Total    *    ---    * 19272    * Var.   *    Variable         *
 ******************************************************************
 * Blocks = 5            Variables = 23      Max. Columns = 4812  *
 ******************************************************************