Figure: Schematic presentation of the various steps in the data analysis chain
Figure: Schematic presentation of the various steps in the data analysis chain
Although it is possible to define histograms interactively in a PAW session, and then read the (many thousands of) events, in general for large data samples the relevant variables are extracted from the Data Summary Files or DSTs and stored in histograms or an Ntuple. Histograms require to make a certain choice as to the range of values for the plotted parameter, because the binning, or the coarseness, of the distribution has to be specified when the histogram is defined (booked). Also only one- and two-dimensional histograms are possible, hence the correlations between various parameters can be difficult to study. Hence in many cases it is more appropriate to store the value of the important parameters for each event in an Ntuple. This approach preserves the correlation between the parameters and allows selection criteria to be applied on the (reduced) data sample at a later stage.
In general, the time consuming job of analysing all events available on tape is run on a mainframe or CPU server, and the important event parameters are stored in a Ntuple to allow further detailed study. For convenience the Ntuple can be output to disk for each run, and then at a later stage the Ntuples can be merged in order to allow a global interactive analysis of the complete data sample (see Figure [more info]).
A typical batch job in which data are analysed offline and some characteristics are stored in HBOOK is shown in Figure [more info]. HBOOK is initialised by a call to HLIMIT, which declares a length of 20000 words for the length of the /PAWC/ dynamic store. Then the one- and two- dimensional histograms 110 and 210 are filled respectively according to the functions HTFUN1 and HTFUN2 and the histograms are output to a newly created file HTEST.DAT. The output generated by the program is shown in Figure [more info].
PROGRAM HTEST PARAMETER (NWPAWC=20000) COMMON/PAWC/H(NWPAWC) EXTERNAL HTFUN1,HTFUN2 *.------------------------------------------------------------ CALL HLIMIT(NWPAWC) * Book histograms and declare functions CALL HBFUN1(100,'Test of HRNDM1',100,0.,1.,HTFUN1) CALL HBOOK1(110,'Filled according to HTFUN1',100,0.,1.,1000.) CALL HBFUN2(200,'Test of HRNDM2',100,0.,1.,40,0.,1.,HTFUN2) CALL HSCALE(200,0.) CALL HBOOK2(210,'Filled according to HTFUN2',100,0.,1.,40,0.,1.,30.) * Fill histograms DO 10 I=1,10000 X=HRNDM1(100) CALL HFILL(110,X,0.,1.) CALL HRNDM2(200,X,Y) CALL HFILL(210,X,Y,1.) 10 CONTINUE * Save all histograms on file HTEST.DAT CALL HRPUT(0,'HTEST.DAT','N') CALL HDELET(100) CALL HDELET(200) CALL HPRINT(0) END FUNCTION HTFUN2(X,Y) * Two-dimensional gaussian HTFUN2=HTFUN1(X)*HTFUN1(Y) RETURN END FUNCTION HTFUN1(X) * Constants for gaussians DATA C1,C2/1.,0.5/ DATA XM1,XM2/0.3,0.7/ DATA XS1,XS2/0.07,0.12/ * Calculate the gaussians A1=-0.5*((X-XM1)/XS1)**2 A2=-0.5*((X-XM2)/XS2)**2 X1=C1 X2=C2 IF(ABS(A1).GT.0.0001)X1=C1*EXP(A1) IF(ABS(A2).GT.0.0001)X2=C2*EXP(A2) * Return function value HTFUN1=X1+X2 RETURN END Figure: Writing data to HBOOK with the creation of a HBOOK RZ file
[FEX1IN] Filled according to HTFUN1 HBOOK ID = 110 DATE 02/09/89 NO = 2 340 - 330 I - 320 I I 310 I I 300 I-I- 290 --I I 280 -I I- 270 I I 260 I I 250 -I I- 240 I I 230 -I I 220 I I- 210 -I I 200 I I - 190 I I-I 180 -I I 170 I I - 160 I I - -I- - 150 I I- I --I I- -I - 140 I I- -I--I I-II-I- 130 --I I- -I I 120 I I - -I I 110 I I I-I I-- 100 I I- -I I 90 -I I- -I I---- 80 -I I --I I- 70 I I -I I 60 -I I-- - I I- - 50 -I I-- ----I-I I-I- 40 I I-I I--- 30 --I I-- 20 --I I -- 10 -------I I-II-- CHANNELS 100 0 1 10 0 1 2 3 4 5 6 7 8 9 0 1 1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890 CONTENTS 100 111222222323222211111 1111111111111111111111 10 1 12224578227034888392975189442985544344445467789101235335456543453430088887545443322111 1. 22345055038484428230601947383077660674994445157562761227948358021717653142735611669210337304276 LOW-EDGE 1. 111111111122222222223333333333444444444455555555556666666666777777777788888888889999999999 *10** 1 0 0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789 * ENTRIES = 10000 * ALL CHANNELS = 0.1000E+05 * UNDERFLOW = 0.0000E+00 * OVERFLOW = 0.0000E+00 * BIN WID = 0.1000E-01 * MEAN VALUE = 0.4846E+00 * R . M . S = 0.2199E+00 Fill according to HTFUN2 HBOOK ID = 210 DATE 02/09/89 NO = 4 CHANNELS 100 0 1 10 0 1 2 3 4 5 6 7 8 9 0 1 1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890 ******************************************************************************************************** OVE * * OVE .975 * * 40 .95 * ++ 2 2 2++ +3 + ++ + + 2+ 3 2 + 2++++ + 2 + * 39 .925 * + + 2 ++ 32+++ +22 22+ +++ + + + + 22+2+++ +2++ + + + * 38 .9 * 223 +3+ +3 3++333223 +2 2 + + ++2+ + 232+322 2+++ +24+ + * 37 .875 * + ++ +2++++ 342533 443224++2 2 + + ++23 + +42+3222233+++3+++2 22+ ++ + + + * 36 .85 * ++ + 5+35+3333483475 65+2+ + ++ + +33+3 +2 +2335222+235 522 24+ ++ 2 * 35 .825 * ++ 2+2 558335876736583+ 2 +2+ + + 3 224+533623+35252+54 32+452++3 332 +++++ * 34 .8 * ++ + 532 656562546C8A88936324332+ +2+23 +332+2236433657234455556+4635+222 +23 +3 + * 33 .775 * +2 33 375B7274C6A66A782+323++2+23 +5++3+5222256768365258276374+86334+ 32 +++ + * 32 .75 * + 2+ 2 45523786A79FB98B6AD4855224+ + ++23323+5755552468283746644543 443324 5223++ 2 * 31 .725 * + ++4+22+637A785B8BBBA6B4656922++ 2 23 24 2+5464+435552843286C6246623636+3+ 2 3 2 3+2 * 30 .7 * + 22 +2 735ABCA89G8C8A6DA5765+3+322 2+2++52234445475+355864768724+B74632+23 +3 3+ + * 29 .675 * 23 +4+3364HBBAFCFCBB98945C7933++ 2 5+3 +4225243752 75787896C367+475443+32242422 2 + * 28 .65 * + + ++5+3795498GAC96CB9A79E6645 34 3+3 ++24537234424532777657445+4746235+2+3++ 4+2 2 * 27 .625 * + 3 647774A9CE67G99BAB6B233233 4+ 2 322 42 44364+657735+735736733+4+23234 +++++2 + * 26 .6 * + ++3+342233874B8C966896565+5242+5 +2+++++2+5225+42544535456A265357253+2222+ 2+2++ + +2 * 25 .575 * ++ + +5 74535525677984573453422 +2 ++ 2 +++4+2 3526525235+4243342+32+ 23 2+ * 24 .55 * ++ +226+584568349865+433 +2222 + ++ +4444352326542332823+444332 +2 2 + + * 23 .525 * ++++2+65436+3A753535+22+++2+++ ++ + ++2 +2 ++4++2+ 224224+32 2+ ++++ 2 + * 22 .5 * 22 4+23+6425 84543+++42 +2 +++2 2 + 2+2+ 3+ 24++2334223+ 223 +2 + + * 21 .475 * + +5334+7333+22 ++2+ + 3+ 2 +4 +32 2 222+2 + 33++ 222 + +3++ + * 20 .45 * + 433244397 2++23232+ 24 +2 ++ ++2+ 2+ +2+33 ++4 +3 ++2+3 + + * 19 .425 * + ++ 2+ 22+24636432646+5+322 4 +++ + 2++ ++ +22+533+3++3+ +432 +322++2+ 2+ ++ + * 18 .4 * +++3237549588A9725H724545++33+33 + + 2 24 4 +A4633 39 25636343322+82++ ++ + +2+ + * 17 .375 * +++3+374879CCCADLD48996CE54365232 +2+2342347+563264636547B47925542444434+2+322 2+ +2 * 16 .35 * +++ +4637549EC87D8IHDICI9B754655432++23233+2554368886H68B9667889677A635C+4+223333+22 + * 15 .325 * + ++++ 2445949CHHDFNHJRHIHKLDD5DC3545422233 24564875549A8E7899B4F4BC3CA7E597842+67242+++++ * 14 .3 * ++++++2667889EDFEHULQHI*IKFIFA878666336+6+48526B79777BCCEBBAEEED58E96997A4674763463++++ 2+ * 13 .275 * + ++++ 3546898BEMPNIURPH*NOECDC8958E442+3542+68554B37466AAGCEEACAC7A476599962365 343++2 +2 * 12 .25 * + 2344658A9DAJPLDENQGDHJEEBAA93 +3225322+4259A576784DA9B98B56A85CD859797A5843523223+ 22 * 11 .225 * 3 256778BA6CEJGIEAICGCHA4A242+43+++52427545466927A78866BB66795655763454656 2 3 +++ * 10 .2 * +2++4357A69BC88AAFAA5665432+434 +++ ++++343233668554584442CA7664745+4++34+++2 + +++ * 9 .175 * + 3 3436344766755264526++3 2+ + ++ +42 22 2+32345++353562 34 33+++4 +3 +++ + * 8 .15 * 2+ + +3+44+262542+4225 232 ++++ 222 + 2+ +23+242 32+222 2++342 22 22+ 2 + * 7 .125 * + +2 +++22+32+ 3+++2 + +42 + 2+ + + 2+ + + ++ * 6 .1 * + + + +2+ ++ + +2+ + ++ +++ + * 5 .075 * + 2 + + + + * 4 .05 * + * 3 .025 * + * 2 * * 1 UND * * UND ******************************************************************************************************** LOW-EDGE 0 0000000000111111111122222222223333333333444444444455555555556666666666777777777788888888889999999999 0 0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789 * I I * ENTRIES = 10000 PLOT ---------I---------I--------- * SATURATION AT= 31 I 9991 I * SCALE .,+,2,3,.,., A,B, STATISTICS ---------I---------I--------- * STEP = 1 * MINIMUM=0 I I
Figure: Output generated by job HTEST
A second run using program HTEST1 below shows how to add some data to the HBOOK RZ file created in the job HTEST (Figure [more info]). After opening the file HTEST.DAT, created in the previous run, in update mode ('U' option) with the name EXAM2, a new directory NTUPLE is created, known as //EXAM2/NTUPLE as seen in the output of HLDIR command at the end of the output. One-dimensional (10) and two-dimensional (20) histograms and an Ntuple (30) are booked. Each Ntuple element or ``event'' is characterised by three variables (labelled 'X', 'Y' and 'Z'). The Ntuple data, when the initial size of 1000 words is exhausted, will be written to the directory on disk specified in the call to HBOOKN, i.e. //EXAM2/NTUPLE, and the data in memory are replaced with those newly read. A one- and a two-dimensional projection of X and X/Y are then made onto histograms 10 and 20 respectively, before they are printed and written on the HBOOK RZ file. At the end the current and parent directories are listed. The contents of the latter shows that the data written in the first job (HTEST) are indeed still present in the file under the top directory //EXAM2. The call to RZSTAT shows usage statistics about the RZ file.
Example of adding data to a HBOOK RZ file
PROGRAM HTEST1
PARAMETER (NWPAWC=20000)
COMMON/PAWC/H(NWPAWC)
DIMENSION X(3)
CHARACTER*8 CHTAGS(3)
DATA CHTAGS/' X ',' Y ',' Z '/
*.----------------------------------------------------
CALL HLIMIT(NWPAWC)
* Reopen data base
LRECL = 0
CALL HROPEN(1,'EXAM2','HTEST.DAT','U',LRECL,ISTAT)
CALL HMDIR('NTUPLE','S')
CALL HBOOK1(10,'TEST1',100,-3.,3.,0.)
CALL HBOOK2(20,'TEST2',30,-3.,3.,30,-3.,3.,250.)
CALL HBOOKN(30,'N-TUPLE',3,'//EXAM2/NTUPLE',
+ 1000,CHTAGS)
*
DO 10 I=1,10000
CALL RANNOR(A,B)
X(1)=A
X(2)=B
X(3)=A*A+B*B
CALL HFN(30,X)
10 CONTINUE
*
CALL HPROJ1(10,30,0,0,1,999999,1)
CALL HPROJ2(20,30,0,0,1,999999,1,2)
CALL HPRINT(0)
CALL HROUT(0,ICYCLE,' ')
CALL HLDIR(' ',' ')
CALL HCDIR('\',' ')
CALL HLDIR(' ',' ')
CALL RZSTAT(' ',999,' ')
CALL HREND('EXAM2')
END
TEST1 HBOOK ID = 10 DATE 02/09/89 NO = 1 280 270 - - 260 I I - 250 - I I I 240 - I I-I- I - 230 I-I--I I I-I- 220 -I I I I- 210 I I I I- 200 I I-I I- 190 - - --I I -- 180 I-I-I I-II-- 170 I I 160 I I-- 150 - -I I -- 140 -I-I I II 130 -I I-II- 120 -I I- 110 --I I-- 100 --I I 90 I I 80 I I---- 70 --I I- 60 -I I-- 50 ---I I-- 40 -----I I-- 30 I I----- 20 - ----I I--- 10 --------I-I I-------- CHANNELS 100 0 1 10 0 1 2 3 4 5 6 7 8 9 0 1 1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890 CONTENTS 100 11111111111111122222222221222222111111111111111 10 1 1111333334446669000123434878888132522637496233109788775524421007777655443322222111 1. 1266487877127932587516069303434644322909949809367004036056844525243975324963516782565365312194856211 LOW-EDGE -------------------------------------------------- 1. 3222222222222222211111111111111111 111111111111111112222222222222222 0 0988776554432211099887665543322100998776654433211000112334456677899001223345566788990112234455677889 0 0482604826048260482604826048260482604826048260482606284062840628406284062840628406284062840628406284 * ENTRIES = 10000 * ALL CHANNELS = 0.9969E+04 * UNDERFLOW = 0.1200E+02 * OVERFLOW = 0.1900E+02 * BIN WID = 0.6000E-01 * MEAN VALUE =-0.3907E-02 * R . M . S = 0.9857E+00 TEST2 HBOOK ID = 20 DATE 02/09/89 NO = 2 CHANNELS 10 U 0 1 2 3 O 1 N 123456789012345678901234567890 V ************************************** OVE * + ++ +232++2+ +++ * OVE 2.8 * ++ 2 +2 + 2 + * 30 2.6 * 2 2+ +34+++ ++ + * 29 2.4 * 2+ 3322343+ 3++ + * 28 2.2 * + 2 247236663524+23++ + * 27 2 * + 2+23769597A75 6+2+ 2 * 26 1.8 * + 5598576EBCDAA53357 2+ + * 25 1.6 * ++3278CC9JFO8F98C86643+2+ * 24 1.4 * 344686AAGJJMEMIDFG964232+ + * 23 1.2 * ++++44BBJGMQOPWNICCGI97322++ + * 22 1 * 2+545BGOMTSX*VYTJMCFA755++2 * 21 .8 * 2+4799DHSRUX****VXRQJC57635+ * 20 .6 * + +25CBEKLZ********MXGGCI4322 3 * 19 .4 * 2 4+779BN*U*********YOIFB862 * 18 .2 * 2 ++266CCLR************OIHA464+2 4 * 17 * + 3238ECX*T***********YKPC772 + * 16 - .2 * + +423D6LDS**X********ZUMGC543+ 2 * 15 - .4 * + 2347CAHSSX*********UMK75D2 3 + * 14 - .6 * 2334AAKML*V**********IIH9773++ + * 13 - .8 * +22565CLJL*X******Z*TL9H948+ + * 12 - 1 * 2 2 32666EMLN****Q*ULLQMABB342+ 2 * 11 - 1.2 * + 22377BDIUS*P***TTUNBDA545+2 * 10 - 1.4 * + + 2 +689E7KKNWUNRIHJCEA472+++ + * 9 - 1.6 * 2+3+74BCMJIGOIKEIAAD6643++ 2 * 8 - 1.8 * + + +2222856AA8HGJACB6786+2+2++ * 7 - 2 * + 2 +273598EDC5977634++ * 6 - 2.2 * + + ++2+274977548883+++2 +++ * 5 - 2.4 * + +3367558445+442+ + * 4 - 2.6 * +2 + 2224+6++7234 + + * 3 - 2.8 * + 33+3+322++ + * 2 - 3 * ++ ++ 22 2 +4+2 2 * 1 UND * + + 23 +2+++ + * UND ************************************** LOW-EDGE --------------- 1. 32222211111 1111122222 0 086420864208642024680246802468 * I 19 I * ENTRIES = 10000 PLOT -------I--------I------- * SATURATION AT= 255 12 I 9936 I 19 * SCALE .,+,2,3,.,., A,B, STATISTICS -------I--------I------- * STEP = 1 * MINIMUM=0 I 14 I ******************************************************** * NTUPLE ID= 30 ENTRIES= 10000 N-TUPLE * ******************************************************** * Var numb * Name * Lower * Upper * ******************************************************** * 1 * X * -.422027E+01 * 0.386411E+01 * * 2 * Y * -.411076E+01 * 0.378366E+01 * * 3 * Z * 0.485187E-04 * 0.179518E+02 * ******************************************************** ===> Directory : //EXAM2/NTUPLE 30 (N) N-TUPLE 10 (1) TEST1 20 (2) TEST2 ===> Directory : //EXAM2 100 (1) Test of HRNDM1 110 (1) Filled according to HTFUN1 200 (2) Test of HRNDM2 210 (2) Fill according to HTFUN2 NREC NWORDS QUOTA(%) FILE(%) DIR. NAME 34 34066 .21 .21 //EXAM2/NTUPLE 41 40438 .26 .26 //EXAM2 Figure: Adding data to a HBOOK RZ file