analog-to-digital-converter we are nowadays often .... yield a drift time measurement free of signal ..... 1, 1980. [8] H.v.d.Schmitt, RTF/68K, Real Time Fortran77,.
IEEE Transactions on Nuclear Science, Vol. NS-34, No. 1, February 1987
182
PARALLEL DATA ANALYSIS IN A MULTICHANNEL FLASH-ADC-SYSTEM
G.Eckerlin, E.Elsen. H.v.d.Schmitt, A.Wagner, P.v.Walter, M.Zimmer
Physikalisches Irnstitut der Universitat Heidelberg Philosophenweg 12, D-6900 Heidelberg Abstract
P'arallel analysis of drift chamber signals with M68000 processors has proven to be an efficient way to deal with the trerneredous data flow generated by high speed (100 MHz) Flash-ADCs in real time. We report on the experience gained with a network of 34 processors, placed in 3 VME crates, to read out the 3072 Flash-ADC channels of the JADE Jet-Chamber at PETRA [1]. The properties of such a system are compared to more conventional readout schemes for drift chambers.
IntrQouctQnk The advent of fast analog-to-digital converters together with ever faster and less expensive microprocessors in recent years has led to a considerable change in the readout philosophy of fast detector signals. Whereas we used to see highly specialized electronics modules performing analog signal filtering, shaping and storage followed by an analog-to-digital-converter we are nowadays often in a situation where even the details of a pulse shape are treated in digital form from the very beginning of the readout. In these schemes the signal is repeatedly sampled by Flash-ADCs (FADC) in very short (WlOns) intervals. FADCs first found their application for high energy physics in small prototype detectors for study of the time development of signals. In these applications the increased amount of data usually did not pose special data handling problems. However, when used in multichannel detector systems, the amount of data generated quickly becomes prohibitive, unless it is immediately condensed to the relevant physical detector information. The latter step usually neccessitates the use of a large number of microprocessors to perform the analysis online.
Th1e bernefits of Flash-ADCs for drift chamber applications are at least twofold when both a time and a charge measurement are required. The
latter allows for the determination of the energy loss and the coordinate along the sense wire (zcoordinate). Single and double track resolution can be optimized to the intrinsic limit given by the chamber itself and are not compromised by the electronics, i.e. by a predefined integration time. At the same time the influence of the diffusion on the time and charge measurement can be largely reduced [2].
The JADE collaboration at PETRA has recently replaced the Jet-Chamber readout electronics by a 100 MHz FADC system, called DL300 [3], controlled by 32+2 microprocessors M68000. The raw data are condensed to the format of the original system: only the charges of the drift chamber signal and its time of arrival are recorded on tape. The FlashADC digitisations are generally discarded. A description of the hardware has already been given elsewhere [4]. Here we emphasize the details of the data reduction achieved by a combination of hardware zero suppression and parallel data processing. Furthermore we present details of the calibration procedure and discuss the resolutions obtained so far.
ExDerimental Setup The JADE central tracking device consists of a Jet-Chamber with 1536 wires arranged in 48 concentric layers parallel to the beam axis. The readout electronics employs a Flash-ADC for each wire end. The data compression which starts from an amount 0.75 MByte of FADC raw data is partly done in hardware (digital discriminator with low threshold) and partly in software (with detailed pulse analysis) to yield an average event length of 2KByte. Three different hardware layers can be
distinguished: The first layer houses the 32 crates of digitising electronics (Flash-ADCs followed by 256 word deep memory) with the FASTSCAN hardware processor for zero suppression (SCANNER). The OOMHz FADCs are operated with a nonlinear response function, extending the dynamic range by a factor of 4 and improving the resolution in the lower amplitude region [7]. The second layer contains 2*16 Front End
Processors (FEP) which are placed In two VME crates. Each CPU (M68000) is connected to one DL300 crate. The CPUs read the relevant data of the FADC system and perform pulse analysis in order to determine the drift time and the signal charge for each hit. The third layer, placed in a separate VME crate, contains the EVent Processor (EVP) which assembles the event record and controls the transfer to the host computer of the experiment via a CAMAC module.
0018-9499/87/0200-0182$01.00 © 1987 IEEE
183
The same VME crate houses another processor, the MOnitoring Processor (MOP), which performs checks on data passing through the EVP and distributes the programs at system startup. Optionally the MOP may request the full Flash-ADC information of selected channels for debug and performance tests of the hardware and algorithms. The system is mainly controlled by standard commands from the host computer of the experiment. Operator intervention is, however, possible both at the terminal of the EVP for debug purposes and at the terminal of the MOP for standard controls and monitoring.
Readout Steps The following
logical readout steps can be
distinguished: a) Trigser and Pulse Recording. With every bunch
b)
c)
d)
e)
f)
g)
crossing (every 4ts) the incoming drift chamber signals are digitised in 256 lOns time bins. The results are stored in FADC memories. A hardware signal immediately resets the Flash-ADCs for the next sampling, unless a valid event is indicated by the fast trigger logic of the experiment. Only for the latter case, an interrupt will be issued to the microprocessors. Hardware Hit Detection. The hardware FASTSCAN [3] for drift chamber hits in a single DL300 crate is completed within 0.5ms. If hits are detected, the data transfer c) has to be invoked. The SCANNER continues the search for new hits while the readout is progressing. Transfer of FADC Data into Local CPU Memory. The CPU attached reads the FADC data within the limits indicated by the SCANNER into local memory. This transfer dominates the pulse recording with roughly 5ms, the exact value depending on the number of hits Enabling the FADCs for the Next Trigger. As soon as the FADC samples for all crates have been read in, the DL300 system is reset for the next trigger. Pulse Shape Analysis. Valid drift chamber hits are searched for in the FADC data read and accurate time and charges are computed. Multiple hits contained in a pulse train are resolved. Data Formatting Valid hits reconstructed for one wire in both the same and in different FADC data blocks are combined to the format of a wire consisting of hit count, wire number, times and charges. Transfer of Data from FEP to Host Computer. The EVP assembles the drift chamber data into groups required by the host computer and synchronizes the data flow with external demands.
Many of these steps proceed fairly independent for the different processors. Even though the steps are strictly sequential for a given processor the time required varies appreciably between the different FEPs. Figure 1 gives a sketch of a possible timing situation for the EVP and two FEPs processing a typical event. It illustrates the main features of the parallel readout:
As soon as the EVP has completed assembly or the previous event data the FEPs often directly initiate the hardware hit detection, since a valid trigger and pulse sampling has occured before. In the given example, FEP1 is supposed to contain no hit and the FASTSCAN is terminated quickly. Its FADC crate is reset for the next trigger. FEP2 has to read, for example, the pulse data of a whole track segment, and many hits have to be transferred. As soon as the first pulses are available in the memory of FEP2, FEP1 starts the hit analysis for FEP2 since it has no own hits to process. In any case, the results of the hit analyses are stored back into the memory of the originating CPU, to facilitate the ordering of hits and assembly of the event record. After completing reading of data and before starting any hit analysis each FEP initialises its FADC crate for the next trigger. The complete reset of all crates is detected by the EVP, which signals this information to the host computer of the experiment controlling the trigger veto. The EVP also recognizes successful hit analysis for the next FEP in the readout sequence and passes the relevant information immediately on to the host computer. In an event with low track multiplicity there will be other FEPs with very few hits in the associated FADC crate, which, therefore, might have completed the hit analysis for FEP2, before FEP2 has even started analysis on any of its hits itself. Transfer of data by the EVP is usually completed shortly after the last pulse in a FEP crate has been analysed.
L.
a)
8)
7-55- -$
'"
I
I
Ib)d)f)
g)
I
EVP FE
e)
d) 55 b+ b) + c) d)
MTrigger - 1--t
Pi
(no hits)
f)
FEP2 (many hits)
Fig.l A possible timing situation showing event triggers and the steps progressing in the EVP and two FEPs with uneven "hit load". For the description of the processes see corresponding letters in the text.
More generally, our method of distributing the main task (analysis of the hits of an event) to N (=32) processors consists of subdividing the data space (raw data of an event) into parts that can be acted upon independently by the algorithm, such that they may be assigned freely to any processor for analysis. The only complication, of having each hit analysed exactly once, is solved with a scheme of self-administrating semaphores. Typical processing times required are 0.4ms/hit for the readout and 2ms/hit for the analysis, the exact values depending on the number of FADC channels with valid data and the number of pulses reconstructed.
184
The effective use of the available CPU power in an application with concurrent processes sharing the same data requires a multiprocessor bus system with high bandwidth, such as the VME System, and dual ported memory in all CPUs. In fact, the processor tasks have to be compute rather than data transfer intensive in order to avoid bus congestion and time losses due to arbitration. With 16 processors per VME crate we observe less than 52 loss in the efficiency to acquire the next pulse data at the processor with the lowest bus priority. The hit analysis task is well matched to the bandwidth of the VME bus. The advantage of using parallel hit analysis is obvious: In this case only the average hit population of 48 wires determines the analysis time per crate, while in the case where each processor only deals with data from its own FADC crate the highest hit population for a group of 48 wires is relevant for the processing time. The gain amounts in JADE to a factor of 4, leaving 6 hits on average for each processor.
Pulse Analysis
When two hits are very close in time, the charges of both hits overlap. We correct for this by fitting a standard average pulse to the measured part of the first hit and extrapolating the pulse into the region of the second hit. The extrapolated part of the fitted pulse Is attributed to the first hit and subtracted from the charge integral of the second hit. Standard 400
200
t [ions]
80)O
out within the limits indicated by the SCANNER is tested for occurences of valid drift chamber hits. Noise spikes with an amplitude slightly above pedestal are rejected. b) Calculation of Drift Time. The accurate value of the drift time is calculated from the pulse shape. The time values of both sides are determined independently and then averaged to yield a drift time measurement free of signal propagation delays along the wire. c) Calculation of Signal Charges. The signal charge is obtained by adding the FADC samples over 15 channels. For overlapping hits the amplitudes are extrapolated into the region of the second hit.
We use the DOS method [6] to detect valid hits and to calculate the accurate time values. The algorithm calculates a weighted center of gravity of the differentiated pulse (lifference Qf Samples).
We find the optimum resolution by using up to four bins of the distribution with weights emphasizing the rising edge. The weighting is applied relative to the bin with the largest increase. A careful choice of the weights is neccessary both to avoid timing biases at the nanosecond level and to achieve the best resolution. The charge is calculated by integrating the signal amplitude for 150ns (15 bins). Figure 2 shows that this fixed integration limit may be applied safely for most track configurations: The drift chamber signal shape deviates only little from the average pulse for extreme z-values or extreme values of the track with respect to the wire direction. Only when the track is strongly inclined to the drift isochrones in the plane perpendicular to the wire (angle a), fluctuations of the cluster ionization become important and extend the average pulse shape to longer drift times. Correction of this systematic contribution requires knowledge of the
track angle, i.e. pattern recognition. We ignore the effect and correct for it offline.
t [10ns] 0.6
