A Two-Level Fanout System for the CDF Silicon Vertex ... - Fermilab

1 downloads 0 Views 203KB Size Report
similar functional blocks. References. 1 S. Belforte, M. Dell'Orso, S. Donati, G. Gagliardi, S. Galeotti,. P. Giannetti, et al. The CDF Trigger SVT", IEEE Trans. Nucl.
Fermilab

FERMILAB-Conf-01/292-E November 2001 1

A Two-Level Fanout System for the CDF Silicon Vertex Tracker A. Bardi, M .Bari, S. Belforte, A. Cerri, M. Dell'Orso, S. Donati, S. Galeotti, P. Giannetti, C. Magazzu, F. Morsani, G. Punzi, L. Ristori, F. Spinella, A. M. Zanetti

Abstract | The Fanout system is part of the Silicon Vertex Tracker, a new trigger processor designed to reconstruct charged particle trajectories at Level 2 of the CDF trigger, with a latency of 10 s and an event rate up to 100 kHz. The core of SVT is organized as 12 identical slices, which process in parallel the data from the 12 independent azimuthal wedges of the Silicon Vertex Detector (SVXII). Each SVT slice links the digitized pulse heights found within one SVXII wedge to the tracks reconstructed by the Level 1 fast track nder (XFT) in the corresponding 30 angular region of the Central Outer Tracker. Since the XFT tracks are transmitted to SVT as a single data stream, their distribution to the proper SVT slices requires dedicated fanout logic. The Fanout system has been implemented as a multiboard project running on a common 20 MHz clock. Track fanout is performed in two steps by one \Fanout A" and two \Fanout B" boards. The architecture, design, and implementation of this system are described.

I. Introduction

The main functional blocks of each Silicon Vertex Tracker slice are the Hit Finders, the Associative Memory system, the Hit Bu er and the Track Fitter [1] [2]. Every time an event is accepted by the Level 1 trigger, the digitized pulse heights in the Silicon Vertex Detector [3] are sent to the Hit Finders which calculate hit positions. The hits found by the Hit Finders and the raw tracks found in the Central Outer tracker by the Level 1 Fast Track nder [4] are then fed to both the Associative Memory system and to the Hit Bu er [5]. The Associative Memory system performs pattern recognition by selecting for further processing the combinations of XFT tracks and SVXII hits that represent good track candidates. This is done by comparing the input data with a stored set of patterns in a completely parallel way, using a dedicated custom VLSI chip (AM chip [6]). The AM system outputs a list of Roads. Each Road is de ned as a combination of SuperStrips on ve di erent detector layers that can be traversed by a single track. The SuperStrips used to de ne the Roads correspond to the hit positions on four silicon layers, while the fth SuperStrip is a function of the track parameters reconstructed by the XFT (track curvature and azimuthal Manuscript received on November 4, 2000. Corresponding author S. Donati is with INFN-Pisa, Via Livornese 1291, 56010 San Piero a Grado, Pisa, Italy (phone +39 050 880239, email [email protected]). M. Dell'Orso, Dipartimento di Fisica, Universita di Pisa, Piazza Torricelli 2, 56100 Pisa, Italy A. Bardi, S. Belforte, S. Galeotti, P. Giannetti, C. Magazzu, F. Morsani, L. Ristori, F. Spinella, INFN Pisa, Via Livornese 1291, 56010 S.Piero A Grado (PI), Italy A. Cerri, G. Punzi, Scuola Normale Superiore, 56100 Pisa, Italy M. Bari, A. M. Zanetti, INFN Trieste, Area di Ricerca - Padriciano 99, 34012 Trieste, Italy

angle measured at the sixth superlayer of the COT at 106 cm from the beamline). To reduce the amount of required memory this pattern recognition process is carried out at a coarser resolution than the full available resolution. The Roads found by the AM system are sent to the Hit Bu er, that retrieves the original full-resolution silicon hit coordinates and the XFT track associated to each Road and delivers them to the Track Fitter system for full-precision computation of track parameters. The Fanout system has two main functions: performing the distribution of the XFT tracks to the proper SVT slices and mapping the two XFT track parameters to a single coordinate which is binned in SuperStrips and is used by the AM system. Simulation studies have shown that the most ecient use of the Associative Memory is obtained by using as SuperStrip the azimuthal angle of the XFT track extrapolated to a distance of about 10 cm from the beamline and a SuperStrip size of 25 mrad. For each XFT track received in input, the Fanout outputs a two word packet: the rst word contains the SuperStrip number used by the AM system, while the second word is a copy of the input word and is used by the Track Fitter for the fullprecision reconstruction of the track. For each event the Fanout receives also from the Global Level 1 trigger the information of which among the 64 CDF Level 1 triggers were red. This information is summarized and output from the Fanout as two bits in the special word used within the SVT system to mark the end of each event and to record diagnostic information (End Event word). II. Architecture and data format

The Fanout system is organized as a set of three (one of type \A" and two of type \B") 9U400 VME boards. Communication between the Fanout A and the two Fanout B boards takes place through a customized P3 backplane, with data owing from A to B. All the input streams to the Fanout system are received by the A board through three connectors placed on its front panel. One stream carries the XFT tracks and two carry the L1 trigger decisions. Data to the rest of the SVT are output only from the Fanout B boards. Each Fanout B has 6 output connectors on the front panel and feeds 6 SVT slices. Both the XFT data and the data output from the B boards conform to the standard SVT data format while L1 trigger data have a di erent format. SVT and XFT data occupy 25 bit words. In each word there are 21 data bits (the 21 least signi cant bits), an End Packet (EP), an End Event (EE), a Data Strobe (DS ) and a Hold (HD) signal

2

(Table I). The EP bit speci es if the same data packet occupies only one or more words. Its value is set to 1 for the XFT tracks since each packet (track) occupies one single word. The EE bit marks the end of data corresponding to one event. The Hold and Data Strobe are used for the communication protocol among boards. The DS signal is driven by the source board and indicates when the data become valid and can be latched by the destination board. The Hold signal is driven by the destination board. When the destination board does not keep up with the incoming data rate, the Hold signal is set active and the source responds by suspending the data ow, which is resumed as soon as the Hold goes out. This protocol is uniform among all the SVT boards. The word with the EE bit set to 1 is called End Event word (EE word), its data eld contains no information related to tracks but it is used to keep track of possible errors occurred during data transmission or processing and to record global information on the event. TABLE I

SVT data word format

24 23 22 21 20...0 HD DS EE EP Data eld The data eld of the XFT word contains 12 bits specifying the azimuthal angle, 6 the curvature and 1 respectively the charge, the isolation and the pseudorapidity of the track. The data eld in the End Event word contains the Event Tag (8 bits), the Parity (PA) of the event (1 bit), the Error Flags (8 bits), the L1 Trigger information (L1T) generated by the Fanout system (2 bits) and the Level 2 Bu er number (L2B) assigned to the event by the Trigger Supervisor (2 bits) [8]. The format of the End Event word data eld is reported in Table II, where the additional EP, EE (both set to 1), DS and HD signals are omitted. The meaning of the information recorded in the End Event word relevant to the Fanout system will be discussed in Sec. II-A.1. TABLE II

End Event data field format

20...19 18...17 16...9 8 7...0 L2B L1T Error Flags PA Event Tag Two streams from the Global Level 1 trigger carry the 64 Level 1 trigger decisions to the Fanout A board. This information is transmitted using one word per stream per event. The format of data is the same in both streams and uses a 33 bit data eld with a DS signal to indicate when data become valid. No End Packet, End Event and Hold signal are present. The data eld contains 32 trigger decisions (the 32 least signi cant bits) and 1 bit of the Level 2 Bu er number (Table 3). The combination of the two L2B bits from the two streams must match the L2B number recorded in the EE word of the XFT data.

TABLE III

Global Level 1 trigger word format

33 32 31...0 DS L2B L1 Trigger decisions For each XFT track received in input, the Fanout system outputs a two word packet through the B boards to the proper SVT slices, depending on the azimuthal angle of the track. The rst word of the packet is used by the AM system while the second one by the Track Fitter. The format of the two output words is the standard SVT data format. In the word used by the AM system only 15 out of the available 21 bits of the data eld are actually used: 12 bits contain the SuperStrip number, 3 bits the Layer number (set to 5) associated to the XFT tracks. The remaining 6 bits of the data eld are spare. The word used by the Track Fitter is a copy of the XFT input word. Since in this case one data packet is logically made of two words, the EP bit is set to 0 in the rst word and to 1 in the second one. The end of the output event is marked by the EE word, which is sent simultaneously to all the SVT slices. All the 12 End Event words are identical except for the Parity bit which can change from stream to stream because di erent streams transmit di erent data sets (Sec. II-B).

A. Fanout A board

The Fanout A board performs the entire data handling within the Fanout system. It receives and processes all the input streams and constructs the output data packets as well as the output End Event word (Sec. II-A.1). The B boards perform simply the fanout function. XFT tracks and L1 trigger bits are received asynchronously and are stored on three sets (1 for the XFT tracks and 2 for the L1 Trigger bits) of two 4K18 bit FiFos each. The set of FiFos receiving XFT tracks provides an Almost Full signal that is sent back to the source meaning Hold. If the Fanout A does not keep up with the incoming data rate, the FiFos become Almost Full and the Hold signal is asserted. The signal is active while the amount of unread data in the FiFos remains above the Almost Full threshold. Although this communication protocol is uniform among all the SVT modules, no handshake is implemented with the Global Level 1 trigger. This is allowed by the fact that the Level 1 trigger information is received at a much lower rate (one word per stream per event) than the XFT tracks. The Fanout A board transmits data to the B board through a customized P3 backplane. The data stream contains one standard SVT format word (21 bit data eld plus EP, EE, DS and HD bits), 12 \enable" bits, used by the B boards to select which SVT slices (one or more) must receive the word, and 2 clock lines routed on the backplane to have each B board receiving one line (Sec. IV). When the Data Strobe bit is active, the B boards receive one word for each clock cycle, validated at the positive edge. The Hold line is driven and is common to both the B boards. It is implemented as a wired-or chain. When the Hold line

3

* C207

R47

R45

R185

R184

R192

R183

R193

C97

C236 C237 R206

R205

R204

P42

P88

P94

C265

C229

IC81

C61

C62

R244 P97

C59

R207

Vbias Fuse: 1A R208 R31

P41 R209

VCC

R32

C64

P98

P91

D1

C260

C60

P92

BYTEBLASTER CONNECTORS P223 R78

C53 C57

RS22

C254

R8

C22

IC39

C267

IC19

R176

R210

R213

C297

C43

C77

C86

C78 C79

C88 C87

C80

C41

C218 C184

R82 C219 C185

C175

C174

R10

P147

R140

C31

C82

C83

C81

C89

P61

P143 P144 R87

P120

C110

C144

R84

C143

C145 P154

P452

IC20 R85

C187

C147

C138

C137

C136

C135

C188 P125

R13

R97

C26

IC44 IC45 IC46

P443

R16

P477

R15

P124

IC93

R14

P183

R96

C28

R141 C149 C148

P480 R134

C111

P481

C11

R44

P479 P485 P182

R95

IC43

C15

IC40 C13

C100

P186 C150

C23

P126

P487

P478

R46

C16 C27 C25

C116

R93

C139

C24

R92

C112

C30

P187 C113

P196

R137

P179 C10

P439

RD

C109

C114

P195

C140

IC72

C164

C12

R86 C115

P15

C141

P194

C163

C151 R136

P193

C162

C152

P462

P438

P192 C142

C153

P466

P155

P191 P28

C161

C154

P437

P475

C160

P476

C146

C155

C268

RS45

IC75

C159

IC47

P416

P503 P505 P417

IC92

P29

P408 P415

R30

IC41 C17

IC42

R139 R133

C158

P489

C36

C29

IC94

R26

C32

C38

R3

C37

R4

P442

IC52

P407

C220

R9 C157

C156

P493

C35 P436

P5

P173

P504 P468

R138

P172

C84

IC83

C176

IC73

J8

P491

RS43

R128

R18

C173

P114 C177

P488

P409

C

P136 R215

P142 R242

IC51

R88

IC76

D

P133

P434

C14

C350 J3

R81 R217

IC82

U10

RS44

P3

R17

R216

C172

C39

R89

IC200

C242 R68

R165

R241

P414

C244

P37

IC84

C171

C178

R240

C34

C90

C85

C247

P132

R161

C1

R164 R163 P134

C170

C179

P512

RS41

P131

C169

C180

R237

U4

R63

C76

R162

C168

C181

C215

R235

R239

P435

C9

IC90

C182

R130

R231 R233

R238

P410

RS42

C8

C183

C18

R232 R234 R236 P411

IC50

C246

P54 C7

IC74

S2 R27

P413

R28

IC85 C2

R175

R174

C298

C48

C42

R172 R173

R171

R157

R169 R170

R166

C166

R194

C19

R168 R167

P540 C165

P552

C33 P412

C217

P541

P538

P553

P554

R129

R156

P544

IC89

C243 R29

C45

IC77

P533 P539

P122

IC35 R155

IC2

R58

P4

IC91

R159

R158

P537

IC37

R142

C40

P483 P492

RD

RD

P1

P10 P8

TEST CLOCK R160

C238

C44

IC36

P128 P129

C20 P293

GND

C245

C221

C222

C223

C224

P495 C186

P542

R55

U1

ST10

P145

P181

P180 P534

C249

P556

P294

IC3

RESET

*

R196

P148

C216

IC99

R197

R195

IC88

IC86

C46

IC22

RS2

P106

R198

R200

C248

C21

C47 R109

P292

IC4

RS3

P149

P130

P146

P543

IC38

TEST

P14 P151

R7 P536

IC107

R201

C269

P535

P105

P301

IC5

R66

ST2

C251 P461

C50

P57

ST1

20 MHz

C258

R21

C55

C51

R6

C49

P300 P299

ST4 ST3

R25

P555 P250

P296 P295

ST6 ST5

FS

IC78

P247

P244 P342

IC6

RS4

IC106

ST8

F02A F13A

J5

P341 P9

P297 RS6

ST7

F01A F12A

F03A

OSCILLATOR

C58

C259

C54

R99 C52

IC78

P509

C108

C257

R103 IC79

RS21

C250

EPC2 EEPROM P207

P245

RS5

F10A F00A F11A

R199

P211 P175

P246

R5 P248 P117

P152

C107

P209

C126

P240

P249

IC7

C106

RS20

C253

ST9

P212

C125

C127

P116

R212

R102

6

IC79

C124

J10

C255

C128

P338 P118

IC100

0

P213 C129

P334 P237

P298

P104

P103

J6

C130

IC23

IC8

J2

IC98

P55

1

P208

RS8

RS7

R211

RS27 5

P243 P340

C56

P273

IC1

EPC2 EEPROM

C123

P7

R108 P272

P256

P242

C131

C256

RS1

C132

P339

P96 R214

4

R22

P43

S3

C252

IC9

P352

C122

RS9 U13

C133

RD

C121

P221 P353

GROUND

all switches to 0

GEOADD GEOADD GEOADD GEOADD GEOADD

J4

C120

P241 P322

R56 U6

K

P89

VIPA crates:set

S1

direct

C66 C134

IC11

P90

3

C213R106R107C214

2

FLEX10K P257 P258

R67

ON

C117

IC29 P336

P93

P113

1

C118 C119

P115

P219 C65

J0

C262 C261

R243

P38

P6

R76

P365

P323

fuse!

F1 P35

P326

IC12

P95

VCC Fuse: 10A !mount only one

P20

C296

P348

IC13

Z

P102

P445

P266

R77

IC27

RS12

B A

C104

C96 C94

R36

C92

R202 R203

C74

C71

C95

C73

C70

C234

C98

R38

R49

R53 R64

R23 C72

C75

C99

P393 R127 C198

F4

R65

C228

R220 R219 R218 C197

R1

P331

RS11

Z

P60 IC95

P157

C203

C196

C199

R2 C63

F3

C206

R54

R222 R221

C69

P161

P33 P39

P455

P460

P32

IC103

P22 P101 P86

F2

R62

R191

R182 IC31

IC58

C263

P40

C91

IC201

C204 C266

R34

C93

P459

P31 P356

P23

P83

P99 P100 P156

IC80

P85 P456

C195

C202 C200

P123

RS13

B A

C231

P84

P30

C6

IC24

C201 P119

P349P71

IC14

C

C233

P171

P19 P330 P328

R111 P325 P58

D

P21

P81

P448

R37

C194

C4

C68

P48

P50

P158

C193

C3

P357

C67

P80

C105

C264

P329

IC104

C235

IC68

R61

IC60 IC54

C5

C192

R39

P404

IC102

P109 P79 P24

P355

P327

IC15

P110

P108

P25 P27

P82 P508 P354

IC28

IC16

P18

P26 P507

P383

R41

R51 R60

IC69

R12 C191

C103

C190

IC30

P351

P324

R19

C232

C189

R223 R110

RS14

Z

P46

R33

P34

GND

S/N

P392

R125

RS10

RS15

B A

R20

P67 R35

august 2000

C230

C226

C225

C227 R112

P402

IC17

P47

P78

R226 R225 R224

RS16

P45

C102

P69

A.BARDI

P394

R229 R228 R227

P347

RS17

C

C101

R181

R189

R180

R188

P112

design:S.DONATI

P121

R230

R24

IC10

D IC101

IC105

INFN PISA

P406

P384 P386

IC18

R179

P2

R187

IC34

P385

R48

C205

RS18

R178

P388

CDF-SVT

P16

U12

J11

There are two identical Fanout B boards whose identity is set by an onboard switch. Their function is to receive the data from the Fanout A board through the P3 backplane and to distribute the 23 data bits to the proper SVT wedges according to the list of enable bits. They have also the

P401

IC33

P391

RS46

B. Fanout B board

IC32

P387

R186

IC53

C212

P53

C208

IC56

P390

R11

U5

P107

XTF-A V1.0 P389

P403

R50

P111 P52

IC59

IC67

R52

C210

R42

R59

C209

C211

R190

RD

U2

R57

A.1 Error handling and data ow monitoring There is a number of error conditions that can be detected by SVT while processing data, for example something can go wrong in the data transfer and the input FiFo on one board can become full, or some of the data received in input by one board are outside the valid range. The system can both set error ags in the VME register of the single board and propagate error ags in the data stream setting appropriate error bits on the End Event word [7]. The Fanout A can detect 4 out of the 8 possible SVT error conditions: the Parity Error (if the parity of the input XFT data does not match the Parity bit of the XFT End Event word), the Lost Synch (if the L2B of the XFT End Event word does not match the L2B of the L1 Trigger words), the FiFo Over ow (if one input FiFo gets full) and the Invalid Data (if the parameters of one XFT track are outside the valid address range of the Look Up Table). When the A board detects one of these conditions, it sets the corresponding error ag in the output End Event word. If one error bit is already asserted in the input EE word, it is propagated in the output EE word. The SVT boards have also a system for data ow monitoring. The input and the output data streams from the boards are continuously copied to circular memories called Spy Bu ers. In the Fanout A board these memories are 128K static RAMs for the XFT data and 64K both for the L1 Trigger data and for the output Spy Bu er, which spies data owing to the P3 connector. These bu ers act as built-in logic state analizers and help system monitoring and diagnostics. As a consequence of error conditions these bu ers can be frozen and read from VME with no interruption of the normal data ow.

RD J1

RD

IC57

is active, the Fanout A stops reading new data from the FiFos. The SuperStrip number (12 bits) inserted in the word used by the AM system and the list of 12 enable bits associated to each XFT track are determined using a Look Up Table. The Look Up Table is implemented as a set of 6 (512K8 bit) static RAMs where 20 bits summarizing the XFT track parameters are used as an address and the SuperStrip number and the list of enables are stored in each location. To obtain a 1M24 bit Look Up table, the 6 memories are organized as two blocks of 3 units, with bit 20 of the XFT parameters actually used as a chip select to enable either the rst or the second block of RAMs. The 2 L1T bits are generated by the A board as a programmable combinatorial function of the 64 Level 1 Trigger decisions. The logic of the board is implemented on seven 240 pin FLEX10K20 FPGAs from ALTERA. The rmware is fully recon gurable on board.

SILKSCREEN TOP LAYER

Fig. 1. PCB drawing of the Fanout A board. Left side is front panel side, right side is VME connectors side.

function to set the Parity bit in the End Event words sent to the 12 outputs. The Parity of each stream is computed by the A board and is transmitted to the B boards: when the EE word is sent from A to B, the 12 enable bits have a special meaning, each of them is the parity bit of the corresponding output stream. The B boards insert each PA bit in the EE word sent to the corresponding output stream. In addition the B boards control the Hold and Data Strobe bits of each output stream. Data ow out of each Fanout B board through the 6 connectors on the front panel. If the Hold signal is received from any output, the line on the P3 backplane meaning Hold is set active and the Fanout A stops reading new data. Each Fanout B receives the clock from the Fanout A board through the P3 backplane and its logical functions are implemented on one 240 pin FLEX10K20 FPGA from Altera. For simplicity this board has no Spy Bu ers and no VME interface. III. Operation modes

The Fanout system has two di erent operation modes: running mode and VME mode. In running mode it processes the data received through the input connectors and outputs the proper data to each SVT wedge. The VME mode is used to write the content of the Look Up Table used to map the XFT tracks into SuperStrips and for testing purposes. In particular in VME mode the content of any Spy Bu er can be written from the VME interface and the board can be forced to process these data as if they were the actual input streams. In this operating mode data contained in the input FiFos are disregarded. This can be done for both the input streams simultaneously as well as for only one. This allows an almost complete test

4

have been optimized for the maximum symmetry among similar functional blocks. References [1] S. Belforte, M. Dell'Orso, S. Donati, G. Gagliardi, S. Galeotti, P. Giannetti, et al. \The CDF Trigger SVT", IEEE Trans. Nucl. Sci., vol. 42, pp. 860 - 864, 1995. [2] W. Ashmanskas, A. Bardi, M. Bari, S. Belforte, J. Berryhill, M. Bogdan, et al. \The CDF Silicon Vertex Tracker: online precision tracking of the CDF Silicon Vertex Detector", Il Nuovo Cimento, vol. 112A, no. 11, pp. 1239 - 1243, 1999. [3] J. A. Valls, \The SVX II Silicon Vertex Detector at CDF", Nucl. Phys. B, vol. 78, pp. 311 - 314, 1999. [4] C. Ciobanu, J. Gerstenslager, J. Hoftiezer, R. Hughes, M. Johnson, P. Koehn, et al. \Online Track Processor for the CDF Upgrade", IEEE Trans. Nucl. Sci., vol. 46, pp. 933 - 937, 1999. [5] S. Belforte, M. Dell'Orso, S. Donati, G. Gagliardi, S. Galeotti, P. Giannetti, et al. \The SVT Hit Bu er", IEEE Trans. Nucl. Sci., vol. 43, pp. 1810 - 1813, 1996. [6] S.R. Amendolia, S. Galeotti, F. Morsani, D. Passuello, L. Ristori, N. Turini, \The AMchip: a Full-custom CMOS VLSI Associative Memory for Pattern Recognition", IEEE Trans. Nucl. Sci., vol. 39, pp. 795 - 797, 1992. [7] M. Bari, S. Belforte, A. Cerri, M. Dell'Orso, S. Donati, S. Galeotti, et al., \Error Handling for the CDF Silicon Vertex Tracker", these Proceedings. [8] CDF II Collaboration, \Technical Design Report", FERMILABPUB-96/390-E (1996). Fig. 2. PCB drawing of the Fanout B board. The orientation of the picture is as in Fig. 1.

of the system in absence of one or both the real input data sources. IV. Clock distribution

Clock signals are generated on the Fanout A board and are distributed to the two Fanout B boards through two dedicated lines of the P3 backplane. Each Fanout B board receives one clock line. Clock generation on the Fanout A is accomplished by using one quartz oscillator connected to one PLL based clock bu er from Quicklogic (\Roboclock"). The Roboclock produces eight outputs with 4 adjustable phase shifts with respect to the input clock, also allowing zero or negative delays. The timing of each phase is adjusted by controlling the individual programmable delays of the Roboclock, in steps of about 1 ns. Each of the three Spy Bu ers uses one Roboclock output to generate the write enable signals to the RAMs. Two Roboclock outputs are sent to the Fanout B boards and one is sent to a set of two PLL logic fanout chips which provide clock signals to the input FiFos and to the Altera FPGAs on the Fanout A. Each Fanout B board is equipped with a system of two Roboclocks which receive in parallel the clock line from the Fanout A board. One Roboclock output is used by the Altera programmable chip. Each of the six output channels from the board uses one Roboblock output to generate the Data Strobe signal on the cable. This allows adjusting the Data Strobe timing with respect to the data bits. V. PCB design

The Fanout A and B boards have been implemented as a six layers Eurocard 9U PCB. Critical lines have controlled impedance. The placement of components and routing