RCE Shelf. ATCA Backplane. â¢Advanced. Telecommunications. Computing Architecture. â¢Commercial product. â¢Dual Star or Full Mesh data interconnect.
Flexible, High-Velocity Data Movement and Manipulation Systems for National Security Applications R. Herbst, R. Claus, M. Freytag, G. Haller, M. Huffer, C. Kenney, G. Blaj, S. Maldonado, K. Nishimura, C. O’Grady, J. Panetta, A. Perazzo, B. Reese, L. Ruckman, J. G. Thayer, M. Weaver
Introduction National and homeland security:
• • •
•
Scalable, High Performance Architecture 2 x 10Gbps DPM 0
multiple fuzzy data streams need for next-generation DAQ:
DTM
DTM
Ethernet Switch DPM 2
DPM 0
Data Acquisition Systems:
DPM 3
DPM 1
COB
DPM 2
DPM 3
DPM 3
COB DTM
COB
DPM 1
24
DPM 0
Ethernet Switch
DTM
Ethernet Switch
Multiple information streams
Ultra-high bandwidth
DPM 1
DPM 1
DPM Board 1 (2 x RCE)
DPM 2
DPM 0
DPM Board 2 (2 x RCE)
RTM
Ethernet Switch
COB
DPM 3
DPM Board 0 (2 x RCE)
DPM Board 3 (2 x RCE)
DPM 2
Off shelf link
Fulcrum Ethernet Switch
PCIe
Timing Dist.
10Gbps
IPMB 1Gbps Power & Reset
Switch Control & Timing Dist. Board DTM (1 x RCE)
Debug Bus
Payload Ethernet
ATCA Back Plane
Backplane Timing 1 Gbps Base Ethernet (2)
Data Acquisition Real time analysis
•
Storage
Scalable and integrated:
Mobile platforms
Local area integration
Global streams
Developed by SLAC National Accelerator Laboratory Widely deployed in mission critical environments:
• •
ATLAS Large Hadron Collider
LCLS Free-Electron Laser
Heavy Photon Search
CERN
SLAC
Jefferson Lab
Cluster
Widely Deployed in Demanding Applications
RCE Shelf ATCA Backplane •Advanced Telecommunications Computing Architecture •Commercial product •Dual Star or Full Mesh data interconnect •10 (40) Gbps Ethernet
14 COB Boards (112 RCEs) •
• Long Baseline Neutrino Facility
Large Synoptic Survey Telescope
Fermi Lab
LSST
Multiple shelves:
Conclusions
• Connected through 10 Gbps Ethernet switches into Clusters • Any 2 RCAs on different shelves: interconnected trough maximum 5 switches
•
Large quantity of fuzzy information Acquired and analyzed rapidly Integrating streams from moving platforms Context awareness
• RCE architecture: meets the above requirements
RCE Cluster
• Experience in real time acquisition and analysis of massive data streams • Example - locating SNM in urban areas requires: • • •
Low latency cluster (Any 2 COBs: direct 10 Gbps link) Any 2 RCAs: interconnected through maximum 2 switches
Cluster on Board
Data Processing Module
Reconfigurable Cluster Element
2 RCEs
1 / 10 Gbps Ethernet Input
ATCA Board with 8 RCEs • 4 DPM, 1 DTM modules • 24-port, 10Gbps, Ethernet switch • Low latency < 300ns
4 Data Processing Modules •Mezzanine cards •12 high speed links: RTM (12 Gbps)
12 high speed serial links (12 Gbps) to DTM
Module
Clock, Trigger, Timing
Firmware supporting:
DTM •1 Gbps Ethernet to local switch, PCI-Express •Interface to backplane timing bus and dual 1Gbps base Ethernet •Timing and debugging to DPM
ARM dual core A-9 900 MHz Xilinx 7000 FPGA 12 high speed serial links (12 Gbps)
Each RCE directly connected to Ethernet Switch
DPM 1
•inter-firmware communication •Inter-application messaging (e.g., real time hit maps, veto)
RCE 1 RCE 2
COB 1
DPM ...
RCE Shelf
...
DPM 4
RCE 8
...
COB 14
DPM 56
RCE 112
