Efficient Implementation of High- Energy Physics Processing Using Modified Jet Reconstruction and Active Size Partitioning

Transcription

1 Efficient Implementation of High- Energy Physics Processing Using Modified Jet Reconstruction and Active Size Partitioning Tony Gregerson University of Wisconsin Tony Gregerson, U. Wisconsin, 13 April 2010 Computing at the Large Hadron Collider- 1

2 The Large Hadron Collider 100m below ground Tony Gregerson, U. Wisconsin, 13 April 2010 Computing at the Large Hadron Collider- 2

3 Compact Muon Solenoid (CMS) Experiment Tracker Collision Figure: D. Barney Tony Gregerson, U. Wisconsin, 13 April 2010 Computing at the Large Hadron Collider- 3

4 al Calorimeter Trigger Energy-based Particle Identification Designed at UW-Madison 4, Gigabit links ~300 boards 1000s of custom ASICs Reducing implementation cost is important Tony Gregerson, U. Wisconsin, 13 April 2010 Computing at the Large Hadron Collider- 4

5 Super-LHC Calorimeter Trigger Designed at UW-Madison Specifications New data set every 25 ns 4 Tb/s throughput Computation latency < 1 us 10x increased luminosity Tony Gregerson, U. Wisconsin, 13 April 2010 Computing at the Large Hadron Collider- 5

6 Mapping Calorimeter Data Physical Detector Clustering Grid 3D Cylindrical Tower Mapping 2D Cartesian Tower Mapping 4096 Towers 17-bit towers processed every 25 ns Grid is divided into small (~ 12 x 12) blocks that are processed in parallel on different chips Tony Gregerson, U. Wisconsin, 13 April 2010 Computing at the Large Hadron Collider- 6

7 Parallel Processing Cluster Finder E/P ID and Weighting Overlap Filter Jet Reconstruction Isolation ID ET sums Jet Sorter Cluster Finder E/P ID and Weighting Overlap Filter Jet Reconstruction Isolation ID ET sums egamma, IsoEgamma Tau sorter Cluster Finder E/P ID and Weighting Overlap Filter.. Jet Reconstruction Isolation ID ET sums MET,HT, SumEt calculation Cluster Finder E/P ID and Weighting Overlap Filter Jet Reconstruction Isolation ID ET sums Tony Gregerson, U. Wisconsin, 13 April 2010 Computing at the Large Hadron Collider- 7

8 Jet Reconstruction Challenges Jet Reconstruction operates on a large area (10 x 10 towers) This requires a large amount of padding elements Bandwidth limits the size of the grid in each chip (~ 12 x 12 to 15 x 15) Padding elements are a major hardware overhead Tony Gregerson, U. Wisconsin, 13 April 2010 Computing at the Large Hadron Collider- 8

9 Modified Jet Reconstruction Several ideas to reduce Jet Reconstruction cost by altering the specification Different Jet sizes (6x6 8x8) Operate on raw towers instead of filtered clusters (Old Method+) Build jets using reduced input granularity (-based Jet Finding) Reduces the positional resolution based on region size Evaluated region sizes from 1x1 (no reduction) to 4x4 Tony Gregerson, U. Wisconsin, 13 April 2010 Computing at the Large Hadron Collider- 9

10 -based Jets (2x2) 2x2 -based Jet finder produces half as many jets, but requires 1/3 the input bandwidth to Reconstruction chip Tony Gregerson, U. Wisconsin, 13 April 2010 Computing at the Large Hadron Collider- 10

11 Number of FPGAs to fit algorithm Modified Jet Reconstruction Jet Finders Compared Bandwidth per FPGA (Gbps) 6x6 Jet - 1x1 6x6 Jet - 2x2 6x6 Jet - 3x3 7x7 Jet - 1x1 8x8 Jet - 1x1 8x8 Jet - 2x2 8x8 Jet - 4x4 8x8 Jet - Old Method Tony Gregerson, U. Wisconsin, 13 April 2010 Computing at the Large Hadron Collider- 11

12 Number of FPGAs 8x8 Modified Jet Reconstruction x8 Jet Finders Bandwidth per FPGA (Gbps) 8x8 Jet - 1x1 8x8 Jet - 2x2 8x8 Jet - 4x4 8x8 Jet - Old Method 8x8 Jet - Old Method+ Tony Gregerson, U. Wisconsin, 13 April 2010 Computing at the Large Hadron Collider- 12

13 8x8 Jet Reconstruction on Commercially-Available FPGAs FPGA (Gbps) Original Original+ 1x1 2x2 4x4 V5-LX110T (48) N/A N/A V5-TX240T (240) V6-HX565T (450) FPGA used in system likely to be similar to 240T -based Jet Finder requires 2-4 times fewer FPGAs at this bandwidth 2 x 2 is almost as efficient as 4 x 4 but has 4 times the positional resolution Final choice also depends on physics needs Tony Gregerson, U. Wisconsin, 13 April 2010 Computing at the Large Hadron Collider- 13

14 Active Size Partitioning Algorithm & System partitioning algorithm designed to minimize padding overhead for very-high throughput grid-processing applications See Paper for details Tony Gregerson, U. Wisconsin, 13 April 2010 Computing at the Large Hadron Collider- 14

15 Thanks for your attention. Tony Gregerson, U. Wisconsin, 13 April 2010 Computing at the Large Hadron Collider- 15

16 Backup Slides Backup Slides Tony Gregerson, U. Wisconsin, 13 April 2010 Computing at the Large Hadron Collider- 16

17 Clustering Algorithm - Particles Electrons/Photons Spatially confined in a cluster of 2x2 trigger towers Significantly higher ECAL contribution ECAL Isolated e/γ should have low energy deposits in the surrounding area Taus η HCAL φ e/γ Confined in 2x3 Clusters Small energy leak in surrounding towers ECAL φ τ Jets η HCAL Most of the energy confined in a central core For jets over 20 GeV, the energy is included in a 8x8 region ECAL φ jet Tony Gregerson, U. Wisconsin, 13 April 2010 Computing at the Large Hadron Collider- 17

18 Clustering Algorithm I/O Input [17 bits x 4096 Towers] (3 Tb/s) Electromagnetic energy (ECAL) [8] Hadron energy (HCAL) [8] Finegrain Veto Bit (FG) [1] Output [748 bits] (30 Gb/s) 4 e/p, 4 tau, 12 jets (energy & position) Total energy (ET) Missing energy (MET) Total jet energy (HT) Missing jet energy (MHT) Tony Gregerson, U. Wisconsin, 13 April 2010 Computing at the Large Hadron Collider- 18

19 Clustering Algorithm Subsystems Particle Cluster Finder Filter out low-energy towers Forms clusters Initial energy calculations Electron/Photon ID Analyzes the energy contribution of ECAL & HCAL in a cluster Cluster Weighting Calculates Center-of-Energy position for each cluster Cluster Overlap Filter Removes overlapping towers between clusters, creating local maxima Prunes low energy clusters Tony Gregerson, U. Wisconsin, 13 April 2010 Computing at the Large Hadron Collider- 19

20 Clustering Algorithm Subsystems Particle Isolation Calculates isolation deposits around 2x2, 2x3 clusters Particle ID Characterizes clusters as isolated/non-isolated, electron/photon/tau using data from E/P ID, Weighting, Overlap Filter, and Isolation Jet Reconstruction Groups filtered clusters into jets and calculates jet energy sums Particle Sorters Sort particles and outputs the most energetic ones ET,MET,HT,MHT Calculation Calculates Et Sums, Missing Et from clusters Tony Gregerson, U. Wisconsin, 13 April 2010 Computing at the Large Hadron Collider- 20

21 Cluster Weighting Energy is weighted to give one of the depicted 16 positions inside the cluster Energy weighting Calculate D eta = E1+E3-E0-E2 Calculate D phi = E2+E3-E0-E1 Calculate Sum =E0 +E1+E2+E3 Eta coordinate gives F eta = D eta /Sum Phi coordinate gives F phi = D phi /Sum No division needed in the circuit Fraction values (one /coordinate) 0.5<F<1, 0<F<0.5-1<F<-0.5, -0.5<F< Tony Gregerson, U. Wisconsin, 13 April 2010 Computing at the Large Hadron Collider- 21

22 Cluster Weighting Logic signd eta out[0] E 0 E 1 D eta Shift<<1 Compare (>) out[1] Adders Sum E 2 E 3 D phi Shift<<1 Compare (>) out[3] out[2] Sign D phi Tony Gregerson, U. Wisconsin, 13 April 2010 Computing at the Large Hadron Collider- 22

23 Overlap Filter Central NE E SE S SW W NW N Energy Adder Energy Adder Energy Adder Energy Adder Energy Adder Energy Adder Energy Adder Energy Adder Energy Adder 11b 11b 11b 11b 11b 11b 11b 11b 11b Central < NE? Central <= E? Central <= SE? Central <= S? Central <= SW? Central <W? Central < NW? Central < N? tower bit sequence ( bits 4x9) 1bit 1bit 1bit 1bit 1bit 1bit 1bit 1bit E1 E3 E2 E4 E1 E2 E3 E4 E1 E2 E3 E4 E1 E2 E3 E4 E1 E2 E3 E4 E1 E2 E3 E4 E1 E2 E3 E4 E1 E2 E3 E4 E1 E2 E3 E4 Cluster Threshold E>X? Energy Adder E1+E2+E3+E4 Finegrain,e/γ 2 bits Central ( bit 1) Energy ( 11bits ) Figure courtesy of M. Bachtis Tony Gregerson, U. Wisconsin, 13 April 2010 Computing at the Large Hadron Collider- 23