Boosted Top Tagging with Deep Neural Networks

Transcription

1 Boosted Top Tagging with Deep Neural Networks Jannicke Pearkes University of British Columbia, Engineering Physics Wojtek Fedorko, Alison Lister, Colin Gay Inter-Experimental Machine Learning Workshop March 22 nd, 2017

2 Overview Introduction Method Monte Carlo Samples Network architecture & training Results Preprocessing P T dependence Pileup dependence Learning what is being learnt Next Steps 2

3 Introduction Low top p T High top p T W boost W b Image: Emily Thompson b Train a deep neural network to discriminate between jets originating from top quarks and those originating from QCD background 3

4 Monte Carlo Samples Signal: Z to ttbar Background: Dijet Generated with PYTHIA v8.219 NNPDF23 LO AS 0130 QED PDF DELPHES v3.4.0 using default CMS card Jets clustered using DELPHES energy-flow objects Anti-k T jets selected with R = Trimming performed with k T algorithm and R = 0.2, p T frac = 5% Signal jets are selected where a truth top decays hadronically within ΔR= 0.75 of a large radius jet Jets are required to have η <= 2.0 Jets are subsampled to be flat in p T and signal-matched in eta Looking at jets with p T between GeV ~ 4 million signal jets and ~4 million background jets Sample divided into 80%, 10%, 10% for training, validation and testing 4

5 Examples of Jet Images Signal jet with pt =781GeV 10 0 Signal jet with pt =1480GeV 10 0 Signal jet with pt =2358GeV Translated azimuthal angle Jet pt per pixel [GeV] Translated azimuthal angle Jet pt per pixel [GeV] Translated azimuthal angle Jet pt per pixel [GeV] Translated pseudorapidity Translated pseudorapidity Translated pseudorapidity 10 4 Background jet with pt =702GeV 10 0 Background jet with pt =1370GeV 10 0 Background jet with pt =2376GeV Translated azimuthal angle Jet pt per pixel [GeV] Translated azimuthal angle Jet pt per pixel [GeV] Translated azimuthal angle Jet pt per pixel [GeV] Translated pseudorapidity Translated pseudorapidity Translated pseudorapidity 10 4 Jet images are typically very sparse roughly 5-10% pixel activation on average if using a 0.1x0.1 grid [1] [1] L. de Oliveira, M. Kagan, L. Mackey, B. Nachman, and A. Schwartzman, Jet-images -- deep learning edition, JHEP 07 (2016) 069, arxiv: [hep-ph]. 5

6 Neural Network Inputs Use sequence of jet constituents rather than image Advantages: No loss of information due to pixelization in an image Inputs are more information dense Using 120 constituents average activation is 30%-50% 6

7 Training and Network Architecture Network Type Number of layers Number of free parameters Activation function Optimizer Loss Fully connected 5, [300,150,50,10,5,1] 41,323 Rectified linear units, sigmoid on output Adam Binary Cross-Entropy Early Stopping Patience of 5 Implemented with Keras Initially planned on using an LSTM, but ended up using a fully connected network We found that performance between the LSTM and the fully connected network was very similar, but the deep networks were much faster to train (~10 times) which allowed for faster experimentation with preprocessing techniques and network architectures 7

8 Preprocessing

9 Preprocessing Large radius, R =, jets are trimmed using subjets R = 0.2 found with the k T algorithm with and p T frac = 5% Order subjets by subjet p T and jet constituent p T within each subjet We use only the 120 highest p T jet constituents Perform preprocessing using domain knowledge about the physics at hand 9

10 No Preprocessing 10 3 Jet p T = GeV Trimming only Background Rejection Trimming only AUC = 0.83 R ϵ = 50% = 8.85 R ϵ = 80% = Top Tagging Efficiency 10

11 Scale Scale p T of all jet constituents by a common factor to ensure that the constituent p T is approximately between 0 and 1 11

12 Scale 10 3 Jet p T = GeV Trimming only Scale Background Rejection Scaling AUC = R ϵ = 50% = 21.3 R ϵ = 80% = Top Tagging Efficiency 12

13 Translate Center jet about highest p T subjet in η, φ plane 13

14 Translate 10 3 Jet p T = GeV Trimming only Scale Translation Background Rejection Translation AUC = R ϵ = 50% = 33.2 R ϵ = 80% = Top Tagging Efficiency 14

15 Rotate Designed method of rotations to preserve jet mass Transform p ', η, φ into p ), p *,, p + Rotate so that second highest p T subjet is aligned with negative y-axis: Transform (p ), p *,, p + ) back to p ', η, φ 15

16 Rotate 10 3 Jet p T = GeV Trimming only Scale Translation Rotation Background Rejection Rotation AUC = R ϵ = 50% = 42.3 R ϵ = 80% = Top Tagging Efficiency 16

17 Flip Third subjet is not constrained, but can be moved to right half of plane Flip jet if average p T is in left half of plane 17

18 Flip Background Rejection Flip AUC = R ϵ = 50% = 44.3 R ϵ = 80% = 9.75 Jet p T = GeV Trimming only Scale Translation Rotation Flip Top Tagging Efficiency 18

19 Performance on Truth vs Reconstructed Jets

20 Performance after preprocessing 10 3 Jet p T = GeV DNN, truth 32,truth DNN, reco 32, reco Background Rejection Top Tagging Efficiency 20

21 Performance at 50% overall Signal Efficiency Truth Jets Reconstructed Jets Signal efficiency Signal efficiency Background rejection Jet p T [GeV] Background rejection Signal efficiency Signal efficiency Background rejection Jet p T [GeV] Background rejection AUC = R ϵ = 50% = 66 R ϵ = 80% = 13 AUC = R ϵ = 50% = 44 R ϵ = 80% =

22 Pileup

23 Performance at different levels of pileup 10 3 Jet p T = GeV No pile up Pile up = 23 Pile up = 50 Background Rejection Top Tagging Efficiency Extremely stable performance with respect to pileup 23

24 Performance at different levels of pileup Signal efficiency Background rejection Signal efficiency: No pile up Signal efficiency: Pile up = 23 Signal efficiency: Pile up = Jet p T [GeV] 24 Background rejection: No pile up Background rejection: Pile up = 23 Background rejection: Pile up = 50 p T dependence also stable with respect to pileup 10 0

25 Learning what is being learnt

26 Jet Mass Background Jets Flat p T distribution 600 < jet p T < 2500 GeV DNN output P(Jet mass [GeV] DNN output) Signal Background Jet mass [GeV] Jet mass [GeV] 26

27 Jet Mass Background Jets Flat p T distribution 600 < jet p T < 2500 GeV DNN output P(Jet mass [GeV] DNN output) Signal Background Jet mass [GeV] Jet mass [GeV] 27

28 Next Steps Short term: We plan to revisit LSTMs Thorough Bayesian hyper-parameter optimization Longer term: Both top and W tagging with deep neural networks now reasonably well-established on Monte Carlo But does it work on data? Start working towards evaluating the performance of these techniques on data Investigate effects of systematics and strategies for mitigating the impact of systematics 28

29 Thank you! 29

30 W-tagging performance on truth QCD-Aware Recursive Neural Networks for Jet Physics. Louppe, Cho, Becot, Cranmer 30

31 Zooming Parton Shower Uncertainties in Jet Substructure Analyses with Deep Neural Networks Barnard, Dawe, Dolan, Rajcic 31

32 Performance when trained and tested on different levels of pileup Signal efficiency Signal efficiency: NN trained on µ = 0 tested on µ =0 Signal efficiency: NN trained on µ = 0 tested on µ =23 Signal efficiency: NN trained on µ = 0 tested on µ =50 Background rejection: NN trained on µ = 0 tested on µ =0 Background rejection: NN trained on µ = 0 tested on µ =23 Background rejection: NN trained on µ = 0 tested on µ = Jet p T [GeV] - Examined how a neural network trained at one pileup level performs on another level of pileup - NN seems relatively robust to changes in pileup expected at the LHC in the next few years Background rejection Signal efficiency Signal efficiency Signal efficiency: NN trained on µ = 23 tested on µ =0 Signal efficiency: NN trained on µ = 23 tested on µ =23 Signal efficiency: NN trained on µ = 23 tested on µ =50 Background rejection: NN trained on µ = 23 tested on µ =0 Background rejection: NN trained on µ = 23 tested on µ =23 Background rejection: NN trained on µ = 23 tested on µ = Jet p T [GeV] Signal efficiency: NN trained on µ = 50 tested on µ =0 Signal efficiency: NN trained on µ = 50 tested on µ =23 Signal efficiency: NN trained on µ = 50 tested on µ =50 Background rejection: NN trained on µ = 50 tested on µ =0 Background rejection: NN trained on µ = 50 tested on µ =23 Background rejection: NN trained on µ = 50 tested on µ = Jet p T [GeV] Background rejection Background rejection 32

33 Jet Mass Background Jets Flat p T distribution 600 < jet p T < 2500 GeV DNN output P(Jet mass [GeV] DNN output) Signal Background Jet mass [GeV] Jet mass [GeV] 33

34 32 Background Jets Flat p T distribution 600 < jet p T < 2500 GeV Signal Background DNN output DNN output) P( 32 wta wta