Rare heavy flavor decays at ATLAS & CMS

Rare heavy flavor decays at ATLAS & CMS A. Starodumov a a ETH, 8093 Zurich, Switzerland Two general purpose experiments, ATLAS and CMS, in addition to their main search/discovery program will perform a study of rare beauty hadron decays. An enormous beauty production at LHC, efficient triggers and precise measurements in tracker systems provide a possibility to measure branching ratios of rare B decays down to 10 9. Such a sensitivity allows us to test New Physics contributions in the rare B decays. In ATLAS and CMS a manifestation of the New Physics could be seen after the first couple of years, even before the direct observations of physics phenomena beyond the Standard Model. 1. Introduction Flavor changing neutral current decays (FCNC) are forbidden at a tree level and highly suppressed at higher order penguin or box diagrams in the Standard Model (SM). In various extensions of the SM new particles may contribute to the diagrams changing the decay properties including branching ratios, decay topologies etc. Rare heavy flavor decays discussed in this paper go via FCNC and, hence, provide an opportunity to search for the New Physics (NP) manifestations. Two general purpose experiments at LHC, ATLAS and CMS, in addition to their main search/discovery program will perform a study of rare beauty hadron decays. Such study is possible thanks to several facts. It is a high production rate of 10 12 bb pairs per low luminosity (10 33 cm 2 s 1 ) year. Low p T single and dimuon trigger selects relatively soft b-hadron decays with high efficiency. Tracker system and a vertex pixel detector allows precise reconstruction of tracks and the primary/secondary (decay) vertices. Hence, mass, isolation and flight path cuts can (and are!) strongly suppress a background rate. Rare heavy flavor decays of ATLAS and CMS interest can be split in two groups. First one is decays with a di-muon in the final state: B(Λ b ) (K ±, φ, K, γ)µ + µ. Second group is radiative On leave from ITEP, Moscow, Russia B decays: B (K, φ)γ with a semileptonic decay of the second B hadron in the event. In Section 2 a trigger strategy only for the di-muon final state is discussed. The rest of the paper is organized as follows. In Section 3 two examples of the forward-backward asymmetry A F B measurement illustrate a potential to distinguish between the SM and NP. A study of the radiative decays are briefly described in Section 4. And finally, an expected limit on the branching ratio of the Bs 0 µ + µ decay (Section 5) is presented. 2. Trigger The main efforts ATLAS and CMS will make in the study of the rare heavy flavor decays with the di-muon in the final state. Only this trigger has a high selection efficiency for soft bb events (see, for example, Level 1 trigger menu in [1]). The transverse momentum threshold of such trigger slightly depends on the experiment and running luminosity, but nevertheless stays the lowest possible one with p T > 3 6 GeV/c. The high level trigger (HLT) strategy is similar for both experiments. First, one confirms a presence of the trigger muons. It is done by reconstructing tracks in the Region of Interest around these muons and matching them with the trigger muons. In CMS a further p T cut is applied on the triggered muons: p T 4 GeV/c. Then, primary and secondary vertices are reconstructed. Cuts on the vertex quality χ 2 20 and on the flight 1

2 A. Starodumov path of Bs 0 candidates L xy 200µm (ATLAS) and L 3D 150µm (CMS) are applied. Other selections are channel dependent. The main cuts are the invariant mass of the two muons, the isolation of B hadron candidates, the opening (two muons) and pointing (the B hadron candidate momentum and flight path direction) angles etc. The HLT rate is less than 1.7Hz for CMS. Detail description of the trigger algorithms one can find in [2,3]. 3. Forward-backward asymmetry in semimuonic rare decays As it was mentioned above the forwardbackward asymmetry A F B in some decays can have different values and shape in the SM and NP scenarios. This asymmetry is defined as follows: A F B (s) = 1 0 dẑ d2 Γ(s, ẑ) 0 dẑ d2 Γ(s, ẑ) dsdẑ 1 dsdẑ dγ(s)/ds (1) where ẑ = cosθ - angle between µ + and Λ 0 in µ + µ frame and s = (p µ + + p µ ) 2 /MΛ 2 b q 2 /MΛ 2 b. The following decay channels: Λ b Λµ + µ and Bd 0 K µ + µ - have been investigated in details by ATLAS collaboration using the full detector simulations and reconstruction![4]. Results of this study for 3 years of LHC operation at low luminosity regime of 2 10 33 cm 2 s 1 are shown below. 3.1. Λ b Λµ + µ decays In 30 fb 1 of the integrated luminosity it is expected about 800 fully reconstruct signal events and less than 4000 background events. Such statistics allows to distinguish between the SM and the minimal super symmetric models (MSSM) at high confidence level as one can see on Figure 1. The three points with error bars represent expected measurements with mentioned above signal and background statistics. The upper set of points is the SM prediction and the lower set corresponds to one of the MSSM scenario. Figure 1. Forward-backward asymmetry for Λ b Λµ + µ. 3.2. B 0 d K µ + µ decays The branching ration and the forwardbackward asymmetry of this decay has been measure already by Belle collaboration [5]. However, the present statistics: 113.6±13.0 signal events - is still limited to make any conclusion about a presence/absence of the NP signals. At LHC in 30 fb 1 it is expected to reconstruct 2500 signal events along with less than 12000 background events. Statistical error of ATLAS data will be significantly smaller than the current Belle experimental error. This allows to find deviations from the SM or set the strong constraints on the NP. 4. Radiative decays The observation potential of the Bd 0 K γ and Bs 0 φγ decays has been studied by ATLAS Collaboration [7].The number of triggered events after one year at luminosity 2 10 33 cm 2 s 1 will be of the order of 9400 and 3200 signal events for Bd 0 K γ and Bs 0 φγ respectively. About 70% of these events can be kept after the off-line selection, with signal to noise ratio of the order of at least 10 2. Tnis allows to obtain clear radiative B decay signals after one year at low luminosity and precise branching ratio measurement.

Rare heavy flavor decays at ATLAS & CMS 3 5. B 0 s µ + µ decay The decay modes Bs 0 µ + µ have a highly suppressed rate in the SM [6] since they involve a b s(d) transition and require an internal quark annihilation within the B meson which further suppresses the decay by (f B /m B ) 2 2 10 3 relative to the electroweak penguin b dγ decay. In addition, the decays are helicity suppressed by factors of (m l /m B ) 2. To date this decay has not been observed and the current best limits from D0 Collaboration is B < 7.5 10 8 at 90% C.L. [8]. Since this process is highly suppressed in the SM, it is potentially sensitive probes of NP. In the MSSM the branching fraction for this decay can be enhanced by orders of magnitude, especially at large tan β [9]. For somewhat exotic MSSM models, e.g., with modified minimal flavor violation at large tan β [10], the branching fraction can be increased by up to four orders of magnitude. Bs 0 µ + µ decay is also allowed in specific models containing leptoquarks [11] and super symmetric (SUSY) models without R-parity [12]. 5.1. Background The main challenge in the measurement of the Bs 0 µ + µ decay rate is background suppression. The signal topology is very clean: two isolated muons with common displaced vertex and an invariant mass compatible with the Bs 0 mass. The background which can mimic signal topology are multi-source. First, non-resonant QCD events, where two high p T hadrons misidentified as muons. Second, bb events with b µ decay of both b-hadrons. And finally, rare B d, B +, and B s decays, comprising hadronic, semileptonic, and radiative decays. Some of them are resonant background, like B s K + K, Λ b pk, others have the continuum di-muon invariant mass distribution, like B s K µ + ν. In Table 1 all fully simulated by CMS backgrounds are shown (some of them have been simulated by ATLAS at a parton level what is marked by + or the explicit decay channel is given). In a previous study of CMS [13] it has been shown that bb events with two semi-muonic decays are major source of background. Recent studies [14] with current version of detectors jus- Figure 2. Background invariant mass m µµ distribution before the application any selection criteria (muon identification, in particular) for the B u µ + µ µ + ν decay channel. Signal mass peak (arbitraly normalized) of Bs 0 µ + µ is shown as well. tify this conclusion. To evaluate background from decays with hadrons in the final state misidentified as muons one needs to know the misidentification probability. A special study has been done in CMS. The following numbers has been obtained: the misidentification probability is about 0.5% for pions, 1% for kaons and 0.1% for protons. The misidentification probabilities are used as scaling weights for the background contributions, which are dominated by hadrons that have been misidentified. Background from bb µ + µ + X and QCD have a flat distribution what allows to evaluate this background from the sidebands of the experimental data. Potentially dangerous background from the two-body b hadron decays thanks to good enough mass resolution does not contribute much and is estimated at the level of about 5% with respect to the bb background. Figure 2 illustrates the dimuon invariant distributionfrom of the B u µ + µ µ + ν decay channel.

4 A. Starodumov Table 1 Event samples generated for the study of background contributions from rare decays and hadron misidentification. The visible cross section and the corresponding number of events expected in 10 fb 1 calculated using a muon misidentification probability is given. The numbers do not yet include any selection criteria. Sample Generator cuts/channels σ vis [ fb] N µid (10 fb 1 ) p µ T > 3 GeV/c, ηµ < 2.4 bb µ + µ + X p µµ T > 5 GeV/c, 0.3 < R(µµ) < 1.8 1.74E + 07 1.74 108 5 < m µµ < 6 GeV/c 2 B s decays B s K K + 2.74E + 05 274 B s π π + 9.45E + 03 3 B s K π + 3.08E + 04 16 B s K µ + ν 2.80E + 05 2.80 10 4 B s µ + µ γ (+) 1.29E + 01 130 B s µ + µ π 0 (B d µ + µ π 0 ) 3.77E + 01 377 B d decays B d π π + 8.34E + 04 21 B d π K + (+, B s K + K ) 3.74E + 05 187 B d π µ + ν (+) 1.25E + 06 6.25 10 4 B u decay B u µ + µ µ + ν (+) 2.24E + 03 2.24 10 4 B c decays B c µ + µ µ + ν (+) 2.01E + 01 201 B c J/Ψµ + ν (B u J/Ψµ + ν) 1.89E + 03 1.89 10 4 Λ b decays Λ b pπ 4.22E + 03 1 Λ b pk 1 QCD hadrons 5 < M(hh) < 6 GeV/c 2 2.24E + 11 1.12 10 8 5.2. Off-line analysis Main off-line analysis cuts are summarized in Table 2. The efficiency of the cuts depends on the order they applied. In the Table 2 CMS efficiencies are presented with ATLAS ones in brackets. One can conclude that the most powerful cuts are the pointing angle, isolation and mass cut. For CMS the mass cut efficiency for background is one oder of magnitude higher than for ATLAS since the background has been simulated in the 5 < M(hh) < 6 GeV/c 2 mass window. 5.3. Results Resulting efficiency obtained in CMS is 0.019± 0.002 stat for signal and 2.6 10 7 for main bb background. The number of events expected in 10 fb 1 is 6.1±0.6 stat ±1.5 sys (7.0±2.6) for signal and 14.1 14.1 +22.3 ( 20 ± 12) in CMS (ATLAS). The upper limit is determined as B(B 0 s µ + µ ) N(n obs, n B, n S ) ε gen ε total N Bs, where N(n obs, n B, n S ) is the number of signal candidate B 0 s µ + µ decays at the 90% C.L. estimated using the Bayesian approach of Ref. [15]. Using the above numbers the resulting upper limit on the branching fraction is B(B 0 s µ + µ ) 1.4(1.2) 10 8 at 90% C.L. for CMS and ATLAS respectively. While this upper limit is about three times above the SM expectation, it allows already stringent constraint on new physics models with the first 10.0 fb 1 of integrated luminosity. The optimization of the analysis with background events from data sidebands will allow an increased sensitivity. 6. Conclusion The CMS and ATLAS experiments have a broad program of rare heavy flavor decays to search for NP manifestations in semileptonic decay channels. Main triggers for these decays are dimuon or single muon. The study of the radiative rare B decays looks promissing as well.

Rare heavy flavor decays at ATLAS & CMS 5 Table 2 Main selection cuts and their efficiency for Signal and Background of CMS (ATLAS) Cut CMS ATLAS Signal Bckg separation, R µµ 0.3 < R µµ < 1.2 R µµ < 0.9 94% 83% α( P T, V T ) < 5.7 < 1 63% 6% L xy /σ xy > 18.0 > 11 37% 2.4% vertex χ 2 < 1.0 < 15 62% 27% p isolation, I I = T (B s) p T (B s)+ p T trk no tracks p T > 0.8 GeV/c 29%(36%) 2.3%(5%) R < 1.0, p T > 0.9 GeV/c Θ < 15 mass window m µµ = m Bs ± 100 MeV/c 2 +140 m µµ = m Bs 70 MeV/c2 99%(77%) 29%(2%) First results are planned in 2009/2010 when the integrated luminosity is expected at the level of 10 30 fb 1. Both experiments have very similar rare heavy flavor decays program. This means a possibility to make a cross check of measurements and to increase a sensitivity to NP by combining results of two experiments. New channels may appear in the program like, for example, Bs 0 µ + µ γ which is under study now. 7. Acknowledgment 10. C. Bobeth, T. Ewerth, F. Kruger and J. Urban, Phys. Rev. D 66, 074021 (2002). 11. S. Davidson, D. C. Bailey and B. A. Campbell, Z. Phys. C 61, 613 (1994). 12. D. P. Roy, Phys. Lett. B 283, 270 (1992). 13. A. Nikitenko, A. Starodumov and N. Stepanov, arxiv:hep-ph/9907256. 14. Ch. Eggel, U. Langenegger, A. Starodumov, CMS AN 2006/097 15. J. Heinrich at al, arxiv:physics/0409129. The author would like to express gratitude to all colleagues from the CMS and ATLAS collaborations for providing a necessary material used in this paper. Specially, I would like to thank Urs Langenegger, Christina Eggel from CMS and Maria Smizanska, Sergey Sivoklokov and Pavel Reznicek from ATLAS. REFERENCES 1. CMS Collaboration, CERN-LHCC-2006-021 2. N. Panikashvili [for ATLAS Collaboration], Nucl. Phys. B, Proc. Suppl. 156 (2006) 129 3. CMS Collaboration, CERN-LHCC-2002-026 4. N. Nikitin et al, Nucl. Phys. Proc. Suppl. 163 (2007) 147. 5. Belle Collaboration, hep-ex/0603018 6. A. J. Buras, Phys. Lett. B 566, 115 (2003). 7. S. Viret, F. Ohlsson and M. Smizanska, ATL- PHYS-PUB-2005-006 8. D0 note 5344-CONF (07-03-10). 9. K. S. Babu and C. F. Kolda, Phys. Rev. Lett. 84, 228 (2000).