LHC Commissioning The good, the bad the ugly

Transcription

1 LHC Commissioning The good, the bad the ugly eefact2018 Hong-Kong, September 2018 J. Wenninger / R. Giachino CERN Beams Department Operation Group / LHC On behalf of the LHC operation & commissioning teams

2 Outline Introduction 2

3 3 Installed in 26.7 km LEP tunnel Depth of m Lake of Geneva Moriond WS - La Thuile Control Room 3

4 4 Lake of Geneva LHCb CMS Moriond WS - La Thuile Control Room ALICE ATLAS 25th September

5 5 LHC ring layout Total length km, in the former LEP tunnel. 8 arcs (sectors), ~3 km each. 8 straight sections of 700 m. Beams cross in 4 points. Design energy 7 TeV obtained with superconducting magnets operating at 8.3 T. 2-in-1 magnet design with separate vacuum chambers. 2 COUPLED rings. The LHC can be operated with protons and ions (so far Pb and Xe). 5

6 The LHC experiments ATLAS and CMS are the two high luminosity experiments, L ~ cm -2 s -1. LHCb is a medium luminosity experiment, L ~ cm -2 s -1. ALICE is a low luminosity / ion experiment, L ~ cm -2 s -1. LHCb and ALICE are luminosity levelled by beam separation. TOTEM, ALFA and AFP are forward physics experiments. 6

7 LHC injector complex LHC proton path Beam 1 Beam 2 Max. P (GeV/c) Length / Circ. (m) LINAC2* Booster* PS SPS LHC *: kinetic energy 7

8 LHC main machine phases Powering tests Commissioning of every LHC circuit (~1600 in total) to nominal current. Machine check out Machine equipment testing without beam, hardware and software. Partly interleaved with powering tests & magnet training. Full equipment integration and machine operation without beam. Beam commissioning Setting up of the machine with low intensities, commissioning of equipment with beam. Beam operation Intensity ramp up and regular physics production at highest energy and intensity. 8

9 Outline Powering tests 9

10 Powering tests Powering tests of the first LHC circuits (=power converter, busbar, magnet, interlocks and quench protection) began in Segmentation of the LHC into 8 main sub-sectors allows parallel circuit commissioning and installation. Crucial integration exercise for power converters, quench protection and circuit interlock systems. Segmented into individual test campaigns for every circuit type. Sequenced tests with expert / automated validation (as of 2012). A full commissioning campaign involves ~ tests 2-3 months. Ends with magnet (~ dipole) training campaigns to nominal field. Commissioning campaigns are repeated after every shutdown or intervention on a circuit. Over the years a high level software to orchestrate and automate test campaigns was developed. 10

11 Powering tests The powering tests of the LHC super-conducting magnet system: o Predefined and approved test sequences, o Automated execution of the tests that are ready, o Test sequence blocked until tests are signed, o Tracking of results no step is missed! Test order encoding in a test sequence 1 block = 1 test 11

12 Incident! On 19 th September, 9 days after first beams, magnet interconnections became the hot topic for more than 1 year 12

13 Damage On September 19 th 2008 an electrical arc in a non-conform interconnection provoked a He pressure wave that damaged ~700 m of the LHC and polluted the beam vacuum over more than 2 km. Arcing in the interconnection Over-pressure 53 magnets had to be repaired Magnet displacement 13

14 Collateral damage Beam vacuum was affected over entire 2.7 km length of the arc. Beam Screen (BS) : The red color is characteristic of a clean copper Clean Copper surface. surface BS Contamination with some contamination with multilayer magnet (MLI insulation multi layer by super-isolation insulation) debris. BS with soot contamination. The grey color varies depending on the Contamination with sooth. thickness of the soot, from grey to dark. 60% of the chambers 20% of the chambers 14

15 Bus-bar interconnects Inspection of the magnets in the damaged sector during repair work revealed systematic QA issues on the bus-bar interconnections. As a consequence the LHC was operated at 3.5 / 4 TeV until Inspection, repair and consolidation of over high current interconnections (12 ka) required a two year long shutdown and recommissioning period (spring 2013 spring 2015). In large machines never short-circuit your Quality Control!! 15

16 LHC beam energy Energy (TeV) 7 TeV Design 5 TeV Joint problems, incident Consolidation delays Magnet de-training after installation 3.5 TeV 1.18 TeV 3.5 TeV Energy increase no quench at 3.5 TeV 4 TeV Operation Consolidation of all interconnections Operation Long Shutdown 1 (LS1) > 6.5 TeV

17 Dipole training and energy Significant de-training was observed on the 1232 LHC dipole magnets. The dipole magnets were finally trained for 6.5 TeV operation in Over 150 training quenches were required to reach e TeV. The spread in number of quenches between the sectors is due to the mixture of magnets from the 3 producers the magnets of one producer are particularly affected. After the next shutdown ( ) it is planned to push the magnets to 7 TeV, expect ~ 500 training quenches! 8 LHC sectors (~ arcs) 17

18 Outline Beam commissioning 18

19 Beam commissioning preparation The core LHC operation team with experience from LEP drove the preparation for the commissioning: Software, software, software! Commission procedures. Planning. Test, test, test! The LHC control system was put in place on other CERN machines as early as 2005 (-3 years) giving ample time for debugging. Dry tests (machine checkout) of all components including the control system started in 2007 (-1 years). First beam tests of the 2.7 km long transfer lines to the LHC took place in 2007 (beam at the door to the LHC). Preparation of the beams in the LHC injector chain. 19

20 Beam preparation August September 2008: Injection tests of up to 4 adjacent sectors. Almost all HW systems involved in tests. Essential checks for: o o o Control system. Beam instrumentation. Optics (magnetic model) and aperture. 22 nd 24 th of August 8 th 10 th of August 5 th 7 th of September Evening of August 8 th 2008: First beam in the LHC after ~25 years of design and construction. 20

21 D-day for LHC 10 th September 2008 Avoid such shows! Great if there is success, but if it fails The incident happened 9 days later! 21

22 Run 1 timeline 3.5 / 4 TeV August 2008 First Injection tests September 10, 2008 Circulating beams November 20, 2009 Beams back June 28, bunches 1380 December fb -1 4 TeV March TeV September 19, 2008 Incident March 30, 2010 First collisions at 7 TeV CM November 2010 First Lead ion run July 4, 2012 Higgs Seminar 22

23 Run 2 timeline 6.5 TeV April 10 First beam at April 25, SPS beam dump vacuum leak, limited bunch train length L > 2 x cm -2 s -1 October, 2 x design luminosity April, 16L2 beam loss events are back! Easter, First eam circulating June 3, Start of physics operation for run 2 July, Design luminosity L >10 34 cm -2 s -1 June, First signs of 16L2 beam loss events July, 150 fb -1! 23

24 Operation The LHC is operated by a stable crew of 8 physicists / engineers and 7 operators supported by some key equipment experts. Gain and share experience, Very well trained and flexible crews that know the machine and its limits. Injection Ring collimation Beam dump Re-commissioning after a winter stop is now done routinely in ~2 weeks with well defined steps. 24

25 Beam types The LHC is designed for beam with 25 ns bunch spacing, but the flexible nature its injection chain, RF system and control system allows operation of any complex pattern from a single bunch to various trains. 25 ns, 50 ns, (75 ns), 100 ns, 150 ns, 525 ns, etc. This has been essential to control election cloud effects and provide a gentle path to higher intensity. Production beam types with gradual increase in complexity helped and provided smooth increase in complexity. Year Beam type No bunches Isolated bunches (up to 40) ns spacing ns and 50 ns ns spacing, including low e-cloud variants

26 Peak performance Progressively more bunches, lower b*, smaller crossing angles etc Peak luminosity limited to cm -2 s -1 by the cryogenic cooling capacity of the low-beta quadrupoles Peak luminosity: Run 1: cm -2 s -1 Run 2: cm -2 s -1 Design luminosity: cm -2 s -1 26

27 Integrated performance Integrated luminosity: 30 fb -1 at 3.5 TeV & 4 TeV Run 1, 147 fb -1 at 6.5 TeV Run 2. Initial target of 100 fb -1 in Run 2 will be exceeded by ~50% 27

28 LHC availability Excellent improvement of availability in : Pre-cycle 2% Increased operational efficiency Enhanced system availability Faster magnet cycling strategy Availability for physics during the high luminosity production period reached ~60% Non-availability of beams from the injector complex is the largest source of LHC downtime Downtime 26% Operation 23% Physics 49% Cryogenics system availability ~95% 2016 availability 28

29 LHC optics The machine optics is reproducible from one year to the next and the beta-beating is corrected down to the % level at 6.5 TeV. Improving optics control including NL correction in low beta sections allowed a progressive reduction of b* to 30 cm (design 55 cm). Virgin machine, b* = 40 cm Beta-beating % Corrected machine, b* = 40 cm Beta-beating 2% 29

30 Collimation The performance of over 100 collimators is excellent and very stable, with inefficiencies of 0.03% for a stored energy of 320 MJ/beam. No beam induced quench from collimation losses in operation. A single setup per year is sufficient machine reproducibility. Tightening the collimation hierarchy (reduced retractions between collimators) coupled to good understanding of the machine aperture allowed to lower b* over time With only 1 alignment per year! p-p p-pb Pb-p 30

31 Machine Protection With over 300 MJ of stored energy (> 100 times Tevatron) each LHC beam has a tremendous destruction power. Rigorous design, implementation, testing and operation of the MP system ensured that so far no beam incident was recorded. An occasional quench is of course part of the life of a super-conducting machine. After any stop or intervention with important impact, MP tests and intensity ramp-ups are scheduled by the MP team. Excellent MP culture shared by all teams! 31

32 Outline The unexpected 32

33 Electron clouds At high intensity the LHC is operated in the presence of electron clouds. Since Run 2 there are differences in heat load (= electron cloud activity) in the different sectors (arcs) more than a factor 2 differences! This was not present in Run 1, it appeared in 2015 cause not understood. The high load sectors may be limiting the LHC beam intensity in Run 3. Beam intensity The 8 sectors (arcs) behave differently Heat load (W/ 100 m) 33

34 No. UFOs per hour Unidentified Falling Objects - UFOs The most credible theory for the Unidentified Falling Objects observed at the LHC are dust particles that fall into the beam and generate beam losses due to inelastic collisions with the beam. These losses can quench a superconducting magnet. Vacuum chamber UFOs cause dumps per year, mostly intercepted by beam loss monitors Loss monitor thresholds were adjusted to balance the risk of spurious dumps and the need for quench prevention & recovery (~5-8 hours). A clear conditioning has been observed along the years 34

35 16L2 During the extended winter shutdown , one LHC sector (S12) was brought to room temperature to exchange a dipole with a suspected interturn short (which was confirmed on the test bench). During the cool down an issue during the disconnection of vacuum pumps led to an air inlet (~few liters) into the cold vacuum chamber. The event and its consequences became only clear a few months later. The air condensed as ice on the vacuum chamber. In June 2017 very strange beam loss events were observed in conjunction with small UFO-like losses in one cell (16L2), eventually operation could only be sustained with a low e-cloud beam and limited beam intensity. Side effect: fewer bunches and higher pile-up, requiring levelling of the luminosity. Partial warm up of the sector to 80K in the winter stop , pumping of the N2 gas present in the cell. In 2018 the loss events are back, partial warm up was insufficient, but operation with 25 ns beams was possible better, but something left over 35

36 16L2 dynamics model The problems in 16L2 is now understood to be caused by air frozen inside the beam chamber, through the following sequence of events: frozen air beam 36

37 16L2 dynamics model The problems in 16L2 is now understood to be caused by air frozen inside the beam chamber, through the following sequence of events: A macro-particle of frozen air (N 2, O 2 ) is detached, triggered by the passage of the beam The macro-particle interacts with the beam, generating a beam loss spike, and disintegrates due to the heat deposition from the beam frozen air beam electron macro-particle Measured local beam loss pattern 37

38 16L2 dynamics model The problems in 16L2 is now understood to be caused by air frozen inside the beam chamber, through the following sequence of events: Gas from the evaporated macro-particle fills the vacuum chamber At the location of the beam a plasma is formed The fast moving plasma electrons destabilize the beam that has to be dumped due to excessive losses at collimators Measured local beam loss pattern frozen air beam electron macro-particle gas ion 38

39 16L2 dynamics model The problems in 16L2 is now understood to be caused by air frozen inside the beam chamber, through the following sequence of events: The fast moving plasma electrons destabilize the beam that has to be dumped due to excessive losses at collimators frozen air beam electron macro-particle gas ion Growing transverse beam oscillations 39

40 Outline Outlook 40

41 Outlook After the false start due to the interconnection problem, the LHC was commissioned steadily over 8 years to twice the design luminosity (@ ~ nominal intensity). Rigorous commissioning and operation were keys to the success. A carefully designed MP system, rigorous MP guidelines and a well working multi-stage collimation system enabled us to safely store over 300 MJ per beam. Impact of the beam MP on availability is modest (~%). UFOs, e-cloud are more or less under control, but they will be with us for the lifetime of the LHC. The 16L2 event highlights that in a large superconducting machine major issues can be triggered by trivial problems. 41

42 Outlook After a two year long shutdown in , the LHC will be back for Run 3 with upgraded injectors that should be able to provide roughly twice higher bunch currents (~ constant brightness) by ~2023. The projected peak luminosity of Run 3 is more than a factor 2 above the cryogenic limit, opening an area of luminosity levelled operation at the LHC! Parameter Design 2018 Run 3 Bunch population N b (10 11 p) 1.15 ~ No. bunches k 2780 ~ Emittance e (mm mrad) 3.5 ~ b* (cm) ~30 Full crossing angle (mrad) Peak luminosity (10 34 cm -2 s -1 ) 1.0 ~2.1 ~4-5 42

43 Thank you for your attention 43

44 Re-commissioning Example of re-commissioning plan in 2018, skeleton is reused every year 44

45 Injector beams The standard LHC beam with 25 ns bunch spacing is obtained in the Proton Synchrotron by splitting of 6 booster bunches into 72 bunches at extraction. Triple splitting at low energy, 2x double splitting at high energy. Emittance at injection into LHC ~ 2.8 mm. A lower emittance variant is obtained from 8 booster bunches that are first compressed and merged longitudinally into 4 bunches (Batch Compression Merging and Splitting, BCMS), followed by splitting into 48 bunches at extraction. Emittance at injection into LHC ~ 1.5 mm. BCMS (8 PSB b.) Standard (6 PSB b.) 45

46 Cryogenic system availability The cryogenics system availability reached 98.6%, 94% when external failures water, electricity are included. Feed-forward actions were essential in smoothing the thermal reactions related to electron cloud and the start of collisions. 46

47 Event pileup The peak event pile-up reaches ~70 events / crossing at the start of fills. Design ~ 25 event/crossing Special high pile-up tests were organized as preview for HL- LHC upgrade. Here an example event with pile up of ~90 in the CMS detector. 47

48 Emittance evolution Low emittance beam Emittance preservation for the small emittance beams: Injected: ~1.5 mm Start of collisions: ~2-2.5 mm More blow-up in the horizontal plane, and largest blow-up observed during ramp. Additional blow-up under investigation no apparent correlation with brightness. With collisions additional blow up of ~0.05 mm/h. Emittance blow up Injection Ramp 6.5 TeV Horizontal mm mm 0.05 mm Vertical mm 0.25 mm 0.05 mm 48

49 Luminosity levelling Under certain conditions and depending on the experiments request, it is desirable to adapt the luminosity dynamically with beams in collision luminosity levelling. Each levelling technique has its advantages and drawbacks. Levelling by beam offset Levelling by crossing angle Complexity Levelling by b* (= beam size at IP) 49

50 Luminosity levelling in 2018 All three levelling techniques were in place during regular operation. Crossing angle and b* levelling are used to enhance luminosity at lower intensity, when the pile up is reduced. b* levelling demonstrates a technique that will be central to operation in Run 3 and later for HL-LHC. Crossing angle levelling (not visible due to luminosity decay) Offset levelling b* levelling 50

51 Aperture restriction At the beginning of Run 2 an aperture restriction was localized in one arc. The object is moving from one year to the next, fortunately there is enough aperture to bump the beam around the obstacle. Clear for beam Shifted beam orbit Edge of the object Vacuum chamber 51

52 Electron clouds Evolution from the heat load normalized by the total beam intensity: Conditioning observed in 2015 continued over the first two months of 2016, Very little change in the following months, No correlation of this evolution with changes of settings and beam configuration