Spring Cleaning and a Scary Moment for the LHC

Spring is very much in the air here in Europe with warm temperatures and sunny skies over most areas. As many of us spring clean our homes the Large Hadron Collider is also undergoing a thorough scrubbing of its pipes to make it ready for it’s first full scale physics runs. For the collider this does not involve teams of cleaning ladies with dusters and scrubbing brushes. The pipes are scoured clean by sending high intensity beams through the pipes for a period of several days.

Before the scrubbing could begin the LHC had to recover from a scheduled technical stop at the start of the month. This process can sometimes seem painfully slow to us outsiders watching the progress but on Wednesday we were given a stark reminder of why they have to be so cautious. During the build up to the scrubbing runs as they attempted to pump in a record 600 bunches to produce the high intensity needed for the scrubbing runs a signal was generated that triggered an immediate quench of the magnets in one sector, sending high currents through the high tension lines. As far as I know it was the first time such an unplanned quench had happened since the disastrous events of 2008 when the splices proved unable to take the high load leading to an explosive release of helium that shut down the collider for over a year. This time round they had the added complication of high intensity beams in the rings which had to be dumped safely. Failure to do so could have sent trains of proton bunches into sensitive areas of the collider causing damage that would have meant another long shutdown. The problem was caused by a faulty temperature sensor where some cables had been plugged in the wrong way. It was lucky that there were two identical systems functioning and the problem only existed with one of them. This made it possible to detect the fault and fortunately the protection systems all worked as designed and no harm was done. It must have been a heart stopping moment for the operators who reported that without the double protection system they could have seen “beyond repair damage”

The collider was stopped for more than a day while they checked all similar connections to ensure that no further faults were still present. The scrubbing runs have now restarted.

These runs were planned at the end of last year after they first reached high intensity with the proton beam runs. They noticed excess background events in the ATLAS detector which were found to be caused by a build-up of an electron cloud in the pipes where the beams approached the detectors. The electrons are released from residual gas molecules in the walls of the pipes. These electrons then strike back on the surface releasing a cascade of further electrons. The circulating proton beams collide with the electrons sending particles towards the detectors that have to be filtered out of the analysis. Too much background of this sort can hide the signals they are looking for.

One step taken to resolve the problem is the installation of solenoids that produce electric fields around the affected areas steering the electrons away from the pipe to avoid the cascades. But in a addition they scheduled these scrubbing runs in which high intensity proton beams are circulated at 450 GeV to tare the residual gas molecules out from the walls of the pipes. They can then be pumped out using the vacuum systems.

The process will last about a week and could see as many as 1200 bunches being circulated. The protons will not be ramped up to the full energy of 3.5 TeV at which collisions have been made but the runs will also allow the operators to see how well the injection systems work to produce the beam intensities that could be used in physics runs later this year.

The extent to which these scrubbing runs are successful will determine how much collision data can be collected this year. If they manage to remove most of the electron clouds they will progress to running with the 50ns bunch spacing that will make it possible to inject up to 1400 bunches for the physics runs. Otherwise they will stick with the 75ns spacing currently in use for physics but that would limit the number of bunches to about 900 and will mean correspondingly lower luminosities for the runs this year.

update 11-apr-2011: For the scrubbing runs they have injected 1020 bunches per beam. This is near to the maximum intensities that they can get to with 50ns spacing. Although they are not ramping up to 3.5 TeV or colliding particles at this time it is good to see that this step can already be done successfully.

5 Responses to Spring Cleaning and a Scary Moment for the LHC

  1. Bill K says:

    “… a signal was generated that triggered an immediate quench of the magnets in one sector, sending high currents through the high tension lines.”
    Actually Phil I believe the quench was not in the magnets but in one of the powering leads which connect the magnets to the power supply cables. The lead in question is made of HTS (high temperature superconductor) and operate at an intermediate temperature, the purpose of HTS being to reduce the heat load. They are mounted in cryogenic DFBs (distribution feed boxes). For some longer distance connections, the DFB is connected to the magnets using DSLs (distribution supeconducting links) which operate at around 5 K.

  2. David W. says:

    To be precise, the pins sample the voltages at various points along the HTS current leads. The cable connects to the voltage-measuring equipment. By giving a falsely low voltage drop (and thus resistance) reading, the error led to the thermal controller shutting off cooling to part of the lead. If the backup hadn’t caught this, the lead could have overheated, and the overheating would not have been detected in time. Possibly damaging the DFB, and they have no spare DFBs.

    Full details here and here. The bad readings are fairly easy to see in the measured data if one is looking for them, so they searched for other errors without unplugging and replugging a thousand-odd of cables. (Which would possibly screw something else up.)

  3. Compyblog says:


    Die Meldung als solche habe ich zwar auch im Podcast heute verstaut, aber da hatte ich nur einen Teil des Problems verstanden. Wie es aussieht, gab es am Mittwoch im LHC einen unerwarteten Quench, das heißt, dass an einer Stelle die Supraleitung für di…

  4. Carl Brannen says:

    I’m told that there are engineers over here in the Seattle area whose only job at Boeing is to make sure that no two cables have pinouts that are sufficiently compatible so as to allow them to be incorrectly mated.

%d bloggers like this: