This morning at 5:30am the T2 cooling package expansion tank switched to WARNING LEVEL NOT OK.  There was no change in BL1A vacuum at this time.  The change in level was confirmed by inspecting the water level visually.  Operators suggested it may have happened because the cyclotron was shut off and the temperature of all water systems dropped, however T1 did not experience a noticeable change in level, so I do not think this is the cause.  It is possible there is a leak on the cooling system somewhere, but the evidence does not suggest a leak into the vacuum volume.

At 11:55am the BL1A vacuum spiked tripping off both cooling packages.  Valve 1AV8 is closed.  There is no pumping on the T1 volume, so the pressure is slowly increasing.  T2 is at approximately 50mTorr.  Operators believe it may have been caused by work at M15 or M13.  

This appears to be unrelated to the water loss at T2, but proper diagnosis is not possible because Vacuum Group staff are away for the holidays.  The cooling packages will be left off until proper diagnosis can be performed in the new year.

Pumping was restored to 1A today by Vacuum Group and the beam line returned to normal vacuum levels.  Both cooling packages were started with no change in vacuum (no water leaks inside the vacuum volume).  The T2 package was left on for approximately 1 minute, and then the expansion tank level was checked.  Approximately 1cm of water was lost.  The T2 package was turned off, and will remain off until the package can be inspected later this week after block removal.  The T1 package will be left running until T1 MRO work begins.

The T2 cooling package was visually inspected today after blocks were removed.  With the cooling package off the following leaks were observed:

- Collimator A return line ball valve: ~ 2 drops / second
- Collimator B return line ball valve: ~ 1 drop / 3 seconds

The cooling package was then started and the following leaks were observed:

- T2 heat exchanger secondary side, north end ball valve (CuALCW):  ~ 1 drop / sec
- Target water return line ball valve above reservoir: very slow leak, wet but no drips visible

These leaks explain the water found in the tunnel, and the slow drop in T2 expansion tank level.  However it is a little mysterious why four valves all failed around the same time on two different water systems..

The copper active supply to the T2 area will be valved off to stop the leaks, and the valves will be serviced or replaced during this shutdown.

Since the previous E-Log the following tasks have been completed:

- Panel spacer rods installed at 4 locations in each tank (two top, two bottom at approximately 1/3 and 2/3 the way across)
- Spacer rods expanded only slightly to begin compressing the rubber window spacers while still leaving at least a 8mm gap between window and frame
- Window and frame around seal area masked off with masking tape
- Sealant was applied from the outside using a plastic cone tip with a copper tube attachment to allow sealant injection to the bottom of the sealant cavity.  The process involved having one person inside the tank to make sure the sealant cavity was completely filled with no voids. A total of 42 of the 20oz. sausages were required.  Two spare unopened sausages remain.

The sealant will now be left for 2 weeks to cure before water is added to the tank.

The warm cell water window tanks were filled on September 17 after 2 weeks of curing time.  A water leak was found coming from the seals of a plastic pipe joining the two tanks near the top.  This was repaired using rubber gaskets and quick cure caulking.  The tank was refilled on September 18 and no longer leaked from that location.  The sand filter was backflushed with the water exhausted to the grass area outside the hot cell lab until the water ran clear.  The pump was then started with the water circulating normally through the filter.  Small water leaks were observed coming from 3 of the 4 plastic ball valves on the filter loop piping.  These leaks stopped after approximately 3 days of continuous running, and are still leak tight now.

During this repair job it was observed that the support wheels for the warm cell manipulators are crumbling and should be replaced before the next time the manipulators are used.

Upon uncovering the T1 and T2 areas to prepare for regular cooling package maintenance, it was discovered that multiple ball valves were leaking. Primarily the Swagelok SS-45S12 parts, but some SS-45S8 parts have failed as well. There were small puddles of water underneath many of the valves. It appears that the leak is due to a compromised seal. It is suspected that these parts failed from radiation exposure. They have PTFE packing material which is highly susceptible to decomposition under irradiation, and as far as we can tell they have not really been repaired/replaced (aside from a written note on one saying the seal was changed in 1997).

It is recommended to add the regular repair/replacement of these valves to the annual shutdown MRO checklist.

Once replaced, this e-log will be updated to document which ones were replaced.

5 and 3 Swagelok ball valves were replaced from the T1 and T2 cooling packages respectively.

These are compression fit valves, 1/2" or 3/4" size.

1 valve in T2 was replaced previously, bringing the total # of replaced valves in T2 to 4.

Remaining valves that need to be replaced:

• T1:
  • 1x heat exchanger valve was not replaced because the heat exchanger line was not drained prior to the operation.
• T2:
  • 2x valves for the heat exchanger line were not replaced because the heat exchanger line was not drained prior to the operation.
  • 1x M8 beam blocker cooling line was not replaced for the same reasion.

Remaining valves that may need to be replaced (verify after next leak check):

• T1:
  • ~4 miscelaneous small valves (likely 3/8") were not replaced - want to check if they actually leak beforehand.
  • 2x panel mounted valves need to be inspected for leaks.
• T2:
  
  • ~3 miscelaneous small valves (likely 3/8") were not replaced - want to check if they actually leak beforehand.
  • 3x panel mounted valves need to be inspected for leaks.
  • 1x valve from the resin flask outlet has threaded connections with plumbing tape - this valve may not be leaky.

Next week, will drain all water lines associated with the cooling package before replacing remaining valves - then perform leak check. 

 (attached hand written notes)

 Update: remaining valves replaced (see: https://elog.triumf.ca/TIS/RH-Meson+Hall/320)