- Both 3/4" ball valves on the T1 cooling package target water circuit were rebuilt, leak tested with house air pressure, and reinstalled

- The 3/4" ball valve on the T1 heat exchanger secondary side was replaced with a new unit

- The 3/4" needle valve on the T1 heat exchanger secondary side was rebuilt, leak tested with house air pressure, and reinstalled

- The T1 cooling package was refilled and restarted using a bypass across the inlet/outlet Hansen fittings

- The package was inspected while running and no leaks were visible (video survey taken with file stored on Isaac's PC)

- All T1 Cooling Package MRO activities are now complete

Both defeats applied to T2 cooling package thermistors were removed today (Col. B interlock was jumpered, and the target water inlet and outlet signals had been swapped).  It was confirmed that all thermistors are now wired correctly.  The readback of both target water thermistors is currently unstable.  Inspection of the Lemo connectors showed that the internal plastic parts of all connectors were damaged or missing.  If the T1/T2 controls upgrade proceeds this shutdown, then all thermistors will be replaced with thermocouples, so after a decision is made on that project (sometime in Feb 2015) these thermistors will either be repaired or replaced.

The active item discovered on January 9, 2015 outside the BL1A bootbox has been moved to the Meson Hall east hot cell for storage.  The item was analyzed by RPG and the radiation was found to be coming from activated metal contained by dried paint in the bottom of a plastic container.  The solidified paint puck was removed from the container which is not active.  The item is approximately 20microSv/hr at 0.5m, and 1.3mSv/hr on contact.

The T1-MK2 target was transferred from the beam line to the storage pit in hole #5 using the transfer flask.  A maximum field of 7.2mSv/hr was measured at 0.5 meters from the target.  A video was taken while the target was lifted from the monolith and can be found on Isaac's PC (too large to upload).

The blocks to the west and east side of the T1 monolith (level with the monolith) were swiped by Danka Krsmanovic and no contamination was found.  These blocks will therefore not be painted this shutdown.

The three active filters removed from the T1 cooling package have been labelled and stored in the area north of the BL1A blocks boot box with permission from Danka.  The bag with the filters is 100microSv/hr at 0.5m and 2mSv/hr on contact.

The following tasks have been performed at T1:

- T1 monolith painted using grey two part epoxy paint
- Video inspection taken while package running. Small leak from both valves on H.E. secondary side (video on Isaac's PC, too large to upload)
- Cooling package sampled and drained, drain valve closed
- All reservoir o-rings changed
- All Hansen fitting o-rings changed (4x #6 size, 2x #4 size)
- All three cuno filters and filter housing o-rings changed (used filters 2mSv/hr on contact, temporarily stored on blocks by T1 package)
- Failed T1 P1 (inlet) pressure transducer replaced
- Supply and return valves for T1 CuALCW closed in tunnel to allow leaking valves to be replaced

- New resin can installed at T1 (this note added oct 15 2015 - was previously undocumented)
 

The following tasks have been performed at T2:

- Video inspection taken while package running. Documented leaks mentioned in E-Log #133 (video on Isaac's PC, too large to upload)
- Cooling package sampled and drained, drain valve closed

RPG has analyzed the water samples and given approval for water from both packages to be released.

The T2 cooling package was visually inspected today after blocks were removed.  With the cooling package off the following leaks were observed:

- Collimator A return line ball valve: ~ 2 drops / second
- Collimator B return line ball valve: ~ 1 drop / 3 seconds

The cooling package was then started and the following leaks were observed:

- T2 heat exchanger secondary side, north end ball valve (CuALCW):  ~ 1 drop / sec
- Target water return line ball valve above reservoir: very slow leak, wet but no drips visible

These leaks explain the water found in the tunnel, and the slow drop in T2 expansion tank level.  However it is a little mysterious why four valves all failed around the same time on two different water systems..

The copper active supply to the T2 area will be valved off to stop the leaks, and the valves will be serviced or replaced during this shutdown.

Pumping was restored to 1A today by Vacuum Group and the beam line returned to normal vacuum levels.  Both cooling packages were started with no change in vacuum (no water leaks inside the vacuum volume).  The T2 package was left on for approximately 1 minute, and then the expansion tank level was checked.  Approximately 1cm of water was lost.  The T2 package was turned off, and will remain off until the package can be inspected later this week after block removal.  The T1 package will be left running until T1 MRO work begins.

This morning at 5:30am the T2 cooling package expansion tank switched to WARNING LEVEL NOT OK.  There was no change in BL1A vacuum at this time.  The change in level was confirmed by inspecting the water level visually.  Operators suggested it may have happened because the cyclotron was shut off and the temperature of all water systems dropped, however T1 did not experience a noticeable change in level, so I do not think this is the cause.  It is possible there is a leak on the cooling system somewhere, but the evidence does not suggest a leak into the vacuum volume.

At 11:55am the BL1A vacuum spiked tripping off both cooling packages.  Valve 1AV8 is closed.  There is no pumping on the T1 volume, so the pressure is slowly increasing.  T2 is at approximately 50mTorr.  Operators believe it may have been caused by work at M15 or M13.  

This appears to be unrelated to the water loss at T2, but proper diagnosis is not possible because Vacuum Group staff are away for the holidays.  The cooling packages will be left off until proper diagnosis can be performed in the new year.

The BL1A amplified air system has been shut off, and will remain off until required for BL1A start-up in Spring 2015 (the lower of the two units was in use, with the upper unit on standby as a spare).

 Between Nov 14 ~ Dec 9 the following activities took place:

- The leaky 10mm Be target at pos'n 3 was removed and plugged
- The loose protect monitor electrical conduit was secured in place using an aluminum shim
- David Cameron performed an electrical check on the protect and profile monitors and got the expected response from both
- The target was flushed with water, then air
- The target was pumped down using a cold trap to capture remaining moisture
- The target reached a minimum pressure of 80mTorr on the leak detector Hastings gauge
- Helium leak check revealed a leak at both of the position 1 Swagelok caps (up to 80 on 50x scale on left side, and 100 on 100x scale on right side when facing ladder with 1s spray of 5psi helium)
- The leak rate did not improve with tightening of the caps
- The caps were removed, ladder side threads inspected (no obvious damage observed), and new caps installed
- Helium leak check repeated: Pos'n 1 left side now leak tight, right side leak rate was worse (could not completely open throttle valve on leak detector)
- Torque was increased on the cap with no improvement in leak rate
- The right side cap was removed and when trying a new cap it would not spin freely
- The ladder side threads were inspected more carefully with the Nikon level:  a small dent across the first thread at approximately 2 o'clock position was seen as well as some material build-up or possible galling seen at the 1st and 2nd threads around the 4 o'clock position.
- The threads were filed to improve their profile which allowed a new cap to be installed with light resistance (still would not spin freely)
- Leak check was performed with a minimum pressure of 70mTorr reached (throttle valve fully open, roughing closed), with large response still at the right side pos'n 1 cap, No change with moderate tightening

It is suspected that the ladder side fitting sealing face is damaged causing a poor seal.  When a cap was installed and torqued this may have caused the galling or material build up observed at the 4 o'clock position (but not the dent on the 1st thread at the 2 o'clock position)

Various experts on-site will be consulted about how to best proceed before doing further work.  One option is to perform a static water test.  If the target is water leak tight then it may be used in the beam line.  If repair efforts are not successful and the target is not water leak tight then the target ladder will have to be replaced which is approximately a 2 week job.

Release of the Remote Handling Hot Cell Lab active sump water was completed today by Hua Yang.  During the procedure fields of up to 0.50 micro Sv/hr were measured around the newly installed filter housing.  After completion the filter was removed and replaced.  The filter measured 450cpm on the RM-14 monitor and ~5000cpm on the 44-2 monitor (both on contact).  RPG has been notified and the filter has been bagged, labeled, and left for pick-up by RPG.

While the 1A blocks over the T1 area were removed last week the opportunity was used to investigate an air leak on the amplified air system discovered on August 11, 2014.

The leak was found to be coming from a device related to the septum polarity switch located behind (south of) the septum power supply on the 1A blocks.  This device is not in use, so the amplified air to the device was disconnected by plugging the relevant branch of a brass 3/8" Swagelok T fitting in the area.  This stopped the leak, and the air amplifier is cycling at usual speed after being restarted.

A filter housing and filter were installed by Dan McDonald on November 14 in the piping section between the Remote Handling Hot Cell Lab active sump and the city sewer drain as shown in the attached picture.  The purpose of the new filter is to prevent pieces of active material from being released to the drain when the sump is pumped out.  If small pieces of active material enter the sump they may sink to the bottom, and not be captured in a water sample which is taken from the top of the sump.

The filter housing is a Waterite HP1034CLUR-KIT, and the installed filter is a 10" Cuno Microwynd 25micron filter.

This modification was approved by Curtis Ballard, Joe Mildenberg, and Grant Minor prior to installation.

 - On Nov 13 at ~1am the BL1A vacuum went bad.  The T1 and T2 volumes were isolated and pumped on separately, the results indicated a water leak from the T1 target.  The level of the expansion tank was slightly below normal at this time, but not enough to be conclusive.  The leak was confirmed at ~9am by starting the T1 cooling package which caused a dramatic vacuum spike.  Approximately 1" was lost from the expansion tank when the package was started, equal to approximately 2.5L of water.

- The T1-MK1 target was removed from the beam line and transported to the hot cell.  A mound of ice was visible on the target lower plate while it was being lifted.  A radiation field of 160mSv/hr was measured at 0.5m from the target during removal.  Upon inspection in the hot cell, a small but clear hole was visible on the entry window of the position 3 target cassette just slightly above and to the right of center if facing downstream.  No water was visible on the profile or protect monitors.  Approximately 1" of water was visible in the base of the T1 monolith.  The monolith was left open to air overnight, and approximately 0.5" of water was visible in the morning on Nov 14.

- Vacuum group was consulted, and given the relatively small amount of water remaining they agreed to pump the water out of the beam line rather than have Remote Handling group try to remove the water before target installation.  The T1-MK2 target was transported from Hole #3 in the storage pit to the beam line.  A field of 5mSv/hr was measured at 0.5m from the target.  All services were attached.

- An electrical check was performed on the profile and protect monitors by David Cameron from the 1A mezzanine.  Both devices appeared normal.  Plant Group then began to replace the blocks over T1.  Graham Waters updated the control software so that the correct target ladder potentiometer values are referenced.  An updated target information sheet was prepared and delivered to Operations (PDF File Attached)

Summary: Clarification of T2 water temperature readouts

Detail: This e-log entry by Doug Preddy is being made to clarify the T2 water package temperature read outs. The temperature read outs on the T2 water package have been switched. The read out for inlet temperature is actually the outlet temperature and the read out for outlet temperature is actually the inlet temperature. On Sunday Oct 19 2014 at 19:45, the T2 water outlet Temp (T2) started to climb giving warning messages at 20:05. Operators tried to decrease the BL1A current, but the temp. continued to climb and eventually tripped the beam off. Operators defined BL1A OFF, and continued to inject to BL2A only. They tried to contact I. Earle, but were unable to. They left message with D. Preddy. The T2 temp. stayed at 45 deg., operators tried to reset the water package and start it, but it tripped almost right away, and increased the temp. by 0.5 deg. Beam was left off. On Monday Oct. 20 Ron Kuramoto found a replacement thermister. It was wired in to the BOP over the south catwalk. The T2 outlet temperature immediately dropped to ambiant. From this test it was concluded that the installed thermistor on the T2 outlet water pipe had failed. The wiring was restored to the original configuration. It was decided to switch T2 Water Inlet Temp and Outlet Temp sensors at the T2 electronics rack to provide a trip on the Inlet Water temperature instead of the outlet Water Temperature. As the T2 water package runs in a closed loop, the outlet temperature signal has been swapped with the inlet temperature by Doug P. and Tony T. This will provide thermal protection (trip at 45C) in addition to the T2 Target high limit bit (trip at 60C). The water flow to this circuit is also monitored and will trip if the flow drops. This action was approved in a meeting with Doug Preddy, Ron Kuramoto, Grant Minor, Yuri Bylinski, Lia Merminga, and Bob Laxdal present on Monday Oct 20th. There is a second, redundant temperature sensor on the outlet of the target water which was installed recently as a safety upgrade to protect the heat exchanger and meet CSA pressure vessel code, see dwg TBP0902. There is a controller for this sensor with a hard-trip switch for the cyclotron, and a temperature read-back display on the Meson Hall mezzanine racks. This read-back should be monitored by operations once per shift while beam is on target to ensure that outlet water temperature is stable and at an expected value. Sensor cabling swapped at Meson Hall, south mezzanine, bay 1, B.O.P. #3. Sensor T1, cable #12102 moved to TB1, terminals 3 & 4. Sensor T2, cable #12105 moved to TB1, terminals 1 & 2. Defeat #7131 removed. Defeat# 7133 Fault report #7722   Will look into replacing the faulty Thermister in the 2015 major shutdown.

On 14Oct2014, the 1AT2 Collimator B water outlet temperature rose to 100^o C. 

On Oct 7^th, during the Mini Shutdown, the T2 cooling package was uncovered to investigate the Cu-ALCW water leak dripping into the 1A Tunnel.

The Target shield block was lifted and found that top of the monolith was dry.  Uncovering the east side of the cooling package directly above the vertical services on the south wall revealed a pin hole leak at a 1/2” 45^o copper elbow on the return line circuit of the T2 cooling package.

The cooling package was turned off followed by valving off the Cu-ALCW service lines leading up to the cooling package to let the Tunnel floor dry over night.

On Oct 8^th, the Tunnel floor was dry and Cu-ALCW service lines feeding the cooling package were drained from the Tunnel by placing a 5 gallon pail under the drain lines filling it about 1/3 full until it stopped.

Mel Good was brought in to replace the faulty elbow.  The Cu-ALCW was then turned back on with no obvious leaks.  At 16:00 approximately 1.5 hours after the repair, I went into the 1A tunnel to inspect for water leaks at the isolation valves and saw water dripping down the vertical piping on the south wall again.   

Another leak was found on the 90^o elbow just ahead of the replaced 45^o elbow. There appeared to 2-3 fine mists spraying out towards the adjacent copper elbow (look at the end of the cable tie on the following photos).

The Cu-ALCW lines leading up to cooling package were again valved off to let the Tunnel floor dry up over night.

On Oct 9^th, the Cu-ALCW lines leading up to the cooling package were again drained followed by replacing the faulty 90^o elbow. The elbow was fitted with new pieces to improve the configuration. We noticed that old copper pieces were worn thin. These damaged parts measured 20,000 c/m on the 44-2 and 40 c/m on the RM-14 monitors and were taken to the RH H/cell lab for inspection.

The T2 cooling package turned on ok about 1.5 hours after repairing the 2^nd water leak.

Mel Good received 0.03mSv over the two day repair. 

I received 0.10 mSv over three days, picking up 0.05mSv on Oct 7^th in the Tunnel where the fields were 200-300uSv/Hr by the isolation valves.  The next the day levels dropped to 40uSv/Hr.  There was no contamination found on the Tunnel floor, however 80,000 c/m on the RM14  and 50,000 c/m on the 44-2 monitors were found on the swipes taken at the bottom edge of the concrete shield blocks.

NOTE: the Cu-ALCW service lines leading up to the T2 cooling package feeds the following 4 devices; Collimator A, Collimator B, the T2 monolith cooling and the Target water heat exchanger.

On Sep. 3^rd during the 6 hr maintenance day, Doug Preddy & I went into the 1A tunnel under “controlled access” to investigate a water leak that triggered the Water Detection Sensors.  It was also noticed that the Cu-ALCW tank level was losing water.

Water was found to be leaking down the south wall chase where the T2 water package goes up the wall. Drip rate was ~1 drip every 15 seconds.

In the late afternoon of Sep. 9^th;

Day one of the 1.5 day maintenance, the tunnel was surveyed by Fiona H. see RPG for field map & floor swipe results. 

Doug, John McKinnon & I set up a plastic sheet with a funnel connected to a 8ft. lg. poly-flow tubing to direct leaking water from the T2 into the south gutter.  This set up was left over night for the floor to dry up and resume the water leak investigation. See photos.

On Sep. 10^th;

@ 08:30 about 70% of the 1A tunnel floor had dried up and the diverted water into the south side gutter had travelled to the T1 area.

The T2 cooling package was then turned off followed by valving off the Cu-ALCW feeding the T2 cooling package. A few more strips of red tuck tape were added to secure the temporary plastic diversion sheet.

@ 11:30 the x-strip graph (T/W  MS 230) of the Cu-ALCW tank level showed that the leak had stopped and leveled off.

@ 13:15 the 1A tunnel floor had dried up with no water seen dripping from the vertical chase.  Doug also mentioned that the “water sensors” that were activated reflected this. The Cu-ALCW was then valved back on to the T2 cooling package and within 3 minutes water began dripping from the south wall vertical service chase which now appeared to about  1 drip/sec.

Fiona took a water sample (20 ml vial) from the drips and also found ~ 1000 c/m (44-2 monitor) on the tacky mat just on the inside of 1A tunnel door.  The 1A Tunnel and South TNF were then locked-up by Derek H., Violeta T. & Doug.   The T2 cooling package was turned back on ok.  Will try to repair this water leak during the Oct. mini shutdown.

 During the maintenance day today, amplified air to the T1 area was valved off in an attempt to pin point the location of the air leak.  The air amplifier cycle rate returned to normal with the T1 line valved off, indicating the leak is somewhere on the T1 profile monitor air cylinders, the T1 septum polarity switch, or on the supply lines to these devices.  Because these devices will not be used in the near future, the line will be left valved off.  Eric Chapman has applied a defeat tag to the valve and made an entry on the Cyclotron Logbook.

The cause of the leak will be investigated further during the 2014 Fall Mini-Shutdown

On Friday August 8th Ron Kuramoto observed that the Meson Hall air amplifier cycle speed had increased.  Cycle time measurements were taken on Monday August 11th.  The lower amplifier was found to be cycling 17 times per minute, and the upper air amplifier 16 times per minute (approximately 4s cycle period).  The cycle period for both amplifiers was measured in June 2012 and found to be approximately 25s.  Note that a cycle is defined as the time between air exhaust events (this may actually only be 1/2 of a full cycle of the amplifier mechanism).  Both amplifiers were delivering air pressure approximately 134psi during the test.

The upper air amplifier was rebuilt in 2012 and has not been used since.  This test shows that the amplifiers are performing normally and that the increased cycle speed is due to a leak elsewhere in the system.  The location of the air leak will be investigated further on maintenance days or during the September mini shutdown.  The exhaust pipes of the amplifier assembly were warm to the touch due to the increased cycle speed, but not excessively so, and the current cycle speed is not thought to be of immediate concern.

The lower amplifier was left running and the exhaust pressure was turned down to 125psi to slightly reduce the cycle speed (it is not thought to be necessary to have air pressure above 120psi)