Logbooks Lab Maintenance Evaporator_1 Evaporator_2 Laser cutter Target Production Test-Stand RH-ISAC RH-Cyclotron RH-Meson Hall RH-Beamlines RH-ARIEL
  RH-Meson Hall, Page 12 of 19  Not logged in ELOG logo
Entry  Wednesday, February 18, 2015, 14:49, Isaac Earle, Repair, Target 1, Target, T1-MK1 Repair Complete IMG_4638.JPGIMG_4646.JPGIMG_4647.JPG

This is a continuation of work detailed in E-Log #129 and #140.

- Static water test performed on the target cooling lines:
     - Test setup assembled and leak checked by pressurising with house air, submerging in water and checking for bubbles (see photos)
     - System filled, pressurized, and valved off on Friday Feb 13 at 5:30pm,  Gauge read 60psi
     - Checked Monday Feb 16 @ 9:00am.  Gauge read 56psi
     - Checked Monday Feb 16 @ 5:00pm.  Gauge read 56psi
     - Checked Tuesday Feb 17 @ 9:30am.  Gauge read 55.5psi  (0.5psi / 24hrs rate)

- Checked results with Dimo and he requested a helium leak check be performed with a newer model leak detector to quantify leak rate:
     - Using Varian 979 leak detector, pumped ~1.5hrs w/ cold trap to remove water
     - TP press: 0.0*10^-4 torr,  Base leak rate ~7.0*10^-9 atm cc / sec
     - Tested all normal leak check locations and did general helium flood with no response at all  (1.5psi helium pressure)
     - Over ~ 20 minute period the base leak rate gradually rose to 8.8*10^-9 atm cc / sec, at one point spiked to high 10^-9 level, then returned to base line
     - Reviewed results with Dimo, he requested test be repeated after the target sits vented overnight

- Target vented and left overnight, leak check repeated Feb 18 starting 9am:
     - 0.0*10^-4 torr TP pressure after ~5 mins
     - 10^-9 atm cc / sec leak rate level after ~20 mins
     - All locations sprayed with 1.5psi helium, general flood in ladder area --> no response (Dimo and Edi present)
     - Target deemed OK for use in beam line by Dimo, he requested the cooling lines be filled with helium after installation in b/l while leak checking T1 volume

- Target will be transported to the storage pit tomorrow (Feb 19) and is ready for use as the spare T1 target

 

The target had an obvious helium leak when first tested (E-Log #129 and #140), which could not be found when helium leak checking again after the static water test.  Some possible explanations are:  water remaining in the lines blocked the leak path,  water remaining in the lines froze when vacuum was pulled blocking the leak path, having water in the lines caused some corrosion or other deposit which plugged the leak.

The helium leak at plugged position 1 right side measured before the static water test is the same, or slightly less than the leak rate when the target was first removed from the beam line.  There were no T1 vacuum issues during the running period, therefore no issues are expected when this target is put in service again.

Entry  Friday, February 06, 2015, 12:39, Isaac Earle, Repair, Target 1, Target, T1-MK1 Repair Status (Pos'n 1 Plug Leak) IMG_4603.JPG

 The T1-MK1 target is currently in the hot cell for repair of a leak at position 1.  See E-Log #129 for details on previous work.

- Pos'n 1 right nut removed (a little stiff, but no excessive force required)
- Exterior threads cleaned w/ Scotchbrite disc on Dremel tool
- Inside sealing face inspected w/ Nikon level.  Possible brownish material on sealing face at 10 o'clock and 12/1 o'clock positions
- Threads inspected w/ Nikon level: small dent at 1st thread 2 o'clock position as seen before and small amount of material build-up or galling at 1st & 2nd thread 4 o'clock position.  Otherwise threads looked clean and straight
- Constructed Dremel attachment using ~1/2" thick, ~4" long piece of Scotchbrite folded over a 1/16" diameter steel rod, secured with zap-straps and shaped w/ scissors to fit inside the ladder port (see photo)
- Cleaned and polished the sealing surface w/ new Scotchbrite Dremel tool
- Inspected inside sealing surface: brown coloured material no longer visible.  A clear scratch/dent is visible at 10 o'clock position.
- Blew out hole and new nut with compressed air
- Installed nut (went on much easier than before which indicates that the Scotchbrite disc thread cleaning method is effective)
- Leak check performed:  pumped down to ~70mTorr on Hastings gauge w/ throttle valve fully open and roughing valve closed
- Sizeable leak at Pos'n 1 right side nut, as before.  No improvement with tightening

The next step will be to perform a static water test on the cooling lines to see if the plug is water leak tight.  If it is, no further action is necessary.  If not, either the ladder must be replaced, or other methods to achieve a seal investigated.
 

Entry  Monday, February 02, 2015, 16:06, Isaac Earle, Maintenance, Target 2, Cooling, T2 MRO Status 

- T2-MK2 target (currently in the T2 monolith) purged with air

- All cooling package reservoir o-rings changed
- Target water line #6 Hansen o-rings changed at both ends
- M8 BB / Col. A supply hose Hansen o-rings changed at both ends

- Target water and demin circuit Cuno filters changed
- Used filters double-bagged, labeled, and stored on 1A blocks north of boot box area

- The following Swagelok ball valves removed for service:  Tgt return (X2), Sec return (X2), Demin supply, Demin return (X2), Col A return, Col B return

*** Four #4 Hansen o-rings on package panel still require changing (size was not available from stores)

Entry  Tuesday, January 27, 2015, 17:38, Isaac Earle, Repair, Target 1, Cooling, T1 Cooling Package 3/4" Valves Rebuilt - T1 MRO Complete 

- Both 3/4" ball valves on the T1 cooling package target water circuit were rebuilt, leak tested with house air pressure, and reinstalled

- The 3/4" ball valve on the T1 heat exchanger secondary side was replaced with a new unit

- The 3/4" needle valve on the T1 heat exchanger secondary side was rebuilt, leak tested with house air pressure, and reinstalled

- The T1 cooling package was refilled and restarted using a bypass across the inlet/outlet Hansen fittings

- The package was inspected while running and no leaks were visible (video survey taken with file stored on Isaac's PC)

- All T1 Cooling Package MRO activities are now complete

Entry  Friday, January 23, 2015, 13:28, Isaac Earle, Repair, Target 2, Cooling, T2 Cooling Package Thermistor Defeats 

Both defeats applied to T2 cooling package thermistors were removed today (Col. B interlock was jumpered, and the target water inlet and outlet signals had been swapped).  It was confirmed that all thermistors are now wired correctly.  The readback of both target water thermistors is currently unstable.  Inspection of the Lemo connectors showed that the internal plastic parts of all connectors were damaged or missing.  If the T1/T2 controls upgrade proceeds this shutdown, then all thermistors will be replaced with thermocouples, so after a decision is made on that project (sometime in Feb 2015) these thermistors will either be repaired or replaced.

Entry  Wednesday, January 14, 2015, 15:06, Isaac Earle, Standard Operation, Target 1, Other, Active Item Stored in East Hot Cell 

The active item discovered on January 9, 2015 outside the BL1A bootbox has been moved to the Meson Hall east hot cell for storage.  The item was analyzed by RPG and the radiation was found to be coming from activated metal contained by dried paint in the bottom of a plastic container.  The solidified paint puck was removed from the container which is not active.  The item is approximately 20microSv/hr at 0.5m, and 1.3mSv/hr on contact.

Entry  Wednesday, January 14, 2015, 13:32, Isaac Earle, Standard Operation, Target 1, Target, T1-MK2 Moved from B/L to Storage Pit 

The T1-MK2 target was transferred from the beam line to the storage pit in hole #5 using the transfer flask.  A maximum field of 7.2mSv/hr was measured at 0.5 meters from the target.  A video was taken while the target was lifted from the monolith and can be found on Isaac's PC (too large to upload).

The blocks to the west and east side of the T1 monolith (level with the monolith) were swiped by Danka Krsmanovic and no contamination was found.  These blocks will therefore not be painted this shutdown.

The three active filters removed from the T1 cooling package have been labelled and stored in the area north of the BL1A blocks boot box with permission from Danka.  The bag with the filters is 100microSv/hr at 0.5m and 2mSv/hr on contact.

Entry  Monday, January 12, 2015, 16:35, Isaac Earle, Standard Operation, Target 1, Cooling, T1 & T2 MRO Status 

The following tasks have been performed at T1:

- T1 monolith painted using grey two part epoxy paint
- Video inspection taken while package running. Small leak from both valves on H.E. secondary side (video on Isaac's PC, too large to upload)
- Cooling package sampled and drained, drain valve closed
- All reservoir o-rings changed
- All Hansen fitting o-rings changed (4x #6 size, 2x #4 size)
- All three cuno filters and filter housing o-rings changed (used filters 2mSv/hr on contact, temporarily stored on blocks by T1 package)
- Failed T1 P1 (inlet) pressure transducer replaced
- Supply and return valves for T1 CuALCW closed in tunnel to allow leaking valves to be replaced

- New resin can installed at T1 (this note added oct 15 2015 - was previously undocumented)
 

The following tasks have been performed at T2:

- Video inspection taken while package running. Documented leaks mentioned in E-Log #133 (video on Isaac's PC, too large to upload)
- Cooling package sampled and drained, drain valve closed

 

RPG has analyzed the water samples and given approval for water from both packages to be released.

Entry  Wednesday, January 07, 2015, 17:27, Isaac Earle, Repair, Target 2, Water Leak, Water Leaks at T2 Cooling Package 

The T2 cooling package was visually inspected today after blocks were removed.  With the cooling package off the following leaks were observed:

- Collimator A return line ball valve: ~ 2 drops / second
- Collimator B return line ball valve: ~ 1 drop / 3 seconds

The cooling package was then started and the following leaks were observed:

- T2 heat exchanger secondary side, north end ball valve (CuALCW):  ~ 1 drop / sec
- Target water return line ball valve above reservoir: very slow leak, wet but no drips visible

These leaks explain the water found in the tunnel, and the slow drop in T2 expansion tank level.  However it is a little mysterious why four valves all failed around the same time on two different water systems..

The copper active supply to the T2 area will be valved off to stop the leaks, and the valves will be serviced or replaced during this shutdown.

Entry  Monday, January 05, 2015, 15:08, Isaac Earle, Standard Operation, Target 2, Water Leak, Cooling Packages Re-Started; Suspect External Water Leak at T2 Package 

Pumping was restored to 1A today by Vacuum Group and the beam line returned to normal vacuum levels.  Both cooling packages were started with no change in vacuum (no water leaks inside the vacuum volume).  The T2 package was left on for approximately 1 minute, and then the expansion tank level was checked.  Approximately 1cm of water was lost.  The T2 package was turned off, and will remain off until the package can be inspected later this week after block removal.  The T1 package will be left running until T1 MRO work begins.

Entry  Monday, December 22, 2014, 14:31, Isaac Earle, Maintenance, Target 2, Water Leak, Drop in T2 water level 

This morning at 5:30am the T2 cooling package expansion tank switched to WARNING LEVEL NOT OK.  There was no change in BL1A vacuum at this time.  The change in level was confirmed by inspecting the water level visually.  Operators suggested it may have happened because the cyclotron was shut off and the temperature of all water systems dropped, however T1 did not experience a noticeable change in level, so I do not think this is the cause.  It is possible there is a leak on the cooling system somewhere, but the evidence does not suggest a leak into the vacuum volume.

At 11:55am the BL1A vacuum spiked tripping off both cooling packages.  Valve 1AV8 is closed.  There is no pumping on the T1 volume, so the pressure is slowly increasing.  T2 is at approximately 50mTorr.  Operators believe it may have been caused by work at M15 or M13.  

This appears to be unrelated to the water loss at T2, but proper diagnosis is not possible because Vacuum Group staff are away for the holidays.  The cooling packages will be left off until proper diagnosis can be performed in the new year.

Entry  Friday, December 19, 2014, 14:35, Isaac Earle, Standard Operation, Target 2, Other, Amplified Air Shut Off 

The BL1A amplified air system has been shut off, and will remain off until required for BL1A start-up in Spring 2015 (the lower of the two units was in use, with the upper unit on standby as a spare).

Entry  Tuesday, December 09, 2014, 14:45, Isaac Earle, Repair, Target 1, Target, T1-MK1 Repair Status (Pos'n 1 Plug Leak) 

 Between Nov 14 ~ Dec 9 the following activities took place:

- The leaky 10mm Be target at pos'n 3 was removed and plugged
- The loose protect monitor electrical conduit was secured in place using an aluminum shim
- David Cameron performed an electrical check on the protect and profile monitors and got the expected response from both
- The target was flushed with water, then air
- The target was pumped down using a cold trap to capture remaining moisture
- The target reached a minimum pressure of 80mTorr on the leak detector Hastings gauge
- Helium leak check revealed a leak at both of the position 1 Swagelok caps (up to 80 on 50x scale on left side, and 100 on 100x scale on right side when facing ladder with 1s spray of 5psi helium)
- The leak rate did not improve with tightening of the caps
- The caps were removed, ladder side threads inspected (no obvious damage observed), and new caps installed
- Helium leak check repeated: Pos'n 1 left side now leak tight, right side leak rate was worse (could not completely open throttle valve on leak detector)
- Torque was increased on the cap with no improvement in leak rate
- The right side cap was removed and when trying a new cap it would not spin freely
- The ladder side threads were inspected more carefully with the Nikon level:  a small dent across the first thread at approximately 2 o'clock position was seen as well as some material build-up or possible galling seen at the 1st and 2nd threads around the 4 o'clock position.
- The threads were filed to improve their profile which allowed a new cap to be installed with light resistance (still would not spin freely)
- Leak check was performed with a minimum pressure of 70mTorr reached (throttle valve fully open, roughing closed), with large response still at the right side pos'n 1 cap, No change with moderate tightening

It is suspected that the ladder side fitting sealing face is damaged causing a poor seal.  When a cap was installed and torqued this may have caused the galling or material build up observed at the 4 o'clock position (but not the dent on the 1st thread at the 2 o'clock position)

Various experts on-site will be consulted about how to best proceed before doing further work.  One option is to perform a static water test.  If the target is water leak tight then it may be used in the beam line.  If repair efforts are not successful and the target is not water leak tight then the target ladder will have to be replaced which is approximately a 2 week job.

Entry  Tuesday, November 18, 2014, 10:52, Isaac Earle, Standard Operation, Target 1, Active Sump, RH Hot Cell Lab Active Sump Released 

Release of the Remote Handling Hot Cell Lab active sump water was completed today by Hua Yang.  During the procedure fields of up to 0.50 micro Sv/hr were measured around the newly installed filter housing.  After completion the filter was removed and replaced.  The filter measured 450cpm on the RM-14 monitor and ~5000cpm on the 44-2 monitor (both on contact).  RPG has been notified and the filter has been bagged, labeled, and left for pick-up by RPG.

Entry  Monday, November 17, 2014, 10:25, Isaac Earle, Repair, Target 1, Other, Amplified Air Leak In T1 Area Repaired IMG_4263.JPG

While the 1A blocks over the T1 area were removed last week the opportunity was used to investigate an air leak on the amplified air system discovered on August 11, 2014.

The leak was found to be coming from a device related to the septum polarity switch located behind (south of) the septum power supply on the 1A blocks.  This device is not in use, so the amplified air to the device was disconnected by plugging the relevant branch of a brass 3/8" Swagelok T fitting in the area.  This stopped the leak, and the air amplifier is cycling at usual speed after being restarted.

Entry  Monday, November 17, 2014, 10:14, Isaac Earle, Development, Target 1, Active Sump, Filter Installed Downstream of RH HC Lab Active Sump IMG_4273.JPG

A filter housing and filter were installed by Dan McDonald on November 14 in the piping section between the Remote Handling Hot Cell Lab active sump and the city sewer drain as shown in the attached picture.  The purpose of the new filter is to prevent pieces of active material from being released to the drain when the sump is pumped out.  If small pieces of active material enter the sump they may sink to the bottom, and not be captured in a water sample which is taken from the top of the sump.

The filter housing is a Waterite HP1034CLUR-KIT, and the installed filter is a 10" Cuno Microwynd 25micron filter.

This modification was approved by Curtis Ballard, Joe Mildenberg, and Grant Minor prior to installation.

Entry  Friday, November 14, 2014, 17:30, Isaac Earle, Repair, Target 1, Target, T1-MK1 Target Failure / T1 Target Swap T1MK2_Nov_14_2014.pdf

 - On Nov 13 at ~1am the BL1A vacuum went bad.  The T1 and T2 volumes were isolated and pumped on separately, the results indicated a water leak from the T1 target.  The level of the expansion tank was slightly below normal at this time, but not enough to be conclusive.  The leak was confirmed at ~9am by starting the T1 cooling package which caused a dramatic vacuum spike.  Approximately 1" was lost from the expansion tank when the package was started, equal to approximately 2.5L of water.

- The T1-MK1 target was removed from the beam line and transported to the hot cell.  A mound of ice was visible on the target lower plate while it was being lifted.  A radiation field of 160mSv/hr was measured at 0.5m from the target during removal.  Upon inspection in the hot cell, a small but clear hole was visible on the entry window of the position 3 target cassette just slightly above and to the right of center if facing downstream.  No water was visible on the profile or protect monitors.  Approximately 1" of water was visible in the base of the T1 monolith.  The monolith was left open to air overnight, and approximately 0.5" of water was visible in the morning on Nov 14.

- Vacuum group was consulted, and given the relatively small amount of water remaining they agreed to pump the water out of the beam line rather than have Remote Handling group try to remove the water before target installation.  The T1-MK2 target was transported from Hole #3 in the storage pit to the beam line.  A field of 5mSv/hr was measured at 0.5m from the target.  All services were attached.

- An electrical check was performed on the profile and protect monitors by David Cameron from the 1A mezzanine.  Both devices appeared normal.  Plant Group then began to replace the blocks over T1.  Graham Waters updated the control software so that the correct target ladder potentiometer values are referenced.  An updated target information sheet was prepared and delivered to Operations (PDF File Attached)

Entry  Thursday, November 06, 2014, 17:22, Ron Kuramoto, Repair, Target 2, Cooling, 1AT2, faulty Thermister water out temperture 

Summary: Clarification of T2 water temperature readouts
Detail: This e-log entry by Doug Preddy is being made to clarify the T2 water package temperature read outs. The temperature read outs on the T2 water package have been switched. The read out for inlet temperature is actually the outlet temperature and the read out for outlet temperature is actually the inlet temperature.

On Sunday Oct 19 2014 at 19:45, the T2 water outlet Temp (T2) started to climb giving warning messages at 20:05. Operators tried to decrease the BL1A current, but the temp. continued to climb and eventually tripped the beam off. Operators defined BL1A OFF, and continued to inject to BL2A only. They tried to contact I. Earle, but were unable to. They left message with D. Preddy. The T2 temp. stayed at 45 deg., operators tried to reset the water package and start it, but it tripped almost right away, and increased the temp. by 0.5 deg. Beam was left off.

On Monday Oct. 20 Ron Kuramoto found a replacement thermister. It was wired in to the BOP over the south catwalk. The T2 outlet temperature immediately dropped to ambiant. From this test it was concluded that the installed thermistor on the T2 outlet water pipe had failed. The wiring was restored to the original configuration.
It was decided to switch T2 Water Inlet Temp and Outlet Temp sensors at the T2 electronics rack to provide a trip on the Inlet Water temperature instead of the outlet Water Temperature. As the T2 water package runs in a closed loop, the outlet temperature signal has been swapped with the inlet temperature by Doug P. and Tony T. This will provide thermal protection (trip at 45C) in addition to the T2 Target high limit bit (trip at 60C). The water flow to this circuit is also monitored and will trip if the flow drops.

This action was approved in a meeting with Doug Preddy, Ron Kuramoto, Grant Minor, Yuri Bylinski, Lia Merminga, and Bob Laxdal present on Monday Oct 20th. There is a second, redundant temperature sensor on the outlet of the target water which was installed recently as a safety upgrade to protect the heat exchanger and meet CSA pressure vessel code, see dwg TBP0902. There is a controller for this sensor with a hard-trip switch for the cyclotron, and a temperature read-back display on the Meson Hall mezzanine racks. This read-back should be monitored by operations once per shift while beam is on target to ensure that outlet water temperature is stable and at an expected value.

Sensor cabling swapped at Meson Hall, south mezzanine, bay 1, B.O.P. #3. Sensor T1, cable #12102 moved to TB1, terminals 3 & 4. Sensor T2, cable #12105 moved to TB1, terminals 1 & 2. Defeat #7131 removed.

Defeat# 7133
Fault report #7722
 
Will look into replacing the faulty Thermister in the 2015 major shutdown.

Entry  Thursday, November 06, 2014, 17:18, Ron Kuramoto, Repair, Target 2, Cooling, 1AT2 Collimator B, faulty Thermister water out temperature  

On 14Oct2014, the 1AT2 Collimator B water outlet temperature rose to 100o C. 

Troubleshooting the T2 Collimator B water out temperature (T7) by swapping it's analog signal in PIE box #2 with T2 Collimator A water out temperature (T6) to see if problem follows. See if unplugging T7 signal causes Safety Page SV7 1A Collimators error signal goes okay.
 
Tony Tateyama  2014-10-14 11:29:36
PIE box channel for T7 appears to be okay as a known good level signal (T6) was connected, and channel warning & trip went okay. Safety Vertical page SV7 1A Collimator signal went okay after removing bad signal from PIE box, and resetting trips on two panels at T1/T2 racks. It appears that the sensor for T7 has become faulty.
 
See interlock defeat #7128
Approved by Doug Preddy
Shorting 2 pin lemo plug K3 of #1 046418 Interlock units in Rack 1-A south side mezzanine in the Meson Hall. Cable # 12708 to removed to insert short.
 
Will look into replacing the faulty Thermister in the 2015 major shutdown.
Entry  Thursday, October 16, 2014, 17:44, Ron Kuramoto, Repair, Target 2, Water Leak, 1A Tunnel T2 Cu-ALCW water leak repair 6x

 

On Oct 7th, during the Mini Shutdown, the T2 cooling package was uncovered to investigate the Cu-ALCW water leak dripping into the 1A Tunnel.

The Target shield block was lifted and found that top of the monolith was dry.  Uncovering the east side of the cooling package directly above the vertical services on the south wall revealed a pin hole leak at a 1/2” 45o copper elbow on the return line circuit of the T2 cooling package.

The cooling package was turned off followed by valving off the Cu-ALCW service lines leading up to the cooling package to let the Tunnel floor dry over night.

 

On Oct 8th, the Tunnel floor was dry and Cu-ALCW service lines feeding the cooling package were drained from the Tunnel by placing a 5 gallon pail under the drain lines filling it about 1/3 full until it stopped.

Mel Good was brought in to replace the faulty elbow.  The Cu-ALCW was then turned back on with no obvious leaks.  At 16:00 approximately 1.5 hours after the repair, I went into the 1A tunnel to inspect for water leaks at the isolation valves and saw water dripping down the vertical piping on the south wall again.   

Another leak was found on the 90o elbow just ahead of the replaced 45o elbow. There appeared to 2-3 fine mists spraying out towards the adjacent copper elbow (look at the end of the cable tie on the following photos).

The Cu-ALCW lines leading up to cooling package were again valved off to let the Tunnel floor dry up over night.

 

On Oct 9th, the Cu-ALCW lines leading up to the cooling package were again drained followed by replacing the faulty 90o elbow. The elbow was fitted with new pieces to improve the configuration. We noticed that old copper pieces were worn thin. These damaged parts measured 20,000 c/m on the 44-2 and 40 c/m on the RM-14 monitors and were taken to the RH H/cell lab for inspection.

 

The T2 cooling package turned on ok about 1.5 hours after repairing the 2nd water leak.

 

Mel Good received 0.03mSv over the two day repair. 

I received 0.10 mSv over three days, picking up 0.05mSv on Oct 7th in the Tunnel where the fields were 200-300uSv/Hr by the isolation valves.  The next the day levels dropped to 40uSv/Hr.  There was no contamination found on the Tunnel floor, however 80,000 c/m on the RM14  and 50,000 c/m on the 44-2 monitors were found on the swipes taken at the bottom edge of the concrete shield blocks.

 

NOTE: the Cu-ALCW service lines leading up to the T2 cooling package feeds the following 4 devices; Collimator A, Collimator B, the T2 monolith cooling and the Target water heat exchanger.

IMG_1056s.JPG

ELOG V2.9.2-2455