Logbooks Lab Maintenance Evaporator_1 Evaporator_2 Laser cutter Target Production Test-Stand RH-ISAC RH-Cyclotron RH-Meson Hall RH-Beamlines RH-ARIEL
  RH-Meson Hall, Page 19 of 19  Not logged in ELOG logo
ID Date Author Categoryup Type Specific Subject
  352   Thursday, August 22, 2024, 15:16 Adam NewsomeStandard OperationOtherActive SumpActive sump filled ~600L with grey water from RCR1 sump replacement

During a replacement of the RCR1 active sump (mechanical services group), approximately 600L of non-active grey water was released into the RH active sump. At this time, this should be all of the water in the RH sump (approximately 600L total).

  354   Monday, September 09, 2024, 16:31 Albert KongStandard OperationOtherActive SumpActive Sump High Level Sensor Testing

The active sump high level sensor was tested and correctly alarmed locally. However the alarm in the MHESA RCR lab did not make any sound and no notification was recieved at the main control room. After some investigation, it is believed that the RCR lab alarm 'had already produced a notification at the control room and so no new notification was produced by testing the remote handling sump. Alternatively, the RCR sump alarm may have been silenced in the control room and so the notification was not seen. 

 

We will look to modify the system so that our sump's alarm does not piggy-back off the RCR sump alarm when notifying the control room.

 

  356   Wednesday, October 30, 2024, 15:10 Albert KongStandard OperationTarget 1TargetT1-MK1 Electrical Check on Hot Cell

T1-MK1 electrical checks were completed on the hot cell. Specifically the profile and protect monitors were checked (assistance from Micheal Donohoe and Holden Jones from 'probes group'/accelerator systems).

Reference WP C2024-10-30-3.

"WORK INSTRUCTIONS / REFERENCE DOCUMENTS: Perform blip test of target protect module by attaching HV oscillator to protect HV bias. Check that the oscillator signal is readable from U,D, L, R and HALO signal cables.

For profile monitor, connect PM cable assembly to 0518 MWC module in portable NIM crate. Connect 0518 "START" to ch2 of an oscilloscope and "ANALOG" to ch1. Set scope to trigger on Ch2. Connect HV oscillator to profile HV bias. Observe signals on all channel devisions, note any channels not responding."

----------------------------------------

Test outcome (paraphrased from text discussion with Micheal Donohoe)

Profile monitor functioning properly from test, but right 'plate' on protect monitor yielded different results to other plates. May need to be re-checked, pending comparisons with results from previous tests (issues with this because "different oscillator was used"). We may have also not used couplings for cables that were correct for that particular protect monitor plate.

----------------------------------------

In images attached, red cable used for profile monitor (connect to the vertical connector at the rear), the dark green wire is used to ground the system when the profile monitor is being checked. The grey cable bundle (with 'box') is used to check the protect monitor, with vertical pins located by the main lifting flange support bars in the front left of the target (see bottom right of first image).

----------------------------------------

Micheal and Holden returned on the 31st with Thomas Manson, they performed direct testing on the leads of the protect monitor connector (directly hook up leads on protect monitor connector to oscilloscope/multi-meter). All direct checks went well, the waveform issue from yesterday's testing likely resulted from the testing cable having issues.

Electrical testing on T1-MK1 is now complete. Probes group will work to update their documentation and possibly develop a process change to fabricate new cables to make the testing happen fully on the hot cell roof and have it complete faster.

Note: it may be a good idea to perform preventative maintenance on the protect monitor @ T1-MK1 the next time it is on the hot cell. Probes group members noted that the protect monitor components are nearing their end-of-life and it would be good to replace old components to avoid the protect monitor failing in the target station.

 

 

Attachment 1: IMG_0368.JPEG
IMG_0368.JPEG
Attachment 2: IMG_0369.JPEG
IMG_0369.JPEG
Attachment 3: IMG_0360.JPEG
IMG_0360.JPEG
Attachment 4: IMG_0374.JPEG
IMG_0374.JPEG
Attachment 5: 78c73bfe-d624-4ca7-95e8-9d4a3d921f8e.jpg
78c73bfe-d624-4ca7-95e8-9d4a3d921f8e.jpg
Attachment 6: Media_(1).jpg
Media_(1).jpg
Attachment 7: Media_(3).jpg
Media_(3).jpg
  358   Friday, January 10, 2025, 18:36 Albert KongStandard OperationOtherCoolingT1/T2 cooling packages drained

The T1 and T2 cooling packages were drained starting at 14:36, by 14:48 both expansion tanks had dried, meaning the draining rate for both tanks were ~80L/hr, generally it will only take 2 hours to fully drain the system.

Tritium samples were collected after allowing the system to drain for ~5 minutes to clear out water in the drain lines and actually collect samples from the reservoirs. The samples were passed onto RPG for analysis.

The drain valves on the cooling packages and in the BL1A tunnel is left open.

Update Jan 23: the drain valves in the 1A tunnel and the valves on the cooling packages were closed.

 

 

Attachment 1: CALCS_T1_T2_cooling_system_drain_speed.html
Attachment 2: Tritium_Sample_and_Analysis_Form_-_T1_Cooling_Package_-_2025-01-10.pdf
Tritium_Sample_and_Analysis_Form_-_T1_Cooling_Package_-_2025-01-10.pdf Tritium_Sample_and_Analysis_Form_-_T1_Cooling_Package_-_2025-01-10.pdf
Attachment 3: Tritium_Sample_and_Analysis_Form_-_T2_Cooling_Package_-_2025-01-10.pdf
Tritium_Sample_and_Analysis_Form_-_T2_Cooling_Package_-_2025-01-10.pdf Tritium_Sample_and_Analysis_Form_-_T2_Cooling_Package_-_2025-01-10.pdf
Attachment 4: IMG_0942.JPEG
IMG_0942.JPEG
  361   Monday, January 27, 2025, 12:53 Albert KongStandard OperationOtherTargetT1-MK1 Moved to Storage Pit #4, M9BB Servicing in Hot Cell, T2-MK1 Servicing in Hot Cell, T2-MK2 from Storage Pit #3 to Station

DRAWINGS UPLOADED IN PDF, DOWNLOAD TO VIEW ALL DRAWINGS, PREVI

---------------------------------------------

Jan 27:

T1-MK1 was moved to the storage pit in position #4. M9BB was moved to the hot cell for shutdown maintenance (o-ring replacement, air cylinder servicing, etc.).

When M9BB work is done, it will be returned to the target station and replaced with T2-MK1 for servicing (target exchange, measurement target installation, etc.).

---------------------------------------------

Jan 29:

Maintenance operations on the M9BB were completed on the hot cell.

  • The air cylinders actuated and lowered smoothly, and at 42 psi as indicated in the instructions document.
  • The felt wiper on the shaft seal was replaced.
  • 3 o-rings were replaced on the bronze shaft seal bracket and lubricated with vacuum grease
  • upper limit switch actuation was confirmed after reassembly
  • BB returned to the adapter plate on the turntable and reoriented for pickup

---------------------------------------------

Feb 03:

M9BB returned to target station:

  • o-ring groove on monolith cleaned,
  • replacement o-ring with vacuum grease placed
  • during return, we noticed pieces of 'scrap metal' on the bottom of the beamblocker, it was removed and when scanned registered a count that exceeded the radiation cave monitor's dial, this will be stored/disposed as active waste.
  • M9 ASU still disconnected, could not 'push-down' M9BB with gas line connected, gas line left disconnected for now.
  • T handle plate removed, limit switch rod returned

T2-MK1 moved from target station to hot cell roof, north hook used for flask, south hook used to adjust the position of the alignment frame:

  • section of hot cell roof railing had to be removed for the flask to clear when handled by the north hook, there is also a similar functioning cutout on the wall by the storage pits
  • peak fields are 1100 uSv/hr at the hot cell opening by the target.

T2-MK1 servicing started on hot cell roof:

  • T2-MK1 water flush completed, ~1 min air purge cycles completed for all target positions, left at position 5 (plugged) for longer term air purging (~3 hours).
  • A broken ceramic cap was found on the top of the target vacuum flange.
  • 4 quarter circle segments from what looks like the old-style graphite targets were found on the base of the target ladder.
  • The coupling between the potentiometer gearbox and the ladder drive screw has a rubber section, we will have to inspect it and decide whether to replace it with a metal coupling during this shutdown, if so, we should also replace the plastic ferrules and update the bulkhead plate to be more servicable.

---------------------------------------------

Feb 05:

T2 station was surveyed, peak fields were found by the blanked off T2 opening: 4mSv/hr. Immediately at the cover fields were around 16 mSv/hr.

No fallen pieces of suspected graphite target segments were found on the monolith, it may be a good idea to check inside the monolith hole with a remote viewing camera to make sure nothing is obstructing the locating feature on the base of the target ladder (to be included in the T2 return work permit).

---------------------------------------------

Feb 10:

The T2-MK1 target ladder underwent a water rinse and several days of air purging. 

5 psi compressed air was run through each target position (including position 5 which is plugged) for several hours each. Then, the ladder was fully dried using the leak detector and  cold trap on Feb 07.

The leak detector was calibrated with Vacuum Group's external source to 8.7e-8 mbar-L/s prior to operation. The leak detector and vacuum line up to the target service panel connections were checked to be leak tight prior to testing (0 leak rate and <1e-3 Torr pressure).

A single pump-down cycle was sufficient to get vacuum to establish within reasonable time:

  • Total pump-down duration was ~55 minutes
  • 60 seconds to go into 'fine vacuum' mode.
  • 2 min 13 seconds to get to ~1e-3 Torr vacuum in ladder
  • leak rate was initially ~1e-8 Torr-L/s and slowly dropped to 0.0e-10 Torr-L/s over the course of the pumpdown (48 minutes to bottom out)
  • the ladder was left at the bottomed out leak rate for around 5 minutes prior to venting. 
  • approximately a 1cm thick, 10cm diameger puck of ice formed on the side of the LN2 vessel of the cold trap.

 

Upon closer inspection with Isaac Earle, the following were observed for graphite pieces on the T2-MK1 target ladder base:

  • The surface finish on the chips strongly suggest that they are graphite target materials
  • Curiously the graphite chips are all of different thicknesses, to our understanding, the graphite material used in the old targets are of consistent thickness.
  • The rounded faces of the chips had no clear sign/evidence of brazing when compared to sample pieces that can be found in the Hot Cell Lab office.
  • No clear signs of beam spots can be found on the flat faces of the chips, near the corner edge.
  • No clear signs of brazing on the targets were observed.

The above observations suggest that the chips did not originate from an operational target that has seen beam.

---------------------------------------------

Mar 21:

The T2-MK1 target ladder was leak tested with 3psi x 0.5s doses of He.

The baseline initial leak rate of the ladder was ~1e-8 Torr-L/s but reached around 1e-10 Torr-L/s when allowed to pump down over ~1 hour. Fine test was reached after 30s of pumping down (with cold trap), ladder pressure was stable at ~1e-3 Torr throughout.

See attached image for detected leak rates, overall all joints performed well, with the highest leak rate of 4.2e-9 Torr-L/s registered by one of the joints on ladder position 1.

---------------------------------------------

April 02:

The T2-MK2 target ladder was moved to the target station from storage pit #3.

During the return procedure, from visual inspection of the target tube, there doesn't seem to be any dropped material in base of the tube.

We encountered some difficulty seating the o-ring on the monolith flange's groove because the o-ring was stored in a coil. Extra vacuum grease was used to keep the o-ring stuck flush in the groove.

When lowering the ladder onto the monolith flange, the chain hoist bottomed out just short and we had to lower the flask with the crane for the target ladder to make contact. Edi from vacuum group helped us pump down on the vacuum volume and at first glance it seems to be pumping down ok.

We will return in the coming days to re-do the service connections to the target ladder and do a final check of the system.

---------------------------------------------

April 03:

The plug in pos#5 and the 5cm target (SER# 301) in pos#3 on the T2-MK1 target ladder were removed in the hot cell and bagged and moved to the East hot cell. 

The loose graphite material on the base of T2-MK1 was also bagged and labelled and moved to the East cell. 

The measurement target bracket TRH1766 was installed in position 5 and a 5cm target cassette (SER # 308) was installed in position 3.

The target bracket was cleaned in an ultrasonic cleaner with simple green, then tap water before being dried. The tube screws were switched with DIY vented screws.

The pos 3 and 5 male Swagelok threads and conical sealing face were 'cleaned' with the conical scotch-brite tools (cone and cup) with the air ratchet and blown out with a compressed air can.

The Swagelok nuts on the new target cassette and bracket was first tightened till the Swagelok gap inspection gauge could not fit in the gap.

The ladder was then leak checked and initially only pumped down to 3e-3 Torr with 2.5e-8 T-L/s leak rate. After some tightening of the Swagelok nuts on pos 3 and pos 5, the pressure and baseline leak rate eventually bottomed out to 1e-3 Torr and 0.0e-10 T-L/s respectively.

With 3psi x 0.5s He, the highest leak rates were found in position 2b, 3b, and 5a, all being less than 3e-9 Torr-L/s.

Of note, there is a 'ding' on the 'right' side bellows (as viewed from hot cell widow with target ladder rotated closer toward the window), the highest leak rate was found when we sprayed He directly at this 'ding'.

 ---------------------------------------------

April 03 (Continued):

We moved the T2 flask alignment frame off from the T2 station, re-connected all target services to T2-MK2 and re-connected the compressed air line to the T2/M9 blocker.

The T2-MK2 profile monitor drives in/out properly without timing out.

The target ladder drives up and down smoothly between pos 0-3.

We tuned the demin flowrate to 1.1-1.2 gpm for the T2 target in position 3 (5cm Be), and for the T1 target in position 1 (12mm Be, actually done end of Mar).

After some time allowing the pump to flow, all process variables in the cooling package reached operational ('green') levels.

As normal though, the T2 target flowrate was relatively low and close to the lower warn limit of 3gpm. We should look into upgrading the pump on this cooling package or adjust the warn/trip limits according to some Engineering analysis.

 ---------------------------------------------

April 09:

  • We attempted to open the needle valve on the T1 heat exchange further but it was already fully open. Flow through the heat exchange is still fluctuating around 6.3 GPM, occasionally dipping below 6 GPM. It may be possible that the heat exchange paddle wheel needs to be changed for the flow rate to increase again.

 ---------------------------------------------

April 11:

  • After re-inspecing the supply and return lines for the T1 heat exchanger in the BL1A tunnels, no metering/flow adjusting valve was found (also true for the T2 line). The flow through the T1 heat exchange will have to hover around the 6gpm mark (flow changed likely because other upstream systems are receiving flow).
  • The flow through the T2 target (12mm Be) hovers around 3GPM, for both position 3 and 5 so we will have to deal with it hovering around the warn limit for the coming operational period (pending thermal analysis to justify changes to the warn limit)
  • The M9/T2 and M20 blocker movement was tested and confirmed.
  • The profile monitor movement for both the T1 and T2 target ladders were confirmed.
  • We also tested a method of powering off the cooling pump by disconnecting the PLC signal/control cable to the 480V power box on the mezzanine.After disconnecting the signal cable, the T2 pump successfully turns off and upon reconnecting, the control room successfully regains control over the pump.

 ---------------------------------------------

Jun 25:

  • T2-MK1 target elevations was measured from Jun 20-24. 
  • The target flange was levelled to within 0.2 degrees during the measurements.
  • Generally, the target centres line up with past measurements from 2020. 
  • The new completed form can be found here.
  • The measurement jig centres was also measured (see hand written note). 
  • During measuring there was a brief instance where the profile monitor made contact with the target in position 1, we were able to verify that the profile monitor centre did not change (measurement immediately before and after stayed the same). 
  • We will re-verify that the electrical system of the profile monitor and that the vacuum of the target ladder is not compromised.

 

Attachment 1: M9BBToHotCellT1-MK1ToPit4_27Jan2025.pdf
M9BBToHotCellT1-MK1ToPit4_27Jan2025.pdf M9BBToHotCellT1-MK1ToPit4_27Jan2025.pdf M9BBToHotCellT1-MK1ToPit4_27Jan2025.pdf M9BBToHotCellT1-MK1ToPit4_27Jan2025.pdf M9BBToHotCellT1-MK1ToPit4_27Jan2025.pdf M9BBToHotCellT1-MK1ToPit4_27Jan2025.pdf M9BBToHotCellT1-MK1ToPit4_27Jan2025.pdf M9BBToHotCellT1-MK1ToPit4_27Jan2025.pdf
Attachment 2: Jan_28_M9BB_Servicing.pdf
Jan_28_M9BB_Servicing.pdf Jan_28_M9BB_Servicing.pdf Jan_28_M9BB_Servicing.pdf Jan_28_M9BB_Servicing.pdf Jan_28_M9BB_Servicing.pdf Jan_28_M9BB_Servicing.pdf
Attachment 3: Feb03_M9BB_return_T2-MK1_to_hot_cell.pdf
Feb03_M9BB_return_T2-MK1_to_hot_cell.pdf Feb03_M9BB_return_T2-MK1_to_hot_cell.pdf Feb03_M9BB_return_T2-MK1_to_hot_cell.pdf Feb03_M9BB_return_T2-MK1_to_hot_cell.pdf Feb03_M9BB_return_T2-MK1_to_hot_cell.pdf Feb03_M9BB_return_T2-MK1_to_hot_cell.pdf Feb03_M9BB_return_T2-MK1_to_hot_cell.pdf Feb03_M9BB_return_T2-MK1_to_hot_cell.pdf
Attachment 4: Feb05_T2MonolithSurvey.pdf
Feb05_T2MonolithSurvey.pdf Feb05_T2MonolithSurvey.pdf Feb05_T2MonolithSurvey.pdf Feb05_T2MonolithSurvey.pdf Feb05_T2MonolithSurvey.pdf Feb05_T2MonolithSurvey.pdf Feb05_T2MonolithSurvey.pdf Feb05_T2MonolithSurvey.pdf
Attachment 5: PumpDownFeb07.pdf
PumpDownFeb07.pdf PumpDownFeb07.pdf PumpDownFeb07.pdf PumpDownFeb07.pdf
Attachment 6: IMG_1608.JPEG
IMG_1608.JPEG
Attachment 7: IMG_1654.JPEG
IMG_1654.JPEG
Attachment 8: IMG_1656.JPEG
IMG_1656.JPEG
Attachment 9: Screenshot_2025-04-02_122004.png
Screenshot_2025-04-02_122004.png
Attachment 10: April_03_Update.pdf
April_03_Update.pdf April_03_Update.pdf April_03_Update.pdf April_03_Update.pdf April_03_Update.pdf April_03_Update.pdf April_03_Update.pdf April_03_Update.pdf
Attachment 11: Screenshot_2025-04-03_133035.png
Screenshot_2025-04-03_133035.png
Attachment 12: Screenshot_2025-04-09_181524.png
Screenshot_2025-04-09_181524.png
Attachment 13: April11.pdf
April11.pdf April11.pdf April11.pdf April11.pdf April11.pdf April11.pdf April11.pdf April11.pdf
Attachment 14: IMG_2292.JPEG
IMG_2292.JPEG
Attachment 15: Jun25-2025.pdf
Jun25-2025.pdf Jun25-2025.pdf Jun25-2025.pdf Jun25-2025.pdf Jun25-2025.pdf Jun25-2025.pdf Jun25-2025.pdf Jun25-2025.pdf
  363   Monday, February 10, 2025, 16:52 Albert KongStandard OperationTarget 1CoolingT1\T2 Cooling Packages Refill and Restart

The T1 cooling package was refilled to ~38cm and restarted. Shortly after restarting reservoir level dropped and settled to 37.8cm.

No visible leaks when viewed from the Meson Hall south walkway,level seems steady after ~1 hour of operation, water conductivity also returned to normal after a few minutes of flow.

All water PV green at EOD.

------------------------------------------

UPDATE Feb 11:

At approximately 6.00pm yesterday, the water level in the expansion tank dropped to 36.4 cm, where it remained level overnight.

 All cooling system PVs stayed green overnight.

------------------------------------------

UPDATE Feb 12:

Water level in the expansion tank remained level at 36.3 cm. Cooling package PVs all green.

-------------------------------------------

UPDATE Mar 27:

The T1-MK2 demin water flowrate was tuned to 1.1-1.2 gpm when in position 1 (12mm Be target). The profile monitor was actuated again in position 0 and confirmed to be working properly.

During this test, we found an odd 'stuck' state that the T1 system controls can enter where the control room operators became unable to turn off the cooling system. It turns out they don't have the ability to turn off single devices, like the pump, and can only turn off the whole system.

We likely arrived at this state by raising the ladder further, even though we were already at position 0 (manual jog button was used by the operator). As a result, flow was blocked and the water temperature ahead of the pump kept rising. This is actually quite a critical failure mode, in the future to prevent this problem, we should instruct control room operators to turn off the cooling system before driving the target ladder.

We solves this issue by asking Tony Tateyama to turn off the pump for us, likely his EPICS user account has the ability to turn off individual devices. With the cooling system turned off, we were able to complete the tests and tune the demin water flowrate.

-------------------------------------------

UPDATE Apr 3:

After the T2-MK2 target ladder was moved to the target station, the T2 cooling package was filled to 38.2cm and the cooling package was restarted after connecting services to the ladder. The water level in the expansion tank dropped to 32.8cm after restarting.

The level in the T1 cooling package dropped further to 34.9cm, which we assume is caused by trapped air that escaped the cooling system volume.

-------------------------------------------

UPDATE Apr 11:

T2 cooling package refilled to 38.3 cm, T1 cooling package refilled to 39.7cm.

 

 

Attachment 1: T1-Level.png
T1-Level.png
Attachment 2: T1-CoolingPkgEpics-10Feb2025.png
T1-CoolingPkgEpics-10Feb2025.png
Attachment 3: Overnight_Water_Level_T1.png
Overnight_Water_Level_T1.png
Attachment 4: T1_Water_Level_Stable_(13Feb2025).png
T1_Water_Level_Stable_(13Feb2025).png
Attachment 5: 50adcc22-08a1-4c73-986d-32a80adb1ded.jpg
50adcc22-08a1-4c73-986d-32a80adb1ded.jpg
Attachment 6: Screenshot_2025-04-03_133035.png
Screenshot_2025-04-03_133035.png
  367   Tuesday, June 03, 2025, 11:39 Aaron TamStandard OperationOtherBL1A Holding TankBL1A Active Water Holding Tank

 June 3, 2025 - Albert Kong, Aaron Tam 

RPG finished analysis of the water sample and given approval for the contents of the holding tank to be released to city sewer

Procedure following Document-64834 Release No. 3 performed 

  • Starting volume sighted at 98cm mark ~
  • Approximately 100 minutes (9:05am-10:45am) pumping @ 1.70-1.80 USG/min 
  • Pumping till dry (pump noise noticeably changed when dry) (Dry pumping time max 30 seconds)

BL1A Active Water Holding Tank ready again to receive water from T1/T2/TNF.

 

 

Attachment 1: Sight_Glass_1.jpg
Sight_Glass_1.jpg
Attachment 2: Sight_Glass_2.jpg
Sight_Glass_2.jpg
Attachment 3: Tank_conditions_to_start.jpg
Tank_conditions_to_start.jpg
Attachment 4: Tritium_Sample_and_Analysis_Form_-_T2_Cooling_Package_-_2025-01-10_-_Completed_2025-05-27.pdf
Tritium_Sample_and_Analysis_Form_-_T2_Cooling_Package_-_2025-01-10_-_Completed_2025-05-27.pdf Tritium_Sample_and_Analysis_Form_-_T2_Cooling_Package_-_2025-01-10_-_Completed_2025-05-27.pdf
ELOG V2.9.2-2455