A fault was reported June 21, 2022 (Fault 15033) regarding T1 inlet pressure warnings... the pressure reading slowly dropped over time as seen in the attached strip tool screenshot (B1A:T1CS:PGIN). The pressure reading issue is attributed to a fault pressure transducer. The transducer was replaced on Oct. 11, 2022 my M. Dalla Valle. It is currently functioning as expected and the system is operational.

The sump level was checked by Robin Swanson on Dec. 31, 2022. It was essentially empty - see attached photo.

It was reported on Dec. 20, 2022 that when Operations attempted to run the target ladder motor, it was unresponsive. A fault report was created: 15477.

During winter shutdown 2023, the T2 area was uncovered and the motor was inspected on 2023-01-11 by Adam Newsome and Maico Dalla Valle. The connectors and cabling appeared OK.

Upon removing the motor from its mounted position, it became clear that the coupling part between the motor and the gearbox had failed. The motor was running properly but not actually connected to the drive shaft and therefore nothing happened when ops tried to move the ladder. We confirmed that the motor runs properly in no-load condition when it is isolated from the rest of the assembly. We were able to move the rest of the drive shaft by hand and it does not appear to have any significant resistance, so it is suspected that this coupling failed due to radiation exposure over a long period of time rather than overload. The coupling part is plastic and designed intentionally to break under high loads (it is thinned out in the middle, where it snapped). Hopefully it did not actually see any high loads (i.e. there is no other root cause of this issue) and it simply degraded so much that it eventually broke.  

To resolve this issue, we will fabricate/purchase and install another one (Albert, Maico) and work with operations to test that all target ladder positions are reachable once it is installed. We will also confirm at that time that the limit switches and potentiometer readback function as expected.

It is recommended to consider the following changes to the drive system going forward to prevent this from happening in the future:

• Replace these plastic coupling parts with metal versions
• Replace the plastic coupling part that is thinned out to intentionally fail under high load with another method of system protection. The same end result could be achieved using a clutch or perhaps a fuse/circuit breaker which would effectively inhibit motion if the motor experienced overload. This would be a significant improvement over breakage because it would not require replacing the part (dose-intensive) and would be more easily recognizable.

For now, this drop-in replacement should suffice because it is not likely that the part will degrade so significantly within the next few years that this would happen again (unless there is a different root cause aside from radiation that we are not aware of!).

[update 2023-01-01] The broken coupler has been replaced by Albert Kong and Adam Newsome. The replacement part was tested - DCR operated the target ladder motor, moving it up and down through all of its positions multiple times. Everything mechanically and electrically appeared OK during motion. DCR confirmed they are able to reach each position and the proper readbacks are present - everything looks OK on their end. The issue is considered resolved and the fault is returned.

Upon uncovering the T1 and T2 areas to prepare for regular cooling package maintenance, it was discovered that multiple ball valves were leaking. Primarily the Swagelok SS-45S12 parts, but some SS-45S8 parts have failed as well. There were small puddles of water underneath many of the valves. It appears that the leak is due to a compromised seal. It is suspected that these parts failed from radiation exposure. They have PTFE packing material which is highly susceptible to decomposition under irradiation, and as far as we can tell they have not really been repaired/replaced (aside from a written note on one saying the seal was changed in 1997).

It is recommended to add the regular repair/replacement of these valves to the annual shutdown MRO checklist.

Once replaced, this e-log will be updated to document which ones were replaced.

On Jan. 13, the T1 and T2 cooling package water reservoirs were drained to the BL1A holding tank. Water samples were obtained for each system and provided to RPG for analysis.

Prior to starting draining, the BL1A holding tank water level was approximately 330L. After draining, the water level was approximately 550L (checked Jan. 16).

T1 Cooling Package MRO began Jan. 16, 2023.

The following maintenance tasks have been completed for T1 (this list will be updated as work progresses):

• Drained water
• Hansen fittings o-rings inspected: OK
• Replaced demineralizing flow metering valve (part number SS-8BK*)... this part was replaced because the equivalent part on T2 exhibited potential signs of wear during 2022 and was replaced in 2022 mini-shutdown (October). There are two things to note about this part: 1) when loosening it, it makes an audible *click* at some point during loosening rather than a smooth continuous feeling like the new replacement part, and 2) it appears to have originally been installed backwards in terms of the flow rate direction arrow. It is now installed correctly.
• Demineralizing resin can - exchanged with fresh resin (2023-01-18... dose rate of old resin which is currently located in the warm cell is approximately 40 uSv/h on contact)
• Replaced filters:
  
  • Target water CUNO filter (it was relatively clean)
  • M15 CUNO filter (it was relatively clean)
  • Demineralizing 90 micron filter (it was very dirty with tiny pieces of debris)
• Replaced leaky valves (see e-log 318 and 320)
• Refilled expansion tank water (see e-log 321)
• Tuned demin flow rate to 1.1 gpm, turned on pump [2023-03-06], confirmed system functionality (all sensors OK, no water leaks observed). See screenshot attached of T1 state at approximately 4:00pm on 2023-03-06 after the system had been running for a significant time.

As of now, the cooling system is ready for beam. Prior to beam-on, actuation tests of key items such as the target ladder and beam blocker need to be performed. It is expected that this will be done within the next 1-2 weeks.

T2 Cooling Package MRO began Jan. 16, 2023.

The following maintenance tasks have been completed for T2 (this list will be updated as work progresses):

• Drained water [2023-01-16]
• Replaced ball valve located between heat exchanger and target flow sensor FGTGT (part number SS-45S12)... this part, as well as numerous other ones, were noted to have been leaking. See e-log 308
• Replaced both CUNO filters (target water and demin. resin circuit filter) [2023-02-01]
• Replaced 90 micron demin. filter (the filter was very dirty - it was last changed in Feb. 2012 as indicated by written note) [2023-02-01]
• Inspected Hansen fittings (o-rings look OK) [2023-02-01]
• Replaced o-ring, shaft, and paddle wheel for flow sensor FGSEC (Q3). The old paddle wheel had significant wear in the shaft hole which resulted in sensor readback issues observed during 2022 operation. It is likely that this was caused by debris or a manufacturing defect which got worse over time. Photos show a comparison between old and new. [2023-02-01]
• Replaced leaky valves (see e-log 318 and 320)
• Refilled expansion tank water (see e-log 322)
• Changed M20 beam block o-ring seals, and performed vacuum check (see e-logs 319, 324, and 323)
• Tuned demin flow rate to 1.1 gpm, turned on pump [2023-03-06], confirmed system functionality (all sensors OK, no water leaks observed). Note: target water conductivity sensor (CONDTGT) took a few hours to ramp up to appropriate value. See screenshot attached of T2 state at approximately 9:15am on 2023-03-07 after the system ran overnight.

As of now, the cooling system is ready for beam. Prior to beam-on, actuation tests of key items such as the target ladder and beam blocker need to be performed. It is expected that this will be done within the next 1-2 weeks.

The flow rate sensor used to monitor the drain rate from BL1A holding tank to active sump, located in the active sump area, was repaired. The paddle wheel, o-ring, and shaft were replaced. This part was significantly worn prior to replacement.

It is recommended to change the filter adjacent to it as it is suspected that this filter is partially clogged, resulting in low flow rate when draining the tank.

Furthermore, it is also recommended to change solenoid valve SV1 as it is suspected that it does not work.

The CUNO filters were changed 2023-01-17 for:

• BL1A Holding Tank drain output, adjacent to the pump
• RH Active Sump, adjacent to the V2 valve

Both of these filters are in-line with the drainage lines from the BL1A holding tank to the active sump. It was suspected that they have not been changed in many years, if ever. Prior to changing the filters, the maximum flow rate measured from the Proteus paddle wheel sensor during draining was around 0.85 gpm (the draining procedure document indicates it should be between 0.9 to 2.5 gpm). After replacing both filters, which each showed significant clogging of sediment, the flow rate significantly increased and the maximum of 2.5 gpm was achievable. Note that when the drain pump was initially turned on, it took some time to reach the desired flow rate due to air in the water lines - the flow rate was quite low until the lines were once again filled with water.

It is suggested that these filters should be replaced every ~3-5 years, depending on usage, or otherwise as-needed (it is possible to see through the clear walls and determine how dirty they are). This MRO item will be added to the checklist of routine T1/T2 related tasks performed to prevent flow rate issues in the future during draining.

Filters were bagged and discarded as active waste.

Attached photos show old filter in bucket, and newly replaced filter, for the RH Active Sump.

It was noticed today (2023-01-17) that the valve V6, used in draining the BL1A Holding Tank to the Active Sump, was leaky. This valve should be replaced as soon as possible. It has been added to the MS Planner maintenance tasks list.

The HEPA filter in the nuclear ventilation system for the Meson Hall Hot Cell was changed and tested by mechanical services under work permit C2023-04-25-14.

They have given approval to operate again. The system is re-energized.

See Cyclotron Fault 15787. The target ladder motor failed to move to the commanded position during a test. The T1 area was uncovered and the motor assembly was inspected by M. Dalla Valle, A. Newsome, and A. Kong. It was determined that the rubber coupler which connects the motor shaft to the drive assembly had degraded and snapped. This is similar to an incident which recently occurred on T2. The coupler was replaced with an all-metal version. The system was tested by the DCR operators. They ran the target ladder to each extreme (position 5 to position 0) multiple times. The target ladder was also moved to a few positions which had targets installed, and the cooling package was energized to run water through and confirm functionality. All feedback sensors for the target ladder assembly and cooling package were observed to behave normally, as confirmed by the DCR. RH group confirmed visually that there were no observable issues. The system is now considered operational.

It is recommended to change all drive system coupler parts on both T1 and T2 during the next shutdown to prevent this issue from happening in the future. These parts should be inspected during the annual inspection of the T1/T2 assemblies as well.

Between 1:00 pm and 6:00 pm on July 17, 2023, the T1 target cooling water flow rate sensor exhibited some noise which resulted in it dropping below the warning/trip limit. The control room requested that the threshold values be lowered - permission was granted. The thresholds were set to 2.3 gpm for both warn and trip levels. See attached screenshot from EPICS sensor data. The occasional spikes and dips continued throughout the night fairly consistently.

While the cooling water temperature (TCOUT) fluctuates, there does not seem to be an obvious correlation between the temperature and the flow rate, which suggests that the flow rate is still adequate.

It is worth noting that overall, the FGTGT signal has been slightly trending downward since around mid-July (see attached pic).

It is suggested to replace this sensor as soon as possible - perhaps the mini shutdown.

As of 2024-04-03, the BL1A Holding Tank water level is approximately 580L. It is recommended to drain the tank prior to the start of the operating year.

A safety walkaround for July 2024 was completed by A. Kong.

Results can be found in the master spreadsheet

Major deficiencies:

• Identified a 1/2 ton chain hoist missing an inspection tag. Mechanical Services will need to be informed about this.

During a replacement of the RCR1 active sump (mechanical services group), approximately 600L of non-active grey water was released into the RH active sump. At this time, this should be all of the water in the RH sump (approximately 600L total).

A safety walkaround was completed for the Meson Hall Hot/Warm Cells.

The resulting spreadsheet can be found on DocuShare as Document-242733.

Main deficiencies identified:

• Hot Cells:
  • Pressure gauge reading is suspect
  • Operator station phone not working (resolved now)
• General:
  • Lifting equipment has overdue inspection

Action has been taken on all deficiencies.

The following maintenance tasks were completed on 2024-11-29 by A. Newsome, A. Kong, A. Tam, M. Dalla Valle:

• Warm cell right window water level topup... the water level was a couple inches lower from the top. Using a tube connected to the nearby sink, the level was topped up fully. See attached before/after picture.
• Hot cell oil level checks:
  • Right side OK. See attached picture.
  • Left side very low compared to July 2022 measurement (suspect internal leak?). The oil was topped up. See attached before/after picture.
• Hot cell scissor table pump oil level check - M. Dalla Valle notes this was topped up approximately July 2022. The table was operated and functioned normally, did not sound concerning. The oil level was visible and acceptable.

Still to do, next maintenance check-up:

• Check HC atmospheric pressure differential gauge
• Lubricate HC turntable (planned for 2025 winter shutdown when target assembly not present)
• Lubricate all telemanipulators