We tested the air amplifiers by actuating the beam blockers and profile monitors on the T1 and T2 target stations. 

The blockers and profile monitors were each actuated (brought up/out and down/in fully) > 5 times and vacuum remained stable throughout (see attached).

The cycle rate of the air amplifiers (time between 'puffs') under steady conditions (target devices kept in the out position) was measured to be between 60-90 seconds, with 90 seconds corresponding to when the 'flow control' ball valve is fully shut and ~60 seconds when the valve is opened (either partly or fully). 

When a target device is brought up, a cycle event will occur shortly after the device reaches the out position.

This cycle time is much longer than last reported in 2014 (see elog #119), by a factor of 2-3. 

The top amplifier sounds different from the bottom amplifier (cycles between being 'rattly' and 'quite' between cycles).

Seeing how the beam blockers and profile monitors were actuated smoothly with the air amplifiers, we will keep the system running off the lower air amplifier with the 'flow control' ball valve kept open (not shown).

At some point we will look into disassembling the top amplifier to see why it sounds differently between cycle events.

For reference, both amplifiers read 20psi at the regulator and 120 psi to the target station. 

The T1 motor controller hearbeat tripped (noticed this morning). 

This does not seem prevent beam delivery or target ladder actuation (see screenshot, target still ready for beam).

An improperly functioning hearbeat led on a controller may indicate that the controller is beginning to fail.

Interestingly, the profile monitor 'out' status registers a warning - the two may be related: i.e. some electrical work interrupted the two signals. 

This problem will be looked at in the coming days.

(Update) 

After consulting with Tony Tateyama from Cyclotorn Controls group, the two trips were re-set and the controller heartbeat is now 'green' again.

The motor controller is likely still healthy, seeing how it was installed only a few years back (~2015). The cause of the trip should be some electrical work on the mezzanine.

(Update - May 08, 2023)

The controller HB tripped again (noticed in the morning). Coordinated with operators to have it re-set. This was done automatically by driving the target ladder to position 2 then back to position 0.

(Update - May 16, 2023)

The controller HB on T2 tripped, and re-set by ops. 

The HEPA filter in the nuclear ventilation system for the Meson Hall Hot Cell was changed and tested by mechanical services under work permit C2023-04-25-14.

They have given approval to operate again. The system is re-energized.

The level sensor for the active sump in the RH lab was tested.
The level sensor is working properly and is operational.
 

Today we re-lubricated the o-rings on the top air amplifier for T1/T2, and replaced the o-rings on the bottom air amplifier with new and also lubricated o-rings. 

We used Haskel lubricant #50866.

The bottom air amplifier spool assembly was noticably dirtier than the top (see attached). and we noticed some worn spots on the o-rings. This was not the case with the top air amplifier spool (it has not been used outside of testing since being fully serviced in 2012).

Replacing the o-rings did not result in any perceptible change to the cycle rate and sound of either air amplifier (~50 sec/cycle, same sounds made by the piston before and after servicing). 

We initially planned to also remove the sleeve on the amplifiers (see image) but could not do so easily. We will contact Haskel to get some input regarding how best to remove the sleeve.

The plan is to perform a full teardown of the lower air amplifier next shutdown (2024).

 The following tasks were performed:

• All electrical, water, and pneumatic connections to the T1 target was reconnected.
• 3x and 2x CUNO filters for T1 and T2 respectively were changed (O-ring for the housings were not changed).
• All hansen fittings on the front mounting plates of the T1 and T2 cooling packages were inspected.
• All proteus paddle wheels (Q1-5) for both T1 and T2 were replaced. 
• The main reservoir drain valves on both cooling packages were closed in preparation for re-filling.
• All other ball valves in the cooling packages were returned to open, also in preparation for re-filling.

Two buckets containing (primarily) active water from the CUNO filters were brought closer to the boot up area, to be moved to the active sink in the MH hot cell lab in the coming days (pending ELOG update). 

Spent shafts, paddle wheels, and o-rings from the proteus flow meters will be checked and appropriately tagged/disposed of, also in the coming days.

------------------------------------------------

Some observations for future shutdown work:

• The drain port at the bottom of the CUNO filter housings can be used to verify that the filter line has been successfully closed off.
• Prepare several lengths of flathead screws to service the proteus flowmeters.
• Prepare spare locknuts and screws for the proteus flowmeters in case they become stripped or a locknut is dropped.
• Stick to well-fitting gloves to make using screwdrivers, picking screws, etc. easier.
• Possibly a pipe or coupling could be used with the tool for the CUNO filters to make it easier to loosen the CUNO filter nut in hard to reach areas.
• If possible, it may be a good idea to replace the o-rings on the CUNO filter housings in future shutdown operations (dash # to be identified). 

The lower amplifier for the T2 target station air supply was removed for teardown, inspection, and rebuild. 

The goal of this operation is to understand wear development in the air amplifier over ~13 years of operation, and potentially determine a recommeded service interval.

---------------------------------------------------------------

The T2 volume was vented for an unrelated maintenance operation during this time.

During testing, prior to removing the lower amplifier, both regulators were set to ~20 psi. 

The upper air amplifier resulted in ~125 psi at the outlet while the lower air amplifier resulted in ~120 psi at the outlet.

The upper air amplifier had more audible air flowing out from the muffler than the lower amplifier. 

'Scratching' sounds in both amplifiers were comparable.

The following cycle times were recorded with the M20 BB raised/out (min:sec):

    UPPER: 1:22 / 1:00 / 2:06

    LOWER: 2:30 / 1:46 / 2:12

The following times were recorded to raise the M20 BB (sec): 

    UPPER: 8.36 / 8.76

    LOWER: 8.56 / 9.10

These times will be compared against after completing the teardown and rebuild of the lower amplifier, at which point this ELOG will be updated.

-----------------------------------------------------------------

UPDATE (Feb 23, 2024):

The lower air amplifier was serviced (photos in 'S:\Albert Kong\Shutdown Files\2024\Feb20 T2 Lower Amplifier Cleanup'):

1. Full assembly cleaning.
2. Piston and barrel was cleaned and lubricated, o-rings and plastic components replaced (with new lubricated ones).
3. Pilot valve components replaced (except plug).
4. Check valves (4x) cleaned and plastic/rubber components replaced (with new lubricated ones). 
5. Muffler cleaned (blown out with compressed air).
6. Spool and sleeve assembly cleaned, o-rings replaced (with new lubricated ones), and rubber stopper replaced.
7. Clamping rods tightened to ~17 ft-lbs. 

Note: the piston o-rings were difficult to seat on the piston body/teflon ring. During assembly, we instead seated the o-ring in the barrel on the piston plates (see picture), which allowed the oring to be seated properly, before placing the piston body onto the piston rod.

After servicing, the amplifier was returned to the station, air connections reconnected, and tested. 

Note: it is recommended to do torque-ing of the clamping rods as a final step to simplify mounting of the amplilfier and re-doing connections to the rest of the compressed air system.

The first observation we made was how silently the lower amplifier now operates when cycled: only the exhaust sound from the muffler can be heard.

Note that the piston's motion can be heard when listening ~5cm away from the amplifier barrel. 

The following times were recorded to raise the M20 BB (sec):

    UPPER: ~8.5 

    LOWER: ~7.7s

The following cycle times were recorded with the M20 BB in the out/raised position (min:sec):

    UPPER: 1:05 / 1:04

    LOWER: 1.22 / 1:45

The outlet pressure from the amplifier registers 120 psi with the regulator set only to 15 psi (improvement from previous performance as well as the upper air amplifier's performance).

We will check in on the amplifier next week to see if it still operates silently and can actuate the beam blockers/profile monitor without issue, at which point this ELOG will be updated. 

-----------------------------------------------------------------

UPDATE (Feb 26, 2024):

The lower amplifier was inspected this morning, higher volume sound could be heard from the drum in concert with the motion of the piston, but still much quieter than before servicing.

The following maintenance tasks were completed on 2024-11-29 by A. Newsome, A. Kong, A. Tam, M. Dalla Valle:

• Warm cell right window water level topup... the water level was a couple inches lower from the top. Using a tube connected to the nearby sink, the level was topped up fully. See attached before/after picture.
• Hot cell oil level checks:
  • Right side OK. See attached picture.
  • Left side very low compared to July 2022 measurement (suspect internal leak?). The oil was topped up. See attached before/after picture.
• Hot cell scissor table pump oil level check - M. Dalla Valle notes this was topped up approximately July 2022. The table was operated and functioned normally, did not sound concerning. The oil level was visible and acceptable.

Still to do, next maintenance check-up:

• Check HC atmospheric pressure differential gauge
• Lubricate HC turntable (planned for 2025 winter shutdown when target assembly not present)
• Lubricate all telemanipulators

blah blah blah.... fjda;jfeowifjoajglfa;nafg

Mike Mouat enabled recording for T1 and T2 expansion tank warning and trip level sensors on XTPAGE/XSTRIP.

XSTRIP output is in discreet integers as follows:
2 = Both sensors OK
1 = Warning level NOT OK
0 = Trip level NOT OK

Mike is now working on having a warning message show up in the control room when the sensors read NOT OK.

 Mike Mouat and Juan Pon have modified the control system so that the control room will receive a message when the warning or trip level sensors for the T1 & T2 expansion tanks read "NOT OK".  The message also displays the current water height in the tank.  This will help with earlier detection of water leaks from the cooling package or from the target ladder in the vacuum volume when the vacuum interlock has been defeated.

All sensors were checked by depressing the level sensor float and the correct warnings came up on the message reader.

 Installation of the seismic restraint clamps has been completed for the MK1 and MK2 target flasks in the Remote Handling lab (photos attached).  Two clamps on the MK1 flask required 3/8" shims due to warpage of the flask baseplate (see photo IMG_0429).

Dragan Mitrovic has been notified so that he can inspect the installation and close the work request.

Installation, wiring, and commissioning of the high temperature limit switches for the T1 and T2 cooling packages was completed today as part of SAS project SASP0120.  The relay output from the switches was wired in series with the Central Safety System signals for T1 and T2 (cable number 13250 and 12250 respectively) in break-out panel #2 on the Meson Hall south mezzanine.  There were already 3 other circuits wired in series with each signal for various other interlock requirements.

After wiring, commissioning of the switches was performed according to the following procedure:

- With the limit switches in a non-tripped state, it was confirmed that the safety signals were reaching the CSS in the main control room as expected (had to jumper T2 circuit to achieve this as the T2 target is not currently installed)
- The limit switch was tripped by adjusting the set point on the switch, it was confirmed that the signal was lost
- The set point was changed to a non-trip level, and the switch trip cleared, confirmed that the signal in MCR returned
- The enable signal from each cooling package was removed by shutting off the cooling package (for T2 just removed jumper), confirmed that safety signal lost as expected
- Reversed this condition and confirmed that the signal returned
- Checked that the thermocouples are performing properly, and the switches actually trip on high temperature by directly applying heat to the thermocouple area on the cooling package using a heat gun;  the switches tripped as expected when the set-points were exceeded

The results were discussed with John Drozdoff (Safety Systems group leader) who approved the method.

The final task remaining for SASP0120 is to route the PRV outlets to the 1A tunnel active drain using flexible hosing.

Yesterday (April 18th) the T1 and T2 cooling package PRV outlet lines were routed to the active drain in the 1A tunnel.  All items for the safety and standards compliance upgrade for T1 and T2 as specified in Document-68861 are now complete.

The T1 and T2 profile monitors were used during today's development shift.  Target ladder movement for T1 was done at the panel on the mezzanine, for T2 the portable control box was used.  Both monitors actuated properly and the scans did not show any "missing" wires.  The profiles at both targets were found to be too broad and were adjusted to span 2 wires horizontally and 6-7 wires vertically (Yi-Nong Rao).

 An end-to-end test was performed on the new T1 and T2 high temperature safety switches yesterday during the maintenance day.  The switch set points were lowered below the current temperature and it was confirmed that all warning messages and XTPAGE status changes occurred as expected.

Water detecting strips were installed around the T1 and T2 areas today with Doug Preddy and contractors from SMT Research.  Four sensors were installed at various locations on each cooling package.  Four sensors were installed on the top surface of the T2 monolith.  In addition, four sensors were routed for future installation at the T1 monolith (it is currently covered by blocks and requires cleaning and painting before sensor installation).

See the attached PDF for sensor ID numbers and placement specifics.

Photos are attached of the sensors installed around the T2 monolith and cooling package.

The T2 target was exercised to position 0 around 6:15pm so that the T2 profile monitor could be used by operations.  The T2 water package was re-enabled when the target was in position 0, however this resulted in some trips of the cooling package.

Operations called around 7:45 to request return of the target to about 2mm above position 4.

The target position was re-set so that the position reading fluctuated roughly between 2.7 and 1.6 mm above position 4 (hovering roughly around 2.0 mm), and so the potentiometer reading was hovering roughly between 0.801 and 0.798.

The cooling package was re-started, and called operations to confirm the system was OK.

A filter housing and filter were installed by Dan McDonald on November 14 in the piping section between the Remote Handling Hot Cell Lab active sump and the city sewer drain as shown in the attached picture.  The purpose of the new filter is to prevent pieces of active material from being released to the drain when the sump is pumped out.  If small pieces of active material enter the sump they may sink to the bottom, and not be captured in a water sample which is taken from the top of the sump.

The filter housing is a Waterite HP1034CLUR-KIT, and the installed filter is a 10" Cuno Microwynd 25micron filter.

This modification was approved by Curtis Ballard, Joe Mildenberg, and Grant Minor prior to installation.

 Ron Kuramoto, Keith Ng, John McKinnon, and myself spent some time today investigating water and air lines related to the T1 Rotary Collimator (Col. A).  We discovered the following:

- There is a panel on the back (south) side of the T1 cooling package with 3 air lines for Col. A actuation, Col A cooling water supply & return, and the Col. Shield water cooling supply and return
- The air lines coming from the panel were decomissioned in 2003 by Tom Lyth, and at least 2 out of 3 were replaced with CuALCW water from the M13 header in the BL1A tunnel
- Motion / actuation of the rotary collimator was disabled in Oct 1994 and it is now used only in one position
- The two water lines coming from the M13 header (which connect to the actuation cylinder) will be disconnected, capped, and tagged.  The M13 header can then be completely removed
- It is suspected, but not yet 100% confirmed that Col. A and the Collimator Shield are still cooled by the lines passing through the panel at the back of the cooling package
- A flow test will be performed to confirm this, which will also confirm that the Q4 flow meter at the cooling package measures cooling water flow to Col. A