Motor contactors for the cyclotron elevating system jacks were inspected on 2025-01-23 by Joel Semilla under work permit C2024-01-19-1. Joel's assessment is as follows:

"All the twelve magnetic contactors have the same contact tips condition. Last year, I have [sic] to take out the contacts from the housing due to severe "pitting" of the contact's silver surface due to excessive "flash-over" which is common to magnetic contactor on inching/jogging duty. Due to the better condition of the contact tips this year, I can re-surface the contact tips on the magnetic housing itself."

Prior to recent inspections, a recurring inspection at an informed frequency had not been set. It is suggested based on the state of the contactors now, in comparison with the previous inspection, to schedule inspections of these contactors every 4 years. Next inspection due: Jan. 2028.

Edit: on 2024-01-24, Joel Semilla updates: "re-surfacing of stationary and movable contacts for the 12 cyclotron elevating motors have been completed".

Because of observed noise on the synchro at jack station 11 during recent shutdown lid up/down operations, the synchro at this station was replaced on Jan. 31, 2024. The procedure in Document-234859 was followed.

Prior to starting the replacement, the following counter values and synchro measurements on the station in question and adjacent stations were noted:
The master synchro count value was measured at: 0000
Station 10: Counter: 99392.5, synchro voltage: 5.89 VAC
Station 11: Counter: 0073, synchro voltage: 0.787 VAC (anomaly)
Station 12: Counter: 0068, synchro voltage: 5.98 VAC
 

During the replacement, the master synchro was jogged such that adjacent stations 10 and 12 read a synchro value of approximately 0 volts (as per the nulling procedure). The following observations were made:
The master synchro count value was measured at: 0072
Station 10: synchro voltage: 0.047 VAC
Station 11: synchro voltage prior to swap: 5.70 VAC (anomaly)
Station 12: synchro voltage: 0.282 VAC

After the replacement, before jogging the master synchro back to its starting value of 0000, the following measurements were made:
Station 10: synchro voltage: 0.038 VAC
Station 11: synchro voltage after swap: 0.12 VAC (now correct)
Station 12: synchro voltage: 0.278 VAC

When jogging the master synchro after the swap was completed, it was confirmed that stations 10, 11, and 12 all responded as expected, and their values were synchronized with respect to each other, indicating a successful replacement. The master synchro was jogged back to its original counter value of 0000. The system is now considered operational.

 The annual elevating system maintenance for the 2023-2024 shutdown following Document-220322, 

Work done after electrical lockout of elevating system and motors electrically disconnected: 

Work done on Station #1: 

• Note readings on station counter (71) and gap clearance on yoke (5/8" wrench thickness) 
• Remove all spares(#13)  from station 1 (two jacks, two highfield gear reducers and one electric motor) 
• Reinstall complete serviced original station into station 1
• Set yoke gap and counter to same as previously recorded

Work done on Station #7: 

• Note readings on station counter (71) and gap clearance on yoke (9/16" wrench thickness) 
• Remove originals from station (two jacks, two highfield gear reducers and one electric motor) 
• Install Spare elevating system (station #13) into station #7 
• Set yoke gap and counter to same as previously recorded 
• Inspected and serviced upper bearing: OK 
• the removed originals are taken to the Remote Handling Active Machine shop where each component will be inspected and serviced to be reinstalled next shutdown

Electric motors have been reconnected at stations #1 and #7.

It was observed when attempting lid up operation on 2024-02-06 that the brakes for each of the jacks were not operating correctly. It appeared that power was not being applied so the brakes were stuck in the engaged state. This ultimately resulted in issues with the lid up operation.

The issue was traced to relay A1 (the master brake power relay) not toggling as expected, despite the relevant control signals being present to try to enable it. This relay is located in the MCC-E 400V panel. Adjacent to the relay is fuse F4 which supplies the relay. This fuse was blown. The fuse was: English Electric CIS50. 50A rated, 600V, H.R.C. Form II. Replacements were ordered, however the part is obsolete due to its age. In the short term, a temporary workaround was implemented by installing a 30A fuse. This will be replaced when the spare fuses arrive.

The system was powered up again today (2024-02-07) and the brakes energized as expected. It appears that there is no issue. The root cause of why the fuse blew in the first place is unknown at this time. It was originally suspected that it was due to accidental mis-wiring of the brakes when they were reconnected after annual servicing, but if this was true, the fuse would have instantly blown again when relay A1 was energized. The best guess at this point is that the fuse failed due to age (it is likely around 60 years old).

This e-log will be updated if any further developments are made on this issue.

It was observed during lid up operation on 2024-02-08, approximately halfway through raising the lid, that the motor contactor for station 10 located within its small motor panel stopped functioning. The symptoms observed were: entire system stopped since station 10 lagged behind (approx. 150 counts on counter), station 10 could not be jogged locally in either direction, the control signal to activate the motor was being sent to the contactor when attempting to jog, but no motion occurred.

After locking out the panel, the contactor was disassembled and reassembled by an electrician. The system was powered back on and the motor functioned normally. The explanation as to why the issue occurred in the first place is unknown but it is likely attributed to years of infrequent usage causing it to "stick" and/or corrosion buildup.

In any case, it is recommended that all contactors be inspected during the year of 2024 and that at least station 10 be replaced, and spares for other stations procured.

The cyclotron lid was raised starting ~10:30am on 2024-02-06 and finishing at ~3:30pm on 2024-02-08, under work permit C2024-02-06-4.

Upon first attempt to raise the lid, issues were encountered. The system seemed to start as expected but quickly tripped, resulting in no appreciable travel. Upon investigation, it was determined that this was due to a blown fuse which feeds the brakes master relay (see e-log  https://elog.triumf.ca/TIS/RH-Cyclotron/165). The result of this was the the brakes were not properly disengaging, leading to additional friction in the system, and preventing regular motion.

After rectifying this issue, a second attempt was made to raise the lid. This time, issues were encountered with some of the stations being out-of-sync with respect to the others (a result of the troubleshooting which took place for the aforementioned issue). After manually jogging some stations, it was decided to lower the lid all the way, jog motors for individual stations so that they physically match each other in terms of travel distance at a known starting point, manually adjust counters on some stations whose counter values were significantly off so that they matched neighbours, and then jog individual stations such that each synchro measurement was close enough to begin lid up operation.

After this work took place, another attempt to raise the lid was made. This time, the lid traveled fairly smoothly, with a few occasional stops. It seemed that some motors were lagging behind others a bit - namely stations 5, 11, and 12. After raising the lid approximately 4-6", it was decided that the best course of action would be to fully lower it and start again, now that the system had some time to "warm up".

The lid was lowered to the bottom limit again, and then raised. The master synchro count was 0 to start. This time, lid travel was the most smooth, and stops near the start of travel were infrequent. The occasional stop occurred but the system could easily be started again. Jogging individual stations was rarely required. Most stops were due to noise observed on the synchro at station 5 (false trips).
When the lid was approximately halfway up, another unexpected issue occurred: station 10 completely stopped, which caused the entire system to stop. Station 10 could not be jogged in either direction in local mode. After some troubleshooting, it was determined that the motor contactor was causing issues - eventually it was rectified (see e-log https://elog.triumf.ca/TIS/RH-Cyclotron/166).

The lid continued to raise smoothly for the remainder of the travel with very few or no stoppages. It raised all the way, as confirmed by the gap measurement.

Final master synchro count when lid was fully up: 47401

 

See attached inspection checklist which contains information about each jack station during lid up operation, as well as pre-job inspection data.

Recommended actions:
- Replace synchro for station 5 next shutdown
- Inspect all 12 motor contactors prior to next lid down
- Order spare motor contactor
- Install replacement fuse (on order)

Due to an issue with resonator latching, there was a need to raise the cyclotron lid a second time this shutdown.

The plan at first was to raise the lid by small increments (approximately 1mm at a time up to approximately 25mm) while RF group performed some checks.

After the RF groups completed their checks, it was determined that the lid needed to be raised the full extent.

The starting master synchro count originally was 52, and it was 2480 at the time when the decision was made to fully raise the lid.

The lid raised entirely with zero stoppages or observed issues. No abnormalities observed with any jack station.

The lid was fully raised by 1:49pm with a master synchro count of 47383.

See attached summary.

The cyclotron lid was raised starting ~11:45am on 2024-05-02 and finishing at ~12:12pm under work permit C2024-05-02-5.

The raising was stopped a few times up until approximately 5" elevation, for RF group to perform checks, and to inspect the tank seal.

The tank seal was stuck to the lid near jack station 2.

The lid raised smoothly with no stoppages due to out-of-sync stations. No stations needed to be jogged.

Starting master synchro count was 66, but was jogged to 2 prior to starting. The final master synchro count when the lid was fully raised was 47387.

Note: the measuring stick which is affixed to the top of the lid and used to indicate full elevation was bent and should be "re-calibrated".

Summary attached.

A third lid down operation took place over May 6-7, 2024.

The system was energized a few minutes ahead of time.

Starting master synchro value: 47387 (after it was jogged up slightly prior to starting).

Upon startup on May 6 at 14:42, the system worked fine and lowered the lid. The lowering was paused at 15:28 (master synchro value of 17593).

Jack 6 exhibited issues similar to previous lid up (see e-log 166). It was likely due to an issue with a motor contactor. Electricians were contacted but could not work on the issue until the following day so the lowering was paused.

At 07:30 on May 7, electrician Randy Boehm determined that a cable connector on the motor contactor was loosened. After tightening the cable, jogging motor 6 was tested - OK.

Lid down commenced again at 08:10. Master synchro: 1915.

The lowering was stopped at approximately 1" gap as per standard procedure for Operations to perform checks. The lowering was stopped again multiple times to check resonator latching - OK.

End time: 08:37

Ending master synchro value: 59

Jogged master synchro value to 2, adjusted station 12 to match.

Notes: It was observed that at lid down Jack 12 did not have any gap. Jack 12 was jogged to match synchro values with the master synchro but still exhibited a 0.125” gap instead of the ideal 0.25” gap. It is recommended to keep an eye on Jack #12 during the next operation and check the motor contactors for loose fittings. In the before and after photos of the motor contactor it looks like the fitting on the bottom right corner of the motor contactor could have either come loose from the vibrations or the screw was sheared off. Further investigation is required.

See attached summary.

 Highfields 7A,7B post shutdown service complete. Oil change completed, new Mobil 600 W Super Cylinder oil added. Sample collected for RPG of used oil. Both found to be mechanically sound and stored in upper mezzanine. 

 Jack 7R pulled apart and inspected for deficiencies, all parts deemed to be in working order, jack was rebuilt and lubricated with Molub-Alloy 777-1 ES. Jack is deemed fully operational for next swap. 

  Jack 7L pulled apart and inspected for deficiencies, alignment monitored with prussian blue and deemed acceptable wear. All parts in working order, jack was rebuilt and lubricated with Molub-Alloy 777-1 ES. Fully operational for next shutdown period swap.

2C has been installed located on all QDLs and all the numbers, continuity check done and passed. Bridge has been removed from the cyclotron tank.

Setup of the HE 1 probe is difficult has we have no mockup of it to properly setup cameras and equipment for it. This is problem for both extraction and installation of the probe. Also with the setup some of the broken micro switch from the HE 1 was found on the HE 1 probe jig after the jig was deconned and moved up the B2 level. Also for setup required camera adjustment to be made with the heavy contaminated probe being very close to the person setting up the cameras.

Now installation of the probe worked but it was very fiddly and touchy feely on the install, which something one doesn't want for remote installation. It required lots of movements and adjustment and a little luck and the probe went in, even with a air cylinder that did decide to work in the end.

On the extraction of HE 1 was mess and was damaged upon extraction, this was due to improper camera setup (no mock up of probe), and to the fact that removal is very fiddly and not east to the current design of the jig, and the way the probe is locked in place on the lid.

Note on contamination on extraction the probe jig was contaminated with about 20 000 cpm in spots it was removable and the jig cleaned. Contamination was also found on the jig for the installation of the jig it was about 800 cpm jig was cleaned again. Plastic was also laid on the platform of the upper ressonator trolley as to prevent the spread of contamination to the trolley, this appeared to work.  AS A NOTE NONE OF REMOTE HANDLING'S TROLLEYS AND CYCLOTRON EQUIPMENT WAS DESIGNED TO HANDLE CONTAMINATION. If the trolley ever get or contamination is ever found on them they will be impossible to clean.

This morning during routine service bridge work in the Cyclotron tank, it was observed by Travis Cave that the cable tray for the Remote Handling service bridge was catching on something on the top inside surface of the Cyclotron tank.  The cable tray was observed to visibly snag on this protrusion and then bounce around.

Isaac Earle and Grant Minor conducted an inspection from the edge of the Cyclotron tank with cameras and binoculars.  The contact point was identified and photos and video were recorded (see attached).

After a meeting with Yuri Bylinski it was identified that the item interfering with the cable tray was a beam scraper foil (part of the Resinator D Gap), close to the center post area.

A number of causes for the snag were suspected:

1. The upper tank lid might not have been at its upper limit (i.e. not raised completely by the elevation jacking system)
2. The center post bearing for the service bridge might not be seated properly
3. The cable tray mounting bracketry or air cylinder may have been displaced

After inspections of the service bridge before and after removal from the tank, it was determined that neither 2 nor 3 were the cause of the problem.

It was observed by the Remote Handling group that the upper limit switches located at stations 1 and 10 had not advanced high enough to mechanically "click", even though their wheels were contacting the reference surface.  There was about 0.5 to 0.75 inch of travel remaining in these switches (see attached photos) before clicking.

It was also observed that the wooden stick normally set flush with the Cyclotron roof beams at max height was about 0.5" below the roof beams (measured with a Starrett scale by Grant Minor).

After further assessment, it was also determined by Dan Louie and Yuri Bylinski that the limit switch lamp on the elevating control system was actually not lit, even though it appeared so.  The dull glow in the lamp was due to the neon style of the bulb, which can glow a small amount even at very low voltages and appear to be lit.  A test of raising the switches while watching the lamp revealed that the lamp switched on brighter after the limit clicked.

The upper tank lid was raised around 6pm by Dale N. with Yuri Bylinski and Grant Minor present.  The stick advanced upwards about 0.5" almost completely to the ceiling (see attached photos).

Until a systematic method of measurement of the vertical displacement of the system is developed (this may be the purpose of the dial indicators at each station, but requires further investigation), this problem can be avoided again by always checking both the limit switches and the distance from the calibrated "wooden stick" to the cyclotron roof beams.

This morning Don Jackson commented to Grant Minor that he believed the tank lid was still not raised high enough to safely operate the Remote Handling bridge and trolleys, despite it having been raised 0.5" yesterday to the point of engaging the limit switches (see e-log from yesterday).  Don said he believed this to be the case as a result of a visual inspection with binoculars from the outside of the tank, as well as from the views achievable with the remote cameras.

Don also believed he had caused further damage to the equipment inside the tank by orbiting with the bridge.

After a teleconference with Klaus Reiniger after lunch (Yuri Bylinski, Dan Louie, Don Jackson, Grant Minor present), the following was confirmed:

• There is a complete set of limit switches at station 1, and duplicate set at station 10 for redundancy.
• Both stations 1 and 10 have an operator upper limit switch, as well as an over-travel limit set slightly higher than the upper limit.
• The lid has traditionally been raised to the point of the upper limit switch tripping, then the lift procedure stops there.
• The wooden stick was traditionally set to just touch the roof beams with the lid at the upper limit.  This means if the stick is contacting the roof beams within 1/32nd inch or so and the upper limit trips, the lid has been raised fully.
• Klaus could only remember one incident of the lid being raised so high before tripping the upper limit that the wooden stick actually bowed under pressure from the roof beams.  In other words, this was not a normal occurrence.
• Klaus also reminded us that the limit switches have some mechanical hysteresis and need to be actuated through their full range of motion to reset them.

Don Jackson indicated that another 1/4" of clearance would be adequate to safely operate the RH equipment in his opinion.

At around 5pm Grant Minor and Ron Kuramoto reset the upper limit switches at both stations 1 and 10 by adjusting the block that both upper limit and over-travel are mounted to, see attached photos.

• A line was scribed in the primer paint to mark the original position of the limit switch mounting block.
• The bolt was loosened, and the block was raised fully to reset the switches, and then set 1/4" higher than the scribed line, measured with a Starrett scale accurate within about 1/64"
• The bolt was re-tightened.

Dan Louie and a shift operator reset the wooden stick 1/4" lower than the roof beams, this was double checked by Grant.

Dan Louie started up the system and confirmed the limit switches were properly reset.  He then raised the lid until the limit switches tripped again, confirmed by checking the illumination of the upper limit lamp (see attached photo).

After the raising of the lid, the wooden stick was confirmed to be contacting the roof beams by Grant Minor.

Dan Louie also confirmed that the master synchro counter had advanced almost exactly 250 counts (proportional to about 250 thousands of an inch).

Mechanical counter readings at each station (where accessible) were taken before and after this 1/4" rise by Dan Wright and Grant Minor:

Grant Minor had dosimeter 208 today, which was signed out at 0.61 mSv and signed in at 0.69 mSv.  Thus elevating system assessments and adjustments resulted in approximately 0.08 mSv dose.

Today Grant Minor, Dan Wright, Arthur Leung, and Dan Louie checked the height difference between the tank lid upper travel limit switch and the tank lid over travel limit switch at both stations 1 and 10.

In the attached photos, the upper travel limit is the lever on the right.  The over travel limit is the lever on the left.

The over travel limit is a safety switch designed to halt the system and lock it out if there is some problem with the upper travel limit.

The following tasks were performed:

1. The new reference positions for the brackets after the March 11th adjustment of 0.25" (see previous e-log) were scribed onto the mounting posts for Stations 1 and 10
2. The brackets at both Station 1 and Station 10 were unbolted and raised up completely to disengage the switches.  This had to be done due to mechanical hysteresis in the switches.  Dan Louie confirmed when the upper limit / over travel limit signal lamps had turned off at the control rack.
3. The Station 1 bracket was bolted up higher temporarily so that both switches were disengaged at that Station.
4. The Station 10 bracket was positioned back at the new scribed reference line from step 1.
5. A 1/8" nominal machinists parallel was slid under the lever arm of the over-travel limit switch at Station 10, and Dan Louie confirmed that the over travel lamp had turned on.
6. The Station 10 bracket was unbolted and raised to disengage both switches, and bolted in this higher position.
7. The Station 1 bracket was then positioned back at the reference line from step 1.
8. Step 5 was repeated for Station 1, confirming that the over travel lamp engaged after the 1/8" parallel was inserted under the switch.
9. Both brackets were positioned back at the reference positions from step 1 and bolted in place.

This procedure confirmed that the over travel limit switches at both stations 1 and 10 engage less than 1/8" higher than the upper limit switches.

In tank survey completed.  A few items of interest were found.  Ladders need to moved manually outboard to #5 resonators and the items noted in attached document need to be addressed.

Prior to the lid-down operation today, the Cyclotron was lowered by 0.75" and then raised again to confirm that the upper limit switches were working.  Dan Louie confirmed this from the control racks.

Maico and Grant walked around the perimeter of the cyclotron to inspect each station and ensure that there were no interferences, e.g. dial indicator rods, tread plates, etc.

The lid was lowered successfully by Gord Roy and Dan Louie.

Mechanical counter numbers recorded after lid down are shown below:

Measurements of the cyclotron magnet face gap were taken on March 15, 2013 at two locations around the tank with the cyclotron lid in the fully raised position (the lid had been raised until the upper limit switches caused the control system to stop travelling).  The purpose of the measurements is to establish an independent means of verifying correct lid height before beginning in-tank remote handling activities at the start of a shutdown period.

The height of the lid at the time of measurement was confirmed to be adequate for the bridge to perform a full orbit without any interference between the bridge or cable tray and devices on the lid.

Measurement locations were labelled using permanent marker.  A plumb-bob was used to ensure the measurement location on the upper and lower faces were aligned vertically.  The measurements were performed using a tape measure hung from the upper magnet face and read at the lower magnet face as shown.

Location #1: RH bridge entry/exit point (~50° position), Measured gap: 46-1/2" +/- 1/16"

Location #2: East side of jack station #1 (~215° position), Measured gap: 46-9/16" +/- 1/16"

Photographs of the upper limit switches on jack stations #1 and #10 were taken as shown below:

At this lid height, the reference stick on the top of the cyclotron lid was found to be touching the concrete ceiling, but was not bowed.  See photo below.

 The distance between the cyclotron tank lid and the RH bridge was then measured using a meter stick attached to the lift trolley.  The trolley was moved into the tank, and the stick raised until it contacted the tank lid.  This position was recorded, the stick was lowered, the trolley was removed from the tank, then the stick raised again to the recorded position.  A piece of aluminum angle was used to translate the measurement from the center of the trolley to the edge of the bridge as shown below:

Total Distance from RH Bridge Rail to Cyclotron Tank Lid: 15-7/16" + 6-5/16" = 21-3/4" +/- 1/8"