Logbooks Lab Maintenance Evaporator_1 Evaporator_2 Laser cutter Target Production Test-Stand RH-ISAC RH-Cyclotron RH-Meson Hall RH-Beamlines RH-ARIEL
  RH-ARIEL, Page 5 of 5  Not logged in ELOG logo
ID Date Author Category Type Subjectdown
  71   Tuesday, November 19, 2024, 15:38 Albert KongHot Cell 1Standard OperationAPTW Front End Moved into Hot Cell

The APTW front end was moved into the hot cell, on the lift table, following procedures outlined in Document-244862.

The mock-up frame was transferred from the east turntable to the west turntable above the front end. The east turntable is covered with the aluminum plate. During covering we accidentally broke the o-ring for the turntable plate flange. The new o-ring seal should be made ~1" longer in circumference to avoid difficulties with seating in the o-ring groove.

When we attempted to move the front-end with the manipulator, we noticed that the front end tended to carry on sliding down the slope of the hot cell (see TRI-DN-23-20 / Document-238536). The manipulators were subsequently unable to arrest the front end, likely because they cannot apply a load towards the CG of the front end. Therefore, for heavy-duty jigs in the hot cell it is advisable to avoid using wheels/rollers that allow the heavy assemblies to move freely on the lift table.

--------------------------------------------------------------------------------

UPDATE Nov 21, 2024

  • FE attachment related tests completed:
    • replacement of shim plates
    • mock-up alignment of FE with shield plug (adjustment made in person, hot cell operator instruct and give direction with remote camera views
  • Difficulties encountered:
    • keeping track of which hole/screws are seen by the cameras during alignment was difficult for the operator
    • shim plate awkward to handle if not familiar
    • incompatible coupling screw and nut threads (M24 course vs fine threads) prevent completion of coupling and uncoupling tests
    • 'loose' service tray requires separate alignment check when raising the FE with the lift table
  • Lessons learned:
    • shim plate replacement easier if FE is semi-attached (screw lifts FE, leaving small gap for shim plate to be inserted)
    • need a cover plate to prevent dropping shim plate and associated screws
    • rotation clearance of FE checked, will clear
    • need to also check service tray alignment when raising
    • magnetic cameras would be beneficial for this operation
  • Follow up items:
    • crane-based positioning of FE for replacement to be tested after concrete pours/hand-testing/when APTW is exchanged for AETE.
    • future design of protective covering when performing shim plate removal
    • optionally implement magnetic camera array to check hole alignment by looking directly at the holes

At end of shift, mock up and FE was separated, markings made on floor to specify where FE needs to be to couple.

--------------------------------------------------------------------------------

UPDATE Nov 25, 2024

Replacement nuts arrived (M24x3) for the coupling screws. Tests were completed to try and couple/uncouple the front end to the cart/mock up.

  • For taking the FE on/off the cart the cart should be able to rotate with the FE so that we can secure the coupling screws.
  • For dropping the FE off from the shield plug, we should make use of the alignment features (cup and cones), once screws are undone and the FE rests on the cones on the cart, it should be secured.
  • If markings on floor is made to assist with alignment, we need to do two sets; one for unloaded cart, one for loaded cart, this is because the lift table will tilt differently when there is a load.
  • The threads on the coupling screws are fine threaded in the real case, there is a slight concern with cross-threading when turning them onto the nuts remotely (from above).
  • Existing camera views sufficient to check for clearance between coupling plates and FE structures.
  • To allow the FE to guide freely into the alignment features (cup/cone), it may be a good idea to still use the air skates, but supplement it with structural 'stopping' features to keep it from rolling off the table.

--------------------------------------------------------------------------------

UPDATE Nov 26, 2024

We returned the front-end to the cart to allow the shield-plug mock up to be rotated by the target hall crane (turntable still not functioning as of this date).

During this operation we noticed that the front end did not fully return to the alignment cones on the cart (see images), somehow the contact pad screws made contact first, and so the two-faced alignment cone did not come into contact.

For the real support frame we should use a minimal number of contact points. Also,more clearance at the coupling screws between the cart to the front end should be added.

--------------------------------------------------------------------------------

UPDATE Nov 27, 2024

We rotated the mock-up frame manually to have the HTV side face the hot cell window. During which the o-ring fell and we decided to cut it for convenience, this should be replaced with an equivalent o-ring but made ~1" longer in circumference.

--------------------------------------------------------------------------------

UPDATE Nov 28, 2024

Today we attempted to re-couple the front end to the mock up frame in the orientation where the HTV faces the hot cell window. The misalignment between the FE on the cart to the mock-up frame is worse in this orientation than in the previous (90 CCW facing down). There was approximately a 1" gap difference between the opposite ended screws. Factors contributing to the misalignment include uneven seating of the FE onto the bosch frame and uneven 'brake' engagement on the bosch frame wheels. During the attachment process the the screws had to be tightened very unevenly, with the HTV side being tightened far more than the extraction electrode side.

Additionally, the service tray shifted significantly from handling in the previous days and it was difficult to align the locating pins/cylinders. This further complicated the coupling process, in the future the service tray attachment should be made a separate operation to the process of coupling the  FE to the shield plug.

Under these conditions, during tightening, the inner screw at the HTV-side somehow jammed against the nut and the nut conversely jammed into the bracket. Subsequently, that screw could not be tightened further and required great difficulty to be removed, even after removing the bracket. In the aftermath, the nut bracket became forced apart (see picture) and the threads on the screw deformed beyond use. This is a critical failure mode for this coupling system. Note that the screw was hand started onto the nuts on the brackets prior to fully tightening the screws.

We believe the factors contributing to this failure include:

  1. bracket gap being too large and allows the nut to wedge itself during tightening,
  2. screw threads not being hardened/of proper grade,
  3. tightening the screws too unevenly and sharply (too large turns per-screw when bringing the FE up, 1/2-1 turn) / allowing some screws being slack,
  4. high misalignment (pitch and roll) between the FE plate and the mock-up frame,
  5. the brackets not being able to accommodate the observed misalignment,
  6. hole for screw is ~short and creates possibility for jamming when there is misalignment (pitch and roll),
  7. aligning the service tray locating features while aligning the screw holes for the FE coupling was difficult,
  8. possibly the extra lead in (chamfer) on the coupling screws is excessive and a non-standard design, which may cause problems starting the nut, likely a standard chamfer on the end of a standard screw would be sufficient for this operation

In the real operation, the long length of the screw will help prevent this failure from happening but some things to keep in mind/improve on would be:

  1. bracket re-design to use standard wrench/nut gaps to avoid nut jamming
  2. use hardened threads
  3. try to see if no-lead in/chamfer/using standard screw profiles is sufficient
  4. re-design the coupling system to not require a nut at the bottom and avoid cross-threading/thread jamming in the hot cell (move threads and nuts all above the FE assembly)
  5. extra: the screw should not be fine threaded in the real case to avoid cross threading
  6. minimize misalignment when trying to couple in the future (rotate turntable to minimum misalignment orientation)
  7. tighten the screws more incrementally to have them go together evenly, don't allow screws to be fully slack if possible to prevent jamming
  8. fully remove the service tray and attach only after the FE is attached

Of the above recommendations we strongly recommend re-designing the nut bracket - even when coupling was smooth (without misalignment) the nuts made a visible dent onto the bracket at the highest point (see image). A smaller gap (standard wrench gap) would prevent such a mark from forming.

We replaced the deformed screw with a standard (off the shelf) one to keep the FE secure to the mock-up.

We also noted after completing coupling that the extraction electrode, farthest out screw had damaged threads (see image).

It is recommended to perform repeated coupling/decoupling operations in the hot cell after some process/design changes are made/implemented to work out any possible failure modes with this operation.

 

 

 

Attachment 1: 4dab0178-cbf6-4af0-b663-67fd0d54c199.jpg
4dab0178-cbf6-4af0-b663-67fd0d54c199.jpg
Attachment 2: 568feb0c-d1a7-46e8-8686-e3e7bd5d1dc1.jpg
568feb0c-d1a7-46e8-8686-e3e7bd5d1dc1.jpg
Attachment 3: 3578f845-4f0c-4776-8fde-7c859ec40bbf.jpg
3578f845-4f0c-4776-8fde-7c859ec40bbf.jpg
Attachment 4: IMG_0419.JPEG
IMG_0419.JPEG
Attachment 5: IMG_0466.JPEG
IMG_0466.JPEG
Attachment 6: IMG_0468.JPEG
IMG_0468.JPEG
Attachment 7: IMG_0454.JPEG
IMG_0454.JPEG
Attachment 8: IMG_0480.JPEG
IMG_0480.JPEG
Attachment 9: IMG_0487.JPEG
IMG_0487.JPEG
Attachment 10: IMG_0483.JPEG
IMG_0483.JPEG
Attachment 11: IMG_0484.JPEG
IMG_0484.JPEG
Attachment 12: IMG_0511.JPEG
IMG_0511.JPEG
Attachment 13: IMG_0512.JPEG
IMG_0512.JPEG
Attachment 14: IMG_0513.JPEG
IMG_0513.JPEG
Attachment 15: IMG_0522.JPEG
IMG_0522.JPEG
Attachment 16: 666c4d22-d093-4570-978e-ee23141e362d.jpg
666c4d22-d093-4570-978e-ee23141e362d.jpg
Attachment 17: bfd8c4e1-f434-47fc-8690-dbd28c9d884c.jpg
bfd8c4e1-f434-47fc-8690-dbd28c9d884c.jpg
Attachment 18: IMG_0615.JPEG
IMG_0615.JPEG
Attachment 19: IMG_0619.JPEG
IMG_0619.JPEG
Attachment 20: IMG_0623.JPEG
IMG_0623.JPEG
Attachment 21: IMG_0624.JPEG
IMG_0624.JPEG
Attachment 22: IMG_0628.JPEG
IMG_0628.JPEG
Attachment 23: IMG_0631.JPEG
IMG_0631.JPEG
  83   Thursday, February 13, 2025, 18:00 Albert KongHot Cell 1DevelopmentAPTW Front End Mid-Height and Upper VCR Joint Access and Service Testing

Feb 13:

The 'hanging' VCR joint on the driver side was successfully exchanged by hand:

  • The adjacent HV shielding had to be removed to do so
  • Access to the screws on the shielding is limited so likely the use of a manual ratchet and socket key is required

All other 'mid-height' VCR joints ok for gasket exchange since they are accessible

The driver side upper VCR line was tested:

  • Issue with bracket for male side on shield plug, the adapter does not thread in fully
  • Flexible hose could be positioned into bracket with manipulators
  • Upper VCR lines will be revisited when correction to bracket is made and when the interfering tube on the HTV side is solved.

 

Attachment 1: Feb13_UpperVCRTesting.pdf
Feb13_UpperVCRTesting.pdf Feb13_UpperVCRTesting.pdf Feb13_UpperVCRTesting.pdf Feb13_UpperVCRTesting.pdf Feb13_UpperVCRTesting.pdf Feb13_UpperVCRTesting.pdf Feb13_UpperVCRTesting.pdf Feb13_UpperVCRTesting.pdf
  84   Thursday, February 20, 2025, 14:44 Albert KongHot Cell 1DevelopmentAPTW Front End KF Gasket Clamps Servicing Tests (By Gate Valve)

Covers testing for dealing with KF joints by the gate valve on the FE.

Photos provided in PDF, download and open to view all, ELOG preview not guaranteed to show all.

Feb 20: Tested two KF clamps joints by the gate valve area

  • When undoing the KF nuts, recommend starting with torque tool, then undoing by hand to prevent excessive loading in the updated retained design or prevent losing the nut if we decide to not have the nut retained.
  • Having a retained design may help by giving the hot cell operator feedback when the nut is brought out as far as it can.
  • During testing, we did not capture possible side loads on the 'free' side of the KF joint, though after reviewing the TISA version of the connectors, they may not be as strongly loaded so this is likely not a problem. Instead, the problem will be preventing the 'free' side from being dropped.
  • A 'captive' KF gasket may be advantageous to better ensure the joint is re-done properly when servicing and to minimize the risk of the gasket falling out (although during testing this risk seemed minimal).
    • After initial search, an off-the-shelf solution does not seem to exist, would likely have to be modified in house, which is undesirable.
  • We were working with 'old' designs for the clamps without captive nut on the clamp and with shorter screw and smaller hex size on the nut.
  • The external clamp had a larger nut than the rear, more realistic to what we understand will be implemented in the redesigned clamps.
  • For testing both, we used only the clamp body with the bigger nut.
    • To move clamp body from the exterior joint to the interior joint, we had to remove the HV shielding on the back
    • We noticed that the screw on the HV shielding was not captive but we recommend for this to be changed to be captive is possible
  • Both KF joints were successfully serviced (undone, gasket accessed, re-done)
    • We did not test replacement of the gaskets, a tool is recommended to aid this
    • Access to clamp nut on exterior joint is ok, interior joint access could be better (orientation not optimal, could be adjusted by rotating the bracket)
    • Larger size KF line that crosses in front of the interior line was not tested due to missing clamp assembly.
    • The larger KF line will likely need to be disconnected and moved out of the way if we need to service the interior KF joint.
    • Again, a caveat with the successful servicing test is that we did not experience any side loads and the 'free' side was held in place by hand when the joint was re done and the clamp closed
      • When holding the 'free' side with the manipulator, it is difficult to position it such that the clamp can close enough to swing the nut back in place (full contact around the gasket difficult)
      • 'Closing' the clamp back up if the lines are aligned and flush is very difficult to do with a single manipulator (if the second manipulator is used to hold the 'free' side in place).
      • Using an allen key to 'fish' the nut on the clamp when re-doing is useful.

Testing will likely be revisited when we receive the updated clamps, specifically that for the larger, untested, KF joint.

Attachment 1: Feb_20_FE_KF_Joint_Clamps.pdf
Feb_20_FE_KF_Joint_Clamps.pdf Feb_20_FE_KF_Joint_Clamps.pdf Feb_20_FE_KF_Joint_Clamps.pdf Feb_20_FE_KF_Joint_Clamps.pdf Feb_20_FE_KF_Joint_Clamps.pdf Feb_20_FE_KF_Joint_Clamps.pdf Feb_20_FE_KF_Joint_Clamps.pdf Feb_20_FE_KF_Joint_Clamps.pdf
  86   Thursday, March 20, 2025, 16:54 Albert KongHot Cell 1DevelopmentAPTW Front End High Voltage Shielding Exchange Testing

Photos compiled in pdf, download to view in case ELOG preview truncates.

------------------

The HV shield covering the beam dump, on the peripheral side of the FE plate was taken off an on successfully with the manipulators:

  • The operation was completed without reorienting the FE towards the HC window and we were still able to perform the removal/installation operations.

 

The 'helmet' shield on top of the FE plate was successfully exchanged:

  • Rough steps: pull the helmet up and back to release from its seated position
  • Both manipulators used during operation
  • The threaded hole to install grabbing features has 'fine pitch' threads
  • Fortunately we were able to replace this shield without the need for additional handling features
  • We moved the helmet out towards the gamma dump direction, but to prevent scraping against ceramic standoffs it may be a good idea to pull out the helmet shield from the other direction, though there's an elevated dropping risk (drop onto service tray) and the manipulators will have to reach farther in
  • The exchange test was completed with the FE plate facing parallel with the HC window, in practice we would like to reorient the FE maybe 15 degrees CW from this orientation
  • Rotating the FE slightly will help with some interference between the left manipulator arm and the shield plug (see photo)
  • Adjacent VCR lines may interfere with reinsertion if not done up

 

HVFT shield was picked up successfully with two manipulators without much issue:

  • Found that the manipulator handling cutout on the shields was a bit too large, a smaller gap across the width of the fingers will make is less necessary to grip tightly to keep the shield in line with the gripper
  • The above observation applies generally to all HV shielding cutouts
  • captive screws where possible on the HV shielding is generally a good idea, the screw on this shield could be captive

 

The HV shield obstructing exchange of a VCR gasket on the driver side was removed with issues:

  • Access to button head screws obstructed by ceramic cylinder, risk of damaging cylinder when accessing these screws with tool, to prevent damage should move position of screws farther away
  • Far screw difficult to see from operator window
  • Raising the turntable/FE up helps with access
  • Remote camera view helpful but due to location, difficult to get head on view of the screws for checking key engagement
  • Caveat: ceramic standoffs that would have further obstructed the screws were not installed (final picture)
  • Changing design to incorporate slots into the screw holes, similar to the gamma dump HV shield would be greatly beneficial
  • Alternatively mount to standoffs to a different plate or add an intermediate adapter plate to make removal and installation of this HV shield easier.

 

Service tray HV shield on multi-pin connector cable side was tested:

  • Existing shield has protrustion that prevents engagement with threaded standoffs and cannot be installed
  • Some change required for this HV shield

 

We did not test the HV shields by the KF clamps for the extraction electrode, we will assume that the KF joint will be made serviceable without needing to remove the shielding and that the VCR joint by that area will never need to be serviced.

 

Testing on other HV shields to resume in following session.

 ------------------

UPDATE April 09:

We ran into some issues turning the East and West turntables.

We tested mounting the service tray shielding by hand and foresee no issues.

The shielding covering the rear busbar connection with ribbon connector by the water service lines to the HTV was tested by hand:

  • There are no locating features to position this shield where it needs to be screwed down.
  • The mounting screws (mount to FE) should be made captive. or employ a similar approach as the ionization dump shielding (slots in and keeps the shielding in place)
  • The threaded holes for mounting grabbing features could be improved by adding additional locating features to 'clock' the grabbing features in the desired orientation, without needing to torque down the screw.
  • The above recommendations apply to all shielding as appropriate.

 ------------------

UPDATE Apr 20:

After the East and West turntables were fixed, we were able to test the removal and replacement of the HV shield covering the rear busbar connection (ATA3540).

  • 3D printed H-handles were attached to both grabbing feature holes on the shield
  • The H-handles rotated with ease despite the screws being moderately tightened, making rotational positioning of the shield difficult
  • It may also be a good idea to add a slight angle to the H-profile on the handles to better clear the manipulator wrist from interfering with other FE structures (see photo)
  • The air ratchet was used with a short socket key to remove the screws on the shielding
  • FE rotated such that the non-gate valve side opening faces the hot cell window
  • The FE was not raised far enough and prevented the left manipulator from accessing the second 3D printed h-handle on the gate-valve-side opening
  • When removing, there was no need to keep the shielding retained since it could not 'drop' and damage anything
  • When returning, it was difficult to tell whether the shield was seated properly, may
    • The shield does not stay engaged on the mating nubs on the FE without the screws tightened
    • Returning the shield onto the FE will likely require two manipulators: one to keep it in place, another to reinsert the screw
    • We would strongly recommend changing the screws on this shield to be captive since dropping a screw in this location will cause it to be stuck deep in the service tray
    • Also, if the mounting design could be modified such that the shield stays put when in place and does not drop, it would be a great help
      • some ideas: substituting One of the screws with a pin, having the slot design that's present on some of the other HV shields, or adding more locating features in the mating interface
  • When doing-up the screws, with the air ratchet (or smaller electrical tool in the future), it was relatively easy for the tool to slip and turn onto the ceramic components.
  • In the real deal, it is imperative for protective impact covers to be placed over the ceramics first.

We tested removal and returning the patch panel shielding by hand:

  • the main thing to note would be that the manipulator handle to be installed may need to be made horizontal with side access

 

Attachment 1: HV_Shield_Testing_Mar_20.pdf
HV_Shield_Testing_Mar_20.pdf HV_Shield_Testing_Mar_20.pdf HV_Shield_Testing_Mar_20.pdf HV_Shield_Testing_Mar_20.pdf HV_Shield_Testing_Mar_20.pdf HV_Shield_Testing_Mar_20.pdf HV_Shield_Testing_Mar_20.pdf HV_Shield_Testing_Mar_20.pdf
Attachment 2: April_16.pdf
April_16.pdf April_16.pdf April_16.pdf April_16.pdf April_16.pdf April_16.pdf April_16.pdf April_16.pdf
  79   Wednesday, January 15, 2025, 12:57 Albert KongHot Cell 1DevelopmentAPTW Front End HTV Water Connector Exchange Testing

The four water connections on the front end plate of the APTW that will connect to the HTV were tested for replacement with the manipulators:

  • The pin on these connectors were fairly easy to remove and replace with the old pin detent tool.
  • The pin detent tool could also be used to manipulate the water lines while protecting the sealing face.
  • The horizontal VCR joint on the DRIVE BEAM CONE (ATA3674) could be undone with the manipulator and standard wrenches, it is advisable to use 'semi-closed' wrenches on the nut to prevent it from dropping when torquing or undoing the joint.
  • Of the four water lines, only the top line could be taken out and returned by hand, testing with manipulator still to be done.
    • It would be nice for this top most line to have a clamp or something to keep the VCR nut on the bottom U-bend from falling.
  • The second line from the top interferes with the bus bar when trying to be removed by hand.
  • The second line from the bottom has a VCR connection obstructed by the same bus bar when trying to remove by hand.
  • The bottom line catches on the other lines when trying to remove by hand, the other lines likely need to be removed first to allow this line to be exchanged.
  • is the intent for the bus bar need to be removed to exchange these lines?
  • The VCR nuts on the water lines located by the bottom of the bus bars will require the turntable to be raised most of the way up to access with the manipulators.

 Testing will resume in the following week for these water lines.

------------------------------------------------------------------------

UPDATE Jan 23, 2025:

We tried to exchange the for water connections on the front end plate again, but this time with the rear busbars removed (by hand, did not have time to test by manipulators):

  • Notes on rear busbar removal:
    • top array of 3 screws should be reachable by the torque tool
    • side screw that fastens the sheet metal peice should also be reachable by the torque tool but with a relatively short attachment
    • optimally, the module would be raised pary-way on the turntable to reach these screws
    • we can keep the busbar attached to the flexible connector at the bottom
    • damage to flexible connector should not happen under normal circumstances, unless something heavy gets dropped onto it from a height
  • General notes on VCR joints on flexible lines:
    • Ideally the male VCR are captured so we can tighten with just 1 manipulator/wrench, particularly those in the cramped spaces behind the FE plate
    • Having the male VCR end at least welded will help make starting the VCR nuts easier, if not, will need to hold it with another wrench
    • May be not a good idea to use side-load gaskets as they fall out more easily and can bridge potential gabs in high voltage carrying conductors
  • Notes on replacement of lines with bus bar undone and rear VCR's undone:
    • Top line can be removed and returned with manipulators without other lines being removed, as with previous testing, a clamp or other feature to keep the VCR nut from falling down the 'U' will help. Also, it would be recommended to not return the pin detent before doing up the VCR in the rear when returning to give the line more adjustment.
    • Second to bottom line is replaceable with the manipulators without other lines being removed.
    • Second to top line interferes slightly with a rigid tube line, it is possible to adjust the bend slightly so no interference is encountered when removing/installing, removal of the line with pin detent tool not possible because the tool interferes with the HTV carriage post, soft cover for direct grabbing with manipulator recommended,
      • The VCR nut for this line is positioned in a way that makes it difficult to get a wrench on and tighten, we only have around 30 degrees of rotation, and the male side is floating which requires another wrench to hold it in place. starting the nut difficult because the line tends to come together misaligned. re-doing VCR accomplished buy quite difficult due to interference and double-wrench operation.
      • From discussion with Michael Genix, the reason for the interference with the hard line is to keep them on the same plane when the drive side is pushed in and out with the piston to prevent interference with other components.
      • Having the detent pin on the HTV end of the water line out helps with establishing the VCR joint for this line.
      • Use of side-load style gasket not encouraged here because there is a strong tendency for the gasket to slip out and fall into the HTV carriage sliding faces and other crevasses in the FE assembly
    • Bottom line requires top line to be removed for replacement with manipulators.
      • having a string on the removed line may help route the new line into place
      • there is a tendency for the line to catch on screws, shoulders, etc. when returned, good camera angles will help
      • the slot on the FE plate for this line does not have much lead in and tends to cause the line to catch, a bit particular when returning
  • Replacement of the VCR's will be tested in the coming session alongside replacing other braided lines in the FE assembly.

 ------------------------------------------------------------------------

UPDATE Jan 29, 2025:

  • The rear busbar attached to the flexible aluminum sheet conductor was successfully taken off with the manipulators:
    • Flexible aluminum sheet conductor may need to be 'tidied' after the busbar is removed and returned (see picture)
    • Washers on screws here are likely to be dropped when removing, it may be a good idea to switch these out to a different style of washer, maybe a retaining push nut (like: https://www.mcmaster.com/94813A200/)
    • The big screw holding the metal sheet on the busbar has already visibly damaged plate at the mounting point, adding a washer to this screw may be a good idea
    • It may not be necessary to have the washer be captive for the bigger screw because the busbar could be removed without needing to fully undo the screw
    • Having the big screw partially threaded on may actually help with reinstalling the busbar by keeping it roughly in the right location for the three smaller screws to be threaded in at the top.
    • Note that the smaller and larger screws were accessed with the FE in different orientations, i.e. there are different optimal positions to access these screws.
    • Access to the small screw was achieved using the extended length socket key that was recently purchased, attached onto the torque tool
    • Access to the larger screw was achieved using a ratcheting wrench and a standard socket key, the torque tool had difficulty accessing the big screw due to interference with other parts of the FE
  • The VCR joint on the second to bottom water line was successfully undone and re-done with the manipulators:
    • an adjustable wrench was used for the male nut on the VCR
    • a custom 12-point secure wrench was used for the female nut on the VCR
    • there was great difficulty getting the two wrenches in place, some contributing factors include:
      • the large custom handle on the 12-point wrench,
      • the short length of the adjustable wrench << this made the operation more cramped when having both wrenches accessing the joint from one opening, also prevents accessing the male and female VCR sections from separate openings
      • small access space to the VCR joint,
      • interference with a HV shielding plate in one access angle,
      • interference with the unfastened busbar in another access angle
      • if the turntable is not raised, the manipulator shafts will interfere slightly with the shielding plates on top of the FE when trying to access these nuts << this issue is mitigated if the turntable is raised
    • It is advised to go from the second access orientation where manipulator interference with the HV shield is avoided
    • there was some difficulty starting the VCR nut, flexible metal sheet was used to raise the nut from the U-bend
      • after much trial and error, a 'pinching' action was used to position the flexible line while keeping the un-welded male side of the VCR joint from rotating, this then allowed the female VCR side to be pushed up and the threads started with the second manipulator
    • range of motion available for the wrenches when re-tightening the nut is limited by interference
      • upon reflection, because of the interference, it may be beneficial to have keep the male side of the VCR joint on the rear un-welded. This will provide an additional option for which side of the joint to turn when re-tightening, and can help alleviate issues with the limited range of motion
    • to replace the VCR gasket, for this joint it is advised to have the water tube on the FE plate unpinned, the hard line can then be swung out at the rear to provide access to the VCR gasket on the male side.
    • during testing for this joint, we found that the slide-on style of gasket stayed on quite well on the male VCR side, this is unlike the horizontal VCR joint on the second to top line where the slide-on style gasket came off and dropped many times during testing
  • Portable magnetic cameras would be good to get better views of the rear VCR joints that are 'tucked away'.
  • For further testing, shaft clamps for the rear female VCR side was purchased to keep the nut from falling (https://www.mcmaster.com/9648t5/), we will need to check whether these clamps affect our ability to extract and replace the water lines.

 ------------------------------------------------------------------------

UPDATE Feb 06, 2025:

  • The VCR joint on the second to bottom water line was re-tested with full-size wrenches:
    • noticed that full size wrench on the male side interfered a bit with the braided section of the water line
    • Having a 'captive' wrench, similar in design to the low clearance VCR wrenches for the HVFT joints may be good to use for the male side of the second to bottom water line (possibly a crows foot and appropriate size of ratchet would work)
    • when un-doing the joint a wrench was dropped that broke a ceramic screw
    • it would be a good idea to have a system of catch trays for the service tray to protect sensitive components against falling tools and to prevent losing VCR gaskets and other small components where they may bridge across high potential and cause issues in the future
    • The addition of clamps (McMaster 96485T) does not affect the ease with which this line is replaced since the female side is hard lined (not part of removable section), and should not affect adjacent line replacements (diameter of clamp not much bigger than nut diameter)
  • The VCR joint on the top most water line was undone and re-done:
    • undoing relatively straightforward since male side is welded
    • when fishing out this line, the gasket has a tendency to drop in a 'pit' by the main pinned support shaft for the service tray that is hard to reach. a catch tray will help prevent this
    • the VCR starter tool was useful to get the VCR nut on this joint to spin off
    • To replace the gasket here, we found that it is optimal for the water line pin to stay in place, we can swing out the flexible hose section and slide in a normal gasket or a side mounted one, assuming a VCR inserter tool is made.
    • To re-do the joint:
      • because the resting position of the female side is misaligned with the male side, a hook tool was necessary to bring them in to alignment.
      • a makeshift 'hook' was used, but a better, full-length one could be made
      • When the FE is 'square' with the hot cell window, the left manipulator interferes greatly with other parts of the FE when trying to access the nut to start
      • this is ultimately a difficult joint to re-start, with great risk of dropping VCR gaskets, but it could be done
      • the VCR starter was used to bring the nut back to finger tight
      • A ratchet and 12-point crows foot might be useful for tightening due to the small angular clearance available in this location, a standard wrench may not have enough rotational configurations at finger tight to be placed fully on the nut
  • Follow up item:
    • long handle 3/8" drive ratches purchased

With this, hot cell testing of the FE plate water lines is complete.

Attachment 1: Screenshot_2025-01-15_121632.png
Screenshot_2025-01-15_121632.png
Attachment 2: Jan15_FE_Plate_Water_Lines.pdf
Jan15_FE_Plate_Water_Lines.pdf Jan15_FE_Plate_Water_Lines.pdf Jan15_FE_Plate_Water_Lines.pdf Jan15_FE_Plate_Water_Lines.pdf Jan15_FE_Plate_Water_Lines.pdf Jan15_FE_Plate_Water_Lines.pdf Jan15_FE_Plate_Water_Lines.pdf Jan15_FE_Plate_Water_Lines.pdf
Attachment 3: Jan23_FE_Plate_Water_Lines.pdf
Jan23_FE_Plate_Water_Lines.pdf Jan23_FE_Plate_Water_Lines.pdf Jan23_FE_Plate_Water_Lines.pdf Jan23_FE_Plate_Water_Lines.pdf Jan23_FE_Plate_Water_Lines.pdf Jan23_FE_Plate_Water_Lines.pdf Jan23_FE_Plate_Water_Lines.pdf Jan23_FE_Plate_Water_Lines.pdf
Attachment 4: Jan29_FE_Plate_Water_Lines.pdf
Jan29_FE_Plate_Water_Lines.pdf Jan29_FE_Plate_Water_Lines.pdf Jan29_FE_Plate_Water_Lines.pdf Jan29_FE_Plate_Water_Lines.pdf Jan29_FE_Plate_Water_Lines.pdf Jan29_FE_Plate_Water_Lines.pdf Jan29_FE_Plate_Water_Lines.pdf Jan29_FE_Plate_Water_Lines.pdf
Attachment 5: Feb06_FinalHTVFEPlateWaterLineExchangeTesting.pdf
Feb06_FinalHTVFEPlateWaterLineExchangeTesting.pdf Feb06_FinalHTVFEPlateWaterLineExchangeTesting.pdf Feb06_FinalHTVFEPlateWaterLineExchangeTesting.pdf Feb06_FinalHTVFEPlateWaterLineExchangeTesting.pdf Feb06_FinalHTVFEPlateWaterLineExchangeTesting.pdf Feb06_FinalHTVFEPlateWaterLineExchangeTesting.pdf Feb06_FinalHTVFEPlateWaterLineExchangeTesting.pdf Feb06_FinalHTVFEPlateWaterLineExchangeTesting.pdf
  72   Wednesday, November 27, 2024, 18:38 Albert KongHot Cell 1Standard OperationAPTW Front End Electrical Cable Replacement Testing

November 27, 2024

Testing completed:

  • Removal and re-insertion of 2x multi-pin coupling cable for HTV (see pictures, left most and right most, middle had no wire at the time).
  • Tested using 'wire' method to pull cable once removed back through the original path.
  • Test without rotating table, on cart, some obstruction that does not reflect reality in operation, rotated cart by hand, will require rotation during operation.

Observations/notes:

  • Right connector has wires come across and may interfere with the middle connector, will require removal of multiple connectors to exchange one depending on configuration.
  • Locating pins on high voltage multi-pin connector needs to be short for voltage bias reasons.
  • Right-most black multi-pin cylindrical connector has higher risk of machined block end catching when fishing from the HTV side.

Recommendations:

  • May be a good idea to add a holder connector to go into the multi-pins and give the manipulators a better grip for handling. Directly grabbing on the cylinder of the multi-pin connector is ok though.
  • Some strain relief at the machined connector blocs (service tray side) will help prevent damage during wire replacement/routing.
  • Possibly re-route some of the electrical connectors to the back side of the HTV column for easier access and shorter length.
  • Will need chart mapping where the cables go and indexing different connections for future hot cell work (will be updated on SW model).
  • Locking pins on multi-pin connectors can be removed and replaced manually with the manipulator.
  • Adding a 'slippy' sheath onto the replacement cable for routing and to prevent catching would be a good idea in the future.

 -----------------------------------------------------------------------

Dec 02, 2024

Testing completed:

  • Removal and re-insertion of 1x ceramic-ended wires specifically the one obstructed by the gas line on the right side of the HTV connection array as viewed from the hot cell window.
  • Removal and re-insertion of main multi-pin connector cable bundle from the HVFT connector array on the service tray.
  • Removal and re-insertion of the gas line obstructing the right-side ceramic electrical connectors << very difficult/impossible to do (recommend modifications + tooling).

Observations/notes:

  • Electrical wires/connectors generally straightforward to exchange.
  • Plastic/ceramic components on connectors are easy to damage during exchange operation.
  • Most connectors require the metal/cylindrical bits to be 'grabbed' by the manipulators if no custom tool is used
  • Rectangular multi-pin connector bracket if single is difficult to return to slot since not supported on both sides
  • Need documentation/reference material to make sure connectors get returned to where they need to be
  • Screws on connector are deep into the FE assembly, creates viewing difficulty
  • Cameras hard to use for these screws because of beam dump being highly active when FE is in operation
  • Ceramic (mass marker) connectors fine to grab on the conductor part
  • When trying to re-align the ceramic bodies to line up the screw holes, found that it was really easy to damage/break the ceramics, avoid using hard tooling/manipulators on the ceramic parts
  • Plastic components on multi-pin rectangular brackets got damaged after handling with manipulators
  • Full sized pin detent tool not compatible with smaller pins, should make smaller sized pin detent tool, smallest clearance found on HVFT side of multi-pin connector bundle
  • Routing HVFT side of multi-pin bundle through wires poses risk of damaging exposed pins

Recommendations:

  • Should buy longer bits/ends for torque control tool to get better view angles on screws.
  • Maybe captive screw for the gas is a good idea to prevent the screw from dropping.
  • Torque tool should be set properly (torque and rotation rate) before performing screw exchanges involving the HTV connectors on the front end
  • Return of the gas line (on bracket) to its location on the FE is difficult, suggested modifications include:
    • Changing orientation of cutout on bracket to let stainless bent/welded tube to be returned separate from the bracket.
    • Changing screws for the stainless part on the gas line to be standard size (M6) for hot cell exchange operations.
    • Changing the unused screw hole on the FE to be pinned and allow for better alignment of the bracket onto the FE
    • Removing burr on sheet metal part on hose/wire opening at the bottom to prevent the braided hose from catching.
  • Should make a rotator tool (3D printed) to help adjust the ceramic end of the mass marker connector and have the screw hole and thread aligned
  • Maybe plastic covers for water/gas connectors to prevent scratching of outer surface
  • Make smaller pin detent tool or settle for old tool already made
  • Possibly design cover to protect exposed pins on multi-pin bundle during replacement

 -----------------------------------------------------------------------

Dec 12, 2024:

Testing completed:

  • Tested removal and return of the gas line obstructing some ceramic electrical cables on the right side of the HTV connector array on the front end, now with 6mm pin (McMaster 97493A162) threaded into the previously unused hole to help guide the previously  unused hole to help guide the bracket.
  • NOTE (20 Feb, 2025): further testing for servicing these lines is documented in a later ELOG (#78)

Observations/notes:

  • Chad Fisher commented that even without the pin it should be possible to complete the removal and return operation if a 3D printed (soft )tool is made to grab onto the tubes and guide it with the bracket into the alignment hole/slot.
  • Old recommendations still stand though + maybe make the pin longer/custom with more lead in.

Recommendations:

  • Remove the retaining tab at the bottom to allow the hose to be positioned more freely.
  • De-burr the sheet metal part to prevent catching on the braided hose.

 

Attachment 1: IMG_0523.JPEG
IMG_0523.JPEG
Attachment 2: IMG_0525.JPEG
IMG_0525.JPEG
Attachment 3: IMG_0526.JPEG
IMG_0526.JPEG
Attachment 4: IMG_0541.JPEG
IMG_0541.JPEG
Attachment 5: IMG_0544.JPEG
IMG_0544.JPEG
Attachment 6: IMG_0555.JPEG
IMG_0555.JPEG
Attachment 7: IMG_0599.JPEG
IMG_0599.JPEG
Attachment 8: IMG_0605.JPEG
IMG_0605.JPEG
Attachment 9: IMG_0659.JPEG
IMG_0659.JPEG
Attachment 10: IMG_0665.JPEG
IMG_0665.JPEG
Attachment 11: IMG_0674.JPEG
IMG_0674.JPEG
Attachment 12: IMG_0685.JPEG
IMG_0685.JPEG
Attachment 13: IMG_0688.JPEG
IMG_0688.JPEG
Attachment 14: IMG_0731.JPEG
IMG_0731.JPEG
Attachment 15: IMG_0701.JPEG
IMG_0701.JPEG
Attachment 16: IMG_0703.JPEG
IMG_0703.JPEG
Attachment 17: IMG_0728.JPEG
IMG_0728.JPEG
Attachment 18: IMG_0745.JPEG
IMG_0745.JPEG
Attachment 19: IMG_0746.JPEG
IMG_0746.JPEG
  42   Tuesday, November 19, 2019, 10:31 Adam NewsomeT-Hall CraneRepair900VFD Encoder Board Replacement
An issue arose in which the ARIEL crane hoist could not be moved. It was determined that the East and West drums were out of synchronization (by more than 0.2 m, which the limit) and the encoder board inside the 900VFD had failed.

Summary: The synchronization and encoder board issues were resolved, and the crane is now functional. The root cause of both of these issues is unknown.

The following is a timeline of events for record, in case a similar issue arises again:

Nov 6:

- Crane hoist not working
- Contact was made to COH to initiate troubleshooting
- Faults were observed: Drive fault for 700VFD, Drive fault for 900VFD and 900VFD not ready, Main Hoist Drum East and West Absolute Encoders Position Difference
- Logging in to the HMI as a maintenance user and viewing hoist feedback showed that: East drum absolute and incremental encoders agree with each other, West drum absolute and incremental encoders agree with each other, West drum and East drum encoder values do NOT agree with each other (approx 0.22 m difference, the threshold for faulting is 0.2 m)


Nov 12:

- Crane hoist mode was switched to operate only the East drum independently, and adjust it so that the encoder readout on the HMI of the difference between East and West drums was approximately 0... the red X indicators changed to a green checkmark, and the encoder difference fault was eliminated
- An attempt was made to operate both drums simultaneously again, but the drive faults persisted... contact was made to COH for further troubleshooting
- Hugo from COH suggested moving the encoder board within the VFD from 900VFD to 700VFD, to see if the "drive not ready" fault followed the board, to narrow down the issue. This swap was made, and it was determined that the issue was in fact due to a faulty encoder board. The VFD itself displayed fault code 4030 (Enc 1 open wire). A replacement as well as a spare were ordered (Allen Bradley 20-750-DENC-1).


Nov 18:

- The encoder board within 700VFD was replaced with a new one (this is the one that was previously in 900VFD)
- The fault messages were eliminated, and both drums could be run independently and simultaneously


Nov 19:

- Thomas Kauss, Travis Cave, and Adam Newsome tested the crane to verify functionality and check for synchronization of drums
- The drums are synchronized, as described in the operator's manual (visual verification of tick marks on hoists, 17 turns of rope when at upper limit)
- The crane was run to its upper limit and down a few times, and the hoists were observed: nothing out of the ordinary was noted
Attachment 1: IMG_20191106_140016.jpg
IMG_20191106_140016.jpg
Attachment 2: IMG_20191106_153311.jpg
IMG_20191106_153311.jpg
Attachment 3: IMG_20191112_114042.jpg
IMG_20191112_114042.jpg
Attachment 4: Hoist_Sync_3.jpg
Hoist_Sync_3.jpg
Attachment 5: Hoist_Sync_5.jpg
Hoist_Sync_5.jpg
  74   Friday, November 29, 2024, 15:23 Adam NewsomeHot Cell 1Maintenance1000 kg crane: y axis position display not functioning

After a recent power cycle, the y-axis referencing was lost for the 1000 kg crane. The position readout was also incorrect, somewhere on the order of > 30,000 mm. On 2024-11-29, A. Newsome re-referenced the y-axis. The position readout correctly reset to zero, and both +y and -y motion is functioning correctly, but the y-axis position readout on the HMI constantly displays 0 and does not change. To be investigated.

 

Update 2024-12-02:

After going online with the PLC, it was determined that the reason for the display of 0 as the position is that the two values used for calibration of the +y and -y limits were actually the same, meaning the scaling factor (the difference between these two values) was 0, which resulted in the displayed value being 0. The root cause of this is that the encoder was not functioning correctly so its value was not changing when the crane moved and it was stuck at one value. Upon investigation of the encoder input card, the red "ERR" light was on. This indicates the encoder signals are not properly reaching the input card. Once this was discovered, it was remembered that this happened in July 2024 as well (no e-log was written). In July, the root cause was identified as being a loose encoder signal wire in a junction box. Junction box CJB1-BC, located on the bridge crane near its disconnects, was opened and investigated. Upon checking each wire, it was determined that the red wire seemed not to be making full contact. The wire was removed and re-inserted, and the "ERR" indicator on the encoder card turned off. The crane's +y and -y limits were re-referenced. The crane is functioning normally after re-referencing. (Note: if something like this happens again, during the re-referencing process, the displayed value on the HMI will be incorrect.. this is because the PLC's scaling factor is not fully adjusted until both +y and -y limits are reached. The actual encoder measurement taken at those limits is used for the scaling factor. This is not an ideal way to program the system, but this is how it works with all ARIEL hot cell subsystems. So incorrectly displayed values can be ignored, in general, until full referencing is completed).

Attachment 1: PXL_20241202_204356508.jpg
PXL_20241202_204356508.jpg
ELOG V2.9.2-2455