Logbooks Lab Maintenance Evaporator_1 Evaporator_2 Laser cutter Target Production Test-Stand RH-ISAC RH-Cyclotron RH-Meson Hall RH-Beamlines RH-ARIEL
  RH-ISAC, Page 78 of 138  Not logged in ELOG logo
Entry  Wednesday, September 04, 2013, 12:21, Bevan Moss, South Hot-Cell, Repair, TM3, , Ante-room work IMG_0983.JPG

 September 3rd

David Wang and myself lowered the leak testing cart into the Ante room and prepared the ante-room for the leak testing of TM3s line. David and Grant then entered the anteroom and inspected the old c-seal and leak tested the the heat shield line. The line pumped down to the lower limits of the leak detector and there was no response from the cart. A presentation with the results of the C-seal inspection results was completed by Grant Minor and email for a design review held on the 4th.

September 4th

David Wang and myself removed the leak detector and surveyed the anteroom and cart. The following results were found with the high spec gamma detector.

Leak Detector body and cart 0 

Leak Detector wheels 200 cpm

Floor off of plastic 300 cpm (there was contamination before)

Plastic in front of HC opening 2000 cpm

Working table after plastic was removed < 50 cpm

Tool box < 50 cpm

First tacky mat 2500 cpm

Second tacky mat 200 cpm

outside of door tacky mats 0 cpm

David then laid fresh plastic on the floor around the HC opening and covered the exposed floor as well.

Following this Travis and David measured the lengths of the wiring and attempted to separate the wiring harness from the Ultem block. It was found that the block was pinned and could not be separated, a new block will need to be made. The measured the following lengths:

Steerering - 26" to 27"

Collimator - 32" - 33"

PNG - 60"

 

Entry  Wednesday, September 04, 2013, 18:53, Grant Minor, South Hot-Cell, Repair, TM3, , Maico's measurements of new c-seals and bench-tested c-seals c-seals_GStanford_file_chronological_4Sept2013.pdfGuyStanford_cseal_file_scanned_30Aug2013.pdf

Today Maico measured the thickness of three indium-plated inconel c-seals, presumably Ultra-Seal P/N 50606.  These seals had NOT been in the Hot Cell.

 

Uncompressed (new) "thin" c-seal

OD .435"

ID .300"

Thickness .094"

Uncompressed (new) "thick" c-seal

OD .438"

ID .280"

Thickness .096"

Compressed (used) "thin" c-seal (Beginning of Life bench test compression only, not long term compression)

OD .438"

ID .294"

Thickness .079"

 

Maico commented that he measured the thickness of a few other compressed "thin" c-seals and they all had .079" thicknesses.

The seal space when the blocks are bolted together (shown on ITA2342 Rev F) should be .070" (steel insert CB) + .002" (recess on module side)  + .002" (recess on source-tray side) = .074"

Thus the compressed seals seem to have .079" - .074" = .005" spring-back.

 

C-seal drawings from Ultra-Seal and Garlock in chronological order from Guy Stanford's design file are attached.  Guy's whole design file is also attached for reference.

Entry  Wednesday, September 04, 2013, 19:01, Grant Minor, South Hot-Cell, Repair, TM3, , Inspection / Ante Room Leak Check Results on TM3 Heat Shield lines (3 Sept 2013) TM3_Heat_Shield_Water_Block_-_c-seal_vs_delta_seal_DRAFT_3Sept2013.pdf

Yesterday, Grant entered the Ante Room with David Wang to visually inspect the two sets of failed c-seals from the Heat Shield circuit, to remove the second set from the Heat Shield water blocks, and to leak check the heat shield circuit with o-rings and a water-block-to-leak-detector fitting.

Some photos of the inspected seals are in the attached design review presentation.  Some unusual marks were observed on both sets of failed seals.

The heat shield lines and water block assembly leak checked successfully to the bottom of the leak rate range ("UNDER" ~1x10-9 atm cc / sec on the Varian 979) with a great flood of helium on all joints with no response anywhere.

Entry  Friday, September 06, 2013, 16:41, Bevan Moss, South Hot-Cell, Development, TM3, , C-Seal crush investigation  6x

 Today Maico completed the new leak testing tool and blank off. He tested them with C-seals without springs and all was leak tight. He then inspected the crushed seals and noticed that there was an area that was crushed less on both seals (more prominent in one,  20130906_tm3sourtrayefurb_P1020975) and that this reduced crushed zone only appears on one side of the seal. This is similar to the failure seen on both sets of the heat shield lines. On the seal with the greatest change in crush zone the average crush zone was ~0.025 and the reduced crush zone was ~0.012". He inspected the leak tester and the blank off and determined they were within tolerance and that they had a total seal goove height of 0.074" which is nominal. New seals were selected and one had a visible dimple prior to crushing (20130906_tm3sourtrayefurb_P1020978)  and the other had some defects on the inside (20130906_tm3sourtrayefurb_P1020962). These defects were marked and crushed using the same leak testing tool and blank off. The seal with the dimple showed a reduced crush zone in the same area (20130906_tm3sourtrayefurb_P1030021) and the seal with the defects on the inside showed less or undetectable change in crush zone. Maico then inspected (20130906_tm3sourtrayefurb_P1020991) and crushed the "thicker seals" (more indium coating). When inserting the seals he noticed that they fit tight on the counter bore of the blank off. He then attempted to crush the seal to the point where the copper faces would touch (as design intent) but the seal locked. The gap between the 2 copper faces was measured to be ~0.001". This setup was leak tested and found to be leak tight. When inspecting the crushed thicker seals it was found that the material had actually been pushed sideways causing a lip to form around the crush zone (20130906_tm3sourtrayefurb_P1030003). Maico then fitted the wires for the testing of the module and the heat shield line. 

    Reply  Friday, September 06, 2013, 17:50, Grant Minor, South Hot-Cell, Development, TM3, , C-Seal crush investigation  

Bevan Moss wrote:

 Today Maico completed the new leak testing tool and blank off. He tested them with C-seals without springs and all was leak tight. He then inspected the crushed seals and noticed that there was an area that was crushed less on both seals (more prominent in one,  20130906_tm3sourtrayefurb_P1020975) and that this reduced crushed zone only appears on one side of the seal. This is similar to the failure seen on both sets of the heat shield lines. On the seal with the greatest change in crush zone the average crush zone was ~0.025 and the reduced crush zone was ~0.012". He inspected the leak tester and the blank off and determined they were within tolerance and that they had a total seal goove height of 0.074" which is nominal. New seals were selected and one had a visible dimple prior to crushing (20130906_tm3sourtrayefurb_P1020978)  and the other had some defects on the inside (20130906_tm3sourtrayefurb_P1020962). These defects were marked and crushed using the same leak testing tool and blank off. The seal with the dimple showed a reduced crush zone in the same area (20130906_tm3sourtrayefurb_P1030021) and the seal with the defects on the inside showed less or undetectable change in crush zone. Maico then inspected (20130906_tm3sourtrayefurb_P1020991) and crushed the "thicker seals" (more indium coating). When inserting the seals he noticed that they fit tight on the counter bore of the blank off. He then attempted to crush the seal to the point where the copper faces would touch (as design intent) but the seal locked. The gap between the 2 copper faces was measured to be ~0.001". This setup was leak tested and found to be leak tight. When inspecting the crushed thicker seals it was found that the material had actually been pushed sideways causing a lip to form around the crush zone (20130906_tm3sourtrayefurb_P1030003). Maico then fitted the wires for the testing of the module and the heat shield line. 

 Hello all,

As an addendum to Bevan's e-Log:

Maico also prepared eight (8) new retainer spring windings out of the .025" diameter stainless welding wire for the next seals that we will attempt in the Hot Cell and Ante Room

Maico, Bevan, Keith and I had a discussion about the bolt torque related to the c-seal and water block compression:
- It was found that not much torque is required to compress the "standard" design "thin" c-seals (Ultra-Seal P/N 50606 .001 - .0015 thou indium plating) - basically hand tight only with an allen wrench
- By hand-tightening the bolts with an allen key until the faces of the blocks came together, and then measuring the torque with a torque wrench, Maico discovered that about 8 foot-lbs = 96 inch-lbs (or 5/8ths of a turn past finger tight) was required
- The c-seals are fully compressed when the faces of the blocks are contacting... any additional torque applied is only to pre-load the bolts to ensure that they do not come loose due to temperature cycling and mechanical vibration
- Chad's Hot Cell torque tool is nominally set to about 168 - 180 inch-lbs (or about 14 to 15 foot lbs) based on an e-mail update from him 3-June-2013 - this torque is normal chart torque for a 1/4"-28 UNF SAE Grade 8 bolt pre-load of 3,250 lbs
- see http://www.imperialsupplies.com/pdf/A_FastenerTorqueCharts.pdf
- This amount of bolt pre-load torque may not be required if the bolts are SAE Grade 5.  I recommend at this point that the minimum possible pre-load torque be applied to achieve the chart recommended pre-load for the grade of the bolt, which should be investigated
- This is to ensure that we do not over-stress the thread inserts on the water blocks... if these inserts are damaged, we will basically render the service chase unusable and ruin the module

Cheers,

Grant

Entry  Tuesday, September 10, 2013, 12:11, Isaac Earle, South Hot-Cell, Repair, TM3, , Module side heat shield water block polishing and blank-off installation 

 The sealing surfaces of the module side heat shield water block were polished on September 9th in the morning using Chad's polishing tool with the following attachments:

- Scotchbrite pads  (30 seconds)  (after this step indium material from the previous seal was no longer visible on the sealing surface)

- 2000 grit sandpaper (30 seconds)

- Felt material with isopropanol (30 seconds)

- Lint free pad (30 seconds)

 

After the final step the surface was rinsed with isopropanol and air dried using canned air.  The water block blank-off was then installed with the manipulators, tightened until snug using the air ratchet, then tightened an additional 1/4 turn while the water block was gripped firmly using one manipulator.  The seal was leak checked and passed successfully.

Entry  Wednesday, September 11, 2013, 13:43, David Wang, Conditioning Station, Development, TM3, , Started 3 turbo pumps on CS for TM3 

TM3 has been connected in CS.Three turbo- pumps have been started. The pumping down is fine so far.

Entry  Wednesday, September 11, 2013, 15:37, Bevan Moss, South Hot-Cell, Repair, TM3, , C-Seal leak checking and repair 

 This e-log is to cover the work completed from September 9th until the 11th. 

September 9th

Today Maico completed the leak checking blank off and leak checking tool (to replace the one that was contaminated). This tool was then leak checked with C-seals and no springs at the machine shop, both halves were determined to be leak tight. 2 sets of small c-seals without divots were selected and used for leak checking the module side and the source tray side. The heat shield line was tested in the ante-room and was determined to leak. Upon further investigation it was determined that the bore of the seal gland was determined to be over sized (0.450"). Isaac then installed the blank off onto the module and pump down was started, the testing car would not stabilize so Isaac tried turning the screw more allowing for further pump down. The leak check was then completed and there was no response. The flow of helium for the target module was checked but was much lower than typical as the second valve was not fully opened. 

September 10th

With the belief that the larger bore in combination with the small c-seals was the cause of the leak Maico found and polished another block that also had a larger bore. Once polished, small c-seals without dimples were selected and tested with the machine shop leak testing cart. These seals were leak tight but upon investigation the crush zone in one section of the seals was significantly reduced. The smallest width of the crush zone was measured to be ~0.007". This result is concerning as it means that larger bore water blocks can be sealed with small c-seals but once installed there is no way to tell one block from the other. There is no data on the lifetime of this combination of seal and bore. 

Following this test a small c-seal with dimples was selected and checked in the large bore. This combination resulted in a significant leak. Following that test the large c-seals were crushed in the larger bore water block. The torque required to crush the seal and to have the faces touch was more than that of a small c-seal regardless of what size bore it was crushed in. However with the larger bore the large c-seals could be completely crushed allowing for the copper faces to touch (the total force on the faces is unknown). The crushed seal was then inspected, it had a nice uniform crush zone that was relatively large when compared to a properly crushed small c-seal. Another testing with the large seal and a 0.025" wire diameter spring was completed. It was found that the wire interferes with the the seal when it exits from the groove. This interference causes a localized increased crushed zone but does not reduce the crush elsewhere or prevent the copper faces from touching. At this point it was decided to repair the heat shield using large seals. Large seals with springs were tested on the actual HS line in the anteroom and yielded that same results. The line was then installed by Maico inside the HC. It pumped down to the lower limit of the leak testing cart and sprayed with helium, no leak was detected. The containment box was then put back on by Isaac and Grant in preparation for a module move.

September 11th

Today the containment box installation was completed and the module moved. Following that Maico and myself entered into the ante-room and retrieved the blank off block from the HC via the tool port. The large c-seal used in the test and the small ones used in the blank off were then collected and bagged. The large c-seal had typical measurements. The small c-seals used in the blank off did not have typical measurements. They had not been compressed as much and the crush zone was near impossible to see (if visible at all). From the measurements it is believed that the block was not sufficiently tightened. It also calls into question the validity of the leak check due to the combination of less helium and reduced crush. 

After these measurements the tools that were contaminated but were to be recovered were bagged and checked by safety. They have been moved to the jacks area where Maico will decontaminate them. The old wiring harness from TM3 was bagged and given to safety for storage in the cyclotron tunnel. The plastic and all of the garbage was then lifted, bagged, and removed from the ante room. Safety surveyed the anteroom following this no alpha contamination was found but there was 5000 counts on the floor of the ante room. The contamination may have existed prior to this work.

 

Many pictures of the cseals have been taken and have been put into \\trwindata\remote handling\Photos\2013\2013_tm3_source_tray_refurb. A report detailing all of the testing and results would be invaluable.  

 

Entry  Friday, September 13, 2013, 07:49, David Wang, Conditioning Station, Development, TM3, , TM3 water lines leak check at CS TM3_water_lines_leak_check_at_CS.pdf

I leak checked TM3 water lines at CS with 60 psi and 80psi helium. The base leak rate is 5.0xe-9 atm.cc/sec. Each circuit 3 minutes helium pressurizing for each time test. No any respose had been found on leak detector during the test. See attachment

Entry  Friday, September 13, 2013, 08:05, David Wang, South Hot-Cell, Repair, TM4, , TM4 coil 1 leak check in SHC TM4_coil_1_leak_check.pdf

Leak checked TM4 coil1 in SHC with Isaac . The leak detector only can pump down to 1.5xE-7 atm.cc/sec. The leak detector got the response once the helium valve is open, not even close to the leak area. The leak detector got a very big response when the helium probe was put close to the coil terminal block on target front side and spray helium for 2 clicks. The leak detector got a small  response when the probe was put close to water block area and spray helium for 2 clicks. The leak is located on the front target side coil line but not from water blocks. The leak is too big to be further pin pointed by this method. See attachment

Entry  Tuesday, September 17, 2013, 14:36, Bevan Moss, South Hot-Cell, Standard Operation, TM3, , Containment Box has been removed 

 The containment box has been removed. Tomorrow the wiring harness will be installed.

Entry  Thursday, September 19, 2013, 15:56, Bevan Moss, South Hot-Cell, Repair, TM3, , Activities for September 18th and 19th IMG_1127.JPGIMG_1134.JPGIMG_1137.JPGIMG_1139.JPGIMG_1143.JPG

September 18th

The wiring harness was completed by Maico and Travis today (see pics) and was installed into the target module. During installation the bracket went on with ease. The junction box appeared to be hung up on something but did engage eventually. The plug for the collimator is not quite long enough for the manipulators to install completely. I was only able to get it to just engage by sneaking the manipulator in from the shutter side and then pushing on it with a long rod. The wiring for the collimator is several inches too long but does not interfere with anything. The remote installation of the wiring to the IMG gauge was easy but there at least 1 foot of extra wire and with the thicker gauge it has a mind of its own. I was eventually able to wrap the wire under the table before so it will not interfere with anything. The module side junction box bracket was not clamped to the table I was able to do up one bolt but other one was blocked by the new steerer strain relief. There is no requirement for both to be done up.

 

September 19th

Travis completed and electrical test this morning and it was found that coil 1 touches coil 2 touch each other very slightly. By inserting a piece of cardboard between the closest water block and the lines we no longer in contact and passed the test. Once the cardboard was removed the lines remained separated but if touched slightly they would spring back into place and make contact. There was some debate as to if we should make a custom part to separate the line or if we should attempt to tweak the lines. I decided that it is known that a very small change in the position of the module will cause separation between the lines, that we should attempt to tweak the lines. I braced the lines with one manipulator to prevent loading the ceramic while Maico "tweaked" the line. The line was successfully moved to cause separation. Maico and I then completed a visual inspection of the remaining water blocks and noticed that the ground electrode cooling lines are estimated 1/8" away from one of the module side coil lines. All other places that can easily be inspected appear to have enough clearance.

Travis and Don then completed an electrical check again (https://elog.triumf.ca/TIS/RH-ISAC/457). This module still failed the test but as noted not due to continuity rather lower than infinite resistance from the coil to coil and positive coil to 60kV common. This indicates that there is dirty or failing insulation that is either from the fiber glass wrap or something within the service tray. From inspection from outside the hot cell the best guess as to where this is happening is just before to after the exit of the water lines from the service tray but there is no way to confirm this.

Although the module may be able to run for a period of time with these resistances the insulation may continue to break down and there is the risk that during a module move the lines may shift and touch again. This combined with the known HV issues of TM3 paints a dark picture of the service tray.

Friedhelm and Lia have been advised of these recent results and have decided to continue with the installation of the target and conditioning of the target. Should the module be able to operate and extract beam it will be installed in ITE. Friedhelm also came for a visual inspection of the module and agreed that the issue is on the module side and that the lines are too close to tell exactly where the issue may lie. The possibility of separating the the blocks with further insulation was discussed but there is limited area that this can be completed remotely. In addition to this there is no indication that further insulation/bracing will improve the problem and there is considerable risk that it could make the issue worse.

Installation of the containment box started this afternoon and is completed other than the tightening of the window VCR joints. 

Entry  Monday, September 23, 2013, 13:55, David Wang, South Hot-Cell, Standard Operation, TM3, , window circuit leak check 

This morning, Bevan and I leak checked window circuit on TM3 at SHC after containment box was installed. It took about 20minutes to pump down the circuit to  0.0xE-9atm.cc/sec. Bevan sprayed helium on both VCR joints and a final splash. No response had been found on leak detector.

Entry  Monday, September 23, 2013, 16:10, Bevan Moss, South Hot-Cell, Repair, TM3, , Activities Sept 20 - 23 2013 

 September 20th

Today was a trying day. When attempting to make the water line connections for the window the VCR seal fell off several times. Each time this happens the seal becomes stuck at the bottom of the containment box. While trying to replace the seal the nut slipped down the line to the bottom of the containment box. When trying to recover said nut with the pick tool it too fell to the bottom of the containment box. Due to the design of the containment box the only way to recover the pick is to remove the containment box (several other attempts were made with no luck). Before anyone asks no the pick cannot be left in the containment box. When attempting to remove the containment box one of the bolts for the shutter panel stripped. I then found the stud removing tool and have successfully removed the bolt. Isaac is now going to finish removing the containment box and that will be as far as we can get today. 

September 22nd 

- containment box removed
- dropped tool and seals removed from box
- containment box reinstalled
- window water lines done up
- target installed (with difficulties detailed below)

The threads for the middle tube heater connection (circuit D) appear to be damaged and the bolt in that location cannot be fully screwed in.  The bolt was removed and showed damage to the threads.

September 23rd

A leak check of the window line was completed (https://elog.triumf.ca/TIS/RH-ISAC/459) and several electrical checks of the coil with and without the target in place. The values for the electrical checks continued to fail the open circuit criteria, an elog on the testing will be reported tomorrow. Isaac and Maico removed the target so that the helicoil could be repaired. The helicoil was repaired by chasing the threads with a modified tap but it was noted during the repair that the bolt holding the line in place and come loose and was no longer clamping the tube heater and ceramic tight. The target was also checked electrically to confirm that the coil was not shorted to common. It showed a open circuit resistance with a ohmmeter.

It was then attempted to re-install the target but as the target was being lifted a cable for the tong motion schematic broke (A-11682). 

Currently the target is sitting in the hot cell with the manipulator still broken. We are attempting to find parts and instructions as to how to replace the cable.

 

Entry  Tuesday, September 24, 2013, 14:46, Bevan Moss, South Hot-Cell, Repair, , , Right manipulator repair 

 The right manipulator A-9885 Handle Cable Assembly has been repaired by Maico and myself. We repaired the manipulator following some verbal instructions from Chad on the modified repair strategy. This allows the repair to be completed without going into the hot cell. The instructions in the repair manual are very vague and can actually cause problems. For example it is best to pin the manipulator before completing any work. Other than that the work went smoothly with the exception that the counter weight hangs up on the manipulator occasionally and can cause the cable to come off of the pulley. Also it appears that the left manipulator has had the cable installed incorrectly as it rides on the housing of the wrist.

Entry  Wednesday, September 25, 2013, 06:59, maicodallavalle, South Hot-Cell, Standard Operation, TM3, , Containment box cover 

The containment box cover has had all  the bolts installed and all bolts were checked with a mirror. TM3 has been raised and is ready for the last electrical  test before  it goes to the conditioning station.

Entry  Thursday, September 26, 2013, 09:46, maicodallavalle, Conditioning Station, Repair, TM3, , Leak check / Pressure Test 

TM3 has had all of the water lines pressurized with Helium and the Helium Leak detector was connected to the turbo pumps. There were no leaks detected.

Entry  Wednesday, October 02, 2013, 13:44, Bevan Moss, South Hot-Cell, Standard Operation, TM4, , Right Coil is leaking tm4leak.jpg

 Today Maico and Travis completed and electrical test to determine the right and left coil on TM4. The left coil is the negative side. From there Maico pumped on both lines for a leak test. The leak cart pumped down to its lower limits on the left line and there was no response when sprayed with helium. The right line would not pump below 2.5 x 10^-2 Torr and had a base leak rate of 3.3 x 10^-7 atm*cc/sec (this indicates a massive leak). Without spraying helium the leak cart responded and peaked at 1.9 x 10^-6 atm*cc/sec. Attached is a picture of the module indicating the line.

 

Entry  Tuesday, October 08, 2013, 18:12, Grant Minor, Crane, Repair, TM4, , Target Hall Crane Stuck in East-West Direction - Update THall_Crane_Bridge_Wheel_at_South_Rail_Splice_8Oct2013.JPGNORELCO_inv13045_joMS-064_31March2013.pdf

Grant Minor wrote this e-mail today:

***

Hello all,

During our planned operation of transporting TM4 from a storage silo to the Hot Cell, we had a problem with the East-West bridge drive on the Target Hall crane.  The crane bridge is currently stuck in position over the silo area.  We were unable to complete the transport operation.  TM4 has been placed back down into the silo and the crane hook block has been disconnected from the module lifting yoke.

We have been on the phone with Norelco this afternoon and have determined the likely cause of the problem.  There is a splice joint in the crane rails at this position, where a small gap exists between the rail sections.  If the crane bridge is stopped during operation such that the wheel comes to rests in this gap, the crane motors have insufficient torque for the wheel to overcome the gap and ride back onto the rail.  The Norelco tech Mike Smith remembers encountering this problem several years ago (before my time) but cannot remember how it was overcome.  The fix likely involved moving the wheel out of the gap with pry bars.

Target Hall Survey Needed at or before 7:00 am, Wednesday Oct 9th (Lynne, Fiona, Danka, Max or Dano?)

Mike Smith and possibly a few others from Norelco will come tomorrow morning at 7:00am, and require access to the Target Hall.  Since we have opened the Hot Cell in preparation for the module move, we will need a survey as early as possible to allow Mike and his crew to enter the hall without a suit and respirator.

Most of the Remote Handling group will be off-site for a Division meeting, but Travis and Maico will stay for the day to accompany the Norelco guys and help with the problem assessment.

Travis: Please take Mike to the control room when he arrives, one of the cameras is currently zoomed in on the wheel and problem rail section.

Maico: Scaffolding may be required to get access to the crane rail.  Unfortunately we don't have an operational crane to transport a man-lift into the hall ;)  Please see what you can do to get this ready in case we need it.  Max's leak check might have to go on hold until we get this resolved.  Please take responsibility for ensuring correct Fall Protection procedures are followed.  This means that nobody should be walking up on the crane rail unless they are tied in with harnesses, or there is scaffolding directly under them.

I can be reached on my cell phone at 778-319-4612.  The Division meeting is on UBC campus so I can return to TRIUMF fairly quickly if needed.

Thank you all for your assistance,

Grant

***

A photo of the THall camera view of the bridge wheel at the rail splice is attached, as well as Mike Smith's crane inspection report from March 2013.

    Reply  Wednesday, October 09, 2013, 19:33, Grant Minor, Crane, Repair, TM4, , Target Hall Crane not operational - likely controls / PLC hardware failure (not stuck wheel) 

Grant Minor wrote:

Grant Minor wrote this e-mail today:

***

Hello all,

During our planned operation of transporting TM4 from a storage silo to the Hot Cell, we had a problem with the East-West bridge drive on the Target Hall crane.  The crane bridge is currently stuck in position over the silo area.  We were unable to complete the transport operation.  TM4 has been placed back down into the silo and the crane hook block has been disconnected from the module lifting yoke.

We have been on the phone with Norelco this afternoon and have determined the likely cause of the problem.  There is a splice joint in the crane rails at this position, where a small gap exists between the rail sections.  If the crane bridge is stopped during operation such that the wheel comes to rests in this gap, the crane motors have insufficient torque for the wheel to overcome the gap and ride back onto the rail.  The Norelco tech Mike Smith remembers encountering this problem several years ago (before my time) but cannot remember how it was overcome.  The fix likely involved moving the wheel out of the gap with pry bars.

Target Hall Survey Needed at or before 7:00 am, Wednesday Oct 9th (Lynne, Fiona, Danka, Max or Dano?)

Mike Smith and possibly a few others from Norelco will come tomorrow morning at 7:00am, and require access to the Target Hall.  Since we have opened the Hot Cell in preparation for the module move, we will need a survey as early as possible to allow Mike and his crew to enter the hall without a suit and respirator.

Most of the Remote Handling group will be off-site for a Division meeting, but Travis and Maico will stay for the day to accompany the Norelco guys and help with the problem assessment.

Travis: Please take Mike to the control room when he arrives, one of the cameras is currently zoomed in on the wheel and problem rail section.

Maico: Scaffolding may be required to get access to the crane rail.  Unfortunately we don't have an operational crane to transport a man-lift into the hall ;)  Please see what you can do to get this ready in case we need it.  Max's leak check might have to go on hold until we get this resolved.  Please take responsibility for ensuring correct Fall Protection procedures are followed.  This means that nobody should be walking up on the crane rail unless they are tied in with harnesses, or there is scaffolding directly under them.

I can be reached on my cell phone at 778-319-4612.  The Division meeting is on UBC campus so I can return to TRIUMF fairly quickly if needed.

Thank you all for your assistance,

Grant

***

A photo of the THall camera view of the bridge wheel at the rail splice is attached, as well as Mike Smith's crane inspection report from March 2013.

E-mail from today:

Hello Dave, Jane, Franco, Michael, Don (controls + Franco), Remy,

We have lost the East-West bridge drive control functionality of the ISAC Target Hall crane, and unfortunately this very much prevents us from operating the ISAC facility.

The problem was originally thought to be a stuck bridge wheel but this was not actually the case.

Mike Smith from Norelco was on-site for most of today working with Michael Rowe in an attempt to diagnose the problem.

Mike believes the problem to be one of the following three items:
- input PLC board on the crane bridge
- output PLC board in the interface panel in B1 level Room 5 (ISAC remote crane control room)
- communication between the two boards (festooning hardware)

We are in a pretty dire situation without the ISAC target hall crane, and we lack the expertise within the Remote Handling group in PLC hardware to diagnose this on our own.  I need to ask for somebody's help from the controls group in diagnosing and repairing this problem.

I have notes from my telephone conversation with Mike Smith which I can discuss with a controls representative in the morning.  There is a pretty muddled history of the design and implementation of these controls but essentially Norelco outsourced the PLCs and festooning to a third party contractor, Seattle Crane in the US, which no longer exists anymore.  Neither Kone, Seattle Crane, or Norelco made appropriate as-built drawings of the wiring of the PLCs.  Mike Smith at Norelco had a stack of hand-written notes about the as-built configuration he made at the time of installation in the late 1990's but these notes have not been located yet.

The PLCs appear to be Allan Bradley but the modules are from approximately 1997 and we're not sure if spare units are available.

Mike Smith says the easiest thing to do at this point is install replacement PLC modules on the crane bridge and in the control room that have confirmed functionality.  If control is not restored then the problem is likely in the festooning.

Control of the crane bridge was temporarily attained by bypassing the controls (speed control / end limits, etc.) and sending drive signal directly to the drive motor controller in order to return the bridge to its home position at the access ladder.  This mode of operation is not safe as the crane has no speed reduction / limits to prevent driving the bridge into the walls at max speed.

Jane / Remy / Franco: please let me know who I should correspond with on the controls side and I will talk with them tomorrow morning.

Thanks,

Grant

ELOG V2.9.2-2455