Six 5 gallon pails were transferred this morning from a temporary storage bunker in the Warm Cell to the Rad Waste Pit for long term storage.  The pails contain various BL1A active waste from the East Hot Cell including dozens of irradiated beryllium and graphite targets.  The field from the pails ranged from 0.1 to 9mSv/hr at 0.5m.  The work was performed according to the clean-out procedure (Document-142965).  Photographs and records of the pail contents, field, weight, etc have been uploaded to Docushare Collection #20185 and hard paper copies were given to RPG.

All remaining waste in the east hot cell is too large to fit into 5 gallon pails, so it must be packaged in a 55 gallon drum.  Some long items will need to be cut in the cell before hand.

The field on the hot cell roof is now less than 5uSv/hr in working areas, and a maximum of 55uSv/hr at 0.5m above the east hot cell opening.  The field was previously > 50uSv/hr in some working areas, and > 200uSv/hr above the opening.

Before and after photos of the hot cell interior are included below:

 The T2-MK2 target was moved from the beamline to the hot cell on January 8th.  The field at 0.5m was 130mSv/hr.  Photographs of the beam spot of the Position 4 target are attached below.  This is a 12mm beryllium target (ID: "JUL.06#4).  This is the only target on the T2 ladder that has seen beam since it was installed on October 10, 2017.  A shielding plug was installed on the T2 flange.  A new 5cm target will be installed on the T2-MK2 ladder after fabrication is complete.  Regular testing and maintenance of the target assembly will also be performed before it is returned to the beamline.

The M20 beam blocker was also moved from the beamline to Hole #7 in the storage pit.  The field at 0.5m was 22mSv/hr.  A blank-off plate was installed on the M20BB flange (no o-ring).  Seal replacement and general inspection/testing will be performed before it is returned to the beamline.

 The RH Lab active sump level reached the high limit yesterday due to an overflow of the ultrasonic cleaner in the Rad Waste Area.  The sump was sampled by RPG, and after analysis the water was approved for release.  The water was pumped out today.  Local and MCR alarms functioned as expected.

 Five 5 gallon metal pails with active waste were moved from the Warm Cell to the Cyclotron East Vault Tunnel today for long term storage.  The pails were labeled with radioactive item tags and storage item numbers (1442 through 1446).  The pails contain used resin from T1, T2, and STF, as well as one pail with active metal parts.

The following is a summary of MRO work done on the T2-MK2 target during the 2018 shutdown:

Jan 8:
- Target moved from beamline to hot cell;  130mSv/hr at 0.5m

Jan 10:
- Position 4 (12mm Be) beam spots photographed.  Spot is slightly high and left (from beam's eye view), also smaller and rounder (less eliptical) than usual - See E-Log #232

Jan 11:
- Flushed position 0,1,2,3,4 with water, then 4,2,1,0 with air
- Left with 10psi air running overnight in Position 0

Jan 12:
- Moved to position 4, 10psi air running through over the weekend to dry cooling lines

Jan 15:
- Pumped out water lines with leak detector
- Frist pump down blanked off tubing and new vac hose (no LN2): ~15 mins to get to 0.0E-10 TorrL/s
- Pumped out target w/ cold trap:  ~7 mins to get to 0.0E-10 TorrL/s (left pumping for 2 hours)

April 2:
- Removed plugs at position 3 and 5
- Pressure tested, leak tested, and recorded alignment dimensions for first batch of new 5cm targets (#301, 307, and 309) - Records stored with targets in RH lab office
- Installed new 5cm beryllium target #309 at Position 3, and target #307 at Position 5 
- Helium leak check on target water system:  Baseline 0.0E-10 TorrL/s leak rate, 1.0E-3Torr, half second helium applications @ 4psi
      Positions 1, 2, 3: no response
      Positions 4, 5: up to 5.0E-10 after delay (appears to be drift)
      Lower 'C' seals: no response
      Upper-left 'C' seal: up to 1.2E-9
      Upper-right 'C' seal: up to 2.4E-9
      Left bellows: up to 6.0E-9
      Right bellows: up to 3.0E-9
- Deemed water leak tight and ready for use (based on previous leak check results and operating experience)
- Tested all target position micro switches: all OK
- Tested profile monitor limits: both OK
- Tested profile monitor actuation: off down limit @ 5psi, on up limit at 35psi
- Tested potentiometer: slightly noisy signal from position 0 to 2 (no change from before)
- Measured Position 3 and 5 elevations

April 3:
- Measured Position 3 and 5 horizontal alignment wrt ladder
- Quick repeat of leak check:  no change

April 6:
- Target returned to beamline
- Vacuum pulled on T2 area (1ACG4 down to 5mTorr in ~2.5hrs, was 4mTorr before venting yesterday)
- T2 cooling package started with target at Position 0 - no change to T2 vacuum
- Target information sheet and elevations data delivered to Ops
- XTPAGE 7L information updated
 

 The following work was performed on the T1 and T2 cooling packages during the 2018 shutdown:

- T1 and T2 cooling packages both sampled and drained to South TNF holding tank
- Water released after testing and approval from RPG

- T1 high active and M15 filters changed (demin. filter to be changed next year)
- Replaced all T1 Proteus paddle wheels, shafts, and o-rings
- Replaced T1 male and female #6 SS Hansen fittings at the package/hose interface
- Replaced T1 #4 female Hansen fittings for the resin can (now all Hansen fittings on the high active loop have been replaced)
- T1 package filled and started, no change to T1 area vacuum
- Demin flow tuned to ~1.0gpm
- Package inspected while running, no leaks observed

- T2 high active filter changed (demin. filter to be changed next year)
- Replaced all T2 Proteus paddle wheels, shafts, and o-rings
- Replaced T2 male and female #6 SS Hansen fittings at the package/hose interface
- Replaced T2 #4 female Hansen fittings for the resin can (now all Hansen fittings on the high active loop have been replaced)
- T2 resin can changed;  old resin can flushed with water then air in Warm Cell and left to decay
- T2 package filled and started, no change to T2 area vacuum
- Demin flow tuned to ~1.0gpm
- Package inspected while running, no leaks observed

The cooling packages are functioning normally and are ready for the running period.

The M20 beam blocker main shaft seals were replaced as per the T1/T2 target device maintenance schedule.  Both air cylinders were found to be leaking past the seals, and the blocker would not actuate in the hot cell even with 100psi (max site air) so they were also replaced.  With the new cylinders the blocker actuated smoothly at about 45psi.  It is recommended that the air cylinders be replaced every second service (once per 10 years).  The blocker was returned to the beam line and smooth actuation and limit switch functionality were confirmed.

The following final checks were performed on T1/T2 devices:

- Lower air amplifier started and tuned to 120psi output
- Air amplifier cycle time measured:  70 seconds for two cylinder turn-overs (1 complete cycle)
- T1 and T2 profile monitor actuation checked:  T1 reached in and out limits but speed unknown because target covered;  T2 traveled smoothly in and out and reached limits as expected
- T1 and T2 target ladders were moved to the various targets that could be used this running period (#1 for T1;  #1, 3, and 4 for T2) - No issues

The T1 and T2 targets are ready for the running period.  Shielding can now be replaced over T2.
 

During first injection of beam down 1A problems were encountered with the T1 protect monitor.  Testing was done by Probes Group from the mezzanine leading to the following investigation and repair work.

May 9th:

- T1 area uncovered, Probes Group performed testing at the top of the monolith confirming that the problem was below the vacuum flange
- T1-MK2 target moved from the beamline to the hot cell.  Measured field 800uSv/hr at ~0.5m
- Visual inspection of protect monitor performed:  No obvious issues with monitor plates.  Some small impact holes visible on exit side of monitor, similar to damage on the previous monitor from Oct 2015
- Electrical check performed on old protect monitor in hot cell, confirming the previous results
- Failed protect monitor removed, resistor pack installed and tested with expected results
- Profile monitor insulation found to be blackened and cracking, so the profile monitor cassette was also removed
- New protect monitor and profile monitors installed
- Electrical check performed on both monitors at the hot cell: Results good for both

May 10th:

- Vertical and horizontal positions measured for the new protect and profile monitors
- Profile monitor actuation tested:  started moving ~10psi, fully actuated ~50psi, monitor limit switches ok
- Measured total resistance of potentiometer: 20.26kOhms
- Target windows inspected and photos taken (stored on Isaac's hard drive), nothing unusual encountered
- T1-MK2 target returned to beamline, vacuum pumped down without issue
- T1 cooling package started fine with no change in vacuum
- T1 profile monitor actuated in and out.  Movement smooth and slow
- Electrical check done on protect and profile monitors from mezzanine: both good

Flow stopped through the T1 collimator cooling loop around the same time as a power bump this afternoon.  The problem was narrowed down to failed solenoid valve B1A:T1:SVCOL by comparing coil resistance to other units without issue (~1.0ohm vs 3.6ohm for working units), measured at the T1/T2 PLC cabinet.  Shielding above the T1 package was removed and the solenoid coil was replaced with a new unit.  After replacement the valve works as expected and B1A:T1:FGCOL reading is normal.  I inspected the T1 package while it was uncovered: no leaks observed.

 The T1 cooling package outlet water pressure sensor (B1A:T1CS:PGOUT) was replaced this morning with a new unit.  Over the last few months the sensor readout had been steadily decreasing (from around 20psi to 4psi) with no corresponding decrease in inlet pressure, and no change in the circuit flowrate (this is how these pressure sensors typically fail).

After installing the new sensor the readback in EPICS returned to the expected value.  Although there are no alarms or trips associated with this signal, it is still useful as a diagnostic tool.

On Saturday July 21st the T1 Collimator solenoid valve would not re-open after tripping closed due to a (planned) loss of vacuum.  Shielding was removed and the valve solenoid was replaced which fixed the problem.  The original valve body was left in place.

This is the second time the solenoid at this location failed (See E-Log #239), and the latest of several failures of these new 24VDC solenoids at the T1 and T2 cooling packages.  Clearly they are not reliable in this radiation environment, so I will investigate an alternative to be installed in the 2018 Fall Shutdown or 2019 Winter Shutdown.

Cyclotron E-Fault #11420 has been returned.

On Wednesday August 15th the T1 cooling package tripped off, and could not be restarted.  On the EPICS screen the pump device went into an immediate time-out each time we attempted to turn it on.  An electrician was involved in the debugging, and issues were found with the 480V supply.  The breaker switch in the 480V supply panel was found to be damaged, and was replaced.  The relay in the contactor box for the pump motor thermal protection was also replaced.  These new parts did not solve the problem.  Resistance measurements between the phases of the pump motor revealed ~10ohms between phases A-B, but ~250ohms from B-C and A-C (measured from inside the contactor box).  The T2 pump and a spare replacement pump were also tested, and had ~10ohms between all phases.  It was therefore concluded that the pump motor, or possibly the wiring between the contactor and motor had failed.  The T1 cooling package was uncovered and drained by the end of the day.

On August 16th the old pump was disconnected and de-wired.  Damaged wire insulation on the pump motor was found, which likely caused a short, leading to failure of the pump motor, and cascading failures of the electrical system and possibly also the control PLC.  A spare pump (Chempump GB-3K-1S) was installed with custom cut gaskets.  The pump was re-wired by an electrician.  The cooling system was re-filled, and start-up was attempted, however there were remaining issues with the electrical system.  By the end of the day various testing by Controls Group and Electrical Group determined that the pump motor contactor also had failed and required replacing.

On August 17th the contactor and over-current protector were replaced by an electrician.  When the breaker was switched on the pump unexpectedly started immediately.  This was found to be caused by the control system drive signal module being faulty causing the output to be stuck on.  Failure of this module may have been caused by excessive current draw due to the damaged contactor.  The failed module was replaced, and the PLC and IOC were both restarted.  After this the system worked normally.  The pump was inspected by Maico Dalle Valle, no leaks were observed and operation seemed normal.  Shielding above the cooling package was replaced, and BL1A was restarted.  Cyclotron Fault #11527 was returned.

A new spare contactor and over-current protector have been ordered (Allied Electronics LC1D09BD and LRD12, Requisition #1037940).  These will be labeled and given to the Electrical Group.  A new spare T1 / T2 pump has also been ordered (Chempump GB-3K-1S, Requisition #1038041) which will be stored in the RH Meson Hall Hot Cell Lab Tool Port Boot Box Area.

A data sheet and quote for the replacement pump are attached.

 The water in the South TNF Holding Tank was released today under work permit #C2018-08-28-3.  The tank contained approximately 100L from the T1 cooling package (drained Aug 16, 2018) and 350L from the 500MeV system (drained April 23, 2018 by Steve Sapriken).

 On Friday Aug 31 at approximately 3am T1CS:FGSEC dropped to 0gpm flow.  This was confirmed to be a real reading when T1CS temperatures continued to rise steadily.  Beam was turned off, and T1CS later turned off by operators (temps continued to rise due to heat from pump).  The cooling package was uncovered in the morning, and the T1CS:SVSEC solenoid was replaced by Doug Preddy and Keith Ng.  This solved the problem and the package was restarted and then covered.

This is the latest of several failures of these new solenoids at T1/T2 since installation in the 2016 winter shutdown.  It is suspected that the "enhanced electronics" in this model of the valves is vulnerable to damage from radiation.  ASCO does not carry 24V DC valves in their "General Service" line which does not have the enhanced electronics.  Valves from other suppliers are being investigated.  If a suitable 24V DC model cannot be found then we can revert to the 110V AC ASCO valves which we know are reliable.  This will require relays and wiring to be done by electricians.

The T1 and T2 cooling packages were inspected today while running.  No water leaks were observed at either package including the high-active loops and copper active. 

The T1 pump was fairly noisy (video was taken to record the sound).  This is the new pump installed in August 2018 (see E-Log #242).  Maico reports that it had a similar noise when started after installation in August, and they were unsure if it was a normal sound or not.  The pump it replaced ran quietly, as well as the pump currently running at T2.  Further investigation will be performed this shutdown to try and determine the cause of the noise and appropriate action.

Both packages have been turned off and pump contactor control signals disconnected in preparation for MRO work and power outage this weekend.

The following work was completed between Jan 16 - Feb 19, 2019:

• T1 and T2 cooling packages drained to South TNF holding tank, water sampled and tested, released after approval from RPG
• T1 resin can swapped
• All Cuno filters changed at T1 (3 total)
• T1 inlet (T1CSPGIN) and outlet (T1CSPGOUT) pressure gauges replaced (archived data on EPICS indicated both starting to fail)
• T2 target water Cuno filter replaced
• All T1 and T2 solenoid valves changed to new models without power management circuit which will hopefully solve the problem of frequent failures (SASP0670)
  Parker brass body valve 73218BN4UN00 with solenoid C222C2 solenoid used for T1CSSVSEC and T2CSSVSEC
  STC stainless steel body valve 2S160-1/2-2-D with solenoid 2W200C-2-D used for T1CSSVTGT, T1SVCOL, T2CSSVTGT, T2SVCOLA, T2SVCOLB
  Necessary changes to PLC BOP wiring made by Tony Tateyama
  All new valves confirmed operational and leak tight
  Cyclotron Fault #11955 returned
• T1 and T2 profile monitor air cyclinder flow control valves replaced with new models tuned to ~1/8 turn open.  Part is Rego F125B (see PO #3044305)
• T1-MK2 profile monitor air cylinders replaced (both were leaking through the top seal)
• T1 and T2 profile monitor actuation checked - both move in and out slowly and smoothly
• M20 BB actuation checked - movement is slow and smooth
• T1 and T2 ladder movement checked - both ok
• T1-MK2 male Hansen fittings replaced as well as female Hansens which connect to them (these are the last remaining Hansen fittings to be replaced at T1/T2)
• Both cooling packages filled and started, now running smoothly.  Inspected for leaks (none found).  No change to 1A vacuum when packages started
   

Note 1: The T1 package water pump is slightly noisy when running (it has been this way since the replacement pump was installed last year) - Will investigate further at a later date

Note 2: In the week of Feb 11-15 Vacuum Group pressurized the T1 target cooling circuit with 30psi helium to try to find the vacuum load on the T1 volume.  This was done twice with the beamline vented for a few hours in between in case water had frozen at a leak location. The conclusion after testing is that the T1 cooling water is not the source of the vacuum load at T1.  Helium was also sprayed all around the top of the T1 monolith (T1 target and M15 permanent magnet) - there was no response on the leak detector

My total dose for this work was 0.15mSv

After extensive testing over the last 2 weeks it was determined that the protect monitor on the T1-MK2 target had an issue with intermittent shorting between the high voltage line and ground (movement of the target ladder was causing the short to come and go).

On the May 21 maintenance day the T1-MK2 target was moved from the beamline to the hot cell (102mSv/hr @ 0.5m).  The T1-MK1 target was then moved from Storage Pit location #3 to the beam line (as in the past, alignment of the flask to the #3 position hole was challenging, and the flask had to be suspended above the alignment pins to achieve alignment with the hole - In the future this storage pit location should be avoided).

Cameras were mounted on the T1 alignment frame for remote viewing of alignment with the hole as well as flask door position and hook latch position.  The camera locations chosen provided useful views and did not interfere with the target or frame.  After procurement of additional mounting brackets the cameras will be permanently installed at these locations (already completed on the T2 frame).

The target moves were completed successfully, vacuum pump-down was normal, and electrical checks of the T1-MK1 profile monitor and protect monitor were normal.  When start-up of the cooling system was attempted problems were encountered with the control system.  Multiple channels on the digital output module were shorting, causing the power supply fuse in the PLC cabinet to blow.  This was eventually tracked down to two damaged cables (B1A:T1CS:SVSEC and B1A:T1:SVCOL) which somehow had been crushed during removal of shielding.  The cables were replaced and all cables secured as much as possible in the vertical chase leading up from T1.  After replacing the cables we discovered that the digital output for the SVSEC signal was stuck in 'drive' status.  The module was replaced and then the control system was working normally (approximately 11am on May 23rd).

The T1 cooling system was started (no change to 1ACG2 vacuum), ladder movement through all target positions was tested, and profile monitor actuation was tested.  Updated target info sheets were submitted to Ops (attached in PDF format) and XTPAGE 7L was updated.

Diagnosis and repair of the T1-MK2 target will take place in the hot cell over the coming weeks.

Cyclotron Fault #12266 was returned.

The following work was completed on both the T1 and T2 target alignment frames

• One of the short taper pins on the top of the frame was swapped for a longer version to simplify flask alignment to the frame (now one pin can be aligned at a time instead of both concurrently)
• Three permanently mounted cameras installed to view east/west alignment, north/south alignment, as well as hook, latch, and door position
• LED strip lights installed on the bottom of the frame for better illumination during alignment of target devices
• All video and power signals routed through a single connector (9-pin CPC) for ease of use
• Frame painted with blue Rustoleum metal paint
• Hole positions, alignment orientation, camera position, etc labeled on frame

The following items were moved today from the Warm Cell to the Rad Waste Pit for long term storage:

1. A 55 gallon drum containing various active components from the East Hot Cell.  The drum was labelled on the body and lid with the identification number RWC-55-01-2019.

2. The old M9 Q1Q2 service stand, which was removed from the beamline in the 2019 winter shutdown

Photos taken during the move are attached, as well as a copy of the loading record form for the drum.  The form has also been given to RPG.

A second drum (RWC-55-02-2019) remains in the Warm Cell, and will later be moved to the cyclotron vault tunnel for storage.  It was resurveyed after the first drum was removed and deemed too low-level for storage in the pit (300uSv/hr at 0.5m)

 The second (and final) 55 gallon drum containing active items from the Meson Hall East Hot Cell was transported today from the Warm Cell to the Cyclotron East Vault Tunnel.   The updated form is attached.