M9sol Logbooks Solenoid Daily log
  Solenoid logbook, Page 1 of 15  ELOG logo
ID Date Authordown Operational Mode Subject
  2   Fri Dec 19 13:07:25 2003 Willy AnderssonMaintenanceTo-Do list for M9SOL
I had distributed this list by email earlier this month. I have added
a few items at the bottom. I am posting it on the M9SOL Elog under
Maintenance. Some of the items may not be achievable or practical but
I thought it would be a useful first attempt at a To-Do list.
Attachment 1: M9Blist3.txt
To-do and To-consider items for M9B superconducting 
solenoid/refrigeration system.

- New pressure guages for the Mycom compressor panel. The 
present guages are damaged from vibration and do not read 
properly. The new guages should be isolated somehow from 
vibration. 

- Investigate designing and building a compressor oil 
heater/conditioner. The helium compressor should be topped up 
with "conditioned" oil. 

- Have contractor check allignment of compressor motor to 
compressor and check flex coupling condition. MYCOM to 
recommend contractor. Check condition of and re-grease 
compressor motor bearings. 

- Replace missing and unreadable identification tags on 
valves, piping, and instrumentation throughout the system. 

- Identify, catalog and label stored spare parts and equipment 
and provide proper storage. 

- Provide "Lamp Check" button for main compressor panel. 

- Investigate LED type replacement bulbs ( Base Style 24XX ) 
for Sulzer control panel. 

- Update and improve operation manual and maintenance instructions. 

- Provide computer control for solenoid power supply to 
simplify operation and make it less prone to operator 
error. 

- Re-arrange locations of vacuum computer and X displays so 
that the opportunity of accidently bumping into buttons on Sulzer 
panel is reduced. 

- Re-write solenoid power supply logic to require that both powerlead 
temperatures must be stable and below XX? Kelvin for 60 minutes before 
enabling arming the solenoid power supply. 

- Increase computer security. Restrict physical access ? 
Virus scanner ? 

- Look into using Ultra-VNC with encryption for security. Stop 
Commute modem access but route Commute access through hard wired 
serial connection to PC running Ultra-VNC. Or keep modem access 
but use call back method. 

- Set up procedure and pipe in convenient sample points for 
dewpoint, N2/O2, oil content readings. Sample readings should be 
documented prior to starting turbine cooling. 

- Upgrade LN2 fill equipment for Purifiers. Can both puifiers 
be operated in parallel during a cool down ? Auto-isolation of 
purifiers' from helium system when LN2 runs out or fails or 
power failure. 

- Provide dedicated LN2 line to M9B compressor building ? Guy Stanford 
has info on relatively inexpensive efficient foam covering for LN2 
lines instead of vacuum jacketed piping. 

- Can anything be done about the epoxy fumes from the buffer tank ? 
A charcoal/molecular seive filter in the buffer tank line 
check-valved for free flow to buffer but filtered flow from buffer ? 

- Fix back-up air compressor system. 

- Remove any unauthorized taps off of the M9B compressed air line to 
other non M9B systems. 

- Clean-up of compressor building and clean-up around Sulzer panel. 

- Route turbine defrost circuits to outside of shielding blocks. 

- Clean-up area behind solenoid power lead flowmeters and pressure 
gauge panel. Remove all unused tubing and cables. 

- Harden Sulzer control panel against water damage from water leaks 
overhead. 

- pipe in precision flow control valve and pressure guages on powerlead 
flow to main compressor suction line. 

- pipe in manual bypass circuit around main coldbox HE in and out 
valves. 

- New vacuum controller for Sulzer panel. Repair old one for a spare. 

- Add in some smart logic to prevent system trip on solenoid vacuum 
during a quench recovery cooldown. When solenoid power supply is 
not armed solenoid vacuum should be allowed to degrade slightly 
( within limits ) from contamination defrosting into solenoid vacuum. 
As soon as solenoid power supply is armed then solenoid vacuum trip 
set to high vacuum again. Solenoid vacuum trip logic should be re-worked. 

- provide vacuum gauge between solenoid vacuum diffpump and BV2. 

- provide computer readable indication for: 

        - CV204 position 
        - LHe heater reading 
        - turbine 1 entrance temp. 
        - Buffer tank line pressure 
        - air pressure 
        - position signal to CV211 and CV212 
        - position of CV151 
        - position of CV141 
        - 80K shield heater setting 
        - purifier LN2 status 
        - backup air compressor OK 
        - solenoid vacuum 
        - solenoid current 
        - solenoid power supply armed ? 
        - others ????? 


- check oil levels, drive belts etc for system mechanical, diff, and 
cryo pumps. 

- check calibration of system pressure and temperature sensors. 

- repairs and upgrades to Sulzer Control panel instrumentation and 
relays 

- re-charge TI148 and TI149 

- follow-up on improvement requests for X displays for vacuum and solenoid 
data. 

- compressor water package maintenance and repairs. 

- repair compressor building roof leak. 

- inspect and clean system air filters and coalesers and water filters. 

- buffer tank clean-up and re-paint. 

- train more system operators for M9B and ISAC II helium refrigeration 
systems ? 

- Vacuum gauges on cold box transfer lines ? Or build small portable LN2 
trapped vacuum pump station ( probably very usefull for many other 
areas at TRIUMF including purifier regeneration in compressor building ) 

- Should we buy Don Dales' first trip logger ? It is a PLC that is networkable 
and may be used for other functions ? Is it obsolete or outdated ? 

- Develope a system using a residual gas analyzer for helium gas purity 
measurements ? 

_____________________________________________________________________________ 

Items added Dec 19,2003 :

- Sniffer Leak check of piping, especially powerlead insulators.

- Top up gas to system. The lowest helium relief valve setting is
SV45 set at 5 kg/cm^2 Gauge  (71.12 psig) so it would be reasonable
to fill the system to 60 psig.

- re-work purifier plumbing, add purge ports at flowmeter. Make
purifiers portable ?

- Make small portable circulation helium compressor + ORS skid ?

- Schedule quarterly steady state vibration measurments for MYCOM 
compressor. (Tom Ries has vibration measurement equipment)

_______________________________________________________End of list for now !
  11   Wed Apr 21 09:57:59 2004 Willy AnderssonStart UpSystem status
Buffer tank pressure is down to about 6 psig. Too low to keep up
with liquefaction requirements to build LHe level. Doug Preddy
requested that I help Curtis Ballard top up system with new
bottle of UHP helium Through purifier. As of 09:45 not yet able 
to find Curtis.

PI229 discharge pressure dropped slightly to 10.45 barA. Set point
is 11 barA. Turbine speeds OK. CV211 is wide open. Energized HIC142
and set to 0.7 to start flow to powerleads. Up in compressor room
checked purifier. LN2 system running OK. In purifier package flowmeter
looks like oil or brownish spots on inside of rotameter glass. Possible
migration of trapped contaminants after correcting flow arrangement
through purifier. Initially buffer tank circulation flow entered
through AV3 but was corrected to enter through AV5.

Have 10:00 meeting, system seems stable and should be OK until I can
contact Curtis to top up system with UHP helium through purifier.
  14   Sun Apr 25 21:41:00 2004 Willy AnderssonQuench RecoveryTrip on PL Temp.
Got pager call at 16:20 this afternoon. Solenoid had tripped on PL 
temperature. Arrived at about 17:00. Did a print screen of the striptool 
graph and solenoid screen. See files Prtscrn1_april25_2004.doc and
Prtscrn2_april25_2004.doc. System was in idle running mode. LHE heater at 
about 30%, buffer tank press at21.5 psig, shield heater at 37.5%. The
jumper is still on the back of the vacuum guage PIC901. The compressor 
building backup air compressor was tripped again, I reset it. HIC 141 was 
set to 0.46 , I set it to 1.1 (closed position). Set solenoid inlet and 
outlet HIC's to valve closed positions. Energized warm return HIC's then 
slowly opened warm return valves to vent pressure in solenoid. Cracked open 
solenoid inlet valves to get abot 1.5 barA pressure on PI143 and PI146. LHe 
level started to drop. Contiued slowly until all LHe boiled away. PI143 and 
PI146 at about 5 barA. Had to adjust turbine speeds and shield heat. Opened 
cold returns then closed warm returns. Gradually increased solenoid inlet 
valves until wide open. Adjust turbine speeds and shiled heater. At about 
21:00 stable // cold return cooling. Temperatures dropping gradually. See 
files april25_2004_2113.BMP and april25_2004_2114.BMP 

Have powerlead supply valve HIC142 open to 0.46 during cooling just to see 
what happens. Time now about 21:42. Leaving system in // cold return 
cooling. Should be ready for series switch over in the morning.
  26   Wed May 5 23:57:38 2004 Willy AnderssonQuench RecoveryPowerleads warming up again.
Got a page from Don Flemming at about 18:50 wondering why the power
lead temperatures were rising. See attachment may5_04_1800.BMP It looked as
though the monitoring signals flat lined for about 2 hours for some reason.
Anyway the powerlead temps were definitely rising. When I arrived HIC142
was wide open but both power lead flows were down to 65 (on the scale) and
the flowmeter float was vibrating rapidly. LHe heater at 33 %. Buffer 
pressure at 10 psig. LIC108 at 54 %. CV147 at 0.45 (HIC141). CV141 at 0.6
TCI121 in auto mode.

Closed HIC142 then opened again. Powerlead temps dropped. Opened CV147
from 0.45 to 0.35 (HIC141) Played with setting of set point of TIC121 and
HIC141 to try to increase excess liquefaction. Reduced shield heat to about 
16%. Difficult to say what is going on with powerleads. Temps had dropped to
good levels but now gradually rising, but very slowly. Hopefully they peak 
and level off. Energized solenoid. Cycled HIC142 a few times but temps
would not trend downwards. Leaving it overnight. Doug, please check and 
adjust the system as required in the morning.
Attachment 1: may5_04_1800.BMP
  27   Thu May 6 09:32:13 2004 Willy AnderssonQuench RecoveryAdditional notes from last night power lead problems.
When performing last nights refrigerator adjustments I also increased the 
compressor discharge pressure to 11.25 barA. Also the backup air compressor 
was tripped off again and I reset it. Finally I opened the city water flow 
to the compressor water package chiller. PI110-1 reads 9.25 barA and PI120-1
reads just over 4 barA. Both power lead flowmeters pegged to top of scale.
Reset the "first trip" device that Don Dale had loaned us.

This morning the power leads had crept up slightly but still well below 
trip settings. Oscillations have not appeared yet. LHe vessel heater at 
about 14 to 15 %. Shield heat at about 16 %. Saved screen shots of 
m9_solenoid.dl and striptool windows to files may6_04_0900.BMP and 
may6_04_0905.BMP. TE106 is 81.6 Kelvin. Solenoid Current is stable at 404 
Amps. (DVM meter reads 20.2) Both power lead flowmeters still pegged to top 
of scale.

Closed HIC142 for 1 second then opened it wide open again to see effect on 
power lead flows. Other than the initial blip they just fall back to their 
previous value, no improvement. See file may6_04_0915.BMP. Adjusting 
turbine speeds slightly. Going to take a check in the compressor building 
then going on to other work.
  29   Fri Jun 25 12:36:04 2004 Willy AnderssonQuench RecoveryCompressor unloaded etc.
At about 09:30 this morning I noticed the sound coming from the compressor 
building seemed louder than usual. After entering the building I noted that 
the compressor was unloaded to about 50% and the compressor suction was 
running at about 15 psig. Down at the SULZER panel the solenoid vacuum 
system was tripped off, the flow was still in series mode but no LHe level,
solenoid temps at over 40 K. Contacted Doug. Cannot figure out why the 
compressor was unloaded to 50 %. Solenoid had not tripped on bad vacuum 
because jumper was still on PIC901. Doug mentioned there was a power bump 
yesterday at about 15:00. May be when the vacuum system tripped but does 
not explain the compressor unloading.

Doug restarted vacuum system. I stopped flow to solenoid. Stabilized 
pressures. We checked the compressor to try to see what would cause the 
Unloaded state. Could not see any oil leaks. 

Back at Sulzer panel loaded compressor to 100 %. Stabilize pressures. STart
cooling flow to solenoid, // warm return. Adjusting turbine speeds and 
shield heat as required. Switched to // cold return flow after solenoid 
temps down to TIC 121 temp of about 20 K.
  30   Fri Jun 25 14:20:58 2004 Willy AnderssonQuench RecoverySwitch to series mode
At about 14:00 solenoid temps dipping below 8 K. Trace of LHe level (dense 
gas) starting to show. Switching to series cooling mode.
  31   Tue Jun 29 14:59:46 2004 Willy AnderssonNormal RunningRemoved jumper on vacuum controller.
Tuesday 14:55

Removed jumper on back of PIC 901. This jumper should be off during
normal operation so the solenoid/refrigerator is protected from bad vacuum.
  40   Sat Aug 7 22:37:04 2004 Willy AnderssonQuench RecoveryCompressor tripped off.
Got a call at about 10:00 this morning, the whole system had tripped off. 
No power bumps or other disturbances. When I arrived the only fault 
indications were "ColdBox Trip" light on the compressor panel and on the 
QRL panel the red "ColdBox" LED was ON. On the SULZER panel the 
yellow "Cryostat Quench" light was ON. On the First Trip logger #1 and #15 
LED's were ON. But no indication about what the actual fault was. Cooling 
water OK, compressed air OK, compressor ventilation OK. Outside ambient 
temperature was cooler than we had last week. BIG1 was off but when I 
turned it back on the vacuum looked good but was slowly rising as the 
solenoid warmed. I put the jumper on the vacuum controller. Prepared system 
for re-start.

Started compressor. Started flow thru coldbox. Raised discharge pressure.
Started turbine flow. Didn't get any freezing but it seemed that T1 speed 
wandered slowly +/-0.1k rev/sec when it got up to about 3.6k revs/sec, but 
this only lasted for a second or two. Not sure if I was just seeing things.
Got turbines up to speed and started // warm return flow. Solenoid had been 
running at 500 Amps and when it quenched and sat warming it had warmed to 
about 40 K. Switched it to //cold return too soon and solenoid started to 
warm slightly. Back to // warm return mode. Had to throttle flow thru 
solenoid with warm return valves to get effective cooling. If had return 
valves wide open TI121 would warm up too much. Discovered that if I 
throttle the flow out of the solenoid slightly then TI121 would fall and 
solenoid would start to cool. Less flow gave more cooling. Don't quite 
understand that one. Switched to // cold return mode when solenoid temps 
falling below 20K.

About 15:00 stable // cold return cooling. Going home for a while.

19:30 solenoid temps bottomed out at about 11K. Switch to series mode. 
Something strange about the first trip detect. I reset it once and when I 
checked it an hour or two later it showed a trip on #1 and #15 LED's even 
though nothing had happened. Something must be causing false indications.

20:15 series cooling, solenoid temps at about 6K, LHe level rising slowly 
at about 10%. Removed jumper on vacuum controller.
  41   Sat Aug 7 23:40:47 2004 Willy AnderssonQuench RecoveryEnergizing solenoid
23:39 Powerleads cold, LHe level over 40% and rising. Energizing solenoid 
to 500 Amps, ramp rate 60. Checking compressor building then going home.
  47   Thu Sep 2 21:58:34 2004 Willy AnderssonQuench RecoverySwitch to series cooling
Solenoid had tripped on powerlead overtemp at about 09:00 this morning. 
Doug recovered the system to // cooling. I switched system to series 
cooling at about 21:30. LHe level at about 10%. Buffer tank pressure is low 
at about 13 psig. Letting LHe level build. Once LHe level rises to about 
35% then I suggest starting flow to the powerleads by opening HIC142 fully 
open. LHe level should continue to build but at a much slower rate. Once 
power lead temps have bottomed out to cr_pl_neg < 9K and cr_pl_pos < 11 K 
(check these values with archived data first) and are stable then can 
energize solenoid.
  86   Sun Nov 28 10:15:18 2004 Willy AnderssonQuench RecoveryPowerlead quench ?
Got page very early this morning. Couldn't come in til 08:30. System was 
in idle running mode and someone had reset the QRL panel so my guess is 
that the solenoid tripped on powerlead overtemp. HIC142 was not fully open 
and I suspect this is why powerleads heated up. Now in // cold return mode.
Solenoid temps down to 25K and slowly going down. Have HIC142 slightly open 
to keep powerleads cool. Strongly recommend that when running HIC142 should 
be fully open to try to reduce the probability of powerlead warmup.
  101   Thu Jun 9 21:41:25 2005 Willy AnderssonQuench RecoveryTrying to achieve stable cooling.
From about 08:00 to 15:00

See Doug's previous entry for quench recovery details.
First the system had been set to // cold return mode because the solenoid 
temps seemed low enough. Temps started to rise instead of cooling so system 
was eventually set back to // warm return mode. With the system in // warm 
return mode the solenoid was still warming instead of cooling. Solenoid 
temps were rising to almost 50K and T1 shield heater was about 90%. T1 
inlet temp was below 75K and falling even with shield heat set so high. 
CV155 kept closed and // warm return flows were full flow. T1 brake valve 
seemed way too open at about 0.8 (compared to previous runs).

15:00 to 19:00

Tried opening CV155, didn't seem to help much. Temps still trending 
upwards. Decided to reduce flow to solenoid to see if the cold box would 
cool down. Started reducing the // warm return outlet valves. TI121 began 
to drop slowly. On graph plotting solenoid temps they started to trend 
downwards. Weird. A reduced flow seemed to produce more cooling ? Don't 
understand this but will continue. Flow temps returning from solenoid were 
about the same temp as TI121, about 35 K. Switching to // cold return mode 
to try to reduce TI121. Keeping CV155 as open as possible, just keeping 
CV204 from closing completely. After switching to // cold return TI121 
starts to slowly fall, T1 inlet temp rises over time so start to drop 
shield heater setting. Solenoid temps now trending downwards. Cracked open 
CV142 to get some flow going through powerleads. Raised compressor 
discharge press to about 11.2 barA. Shield heater down over time to about 
10%. Reduced CV155 down to the point where temps almost stop falling then 
open cv155 just enogh to keep temps falling.

19:30 Solenoid temps down to about 28K. Letting system continue cooling 
overnight.

Guessing the cause of the turbine overspeed quench was caused by 
contamination entering the spinning turbine, causing rapid speed changes or 
possibly the continued saga of the water damaged Jaquet turbine speed 
tachometer giving random false trips.
  107   Tue Jul 19 16:13:32 2005 Willy AnderssonQuench Recoveryswitch to series mode
CR_6KS_M down to 11 K.
Switched to series mode.
Started small flow to power leads to get them cold.
HIC142=0.6        Later open HIC142 wide open.
Raised discharge press to 11.4 barA.
Letting LHe level build.
  108   Tue Jul 19 18:09:56 2005 Willy AnderssonQuench RecoveryQuench II
Machine trip just as I was exiting the TRIUMF gate. QRL said cold box fault 
as did red lite on compressor. At cold box panel all turbine red lites were 
on and compressed air failure lite was on. But there was no problems with 
compressed air. On fist trip logger LED # 8 and 16 were lit. Restarted 
compressor then start flow to cold box, start turbines, start // warm 
return until temps returning back from solenoid were <= TIC121 then 
switched to //cold return mode. Opened cv155 to 0.88
Solenoid temps at 24K and starting to trend down.
  109   Sun Aug 14 10:03:13 2005 Willy AnderssonNormal RunningSystem stable
No experiment running in M9B.

Solenoid current is zero and power supply not armed.

System in stable series cooling mode.
  110   Mon Oct 31 07:48:57 2005 Willy AnderssonQuench Recoverywhat a fine entry
Keep IT safe boys
  111   Wed Nov 2 10:17:27 2005 Willy AnderssonStart UpM9B helium refrigerator and solenoid cooldown
Started M9B MYCOM helium compressor circulating helium through system last 
Friday. Right hand purifier was regenerated previously, LN2 fill system 
started and flow started from buffer through purifier to low pressure 
return line to compressor. Left in this mode over the weekend. Curtis came 
in on weekend to add new full LN2 dewar to purifier.

Started // warm return cooling Monday.
Tuesday morning solenoid temps about 35 K and TI121 about 35 K  so 
switching to // cold cooling.

Note:

PDI 104 guage is very slugish to respond, needs to be replaced or cleaned.
TI148 and TI149 need to be recharged with helium, they do not work yet.
TIC121 may be reading too high a temperature, try to verify it with TE147 
before JT effect begins. Now TI121=20.7 K   TE147=25 K   HIC155 is full 
open still.
LIC108 is reading too high, reads 22% LHE but really is no LHE at this 
point.
  125   Tue Jun 13 15:52:46 2006 Willy AnderssonQuench RecoveryDoug has reduced and now closed CV155.
Now in full // cold return mode.
Had just talked with Curtis, He said thta he has already informed the 
experimentors that the solenoid will not be ready until tommorow, so he 
suggests leaving the system in reduced // cold mode overnight.
  126   Tue Jun 13 16:04:33 2006 Willy AnderssonQuench RecoveryTIC 121 set to auto mode
ELOG V2.9.2-2455