Turbo pump TP6 requires replacement.

Wednesday, November 15, 2017, 09:55: Turbo (TP2) off. Ion gauge off.

Wednesday, November 15, 2017, 10:55: Venting

Power outage over the weekend. Reboot PLC and systems.

Monday, December 04, 2017, 08:22: Turbo pumps TMS:TP6 & TP2 Restarted.

Monday, December 04, 2017, 08:28: Time out on turbo pump TP1B.

Monday, December 04, 2017, 09:25: Turn off/restart TSG:TP1B Maincontrol.

Monday, December 04, 2017, 09:26: TMS:IG6 Restarted.

Monday, December 04, 2017, 14:24: Multi gauge controller to TSG:IG1 burned out. Replaced. Vacuum @ 4.45e-06T. Monitor.

Powered down vacuum gauge and turbo pump controllers

Tuesday, January 02, 2018

Cycled power for TCSIOC and PLCIOC

TP2 and TP6 started

TP1B does not start. TP1B controller does not start up correctly - controller will have to be replaced. E-fault submitted (10739).

Controller for PS for Q1,6 was switching constantly. Unplugged and plugged back in resolved the problem

TP1A started

(Aaron Schmidt)

Wednesday, January 03, 2018, 08:15:  TSG:TP1B Still non-responsive.

Wednesday, January 03, 2018, 08:20: Power/sources on. Add Voltage/Amp and check power supplies.

BIAS - 300V Reads 290V - OK

EE - 300V Reads 298V - OK

EZL - 300V Reads 293 V - OK

CB1 - 10V Reads 10V - OK

VCB1 - 10V Reads 10V - OK

CBC (both)  - 10V Reads 9V - OK

TBHT - 10A Reads 9.9A (009 direct off PS) - OK

TGHT - 10A Reads 9.8A (007 direct off PS) - OK

Optics Check:

Magnet ok - Some optics start, some do not respond. Submit fault report for communications issues.

Fault Report submitted - 10742

Thursday, January 04, 2018, 14:38:

Kyle and Dave were able to get all the canbus optics for ISAC test stand re-configured except one device: TMS:Q2

"We investigated and it looks like the controller board for the TMS:Q2 supply is toast. Unfortunately these boards are a very old style so we don't have a surplus of replacements on hand. Our electronics shop did have some spares but the spares are not ready for use and need to have a few components soldered on. " - Kyle

The repaired controller board should be ready for next week.

Wednesday, January 10, 2018, 13:22: TS optics all fixed. Turbo pump TP1B is still out.

Computer, TS, epics, restarted. Power checks

Monday, January 22, 2018, 09:00: ISAC PLC TLCIOC Restarted. 

Turbo pumps restarted.

Monday, January 22, 2018, 09:26: New controller for TSG:TP1B brought in.

(Connect) Global reset. Resart Turbo pumps.

Monday, January 22, 2018, 09:40: Both turbo pumps @ speed.

Vacuum controls ok

Sources ok

Optics -  Issues. Some restart, others do not.

Friday, January 26, 2018, 11:08: New Helium tank installed inside the Test Stand.

-The 'Full' tag remains. The He tank was not opened.

-The old tank was placed beside the Test Stand beneath/beside the power switch.

Wednesday, January 31, 2018, 09:50: A wire (B - F) was too close (Almost touching) to the EZL and had visible sparks during HV tests 1 & 2. The wire was adjusted to sit higher. and away from the EZL

Monday, February 05, 2018, 09:10: the epics program was in 'White-out'. Turbo pumps tripped.

Unplug, re-plug PLC.

Monday, February 05, 2018, 09:16: Restart Turbo pumps on chamber & beamline.

Monday, February 05, 2018, 09:21: Ion gauges on.

Tuesday, February 06, 2018, 09:00: TSG:CB1 noted to be red. Attempts to restart/reconfigure failed. Alive displays red MAJOR. Fault submitted.

Wednesday, February 07, 2018, 08:40: A 75 kV Glassman power supply was installed in the rack for use in further HV testing.

-The 75kV power supply is linked into the keyed interlock system. The key & switch labelled BIAS/EZL will unlock it.

-The PS is currently not connected to the epics controls and can only be adjusted manually.

-The old Glassman supply is disconnected and removed from the cage.

The 75kV power supply is now in EPICS under the usual TSG:BIAS controls. The upper limit has been set to 70kV to protect the transformer.

Thursday, February 08, 2018, 16:54

Testing the HV bias I found that the einzel lens was behaving strangely. When I tried to put it higher than 1kV, the current would start spiking from 12uA to > 100uA. The voltage would jump around quite a bit as well (+/- 100V). Everything seems the way it was before, except the EZL power supply is still connected in parallel with the old bias power supply, which is no longer powered. Maybe one of the transorbs connecting line and neutral on the bias side is fried? I will unhook the connection to the old PSU and try again.

Friday, February 09, 2018, 08:25    Disconnected the AC power chain from the old bias supply but now cannot turn on the new bias supply in EPICS because the "TSGBIASSAFETY ok" interlock is tripped. No idea what this means - maybe it checks for AC power?

Friday, February 09, 2018, 08:48    Reconnected power to old bias supply, does not fix problem. Talked to Mike, he thinks it is PLC related, so will call Gelo later.

Friday, February 09, 2018, 11:38 TSGBIASSAFETY ok interlock problem seems to have solved itself, it now appears to be working fine. Will turn off power for now and continue tests on Tuesday. (note: turns out Mike told Dave about the problem and he solved it. No one knows what this interlock is for, but it used to be reset using the Global Reset button and now the RST button for the power supply should also reset it.)

Ramped up to 60kV with the new power supply, same as what we did last time with the standard test stand bias supply (see entry #627 ). Much more instability on the bias current readout (see attached screen shot), develops around 54kV. As well, the old supply read out 37uA at 60kV while the new supply reads out ~85uA. This could just be an EPICS read back issue though.

Sparks at 62kV on the bias.

Turned it down, turned it back up and got a spark at 60kV.

Now up to 63kV, no sparks so far (5 mins). Left EE @ -3100V and EZL @ 28.5kV

There is the sound of a low level continuous discharge, like HV power lines, and a very strong ozone smell coming from the HV duct that sticks into the cage and carries the cables from the HV rack. Also seems to be present near the EE insulating standoff. Can't see anything though. Maybe just moisture in the air? Starts at 55kV on the bias, does not change with EE voltage.

Turned EZL up to 29kV, bias stable at 63kV for 11mins.

Sparks at 63.6kV in the EE spark gap, EE @ -3.1kV.

I vacuumed inside the HV cage, though it has many small areas that are difficult to access so I am not sure how great a job I did.

Turned the bias up and again at 55kV I hear small sparks near the end of the HV duct that brings the cables from the rack. Cannot see anything. If it is really sparking where it sounds like sparking, it must be from the edge of the duct to the faraday cage.

Changing the EZL voltage and the EE voltage has no effect on where it starts sparking.

Found TMSTP6 off, restarted it, seems fine now (drawing 0.8A)

The test stand holds up to 63kV, however at voltages over 54kV there are small discharges, rising to a continuous level around 60kV. The best guess right now is that these are coronal discharges coming from the end of the HV duct to the faraday cage. The only way to fix this would be to round out the edges of the duct, or to move the duct farther away. Since I have heard small discharges in other places, we may need to make a variety of modifications. Things that have had no effect:

1. Changing the EZL/EE voltages - break down is happening only between HV bias and ground

2. Cleaning the faraday cage

3. Locating sparks with a thermal camera

4. Conditioning - break down always happens at the same point

Tuesday, February 20, 2018, 09:01: Sparks at 62.5kV after a couple minutes. Sounds like they were in the EE spark gap but don't appear on the EE read current. Ozone smell around 61kV, so there are still some discharges.

Turned back up to 62.5kV but no sparks. Put it up to 63.5kV and there was a spark, but not in the EE spark gap as expected. Took down TP1B but everything is back up and seems fine.

Friday, February 23, 2018, 08:50

Friday, February 23, 2018, 08:53: BIAS / EZL / EE @ 62.5 / 29 / -1 kV

Friday, February 23, 2018, 08:56: Spark* Turbo pump TMS:TP6 restarted,

Friday, February 23, 2018, 09:13: Bias sitting at 61kV for about 15mins now. Still corona discharges from the area of the EE, but the area of the duct seems fine.

1. Al corona shield installed over end of the HV duct to suppress discharges from the sharp edge of the duct.

2. Some moving and shortening of cables to keep them from running close to different voltages.

3. Changed the angle bracket on the 3M bias-EE resistor chain to a plate, same position.

4. Moved the 164K EE resistor chain to sit flat against feedthrough K.

5. Moved the bias-EE spark gap to sit flat against the back of feedthrough K, required tapping a hole in the back of the copper clamp.

6. Cap and top of rack vacuumed.

7. Piece of shielding removed from top of HV rack