Anders Mjos wrote:

The level dropped below 15 in over the weekend. MP2 stopped, but MP3 condtinued pumping.

 e-fault submitted and Niel Wong has been contacted.

Niel Wong came and had a look at the system. We decided to close the circuit as the water is no longer used with the new power supplies. The circuit will be monitored, but not repaired unless it is needed again.

Anders Mjos wrote:

Anders Mjos wrote:  The level dropped below 15 in over the weekend. MP2 stopped, but MP3 condtinued pumping.  e-fault submitted and Niel Wong has been contacted. Niel Wong came and had a look at the system. We decided to close the circuit as the water is no longer used with the new power supplies. The circuit will be monitored, but not repaired unless it is needed again.

 Due to the water leak, the expansion tank needed topping up. It was filled from 14 in to 20 in. See work permit I2014-12-15-2.

Anders Mjos wrote:

Anders Mjos wrote:  The level dropped below 15 in over the weekend. MP2 stopped, but MP3 condtinued pumping.  e-fault submitted and Niel Wong has been contacted. Niel Wong came and had a look at the system. We decided to close the circuit as the water is no longer used with the new power supplies. The circuit will be monitored, but not repaired unless it is needed again.

The valved off water circuit has been valved off and  TCS:WFB2:BIASSW1 and TCS:TCM2:BIASSW1 have been removed from the power supply interlock chain by Dave Morris.

The RGA showed no response to He-spray on the Service Cap. He stable at 3.5E-9 T partial pressure, water at 1.3E-6 T. IG1 at 9.9E-8 T

 by "enable staging" button. MP3 has to be turned off-on to open WPV1

No leak detected on either He-detector (connected directly to Service Cap) or on RGA. See attached screen shots.

A vacuum increase (air only on RGA) was seen when de-pressurizing circuit I.

No Source Tray installed. Loop on cooling circuit D. All other circuits blanked off.

Voltages stated below are set point voltages. These corresponds reasonably well with the local read backs on the power supplies. The read back voltages in EPICS are somewhat lower.

Wednesday, January 07, 2015, 09:16: 20 / 10 kV (BIAS / EL)

Wednesday, January 07, 2015, 09:20: Vacuum responses from BIAS 27 kV. Corresponds to each increase in bias voltage.

Wednesday, January 07, 2015, 09:40: 30 / 15 kV 

Wednesday, January 07, 2015, 10:28: Sparking and vacuum responses continues with every 100 V BIAS increase. If the voltage is lowered and brought back up, no sparks are observed; the module is conditioning.

Wednesday, January 07, 2015, 12:09: 34 / 17.5 kV

 Wednesday, January 07, 2015, 13:05: 35 / 17.5 kV

Wednesday, January 07, 2015, 14:50: 37.5 / 20 kV

Wednesday, January 07, 2015, 15:40: 38.5 / 20 kV

Wednesday, January 07, 2015, 16:02: BIAS and EL power supplies off.

Thursday, January 08, 2015, 10:00: HV condtioning continues

Thursday, January 08, 2015, 10:11: 35 / 20 kV

Thursday, January 08, 2015, 10:29: 38.5 / 20 kV

Thursday, January 08, 2015, 11:43: 40 / 20 kV

about 15 sparks per hour (5 / 20 min)

Thursday, January 08, 2015, 14:41: 42.5 / 22.5 kV

Thursday, January 08, 2015, 15:08: Communication lost with RGA. Could not be recovered.

Thursday, January 08, 2015, 15:58: 45 / 22.5 kV

Thursday, January 08, 2015, 16:27: BIAS and EL power supplies off.

Friday, January 09, 2015, 08:52: 45 / 22.5 kV

Tried to recover RGA by unplugging and plugging power cord, but no success.

Friday, January 09, 2015, 10:45: Increased sparking at 48 kV - reduced to 47.5 kV

Friday, January 09, 2015, 11:53 : 50 / 25 kV

Friday, January 09, 2015, 12:28: ~ 1 spark / minute for the last 10 min

 Friday, January 09, 2015, 15:27: ~25 sparks / hour for the last 20 min

Friday, January 09, 2015, 16:18: ~18 sparks / hour for the last 20 min

Monday, January 12, 2015, 09:31: 50 / 25 kV

Monday, January 12, 2015, 10:31: No sparks for 20 min after ~30 min of conditioning.

Monday, January 12, 2015, 12:24: BIAS / EL power supplies tripped on conductivity (spark)

Monday, January 12, 2015, 13:13: 55 / 27.5 kV

Monday, January 12, 2015, 14:32: 1 spark for the last 10 min

Monday, January 12, 2015, 15:38: 57 / 30 kV

Monday, January 12, 2015, 15:48: ~60 sparks / hour for the last 10 min

Monday, January 12, 2015, 16:00: BIAS and EL power supplies off.

Tuesday, January 13, 2015, 08:45: 55 / 27.5 kV

Tuesday, January 13, 2015, 09:18: ~60 sparks / hour for the last 10 min

Tuesday, January 13, 2015, 10:17: No sparks for the last 10 min

Tuesday, January 13, 2015, 10:30: 57.5 / 30 kV

Tuesday, January 13, 2015, 11:34: 59 / 30 kV

Tuesday, January 13, 2015, 13:23: No sparks for the last 25 min 

Tuesday, January 13, 2015, 14:11: 60 / 30 kV

Tuesday, January 13, 2015, 14:42: Spark tripped TP2 and TP1S and both power supplies

Tuesday, January 13, 2015, 15:02: 60 / 30 kV (local read back 60.5 / 30 kV, 0.1 / 0.15 mA)

Tuesday, January 13, 2015, 15:38: BIAS and EL power supplies off.

Wednesday, January 14, 2015, 08:34: 60 / 30 kV

Wednesday, January 14, 2015, 10:41: 3 sparks / hour for the last 60 min

Wednesday, January 14, 2015, 13:39: 61 / 30 kV

Wednesday, January 14, 2015, 14:29: 61.5 / 30 kV (local read back 62 / 30 kV, 0.1 / 0.15 mA)

Wednesday, January 14, 2015, 15:40: ~ 6 sparks / hour for the last 20 min

Wednesday, January 14, 2015, 16:07: TP2 found to be tripped off. Restarted.

Wednesday, January 14, 2015, 16:23: 60 sparks / hour for the last 20 min

Wednesday, January 14, 2015, 16:28: BIAS and EL power supplies off.

Thursday, January 15, 2015, 08:27: 60 / 30 kV

Thursday, January 15, 2015, 11:24: ~35 sparks / hour for the last 20 min

Thursday, January 15, 2015, 14:09: 12 sparks / hour for the last 60 min

Thursday, January 15, 2015, 15:33: 58 / 30 kV

Thursday, January 15, 2015, 15:49: 56 / 30 kV

Sparks observed both at 58 and 56 kV

Thursday, January 15, 2015, 15:57: BIAS and EL power supplies off.

Monday, January 19, 2015, 09:53: 55 / 30 kV

Monday, January 19, 2015, 11:44: ~20 sparks / hour for the last 20 min

Monday, January 19, 2015, 12:53: ~10 sparks / hour for the last 20 min 

Monday, January 19, 2015, 16:30: 50 / 30 kV

Monday, January 19, 2015, 17:31: BIAS and EL power supplies off.

Tuesday, January 20, 2015, 08:54: 40 / 20 kV

Tuesday, January 20, 2015, 09:05: 50 / 25 kV

Tuesday, January 20, 2015, 13:43: 8 sparks / hour for the last 60 min

Tuesday, January 20, 2015, 16:43: 3 sparks / hour for the last 60 min

Wednesday, January 21, 2015, 09:10: 40 / 20 kV

Wednesday, January 21, 2015, 09:20: No sparks

Wednesday, January 21, 2015, 09:23: 50 / 25 kV

Wednesday, January 21, 2015, 10:33: 3 sparks / hour for the last 60 min

Wednesday, January 21, 2015, 13:43: 6 sparks / hour for the last 60 min

Wednesday, January 21, 2015, 16:07: 5 sparks / hour for the last 60 min

Wednesday, January 21, 2015, 16:09: BIAS and EL power supplies off.

Thursday, January 22, 2015, 09:11: 40 / 20 kV

Thursday, January 22, 2015, 10:14: No sparks

Thursday, January 22, 2015, 10:16: 45 / 25 kV

Thursday, January 22, 2015, 11:26: No sparks

Thursday, January 22, 2015, 16:57: No sparks

Thursday, January 22, 2015, 16:57: BIAS and EL power supplies off.

Friday, January 23, 2015, 09:28: 40 / 20 kV

Friday, January 23, 2015, 09:39: No sparks

Friday, January 23, 2015, 09:40: 50 / 25 kV 

Friday, January 23, 2015, 16:16: No sparks.

Friday, January 23, 2015, 16:17: BIAS and EL power supplies off. 

Monday, January 26, 2015, 09:17: 40 / 20 kV

Monday, January 26, 2015, 09:29: No sparks

Monday, January 26, 2015, 09:30: 50 / 25 kV

Monday, January 26, 2015, 09:52: No sparks

Monday, January 26, 2015, 09:53: 55 / 27.5 kV

Monday, January 26, 2015, 16:55: No sparks for 7 hours. BIAS and EL power supplies off. 

Tuesday, January 27, 2015, 09:10: 40 / 20 kV

Tuesday, January 27, 2015, 09:16: 50 / 25 kV  

Tuesday, January 27, 2015, 09:31: 55 / 27.5 kV

Tuesday, January 27, 2015, 09:51: No Sparks

Tuesday, January 27, 2015, 09:53: 57.5 / 30 kV (one spark while increasing BIAS voltage)

Tuesday, January 27, 2015, 10:14: No sparks

Tuesday, January 27, 2015, 10:18: 58 / 30 kV. Some sparks

Tuesday, January 27, 2015, 10:46: 59 / 30 kV. Some sparks while increasing BIAS voltage

Tuesday, January 27, 2015, 11:43: 59.5 / 30 kV. Some sparks while increasing BIAS voltage

Tuesday, January 27, 2015, 13:24: Spark

Tuesday, January 27, 2015, 13:33: 11 sparks for the last 10 min

Tuesday, January 27, 2015, 13:33: 59 / 30 kV

Tuesday, January 27, 2015, 13:35: 57.5 / 30 kV 

Tuesday, January 27, 2015, 16:14: ~3 sparks / hour for the last 60 min

Tuesday, January 27, 2015, 16:53: ~3 sparks / hour for the last 60 min

Tuesday, January 27, 2015, 16:55: BIAS and EL power supplies off. 

Wednesday, January 28, 2015, 09:11: 40 / 20 kV

Wednesday, January 28, 2015, 09:17: 50 / 25 kV 

Wednesday, January 28, 2015, 09:27: 55 / 27.5 kV 

Wednesday, January 28, 2015, 10:35: 11 sparks observed. BIAS and EL off - preparing for power outage

 Found both VFDs with OF3 fault (over voltage during constant speed operation).

Thursday, January 08, 2015, 16:37: Cycled power on the chiller to clear faults.

The copper bars was installed by David and Anders. After the measurements were completed, the copper bars were removed.

See photo and attached spread sheet.

 See e-log entry for ITE for details. The copper bars have not been removed. They can be used for testing the new power supplies.

Local measurement was done by Ray with a voltmeter. The voltmeter was consistent with EPICS readback and local readback on the power supplies. A voltage drop of 0.137 V @ 580 A was measured across the bus bar from the TGHT power supply to the cooled transmission lines.

Chad, Anders and Alex inspected the target this morning. The target had no significant visible damage to the container. Some material was deposited on the Extraction Electrode.

Phots on DocuShare

Monday, March 30, 2015, 08:59: Reset chiller by switching on off power (fault 2)

Monday, March 30, 2015, 09:02: Switched on CHILL:CP2, TS5 at 34 C

Monday, March 30, 2015, 10:24 : Cage locked. EE to GND. BIAS and EL ON.

Troubleshooting BIAS current limiting. Turned out to be a EPICS initialization issue. Resolved by setting/changing the current limit again in EPICS.

Monday, March 30, 2015, 14:00: BIAS/EL 18 / 9 kV

Monday, March 30, 2015, 14:16: BIAS/EL 24 / 12 kV

Monday, March 30, 2015, 16:03: BIAS/EL 30 / 15 kV. Occasional sparking while stepping up since 26 kV

Monday, March 30, 2015, 16:13: BIAS/EL 30.8 / 15 kV. 110% of scheduled 28 kV. BIAS/EL 0.04 / 0.09 mA

Monday, March 30, 2015, 17:26 : Maintained BIAS over 1 hour. One spark after.

Monday, March 30, 2015, 17:27 : BIAS / EL off.

Tuesday, March 31, 2015, 08:12: Cage access. EE to EE PS. Cage locked.

Tuesday, March 31, 2015, 08:24: BIAS/EL/EE at 30.8 / 15 / 1 kV

Tuesday, March 31, 2015, 08:26 : EE to 2 kV, 1.96 mA

Tuesday, March 31, 2015, 08:46: No sparks for 20 min.

Tuesday, March 31, 2015, 08:48 : BIAS/EL/EE at 20 / 9.3 / 1.5 kV

Tuesday, March 31, 2015, 08:57: Started auto ramping of heaters to 580 / 230 A at 2 A/min.

Tuesday, March 31, 2015, 10:54: TBHT at 230 A

Tuesday, March 31, 2015, 14:05: TGHT at 580 A

Tuesday, March 31, 2015, 14:21: BIAS/EL/EE to 30.8 / 15 / 2 kV

Tuesday, March 31, 2015, 17:48: Source stable for 4h

Tuesday, March 31, 2015, 18:20: Lowered IAS/EL/EE to 20 / 9.3 / 1.5 kV. IG2 at 6.2E-6 T

Tuesday, March 31, 2015, 21:27: Starting ramp down at 15/12 A/min. IG2 at 6.8E-6 T

Tuesday, March 31, 2015, 21:51: BIAS/EL/EE to zero and off. IG2 at 6.6E-6 T

Tuesday, March 31, 2015, 22:17: TGHT and TBHT off

Tuesday, March 31, 2015, 22:41: Turbos off.

Tuesday, March 31, 2015, 22:42: Water off.

The following variable have been added to the archiver (as per work request)

TCS:TS5:RDTEMP 
TCS:TCM1:EE:RDTEMP 
TCS:TCM1:COIL1:RDTEMP 
TCS:TCM1:TBHT:RDTEMP 
TCS:TCM1:TGHT:RDTEMP 
TCS:TCM1:MSP:RDTEMP 
TCS:TCM1:HVC1:RDTEMP 
TCS:TCM1:HS:RDTEMP 
TCS:FC:SCALECUR 
TCS:COL:SCALECUR 
TCS:TK2:RDLEVEL 
TCS:TK2:RDLEVEL 
TCS:DIF1:RDFLOW 
TCS:CHILL:RDHRS

The archiver was restarted.

Monday, April 20, 2015, 09:35: Started cooling water circulation

HV cover did not fit due to interference with the air sampling valve. Target Hall will have to be locked out for conditioning to proceed. WP I2015-04-20-2

Monday, April 20, 2015, 10:09: Started chiller staging

Target hall and HV enclosure has been locked out.

Monday, April 20, 2015, 13:45: 22 kV, 0.04 mA BIAS 

Monday, April 20, 2015, 14:58: No sparks observed. BIAS off

Tuesday, April 21, 2015, 08:26:  Starting heater auto-ramp to 500 / 230 A at 2 A/min

Tuesday, April 21, 2015, 08:32: BIAS/EE to 20 / 1.5 kV

Tuesday, April 21, 2015, 12:59: TGHT / TBHT at 327 / 230 A. Pausing the auto-ramping. Bill will be monitoring TCS while I am off-site.

Tuesday, April 21, 2015, 15:56: Continue auto-ramping

Tuesday, April 21, 2015, 17:34: Auto-ramping finished. TGHT / TBHT at 500 / 230 A

Wednesday, April 22, 2015, 08:43: Vacuum tripped heaters and BIAS over night. Switching off to do a leak check on the service cap.

Wednesday, April 22, 2015, 09:03: Stopping turbo and roughing pump for David. David is preparing a leak check.

Wednesday, April 22, 2015, 09:06: Started roughing pump.

Wednesday, April 22, 2015, 09:22: Roughing pump trips off after a few minutes of pumping.

Wednesday, April 22, 2015, 13:11: Stopped water flow.

Wednesday, April 22, 2015, 16:07: Starting turbos 

Wednesday, April 22, 2015, 16:15: All turbos at speed.

David found a leak on the window line this morning.
https://elog.triumf.ca/TIS/RH-ISAC/896

Thursday, April 23, 2015, 11:12 : Starting cooling pack

Good vacuum limit on IG1 changed from 7E-5 T to 1E-5 T

Thursday, April 23, 2015, 12:53 : Starting heater auto-ramp to 500 / 230 A at 2 A/min. Bad vacuum limit reduced to 8E-6 T

Thursday, April 23, 2015, 13:03 : BIAS/EE to 15/ 1.5 kV

Thursday, April 23, 2015, 13:28 : BIAS/EE to 20/ 1.5 kV. No sparking observed

Thursday, April 23, 2015, 17:23: Auto-ramping finished. TGHT / TBHT at 500 / 230 A

Thursday, April 23, 2015, 17:51: Fault on cooling circuit (Fault 2).. Manually starting CP1

Thursday, April 23, 2015, 18:01: Cycled power of chiller unit. System recovered. Enabled staging.

Thursday, April 23, 2015, 18:42 : One minor spark observed. Starting cool down at 12 and 9 A/min

Thursday, April 23, 2015, 19:51 : Heaters and BIAS off

Friday, April 24, 2015, 08:21 : Water cooling off

Changed by Dave Morris. 

New value is 1E-5 T (old value was 7.09E-5 T)

The new TBHT and TGHT power supplies were brough up to high current; 350 A and 420 A respectively. The TGHT power supply was limited by PLC to 420 A. The resistance calculated for both the TBHT and TGHT copper lines from the electrical services room to the ITW pit was 1.35 mOhm. See attached spreadsheet for details.

Dave Morris has been contacted to correct the current limit for the TGHT supply to 600 A.

Caled cyclotron ops to open P23 and confirmed penthouse valve open.

Filled up TK2 to 21 in water leve.

 The communications chip has been replaced. The electronics box has been grounded (see attached pdf).