David raised the bias to 49.5kV. Module is still sparking mainly on top. Humidity is super high.

Wednesday, July 10, 2019, 09:50 : sitting at 49.7kV for a coupe mins, big spark in vacuum took everything down.

Wednesday, July 10, 2019, 10:16 : Bias at 50kV drawing 0.1mA but looking pretty jumpy. Left it here for ~5 mins then had to turn down bias for TH work.

Wednesday, July 10, 2019, 13:33 : Back at 50kV on bias with no problems.

Wednesday, July 10, 2019, 13:46 : 10 mins at 50kV, will try to go a bit higher

Wednesday, July 10, 2019, 14:06 : Everything sparked off while sitting at 50.6kV. This seems a bit futile....

Wednesday, July 10, 2019, 14:15 : Back at 50.1kV with no problems

 Wednesday, July 10, 2019, 14:33 : Up to 51kV

Wednesday, July 10, 2019, 15:20 : Difficult to go above 51.4. Turning up EZL since there were a couple sparks to it. -> Putting this up causes significant sparks between HV bias and EZL in vacuum at 5kV on the EZL, 51.1kV on bias. Leaving EZL @ 5kV

Wednesday, July 10, 2019, 16:04 : Around 52.2 sparking on the top of the module

Wednesday, July 10, 2019, 16:36 : 53kV

Wednesday, July 10, 2019, 17:16 : up to 54kV without much trouble

Wednesday, July 10, 2019, 17:53 : Sitting at 55kV for about 4mins when a spark tripped off everything. Will try to bring it back to sit stable at 55kV tomorrow

Thursday, July 11, 2019, 10:28 : TM4 up to 54.9kV without too many problems, EZL at 25kV. Will sit here for a little while

Thursday, July 11, 2019, 10:48 : 20mins at 54.9kV, going to 55kV

Thursday, July 11, 2019, 11:14 : 25 mins at 55kV and everything sparked off

Thursday, July 11, 2019, 11:26 : Back up to BIAS / EZL = 54.8kV / 26kV. Checked that the TGHT and TBHT power supplies have not been killed by sparks, everything looks fine.

Thursday, July 11, 2019, 12:56 : Sat at 49.6kV for 1.5hrs with one spark.

Thursday, July 11, 2019, 14:02 : Been sitting at 55kV for 2 hrs now with one spark.

Friday, July 12, 2019, 08:09 : Back up to 52kV easily, then had to reduce step size to 100V to keep going

Friday, July 12, 2019, 08:47 : 56kV

everything sparked off at 56.5 or so

Left TM4 at 40kV overnight with no previous heating. There was only one real spark which turned off TP2 but ops turned it back on. Ramping up again.

Tuesday, July 09, 2019, 09:13 : At 45kV the module is sparking on the top pretty regularly. Turned down and David will remove cover and lock out target hall.

Tuesday, July 09, 2019, 16:30 : Went up to 45kV pretty easily now.

Tuesday, July 09, 2019, 16:32 : Everything tripped at 46.5kV. Looks to be a spark outside the vacuum. TM4 sparking much more outside the vacuum than last checks done without the source tray...

Tuesday, July 09, 2019, 16:43 : Everything easy up to 46.8kV, then sparking a bit. Most of the sparks are outside the vacuum. Sitting at BIAS / EZL = 47.1kV / 28kV for a couple mins before ramping down.

Tuesday, July 09, 2019, 16:57 : 10 mins at 47.1kV with not problems. Continuing up still looks like it is really sparking a lot outside the vacuum.

Ramping up TM4 with new source tray in. Current condition: new source tray but no water running in lines B, D or window. Multipin on the source tray grounded but no lines running into the service tray. Vacuum at 1e-6T.

Small sparks at each BIAS voltage increase starting at 30kV. First real spark around 38kV.

Going up in more than 100V steps causes spikes that take out TP2.

Monday, July 08, 2019, 14:54 : TP2 sparked off at BIAS/EZL = 44kV/25.1kV after 10mins at this point... will leave it here for a couple hours

Monday, July 08, 2019, 16:44 : 6 sparks in last 2hrs, some very strange vacuum response in which the pressure rose steadily and stayed high for 30mins before coming back to normal.

Monday, July 08, 2019, 17:25 : After 30mins at 45kV, started sparking, so will ramp down to 40kV and leave it overnight

 July 3:

New source tray was installed on TM4 yesterday. LD was used to pump down all water lines. Tube heater +D circuit has a big leak  around water block area. At base LR5.5 XE-5 atm.cc/sec, base pressure 170 mtorr, When Chad  moved this helium probe close to  water block area, The leak rate on LD went to maximum level right away. When the probe was moved out of water block area, the LR went back to base level. The test repeated again with same result. Also, Before helium spray test, I applied 80 psi air into the circuit. The air pressure can not be held in the circuit. The pressure drop on the line is quick. This shows a big leak on  D line as well. The rest water lines are fine. I pumped all of them down to LD bottom scale without problem. Chad and I helium sprayed window and MSP water blocks. No leak has been found.  The only line we have problem is the D line.

D line is still leak after repair and installation. We suspected the c-seals issue on this module. So, all water lines have been sprayed with helium for leak check.  Found "B" line target oven negative  is also leak. The rest line are good on helium spray leak check.  The further investigate on thi issue is taken place now.

The AC wiring for the SMP motor was inspected by Randy Boehm (TRIUMF electrician) as recommended by Franco Mammarella (TRIUMF electrical engineer).  After inspecting the system Randy required the following changes:

1. AC wiring inside the control panel be increased to 16awg minimum (14awg preferred)
2. Wiring between the control panel and motor control box be increased to 14awg
3. Power cable to the motor be increased to 14awg
4. Current rating for MS connectors be checked and replaced if below 10A
5. Prefer that motor control box components be relocated to the control panel to reduce number of AC wires and connections (preferred but not mandatory)
6. A dedicated ground wire is required in the cable going to the motor (if not already in place)

After these changes have been made the inspection will be repeated.

July 31, 2019 update:

Items 1, 2, 3, and 6 from above were completed.  14awg wiring was used for wiring inside the control panel (Item 1).  Current rating for the MS connectors was checked and is over 10A (Item 4).  Motor control box components were not relocated to the control panel at this time (Item 5).

Randy repeated his inspection today, and approved of the updated 120V wiring.

Adam updated the SMP electrical and control schematics (IRH1868-1874)

ITE is configured to take proton beam.

I brought up TCS bias to 57.5 KV on TM4 without source tray at TCS. It was kept at this level about 1.0 hours. Some intermediate sparks. No spark trip on devices. 57.5KV is very close to the HV up limit of this module.

 TM#4 has been moved from the conditioning station to the south hot cell. The module has no source tray.

The conditioning station was ramped down overnight and David ramped it back up this morning. Currently trying to get it to sit stable at something higher than 55kV.

Wednesday, June 12, 2019, 08:11 : Sitting at BIAS / EL = 53.9 / 28, 2 sparks to ground, one turned off TP2

Wednesday, June 12, 2019, 08:53 : Been at BIAS / EL = 56 / 28 for >10mins with no real sparks. Will continue up.

Wednesday, June 12, 2019, 08:54 : Spark to EL at 56.1

Wednesday, June 12, 2019, 09:34: Sitting at 55kV for 40 mins. 2 sparks to ground, 1 spark to the EL.

Wednesday, June 12, 2019, 09:49 : Leaving it at 56.6 for a while

Wednesday, June 12, 2019, 09:56 : Either there was a spark to IG1S or there is a vacuum limit that does not accept a 1.2e-6T spike. Everything turned off, but everything seems to still be working.

Monday, June 10, 2019, 16:59 : TM4 back up to BIAS / EL = 56.1kV / 29.5kV

Monday, June 10, 2019, 17:14 : After 57kV on the bias, things seem to be getting sketchy. The EL voltage and current are a bit jumpy and stepping that voltage up seems to produce a slight spike in the bias current.

Monday, June 10, 2019, 17:31 : At BIAS / EL = 58 / 30.3, looks ok.

Monday, June 10, 2019, 18:11 : Sitting at 59.5kV, could probably push further but will sit here for ~30mins and make sure it is relatively stable. Note 59.5kV set point is really 56.9kV on readout.

Monday, June 10, 2019, 18:18 : some current inconsistencies (not really sparks probably corona discharges...) so went down to 59kV

Monday, June 10, 2019, 18:23 : still getting some sparks at 58kV, going down to 57.5.

Monday, June 10, 2019, 18:37 : went back up to 58, spent 10 mins there but was getting these corona-discharge-type-sparks where the vacuum responds and current is drawn but voltage is maintained. Went down to 57kV, still getting them....

Monday, June 10, 2019, 18:40 : Going down to 56 to see if they stop....

Monday, June 10, 2019, 18:42 : Trying 55 kV now... see attached screenshot for example of sparks

Monday, June 10, 2019, 18:48 : No luck at 55 or 54kV. Going down to BIAS / EL = 50 / 27 for a while to see if this resolves

Monday, June 10, 2019, 19:20 : 30 mins at 50kV, a couple little sparks, but no big deal. Ramping up to 55 and let's see if it is ok.

Monday, June 10, 2019, 19:30 : 55 still not working, going down to 54

Monday, June 10, 2019, 20:16 : Held bias at 54kV for about an hour - dropped it down once to alleviate some sparking. Leaving it at 50kV overnight, ops have instructions to keep an eye on it and shut it off if anything seems strange.

Ramped down the bias so David could remove the HV cover and lock out the target hall.

Ramped back up to BIAS / EL = 50kV / 26kV quickly with no sparks.

Monday, June 10, 2019, 15:35 : BIAS / EL = 52kV / 26kV

Monday, June 10, 2019, 15:45 : BIAS / EL = 54kV / 28.5kV. It seems that keeping the EL at 1/2 the bias voltage is right on the edge of what is acceptable - the HV seems to be sparking to the EL instead of to ground (either that or the EM interference from the sparks it producing a reaction in the EL...) I will run the EL higher and see if that helps.

Monday, June 10, 2019, 15:57 : Sat at 54kV for > 10mins, will continue up

Monday, June 10, 2019, 16:11 : At BIAS / EL = 55.8kV / 29kV got some sparks, some to EL and some to ground. Back at 55kV.

Monday, June 10, 2019, 16:24 : Sitting at 56kV for a couple mins

Monday, June 10, 2019, 16:37 : Sitting at 56.3kV bias, a spark tripped off the IG1S, which tripped the turbo and shut down all the HV. Back to zero!

 The old TM#4 source tray in the south hot cell was moved to the spent target vault. The tray had a low of 7.5mSv/hr to a high of 22.1mSv/hr though a 390 degree rotation.

TM4 in conditioning station with no source tray, extraction electrode set to HV bias. Brought up to 30kV over the weekend by ISAC ops with no issues.

BIAS=30kV

EL=15kV

IG1=9.4e-7T

Monday, June 10, 2019, 08:21 : At 32.4kV on bias, EL at 15kV, bias sparked to EL. Every bump of 100V on bias is accompanied by a small vacuum jump and often by a small current increase from ~60uA to 120uA.

* Conditioning station read back and set values are off by 2kV but archive values show this has always been the case. Which to trust? *

Monday, June 10, 2019, 08:54 : Spark while sitting at 38kV on bias, 19kV on EL. Seems to be a spark to EL.  Sitting at 38kV for ~10mins

Monday, June 10, 2019, 09:32 : Sat at 40kV for ~10mins or so, bumping up again.

Monday, June 10, 2019, 09:38 : Spark to EL while sitting at BIAS / EL = 42kV / 20.5kV

Monday, June 10, 2019, 13:57 : Ramped to BIAS / EL = 43 / 21.2. No problems other than what seems to be the usual conditioning station stuff.

 Monday, June 10, 2019, 14:17  : Couple sparks on the way to 48kV, seems to be to the EL.

Monday, June 10, 2019, 14:36 : Up to BIAS/EL = 51/25.5 with no big problems. Will stay here for a couple mins

TM4 has been connected at TCS for HV test. TCS water system and vacuum pumps are switched on.

 TM#4 has been moved from the south hot cell to the conditioning station.

 Results from the electrical check of TM#4 at the south hot cell with no source tray. R&U are shorted and when meggered A&P have a value of 193k ohms. See attached for details.

ITE water signal interlock check is done. Everything is good. Coil 2   circuit no longer used in ITE . PQRU are looped as one circuit fro coil 1 circuit on Module since now.

 TM#4 has been moved from the south west silo to the south hot cell.

 A 55 Gal. drum was remotely lowered into the south hot cell. It was then filled with compressable waste. Then the drum was remotely removed from the south hot cell. The drum measured about 20microSv/hr upon removal from the south hot cell.