Tuesday, June 08, 2021, 08:25: IV4 open. Optics set minus TPS:CB2. Shows as green 'on', set to 500V, however read back shows as 0 V.

Wednesday, June 09, 2021, 08:37: HV off. Attempts made to restart TPS:CB2 unsuccessful. Controls notified.

Wednesday, June 09, 2021, 11:49: Card replaced by controls. Verified "Ok"  in " LB Mode": RDVOL, RDCUR, RDDAC all similar. Operational again.

Monday, June 28, 2021, 09:22: Roughing.

Monday, June 28, 2021, 10:10: TP1B on.

Monday, June 28, 2021, 10:22: TP1A on (Tripped)

Monday, June 28, 2021, 10:30: Both turbo pumps off. Leave it under a rough vacuum.

Fernando

Wednesday, June 30, 2021, 15:09 : Venting

Wednesday, June 30, 2021, 19:08: Roughing

Thursday, July 01, 2021, 08:57: Turbos on 

Thursday, July 01, 2021, 09:11: TP1A timed out. Reset it. Waiting for CG1A to drop a bit more. 

Thursday, July 01, 2021, 09:19: TSG:BV1A interlock bypassed so that can be opened. TSGTP1A open

TP1A at speed

Thursday, July 01, 2021, 09:20: Interlock bypassed was removed.

Thursday, July 01, 2021, 09:20: IG1 on. 

Wednesday, July 07, 2021, 08:52 : IgG1 @ 4.64e-7 T.

Wednesday, July 07, 2021, 08:53: Waterlines open. No vacuum response. RGA does not respond, probably the filament burnt out.

Wednesday, July 07, 2021: Gas system has a tiny leak, probably from my initial checks about two weeks ago

Wednesday, July 07, 2021: During adjustments on the gas system the copper pipe yielded a bit and the leak clearly increases. I asked Maico to prepare new VCR connection for replacement (thanks!)

Thursday, July 08, 2021: The gas system inside the Faraday cage is not leaking anymore. However, the gauge inside the service cap does not respond to any pressure changes. It needs replacement.

Friday, July 09, 2021: Gas system troubleshooted with a small nitrogen bottle. Problem was on the two-stage regulator. Tighter connection fixed it.

Power supplies respond normal

Friday, July 09, 2021, 15:19: IV4 open.

Emittance meter power supply on.

Friday, July 09, 2021, 15:22: IG1 @ 4.40e-7 T.

Friday, July 09, 2021, 15:22: Auto-ramping TGHT/TBHT to 300 A / 150A @ 2A/min

Friday, July 09, 2021, 16:06: Emittance meter was not workin. Mike Rowen rebooted the IOC and that fixed the issue.

 Sunday, July 11, 2021, 16:40: IG1 @ 5.13e-7 T.

Sunday, July 11, 2021, 16:42:  Auto-ramping TGHT/TBHT to 550A / 250A @ 2A/min

Sunday, July 11, 2021, 16:43. Anode @ 100 V and  500 mA current limit. 

Monday, July 12, 2021, 09:15: BIAS / EE / EZL @ 30 / -1.5 / 13 kV. - AS

FM

Monday, July 12, 2021, 09:41:  Auto-ramping TGHT/TBHT to 580A / 280A @ 2A/min

There seems to be an issue with the emittance meter. The scans seems to be successful but the plot is only noise.

Tuesday, July 13, 2021, 11:51: Telescope set on the magnet viewport: Grid is fine and I will check it continuosly.

The gauge inside the service cap is working now, as it actually never broke, a manual valve was still close inside the faraday cage.

I measured the voltage sent from the emittance PS, so the emittance meter receives voltage. I think the problem might be the faraday cup, either internally, or with the signal.

Tuesday, July 13, 2021, 15:54: Venting the beamline after the magnet to re-install the mass slit.

HV off

Tuesday, July 13, 2021, 16:19: TMS:BV6 open.

Tuesday, July 13, 2021, 16:21: TMS:BV2 open.

Tuesday, July 13, 2021, 16:27: TMSTP6 and TMSTP2 on.

Tuesday, July 13, 2021, 16:32: TMSIG6 On.

Tuesday, July 13, 2021, 16:40: HV on.

 Wednesday, July 14, 2021: Troubleshooting the emittance meter. I think I saw some indications of beam so it might be that I just need more intensity for a proper measurement.

Moving on, I tested the code as I want to implement a a couple of things: record a bidimensional anode current to make sure I capture any chance with the coil (if any), use temperature estimation from heating power as well.

When running a magnet-anode scan on sodium I triggered the high efficienct regime. This indicates that the cathode is either hotter or closer. With 280 A I am inclined on the latter.

The regime is unstable as the voltage fluctates as well as the ion current. Over a period of two hours

Friday, July 16, 2021, 12:08: Hv off to increase upstream pressure from |200| to |240| kPa, which is the maximum I can achieve with my two-stage regulator.

Friday, July 16, 2021, 12:15: Mass scan.

Friday, July 16, 2021, 13:54: TBHT to 260 @ -2A/min.

 Friday, July 16, 2021, 14:09: HV off.

Friday, July 16, 2021, 14:17: BIAS / EE / EZL @ 30 / -1.5 / 12.5 kV.

Friday, July 16, 2021, 14:17: Mass scan ongoing. Done for the day (on-site)

Saturday, July 17, 2021: Mass scans in both normal and anomalous regime.

Sunday, July 18, 2021, 07:42: Found TSG:CBC in a red state. Using the cfg button on the canbus utilities does not work. Likely the canbus card needs to be replaced.

Monday, July 19, 2021: AS fixed the CBC

Increased anode current limit to 600 mA. More ion current.

Anode voltage setpoint=500 A, anode voltage readback=250 V.

Moving the setpoint to 250 causes no effect. But when moving the setpoint to 245 V, the ion current, and the drain current decrease.

Putting together previous findings to discuss what to do next.

Grid still fine.

Tuesday, July 20, 2021, 17:22:   TBHT to 240 @ -2A/min.

Around 16:30 there was a slight target voltage drop. This coincided with the anode getting current limited (500mA) at 100V.

 Tuesday, July 20, 2021, 17:47:  IG1 reacts to the "sparks" observed on the anode leackage current.

Lowering the anode voltage to 0 V. I'll go on-site tomorrow to check the state of the grid and to check if there are visible sparks inside the chamber.

Wednesday, July 21, 2021, 12:40: entries from early today lost I'll try to summarize what has happened:

grid still fine

Manually sweeping the anode but no visual of any kind inside the vacuum. IG1 still reacts to sudden increase of leakage current but nothing observed inside the vacuum chamber.

 anode leakage current responds to decreasing the target heating current when lowering to 235A.

When going to 200 A, the anode leakage current drop to 0 mA when the cathode reached 211.

Now the cathode gets current limited as of 1 V.

grid still fine.

Wednesday, July 21, 2021, 12:44: TGHT/TBHT to 0 A @ -2A/min 

 Wednesday, July 21, 2021, 14:43: TBHT @ 0 A, TGHT @ 340 A. Setting the anode voltage to 500 V has a leakage current of 102.5 mA.

Wednesday, July 21, 2021, 22:19: IV4 closed

 Wednesday, July 21, 2021, 22:21:  BIAS / EE / EZL @ 0.000 kV.

Wednesday, July 21, 2021, 22:22:  IG1 off. Turbos off.

Thursday, July 22, 2021, 10:44: Venting:

TP1A tripped off on Fernando, seems the backing pressure got too high. Even with PV1 closed the RP1 pump could not get below 200mTorr. Derek from vacuum group came and had a look, and is swapping RP1 out for one of the vacuum group pumps, then will see if they can repair ours.

After not being able to get beam through the emittance meter. I diagnosed that the plates was not biased (reason yet unknow).

I swapped the cables from the plates such that it looks like the picture below. With this, I was able to take an emittance scan with the previous parameters.

Varying the optics affect the shape and position of the emittance so I believe this is a real signal.

Keeping this configuration until I finish my measurement.

Problem: Some devices turn off unexpectedly and sometimes without any obvious trigger. In the past, the only time this was an issue was during mass scans. Now with the automation of measurements, this is a major problem as it prevents longer parameter scans.

Thursday, August 05, 2021, 08:00: Kevin setup the CANBUS snooper by hooking it up on the magnet card loop port.

This device tracks the network so if any error happen it will be captured.

Thursday, August 05, 2021, 09:39: Test unsuccessful. No device turned off (I should be happy XD)

Thursday, August 05, 2021, 09:40: Will run the test another time

A PC was setup near the old magnet power supply (20210806_150334.jpg ). One of the software run smoothly but the other more useful one fails to connect to the usb adapter.

 Friday, August 06, 2021, 14:52: Fixed issue with  the usb adapter. With this, the CANBUS traffic can be store and inspected to catch any error on the communication (Annotation_2021-08-06_144737.png).

Friday, August 06, 2021, 14:53: Loggers are running. 

 Friday, August 13, 2021, 11:07: I forgot to input the feedback  from Kevin:

"33 errors out of 10,300,00 transaction seems statistically insignificant to me. "

From the devices tripped,

"Most of the tripped supplies are located on the front side of the rack, the original test stand controllers"

So maybe the issues could be

"Compatibility issues with the old and new? The newer supplies are all daisy chained, should we upgrade these to the tap boards?

Should the old controllers (original test stand appear to be MK supplies) be upgraded (do we have enough)?"

Saturday, August 07, 2021, 18:02: As mentions in previous entries, the FC0 current oscillates with no change on ion source parameters.

Hence, I came on-site to see if there was an issue with the hardware. Nothing obvious.

Last night there were some sparks that might have cause some issue.

RPMS are shifted toward negative current reading, similar to what we observed when testing TM4 sourcetray.

To rule out hardware issues I swapped the signals from FC0 to FC6 and visceversa.

When the output of FC0 goes to the FC6 controller the readout makes sense, and this behaves as expected with gas on/off.

When the output of FC6 goes to the FC0 controller there is no readout coming out. This suggest, at least to me, that some damage or error is happening on the DAQ side.

I will reboot the ioc expecting to see something.

Things sort kind of Ok but I don't trust it. FC0 reading makes a bit more sense. RPMS remains with an offset

Saturday, August 07, 2021, 20:31: The FC0 reading increase from 7.54e-11 A to 1.64e-8 A starting at 19:32. Also, the RPMS baseleves are not offset anymore, the -10 rd when using the 1 nA scale is back to 0 (normal)

 Monday, August 23, 2021, 09:53: RP1 was swapped back to our original pump. (see e-log 889)

Derek informed that they test it overnight and performed well, and it's a bit slow but it goes down to 20 mT.

Current is at 36 mTorr and dropping.

Monday, August 23, 2021, 10:49: Aaron removed the Extorr RGA and install a blind instead.

When the new RGA arrives it will be changed.

Wednesday, August 25, 2021, 15:29: Roughing.

Thursday, August 26, 2021, 04:11: IG1 switched on. Vacuum @ 8.35e-06T - AS 

Thursday, August 26, 2021, 14:04 Water on, no spike in vacuum. IG1:2.6e-6T

Thursday, August 26, 2021, 14:37 HV rack closed, BIAS EE and EZL power supplies on.

Thursday, August 26, 2021, 15:03 BIAS/EE/EZL 10000/-1000/9000

Thursday, August 26, 2021, 15:08 BIAS/EE/EZL 30000/-1500/13000 1 spark just when reached 30kV.

Thursday, August 26, 2021, 15:28 Series of sparks restarted the controling PC. Back in control BIAS/EE/EZL 48000/-2500/22000

Thursday, August 26, 2021, 15:34 BIAS/EE/EZL 50000/-2500/22000

Thursday, August 26, 2021, 15:48 BIAS/EE/EZL 55000/-2800/24000

Thursday, August 26, 2021, 15:53 Ramping down.

Monday, August 30, 2021, 10:25 turbos off.

Pump down chamber w/ empty HS

Feedthrough R had a flange installed with a HV feedthrough on it for TM3 service tray prototype HV tests. It was replaced today with the original blank flange.

Thursday, September 23, 2021, 07:15: Roughing. Waterline loop installed.

Monday, September 27, 2021, 10:35: MB1 off. Turbo pumps, IG, RP, all off/closed. DI water closed/purged.

TS shutdown for planned power outage.

Friday, October 01, 2021, 06:03: Roughing pumps on.

Friday, October 01, 2021, 06:04: Roughing TS beam line side.

Friday, October 01, 2021, 06:52: TMS:TP6 & TMS:TP2 on. Venting Chamber side.

Friday, October 01, 2021, 06:57: IG6 on - Filament 2 manually set. Beam line side vacuum @ 1.5e-05T

Friday, October 01, 2021, 07:01: Vacuum @ 9.04e-06T & dropping steady (Beam line side) OK.

Friday, October 01, 2021, 07:05: Reboot ISAC TS PLC/IOC. Reset controls whiteout.

Friday, October 01, 2021, 07:10: MB1 restart ok.

Wednesday, October 06, 2021, 11:56: Roughing.

Thursday, October 14, 2021, 09:07: Venting.

Wednesday, October 20, 2021, 15:01: Roughing.

Thursday, November 04, 2021, 11:54: Venting

water loop removed

Thursday, November 04, 2021, 13:39: Roughing

Tuesday, November 16, 2021, 13:44: Roughing. Water loop installed. Lock out tag removed.

 December 30th, 2021: Water poured down around Isac test stand and probes lab area in service annex extension. problem was traced to a Cracked HVAC coil on SA roof

Fault: 14601 

https://web.accel.triumf.ca/ncr/dbfault.pl?faultno=14601&usertype=unknown&username=&faultgroup=CYCLOTRON&debug=0&btn_submit=Showfault

https://elog.triumf.ca/TIS/Test-Stand/908

Noted on arrival to startup the ISAC TS some cleaning of the area was required. Optics PS & all PS quickly checked. OK / Dry. 

Use caution when HV is to be applied later. Expect conditioning / noise from the cage. 

Connection of TBHT negative to HV common removed (R4 in circuit diagram). Labelled and put in MRO bucket.

Anode connection moved to feedthrough J.

Rewired multipin patch panel. Left some grounded (HVC) wires hanging, should probably make this nice later.

Wednesday, January 12, 2022, 16:08 : HV ramped to 30kV, EZL=15kV, EE=-1.5kV

Ramping up to 250A on TGHT, 200A on TBHT. See MassMarkerTest#1 for ramp up values of the thermocouples.

Seeing some beam on FC0, changing magnet settings to Cs to check FC6 then will ramp up.

Thursday, January 13, 2022, 14:22 :  1.3e-10 approximately Cs on FC6. Optimal EE -2700V, slightly higher than standard on EZL.

 Ramping down to Na on the magnet, found 5 pA:

 Started a ramp up to nominal heater values (TGHT-500 A  TBHT-230 A) but too much HV sparking so aborted at TGHT-299 A and TBHT-230 A. Ramped HV down to 16kV to get sparking to slow down.

Ramped quickly back up to 30kV to get FC readings but since there is so much sparking, we decided to ramp everything down and open the vacuum chamber.

PLAN:

1. ramp down heaters, vent vacuum, open chamber.

2. Try to tape the thermocouples up so that they are further from ground.

3. Try to crimp the bottom of the mass marker tubes, and bend them upwards so they aren't facing ground.

4. Add a dedicated ground to the COIL power supply.

From Fernando:

---

Monday, January 17, 2022, 09:52: IG1 on. Source PS off. HV Cage removed. - AS
Monday, January 17, 2022, 11:11: Cage on. TGHT/TBHT PS on. - AS

Monday, January 17, 2022, 11:23: PS/ Sources on. - AS

Monday, January 17, 2022, 14:00: HV ramped to 20kV, EZL 10kV, EE, -1500V. Ramping TGHT/TBHT to previous values of 250A/200A.

Monday, January 17, 2022, 16:51 : Back to previous heater values, didn't find any Cs on FC6 so went to Na. Tuned and found the tune the opposite of last time, all voltage wanted to go lower. Found 736pA, maybe better than last time because I tuned quite a bit.

Monday, January 17, 2022, 18:00: Ramping to TGHT 300A, TBHT 230A, where we had sparking problems last time.

Monday, January 17, 2022, 18:17 : So far no sparking issues so I will start to heat up the mass markers. One spark about 4 mins after ramping up, after that maybe 10 sparks/40mins or so. Too many....

Monday, January 17, 2022, 19:20 : Got to 20A on the mass markers, will ramp down now.

Friday, January 28, 2022, 18:49 : Yesterday opened the HV rack to re-start data logging on the thermocouples. This morning ramped everything up and started the heaters. Found the Na current around 250A on the target, but abandoned looking for it because the TGHT heating was causing lots of outgassing, which was causing bias sparks. In addition, there is a weird effect where the Na current spikes, then slowly decreases over the course of an hour or something, to values lower than before the heating. Could this be an indication of some thermal issues, like cold spots where things condense? Or something is drifting. The HV common point on the target is at the mass markers now instead of at the exit of the ion source, but I can't imagine you would see such a large difference. The thermocouples do not change in value during these long slow declines. In light of this, decided to just go up and see what the thermocouples read. Settings were close to what is shown above - bit lower on the HV but magnet the same.

Monday, January 31, 2022, 16:23: Power/sources off. Thermocouple reader off. HV rack open. -AS

Wednesday, February 02, 2022, 07:36: IG1 & Turbo pumps off.  - AS