ABCD is short to 60KV bias. Othjer are ok.  see attachment.

Found afterwards that this short was due to sparking through the multipin wire insulation where the wires pass behind the TBHT block "C". See fault 17265.

Tm3 was moved from ITE to TCS this Tuesday, then to SHC  this morning. Both moves are successful.

 TM3 with spent target SiC#46 was disconnected in ITE and ready to move to TCS soon.

 5 shielding blocks have been transferred from ITE to ITW. MAA is secured again after moving.

 The UC#45 target was moved from the south hot cell to the spent target vault. It was place in Pail#292 spot 1A of the vault, was 19.5mSv/hr. Also this pail contains the target material of UC#44. See attached PDF for more details.

A safety walkaround for July 2024 was completed for the B2 level by A. Newsome. No deficiencies to report.

Results can be found in the master spreadsheet

 TM4 with TiC#7 was connected in ITW, all TPs are running at full speed.

TM4 has been moved from TCS to ITW.  the move is smooth.

TM4 has been moved from TCS to ITW.  the move is smooth.

 SS washers were added to all slotted holes on SMP chainguard cover plate

 New spring idlers were installed onto rotational chain drive for SHC roof. Operation ran smoothly in both directions to full limit. Chain was noted to have adequate slack when not on driven side. Periodic checks to confirm operation will continue in the following months to guarantee proper operation.

An annual electrical and mechanical inspection was performed on the SMP today by Adam Newsome, Travis Cave, Riley Sykes, Maico Dalla Valle, and Jason Zhang as per Work Permit  I2024-07-10-1. The following items were checked:

• Check condition of wiring for physical/radiation/UV damage
  • OK, no signs of significant damage
• Check for cable tray debris or damage
  • There were multiple items in the cable tray which do not belong there - misc. pieces of wood and metal... these items were not disposed of but should be in the near future
• Check inside control panel: components and wiring, labeling
  • Labeling intact, components in good condition
  • Misc. random wires pulled to check... OK
• Verify camera views
  • This was not inspected in detail as it is something that would be done prior to remote operation
• Check connectors for damage
  • All connectors OK
• Check pendant for damage and verify labeling intact
  • Pendant OK
• Lid open/close test
  • Lid was opened and close multiple times - smooth operation, no concern
  • Lid close logic was checked - OK
• Vessel rotation and limit switch check
  • Both CW and CCW limits were checked, functioning
  • No concern on rotation - chain drive, rotary limit switch, and cable reel all functioning normally
• Lubricate chains
  • Not done - should be done next year
• Inspect tensioners
  • Done - OK
• Tighten fasteners
  • Not all fasteners were checked - washers will be replaced soon so this was not deemed necessary
  • Approximately 50 washers are missing for socket head screws in slotted parts. Not safety critical but they should be replaced soon.
• Visual inspection
  • All OK

Overall the inspection was successful.

ITE:EL has been tripping off due to HV sparks. Usually this also trips ITE:Q1/2. These are sitting in side-by-side racks.

The working theory is that HV sparks are returning to ground through the EL and quads, tripping them off. Possibly this is the result of a changed low impedance ground path. How this change happened could be either something we did during shutdown or the changes we made in TM3, no clear evidence yet.

The goal of these changes is to stop the EL and quads from tripping, assuming the sparks are travelling through the ground.

Possible routes through ground to the EL power supply:

1. Through controls. EL is controlled by a canbus card. Preliminary investigations by Mike Rowe make it look like the canbus reads the PSU max voltage (60kV), then the staton bit trips, then after a while of trying to talk to the power supply, the statdrv is turned off. It is not clear why the power supply is turning itself off, since it doesn't seem to have a trip or protection circuit inside. To be further investigated.
2. Through the AC ground.
3. Through the shield on the output voltage conductor that runs to the pit.
4. Through the EL wire and conductor itself.

Current attempts to mitigate the tripping:

• Add an isolation step to the AC power, in case it is coming through AC ground (option #2).
• Add ferrites on the controls lines (option #1, this was done earlier and no effect has been seen).
• Add transient voltage suppresors on the Vmon, Imon, Vprog and Iprog control cables (option #1).
• Replace grounding cable with a grounding strip where the grounding wire used to be on ITE:EL. The ground wire was a braid, so probably not too bad, so not sure if this will help.
• Improve the connection between the two grounding bars in the racks. The grounding bar for th EL supply is well connected to the copper ground strip from ITE, but only connected through a small cable to the grounding bar for the quads. A copper foil sheet was added between them and the previously used wire was disconnected on the quad side bar..

TM4 has been moved from SHC to TCS. The move is successfully.

Today (2024-07-05), five flasks were loaded in preparation for shipment with the following targets: SiC#43 (Pail 272), SiC#44 (Pail 275), Ta#63 (Pail 266), UC#40 (Pail 278), UC#38 (Pail 274).

The crimped lifting loop has been applied to each pail. The locks have not yet been applied to the flasks at the time of writing this log, but will be done next week.

The mini-storage vault is now EMPTY. Because some of the pails had contamination (200 - 1500 CPM), it is highly recommended that the mini-storage vault be cleaned or at least swiped for contamination.

The target index spreadsheet has been updated (attached).

TM4 and TiC#7 target electrical check and HS leak check at SHC after target installation have been done.  all results are good. I will attach the checklist when printer is back to work.

TM4 HS line was helium pressure tested at TCS this morning. No leak. Also i leak checked TM4 top all air to  vacuum seals at TCS. Everything is good.

TiC #7 is a febiad target.TM4 top has been configured for febiad target. checked pin 2-2 to bias, target ,and chassis at 250V. they are all OL.  

electrical check  on TM4 UCx#45 target at SHC after operation is done.   see attachment.

TM4 with spent target UCx#45 was disconnected in ITW and moved to SHC.

 TM4 move from ITW to SHC is smooth.

Initial Summary:

TM3 sparking behaviour in ITE has been very disruptive both with and without protons. Protons are at 60uA, target heater at 540A. Max voltage achieved so far is 51kV, limited by time and a current limit on the BIAS supply.

The ITE:EL trips off every couple of minutes. Just needs to be restarted. Mike investigated and found that the trip is happening inside the power supply, then a moment later the canbus is defining the supply off when it cant get it to respond. Trip frequency decreased over time but still disruptive.

All the waterflow signals, including ones controlled in the cyclotron system, were tripping off regularly. These have all been forced, then we implemented a 5s delay in the ISAC waterflow interlocks. This seems to have solved the issue.

The ITE:BIAS supply has also been limited at 500uA instead of 5mA. Probably something hardcoded into the canbus - this was fixed by Keiko and the hardcode limit was set to 20%(?) of 5mA.

TM3 has been very sparky, even at low voltages. I can't understand why. A new ground path would be explicable but the excessive sparking is difficult to understand.

Possible avenues:

1. make sure the module outer surface is connected through a copper sheet to the sheet on the wall.
2. check that special ground still grounds all the gnd power supplies in the electrical room - this I checked but some of the grounds are connected through wires instead of sheets, this can be improved.
3. try running the module with only HV hooked up and nothing else.
4. try running TM3 in ITW.
5. Search work permits and shutdown jobs for any changes made to the system that could affect this.
6. hope there is a software solution for the EL problem, or start moving towards PLC control - no software solution because the canbus card is only reacting to the power supply refusing to turn on, so it seems the issue originates in the PSU. Moving to PLC is possible but may not help and increases the risk of frying the PLC module.
7. Add a spark gap or limiting resistor to the EL somewhere. Resistor only helpful if the EL itself is sparking. Difficult to find a spot for the spark gap.
8. Get a bipolar supply that can sink the current of a spark and replace the EL supply. - Tomislav doesn't seem to think this is viable.
9. Add protection on the controls cables.

Spark recording:

60uA/50kV/540A July 3 02:30 - 05:30 : average of 50 sparks per hour. data file https://isacwserv.triumf.ca/onlylocal/isacdata/ITE_TM3_SiC46_HVConditioning_2_20240704

60uA/32kV/540A July 3 19:15 - July 3 21:45 : average of 8.4 sparks per hour. data file https://isacwserv.triumf.ca/onlylocal/isacdata/ITE_TM3_SiC46_HPSIS_HVConditioning_20240704

50uA/42.8kV/540A July 4 8pm - July 5 2pm : average of 20 sparks per hour, or if you cut out the last two hour that were very sparky, 14 sparks per hour. data file http://isacwserv.triumf.ca/onlylocal/isacdata/TM3commissioning_July4_52024.dat

50uA/42.8kV/540A July 8 00:00 - July 8 18:00 : average of 3.4 sparks per hour. data file http://isacwserv.triumf.ca/onlylocal/isacdata/TM3commissioning_July82024.dat

50uA/42.8kV/540A July 10 10am - 3pm : 3.6 sparks per hour

50uA/53kV/540A July 17 5am - 10am : 4 sparks per hour

45uA/53kV/540A July 20 8pm - July 21 6:30am : 24 sparks in 11 hrs, avg 2.2 sparks per hour.

EL trip recording (first 2 entries from the archiver, verify with ops elog):

50uA.42.8kV/540A July 4 23:00 - July 5 16:00 : 9 trips, average 0.5 trips per hour

50uA/42.8kV/540A July 6 12:00 - July 7 03:00 : 6 trips with 2.5hrs off, average 0.5 trips per hour

50uA/42.8kV/540A July 7 05:00 - July 8 18:00 : no trips in past 30hrs. Last trip July 7 at 3am.

50uA/42.8kV/540A July 10 3pm - still no EL trips

50uA/53kV/540A July 16 8pm - July 17 10am : 3 trips in 6 hrs. Note cable in EL circuit got burned so EL is running at only ~4kV

45uA/53kV/540A July 17 3pm - July 18 9:30am : 7 trips in 18hrs

45uA/53kV/540A July20 14:00 - July 21 6:30 : 5 trips in ~17hrs

2024-11-15 EDIT, Alexander Shkuratoff:

I was not involved in this run, but I have gone through all the Elogs for TM3 up to this point and summarized all the data dumps at the bottom, so I may as well do it here.

	ITE_TM3_SiC46_HPSIS_HVConditioning_20240704	2024-07-04 01:42	4.8M
	ITE_TM3_SiC46_HVConditioning_Vac_20240704	2024-07-04 01:59	5.3M
	ITE_TM3_SiC46_HVConditioning_2_20240704	2024-07-04 05:31	1.1M
	TM3commissioning_July32024.dat	2024-07-04 17:04	11M
	TM3commissioning_July4_52024.dat	2024-07-05 14:18	12M
	TM3commissioning_July82024.dat	2024-07-08 18:03	13M

 Additionally, there is no Elog entry for August 18, 2024 for which TM3 commissioning continued in ITW, but I have found the following StripTool data:

	TM3commissioning_Aug182024.dat	2024-08-18 05:37	14M
	TM3commissioning_Aug182024_2.dat	2024-08-19 06:16	14M

Tm2 has been moved from TCS to SW silo. the move is smooth.