Plant group and us  finished TH floor mopping and garbage collections last Wednesday. Th moping head had 800 count after using. all garbage were bagged , checked, tagged, and sent to radwaste area.

TCS cooling water system was started and tested this Wednesday. Chiller compressor 2 and 3 was in operation cycle twice without issue. TCS:TK2 was refilled to 28 inches. Water system works fine. system was switched off after test.

TM2 HS line helium leak check is done. leak detector base LR 1.8xE-8atm.cc/sec.  helium pressure 70 psi. 5 minutes test. no change on LD leak rate.    I spray lots helium to gas line adaptor and fittings. No leak.has been found  at this base leak rate level.  Since the TCS MP1 has some  issue ,this time the base leak rate on LD only can get to 1.8Xe-8 atm.cc/sec at this moment instead of 0.0xE-9.  The low E-8  level is not perfect, but  i  have no concern on it.

TCS MP1 is replaced due to poor performance( system shows unstable and high reading on TCSCG1A and high TP1,TP2 current draw) . The pump is edward 15i scroll pump. Since vacuum group has no same spare pump, i have to use SMP roughing pump SMP:MP1 which is a  same pump to replace it.. After the replacement ,the TCS:CG1A and TP1, TP2 current draw are back to normal. The original TCSMP1 was surveyed. Swipes on intake and exhaust port are 300 counts and 1000counts. RAM contact field measurement are from 0.2 to 0.5 usv/hr. It is stored in TH. We will try to refurbish this pump first and use as for SMP:MP1. If we can not refurbish it, a new 15i pump will be purchased for SMP:MP1.

2023-05-31: Opened pump tip seal chamber and swiped chamber inside. The contamination level :1200 counts on 44-2.  The tip seal and tip seal chamber o ring need to be replaced . Checked gas ballast knob. The O-ring inside knob is in good condition. Checked exhaust port o-ring. it is in good condition. spare part PO will be issued soon.

2023-06-14:Tip seals and o-ring have been replaced on original TCS:MP1. After, it was tested at TCS for 48 hours. The pump is much quiet on operation, and the pump body temperature is mild. The final  inlet pressure this pump could get with TCS PV1 closed is about 23 mtorr.  The result is good. I installed it back to TCS vacuum station and reinstalled SMP:MP1 back to SMP vacuum station.  

Plant group changed TCS water filters last Friday. 4 filters on resin can inet /outlet sides and filters(4 in one cartridge house)on pump return line have been changed. Triumf maintenance tags on filters have been updated. No water leak after job. The TCS TK2 water level dropped  from 20" to 16" because of the job. I will refill the level to 28" today. 

TCS:TK2 is refilled to 28".

TM2 Nb#10 has been connect at TCS. TCS water system and vacuum system have been started. Everything looks ok so far. TCS Faraday cage is secured for TM2 conditioning at TCS.

TM3 was moved from the SHC to the TCS (no target installed).

Tuesday, January 02, 2024, 15:58 :

Started HV ramp up on TM3 service tray about 3pm today. Had to bypass TCS:BIAS interlocks to make it work - there are some flow restrictions on the water lines that are causing the flow interlock to periodically trigger. The water flow it not very important for this test as long as the water is moving. Turned out that it was also conveninent to have all the interlocks bypassed because sparks tripped off the IG1 gauge and this did not cause the HV bias supply to turn off. Should use same setup next time.

Some sparking around 35-44kV, tripped off turbos/IG1/IG1S 3 times and so decided to ramp down.

Tried again while standing outside TH door to hear if sparking is outside the vacuum. Went to 48kV very slowy with multiple trips of the turbos, mostly TP2. Lots of sparks but easy to ramp back up to previous high point, so likely just a painful conditioning process. Current draw 167uA at 48kV. No sparks audible outside door, and vacuum response looks too large to be electrical noise.

Continued on Wednesday January 03 - went up to 53kV in the same way as before, lots of TP2 trips making its current a bit unstable... data was dumped as TM3_servicetray_conditioning_jan32023. Tried limiting the current on the power supply to very near the drain current to see if that reduces the number of trips. Didn't notice a significant effect.

Thursday, January 04, 2024, 09:57 :

Restarting conditioning, starting at 180uA at 53kV. Got up to around 60kV with many trips of the turbos, and one restart of the IG1 controller needed. Added a grounding hook to the vacuum controller rack to see if that helps reduce the trips. (doesnt appear to have).

Friday, January 05, 2024, 09:51 :

Continuing conditioning. Starting by sitting at 50kV for 10mins. No sparks observed during this time.

Even though we got to 60kV last time, onset of conditioning started at 58.6kV with a trip at 58.9kV... got up to 70kV.

Got to 72kV on Monday January 08 2024.

On Monday Jan. 15, tried to improve the grounding on the vacuum services rack to reduce sparking through the turbo controllers. Added two welding cables from the vacuum rack ground busbar to the copper grounding strip on the ground. If the spark is travelling from the faraday cage to the rack through conduit or grounding cables, maybe this will help. If it is coming through the turbo cable then it likely won't. Tried to add grounding straps on the turbo controller chassis (there is a brass screw which I think is for this purpose) but couldn't get either of the screws out with any resonable force. They undo a bit then get stuck...

 Wednesday, January 17, 2024, 20:39:

David vented TM3 with N, then opened the service cap to install new insulators. We pumped back down and Alexander and I ramped back up today - instabilities started around 55kV and we were back to slow conditioning in the high 50s. Got up to a set point of 65kV. Ray did some work on the TCS bias supply and now neither the set point or readback is correct. The PSU was reading 51.5kV at set point 50kV and 60kV at set point 58kV. Will continue later...

Wednesday, January 31, 2024, 16:23

Ramped up again, some slight instabiities around 56kV but could still go up pretty quick until about 65kV. Since Ray's testing of the TCS:BIAS supply both the readback and the set point are off from the powersupply display. Ran out of time and ramped down at set point 70.5.

Tuesday, February 06, 2024, 14:47

Testing again after venting the module with air. First tiny spark at 48, then some vacuum reactions starting at 49 but still no issues. Bit more reactive around 53.

 stopped at 69.4

Wednesday, February 07, 2024, 15:28 :

Ramped easily to 60kV set point and let it sit there. Didn't spark at all then after 100mins, sparked TP2 off. Otherwise first sparking happened at set point 67.

Found at limit at around 72.5 on the PSU front output, set point 70.6. think it comes from EPICS. Will have to test the power supply.... stopped there.

Thursday, February 08, 2024, 15:03 :

Long term testing. ramped back up, first spark at set point 65.

Let it sit for 2hrs 20mins. Something like 6 sparks during this time. Current also dropped significantly from 0.32A to 0.27A.

EDIT 2024-11-25, Alexander Shkuratoff:

All the StripTool data related to the above tests are on the server as the following below. Those without "Alexander" in the name were dumped by Carla.

	TM3_servicetray_conditioning_Jan32023	2024-01-03 17:29	807K
	20240103_TM3_Service_Tray_TCS_Alexander	2024-01-03 17:31	1.2M
	20240104_TM3_Service_Tray_TCS_Alexander	2024-01-04 16:58	238K
	20240104_TM3_Service_Tray_TCS_Alexander_2.csv	2024-01-05 07:00	13M
	20240104_TM3_Service_Tray_TCS_Alexander_1sec.csv	2024-01-05 12:21	81
	20240105_TM3_Service_Tray_TCS_Alexander.csv	2024-01-05 16:51	13M
	20240105_TM3_Service_Tray_TCS_Alexander_1sec.csv	2024-01-05 16:52	2.7M
	TM3_servicetray_conditioning_Jan52023	2024-01-05 21:24	8.8M
	20240108_TM3_Service_Tray_TCS_Alexander_1sec.csv	2024-01-08 16:56	4.7M
	20240108_TM3_Service_Tray_TCS_Alexander.csv	2024-01-08 16:56	7.1M

	TM3_servicetray_conditioning_Jan172024	2024-01-17 13:29	257K
	20240117_TM3_Service_Tray_TCS_Alexander.csv	2024-01-17 20:11	4.0M

	TM3_servicetray_conditioning_Jan312024	2024-01-31 17:24	2.2M
	20240131TM3_Service_Tray_TCS_Alexander.csv	2024-01-31 17:55	4.5M
	20240206_TM3_Service_Tray_TCS_Alexander.csv	2024-02-06 17:58	2.6M
	TM3_servicetray_conditioning_Feb62024	2024-02-06 19:29	4.6M
	TM3_servicetray_conditioning_Feb82024	2024-02-08 17:55	3.1M

 And the StripTool settings:

Alexander_TCS_HV_Conditioning.stp

2024-01-17 19:57

3.7K

After Tm3 service tray and service cap installation, TM3 service cap air to vacuum seals leak check has been done at TCS. TCS:IG1 and IG1S are at low E-7 torr range.  leak detector base  0.0XE-9 atm.cc/sec,  base pressure0.0X E-4 torr.  I sprayed helium to all new air to vacuum seals on service cap. No leak on seals. The leak check has been repeated  2 times with same result.   

Monday and Tuesday, I started TCS water system and enabled the TCS chillers auto stage in/out. I saw chiller 2 in/out three times at 17C and 21.8C without issue.   TCS water system works fine. The test is finished.

TM3 has been moved  from SHC to TCS. The move is smooth.

TM3 module top helium spray leak test was done yesterday.  No leak on TM3 module top @ LD base leak rate 0.0 x E-9 atm.cc/sec.

TM3 helium pressure test and electrical check with new source tray at TCS were done yesterday. The results are excellent. See attachment

Saturday, April 27, 2024, 09:35 :

Started ramping up HV, EE and EL together. First small spark at 20.5kV on bias, no reaction in other devices.

Note both BIAS and EL are off in their readbacks. See table below

Saturday, April 27, 2024, 10:23 :

Spark on HV only at 25.5, couple sparks from here to 33, then at 33 starts real conditioning.

EL is reacting to some of the sparks, not sure if this is just electrical noise or not.

Saturday, April 27, 2024, 18:14 :

lots of trips above 60kV on the set pt. tripping off turbos and gauges. IG2 needed the controller reset.

RESULT: Got to 64.5 (actual) 62 (set point) with quite a bit of sparking over 60kV. Will go the rest of the way to 73 in the next days. Data dumped as isac/data/TM3_HVtests_Apr272024.dat

Sunday, April 28, 2024, 11:23 :

Ramped back up to 60kV reasonably easily, couple sparks and turbo trips.

Went up to 64kV (set point) with a lot of difficulty. Data dumped as isac/data/TM3_HVtests_Apr282024.dat

Monday, April 29, 2024, 14:58 :

Ramped up to set pt 62 easily, went down to 500V then 100V steps after that.

Went to set pt of 67 and left it there for ~40mins. Not perfectly stable but not too bad.

Tuesday, April 30, 2024, 17:12 :

After rebooting the TCS PLC to fix all the non-responsive devices, everything is working fine again.

Ramped easily to 60kV in 1kV steps.

Still got to 67 without too much trouble. Behaving pretty well...

At 70.2 (set pt, 71.3 readback) the water and vacuum gauge both tripped off and cannot be restarted. I am not sure what the real power supply reading is here but in the past it has been 70.5 set pt = 73 real value, which is as high as the power supply goes.

Stopping here since I'm not sure the water is circulating... the off pump says it is and the on pump says its not... left it at 70.2 for 40mins, still sparking going on (20:28 to 21:10)

Data dumped as isac/data/TM3_HVtests_Apr302024.dat

Sunday, May 05, 2024, 10:50 :

Containment box and target installed.

HV conditioning starting at 47kV.

Spark at 58kV knocked out IG1. Continuing with IG2 and IG1S.

Went up to set point of 63, readback 64. Will continue on Monday.

Data dumped: isac/data/TM3_HVtests_May52024.dat

Monday, May 06, 2024, 09:37 :

Ramping system back up.

First sparks at 50kv while raising the EL voltage (not sure if they are related).

AS: Voltage quickly up to 63 kV from 49 kV. Very smooth, no sparks or instabilities. Ramped at around 250 V steps every 10-20 seconds. First signs of instability at ~62.75 kV and definitely at 63 kV. I guess this is where it was conditioned to last. Let sit at 63 kV for a while while doing something else. It sparked once and turned off a turbo after half an hour. Starting ramping again.

AS: Spark at 65.75 kV killed IG1. Carla manually reset at 69 kV. Still working. A spark at 70.25 kV took out IG1 again.

AS: Maximum voltage reached at about 16:45. Setpoint 70.5 kV, readback 71.7 kV, PS itself ~73 kV. Carla reset IG1.

AS: 18:30, spent about 1.5 hours cumulatively at this maximum voltage. During this time it sparked about 15 times. Not long enough to tell if the spark frequency is decreasing. In the middle of this, Carla did something accidentally and caused the voltage to drop a bunch, so I manually set it to 0 V. She then reset TP1S, which was not staying on. Also, the EE and EL stayed at their respective voltages during this downtime (2 kV and 28.7 kV). I ramped the main BIAS voltage back up to the maximum voltage within 5 minutes and it was smooth all the way but sparked again at the maximum voltage. It seems conditioned up to or near the max voltage. Ramped down and turned off BIAS, EL, and EE.

AS: Dumped data to isac/data/TM3_HVtests_May62024_Alexander.dat

Tuesday, May 07, 2024, 10:02 :

Tested the EE - ramped up easily to 7kV, but then had issues maintaining the voltage. No vacuum response. Ramped down since we never use it that high anyway, though cable should be good for 10kV. More investigations later.

EDIT 2024-11-25, Alexander Shkuratoff:

Summary of StripTool data taken during this round of tests:

	TM3_HVtests_Apr272024.dat	2024-04-27 18:56	12M
	TM3_HVtests_Apr282024.dat	2024-04-28 13:16	1.4M
	TM3_HVtests_Apr302024.dat	2024-04-30 21:11	9.0M
	TM3_HVtests_May52024.dat	2024-05-05 13:06	5.0M
	TM3_HVtests_May62024_Alexander.dat	2024-05-06 18:45	9.5M

TM3 has been moved  from SHC to TCS by David and Frank. The move is smooth.

TCS was argon vented this morning between 7:00 to 8:30 am before module was moved to ITE.

TCS was argon vented this morning between 6:00 and 7:30am for TM4 moving to ITW.

TM2 was moved from silo to TCS yesterday morning. The move is smooth.    TCS vacuum system was started after. TCS and TM2 pumping down is ok so far.

refilled TCS:TK2 to 27.4 inches to compensate water lost during THC filters change.

TCS water system  five filters were changed yesterday by plant group Dan.T. Service labels on filters are updated.  No water leak after job.