I leak checked TM2 at conditioning station this morning. All air to vacuum seals on the top of the module are good. All water lines are pressured with 60 psi helium for minutes each. One of the tube heater + water line( 2lines for tube heater + for supply and return) has a huge response on 60 psi helium pressurizing. The leak rate went from 6.0xE-9atm.cc/sec to maximum leak rate in 5 seconds and I saw a pressure increasing either. when I pressurize the other side tube heater + line ,the leak detector also had a response,but it is slower than first side .Since the top of the water adaptor isn't sealed , so I think the leak should be in one line only. All other water lines are leak tight since I didnt see any obvious response on leak detector at 5 to 6x E-9 atm.cc/sec which is very low on leak rate. 

Firday, November 8 all water blocks (excluding I and K) were blanked off and vacuum leak checked. No responses were found. (First Picture)

I and K could not be blanked off without removal of the water block jig. Also, I dropped one of the blank offs onto the table, so one line could not be blanked off until a new clean blank off was inserted into the cell. In addition, there was still reminace of a bolt in block Ias it aslo could not be removed until the water block jig was removed.

Tuesday, November 12 a new blank off was put into the cell. The water block jig was removed, which was very easy, confirming documentation from installation (in 2008?) that it fit very well / the blocks were bent and soldered to the jig very carefully.

Screw was removed from I and both I and K were blanked off and vacuum leak checked. No response. (Second Piccture)

Containment box was re-installed and module ready to move to the conditioning station by noon.

 High voltage testing was started on TM2 with no source tray today. The maximum stable for 1 minute voltage reached was 47.5kV but the module is still conditioning. With every step up in voltage the current goes up a large amount and slowly bleeds down. Large sparks disrupts the connection of the CANBUS to the PS which is irritating but the module can be brought up to the equivalent voltage in less than 30 seconds. Attached is a screen shot of most of the strip tool. I have also attached all of the dumped data. The module will be left under vacuum with the water running over the weekend and HV testing will continue on Monday. 

 This morning when I went to start HV conditioning CP2 and CP3 both had faults and the temperature of the water was ~27C. I reset the compressors by flipping the system reset switch. This cleared the fault and allowed the compressors to come one. To the best of my knowledge these faults happen as a result of compressor fluid pressure too low or high and temperature too low or high and lack of flow. The system cannot determine which one it was that caused the trip. The system restarted and faulted again. As I was watching this time it can be determined that the fault was the result of an over temperature event. This was the caused by the cooling fans not coming on. Which was traced to the fan VFD controllers which both read OP3 which is an Over voltage during constant speed operation. The operating manual lists these as the possible causes

The input voltage is fluctuating abnormally.

• Power network is greater than 200 kVA

• Power factor capacitor switching

• SCR switching on power network

• The drive controller is regenerating— the load causes the motor to run at a frequency higher than drive controller output frequency.

• Intermittent output phase fault

The proposed solutions are

• Install a line reactor.

• Check the input and output circuits for phase failure and rectify. 

I will continue to monitor over the next few days to see if this continues to happen. If it does I will contact ESC.

 The Glory of TM2 continues 100V at a time. Today I brought the module to 46kV and left it there for 1 hour. During this time there was only 1 spark. I then continues at up to 53kV. At this voltage the canbus continues to be temperamental when the module sparks. The module is still conditioning and I will continue more tomorrow. Attached is the dumped data and a screen shot of the conditioning from 46kV to 53kV.

 HV conditioning of TM2 continued today. At ~53.5kV the module started to spark externally to the HV cover. This is the first time the cover has been tested to this high of a voltage due to the limitations of the modules. As such for the conditioning station to achieve 60kV while keeping the target hall open of personnel the cover and cage will need to be modified. To continue with HV testing of TM2 the cover was removed and the target hall locked out as per the procedure covered in Document 70039 "ISAC Conditioning Station: First Lock-Out and High-Voltage "ON". With the cover off the module was able to achieve ~56.7kV without the Einzel lens on. Eliminate the Einzel lens as a potential source of sparking it was brought to 25kV while the module continued conditioning. Maximum voltage achieve under this configuration was 57.1kV. Increments are now 50V steps. The Canbus continues to be very sensitive to any sparking and is slowing the process down considerably. It does however appear that the module is continuing to condition. I will continue tomorrow after the module moves to attempt to reach 60kV and hold it for 1 hour.

 HV testing without the cover continued today. With the EINZEL lens at 30kV the target module was able to achieve 58.7kV. At this voltage any spark was reeking havoc on the electronics of the TCS. Vacuum gauges signals were occasionally lost, the thermocouple reading would occasionally be lost, turbo pump would turn off. All of these events caused the interlocks to trip resulting in the power supplies to be turned off. As a result I bypassed majority of the interlocks to prevent this from happening. Ultimately as spark took out the CANBUS controller first for the EINZEL lens and then for the BIAS. David Morris has been contacted to have them replaced. The damage of the CANBUS controller for these power supplies is becoming a common problem and should be investigated for if there is a better solution. During testing I experimented with raising the BIAS to near its limit followed by the EINZEL lens and vise versa. From this it was determined that the EINZEL must be raised to high voltage before the BIAS otherwise sparking occurs. Even a step from 0 to 500 volts on the ENIZEL lens with the BIAS at 55kV caused sparking. There are a few combinations of testing along this line I would like to continue with but initially my though is that the BIAS and ENIZEL do not have sufficient clearance for the BIAS to operate above 55kV. Attached is a screen shot of the strip tool and the data. I have not heard from David yet but I do not expect the TCS to be operational today. 

 It was attempted to HV condition TM2 to 55kV and leave it operating there for an hour to demonstrate stability. Unfortunately at 52.65kV a spark destroyed the canbus controller. This controller had shown issues in the past but seemed to magically fixed itself. I will contact Hubert for another card. Attached  is is a screen shot of the stip tools. 

Today 55kV was applied to TM2 and held for over 1 hour with the EINZEL at 27.5kV. There were 2 sparks and the current was unstable at this voltage but this test should be considered a success. Once again while bringing the voltage up the canbus continued to be a problem. After successfully passing the test I continued to raise the BIAS and EINZEL. A spark at 57kV tripped a controller for the vacuum and the canbus. As stated before I believe the module will reach 60kV and that there is an interaction between the EINZEL and the BIAS. I am not certain if that interaction is internal or external to the module. Attached is a screen shot and the dumped data.

While under vacuum Travis and I completed diagnostics of the HV chase. Majority of the lines showed OL at 1kV but there were the following "failures"

A to P @ 250V 716kOhm
A to Q @ 1kV 25.18MOhm
B to C with ohm meter 8Mohm @250V 830kOhm
B to D @ 1kV 15.57MOhm
C to D @1kV 16.18 MOhm
P to Q @1kV 1.069 GOhm
G to J with ohm meter dead short.

Ideally all of these values would be OL and they may improve or get worse when the source tray is installed depending on where the electrical contact is. I need to discuss with Friedhelm but I believe that as these values the module can still operate. Without changing the HV chase there is very little chance of improving these values.

TM#2 has been moved from the conditioning station to the south hot cell.

Chad and I leak checked TM2 tube heater(+) water line yesterday afternoon. The water line was difficult to pump down on base leak rate, but LD  base pressure was fine . The base pressure on leak detector was pumped to 0.0XE-4 torr quickly. The base leak rate only could be pumped  to  4.5 xE-8 atm.cc/sec after 1 hour waiting time. All vacuum connections to leak detector  in the target hall were leak checked and they were good. Chad checked helium flow in SHC. He sprayed helium twice on tube heater(+) water block soldering joints. I did not see any response on leak detector in TH.The base Leak rate and base pressure did not move up during the test.  See attachment pictures.

Chad and I did a pressure test on TM2 tube Heater+(D) water line.I pressurized the D line with shop air from quick connectors(both ends) on service cap.chad applied bubble soap onto the D water block in SHC. The test started from 5psi,and increased 5psi each time to 40 psi. From 5 to 35 psi, no bubble has been found on water block. At 40 psi , bubble appeared from blind blank off seal , but not from soldering joints. Since the o-ring seal for the blank off is  vacuum seal , but not for pressurizing, It is understandable the seal could not hold 40 psi pressure air. No bubble had been found on the D water block soldering joints .See attachment pictures.

Bevan and I did a leak check on TM2 tube heater+ from inside service cap in TH. The leak detector base leak rate 0.0xE-9 atm.cc/sec, base  pressure 0.0xE-4totrr. One of side panel on service cap was opened. Bevan sprayed the helium to the "D" line inside the blue insulator.The helium flow was checked , and he sprayed twice. No response has been found on leak detector.  See attachment pictures

Containment box has been removed and the blank off from circuit D has been removed for inspection of both the blank off and water block sealing surface.

New cleaning and polishing pads have been inserted onto the hot cell as well as rubber bungs and a new blank off.

The inital waterblock cleaning with the "scotch bright" pads has been completed. See attached pictures...the first picture being before and the second after.

Tommorow the water block will have bungs put in and a leak check to check the block will take place. After that, the new blank off will be installed and tested.

Chad and I  did the Leak checked TM2 tube heater (+) water line  with rubber insert plugged water block on " D"  in SHC. The Leak detector was very easy to pump down to the bottom leak rate and pressure. Chad checked the helium flow in SHC, and sprayed 10 second helium on  the block. There is no response had been found during the test. The test was repeated again and no leak has been found. See attachment pictures.

Chad and I did the pressure test on TM2tube heater+(D) water  line at SHC with shop air.We tested from 30psi to 80psi on both side of D line. The bubbles were found at 50-55psi on blank off seal in SHC.

David Wang wrote:

Chad and I did the pressure test on TM2tube heater+(D) water  line at SHC with shop air.We tested from 30psi to 80psi on both side of D line. The bubbles were found at 50-55psi on blank off seal in SHC.

 I would like to correct this E-LOG. There is no real  leaking bubble had been found during the whole test according to the information supplied by Chad  Fisher during  this afternoon meeting. I would like to attach some pictures took from TH during this test.

Chad and I did the leak check on TM2 tube heater (+) water line  with blank off( + elastic o ring seal) installed on "D"  water block in SHC. We got the same no leak result as we got with rubber insert this morning.