02-27mar19 1370hp polB stuck in

The 1370 hipass filter for lbw polB was always inserted in the rf path between 02mar19 and 28mar19..

Background:

Lband wide (lbw) has a filter bank after the lna's and before the postamps. Each polarization has its own set of filters:

lbw filters

The 1370 highpass filter (1370hp) had been giving us trouble:

Some history:

10jan19 the 1370 hipass filter on lbw was acting up (more info). After switching, it took a long time to finally settle down.. when switching to the 1370 hipass filter, lbw output power would jump and then drift slowing for a few minutes.
12jan19: lbw thermal cycled.. but polB amp died.. lbw was down until 20jan19

The problem:

On 27mar19 the a3299 (ips experiment) was run and i noticed trouble with the lower portion of the band on polB.
After some trouble shooting, it turned out that the polB 1370 hipass filter was always inserted in the rf path (even when commanded to be bypassed).
On 28mar19 around 15:00 Felix bypassed the polB filter (even when selecting it, it would be bypassed).

I then went looking for when the problem started.

X111 interference monitoring data jan->mar19 (.ps) (.pdf)

x111 uses the interim correlator to check for rfi
4 25 mhz bands are taken at a time:

60 1 sec records are followed by a 10 second cal on, caloff.

data was taken from 01jan19 -28jan19 (although there is not data for every day).
page 1 plots the calon, caloff spectra (42 in total)

Black is polA, red is polB
Top: 1272.5 MHz polA
2nd: 1272.5 MHz polB
3rd: 1412.5 MHz PolA
bottom: 1412.5 MHz PolB

Page 2 : calDeflection/Tsys total power over the band

Black is polA, red is polB
Top: 1272.5 MHz band
bottom: 1412.5 MHz band
around 12jan19 both polA, and polB cal deflection had trouble. This was right before the lbw warmup.
There was no x111 data between 15jan19 and 3mar19 (i had removed it from the list during the work on lbw).
on 03mar19 the polB caldeflection for the 1272.5 MHz band was zero, while the polB 1412.5 deflection remained around .38 Tsys.
This was caused by the polB 1370 hipass filter cutting out all power below about 1360 MHz.

lbw band from puppi coherent cals (.ps) (.pdf)

Pulsar observations use the puppi backend to measure the entire 600 MHz bw of lbw.
For calibration purposes, a coherent cal is usually taken for a few seconds:

The 25 hz cal is injected into lbw
puppi folds the signal at 25 hz (usually with about 2048 phase bins).
To look at the bandpass, i averaged the cal spectra for about 1 second (over the 2048 phase bins).

The black trace is what puppi calls polA, red is what puppi calls polB (or at least what my idl software thinks is puppi polA,B).
Top: 08jan19 -> 28feb19

both polA and polB cover the entire band

middle:02mar19 ->27mar19

The black trace has no power below 1370 MHz. This is the 1370 hp filter stuck in.
The red trace is normal
Note: The actual bad pol was polB.. not sure why these reversed .. maybe my idl software ?

Bottom:

28mar19 (evening) thru 02apr19

The 1370hp filter has been bypassed. both pols now cover the entire band.

Note:

The pulsar fits file remove a dc offset and then scale the spectra (by channel).
when looking at the scaled data, it is not obvious that there is a problem (the bandpass has been removed). You need to put the scaling back in to see the problem.

Summary:

The lbw 1370 hipass filter was always inserted in polB between 02mar10 and the afternoon of 28mar19.
During this period, polB had no power below 1370 MHz.
The 1370hp polB filter was bypassed on 28mar19 (15:00).

it will not be selectable until the problem is resolved (we need to take lbw offline to bring the filter bank down to the lab to work on it).

For some reason, my idl code that reads the puppi fits files is returning polA,polB reversed.

processing: x101/190329/lbwhp1370_x111.pro, puppi_cal_lbw.pro

23apr19: hp1370 polA,B drive cables switched.

On 23apr19 the polA,B drive cables from the hp3488 to the hp1370 filter were switched. The hp3488 polB drive cable is now driving the polA switch while the hp3488 polA cable is not driving polB. After this switch, both switches started to work correctly.

The switching was tested using the interim correlator.
The setup was:

lbw linear pol, lfb 1 5 requested (hipass and 1120-1750 bandpass.
rf center freq 1405MHz
interim correlator, 1 sec dumps, 25 MHz bw.
Data was taken for about 300 seconds for each scan
The telescope was stationary so the sky was drifting through the beam.
after a scan, the lbw filter bank was switched to another set of filters, and then back to the lbfb 1 5
A new scan was then started.

The plots show the total power vs time for the 7 scans (.ps) (.pdf)

There were 4 separate interim cor sub bands of 25 MHz centered at 1405 MHz (each in a different color).
looking at the spectrum analyzer, the 1370 hipass filter was being inserted in both polA and polB ( i guess i forgot to check if it was being switched out between scans .. but felix assured me that he checked that when he was upstairs....:).
The dotted vertical lines are a start of a new scan.

There was about 15 seconds between the end of 1 scan and the start of the next scan.
During this 15 seconds, the filter bank was switched away from lbfb 1 5 and then back to lbfb 1 5

Top: polA
bottom: polB
The only jump in level was seen between scan 1 and scan 2 in polA (about .5%). PolB showed no jump
after the start of each scan, the power level remained stable (it didn't drift .. except the for the sky changing).

SUMMARY 23APR19:

Switching the polA,B drive cables into the hp1370 filter caused things to start working.
when switching away and then back to the hp1370 filter, the power level returned to the original value (it was making repeatable contact)
The hp1370 is now operational..

although sometime someone in the future may wonder why commanding the polA hp1370 filter causes the polB filter to change.
this will not cause a problem since we normally switch both of the pols together.

processing: x101/190423/chklbw.pro

1370hp polB stuck in.

24apr19