Setup
Processing the data
Plotting the data
Summary

---

Intro

   Monitoring the 12meter total power has shown that it varies. As the temperature increases the total power goes down.

• run winking cal for for 26 hours (after temperature stabilization)

On 8nov21 a peltier cooler was installed to temperature stabilize the cal diode. If the cal diode output is constant, then it can be used to correct for gain variations in the system.

---

Setup:

The analysis used the 25Hz hardware winking cal data taken on 17jan22 when the cal temperature was measured (more info)

•  7 172 Mhz bands were recorded with 2048 freq channels/band
• the band frequencies were:
• 8219.00 8363.00 8505.00 8647.00 8789.00 8931.00 9073.00 MHz.
  
• the 25Hz hardware winking cal cycled with 20 milliseconds cal on  followed by 20 milliseconds cal off
• the spectra were dumped at 2 milliseconds (giving 10 samples per calOn and cal Off.
• the spectra around the transition were excluded giving 8*2ms = 16 ms for cal on and 16 ms caloff for each cal cycle
• Data was taken for 120 seconds tracking blank sky giving 120/.04=3000 cal cycles in the 120 secs of data.
  

---

Processing the data.

    For each 120 seconds of data:

•  The calOn and caloff spectra were input
• they were averaged to 16 milliseconds giving 1 cal on, off spectra per cal cycle
• The rms/mean by freq channel was computed for the 3000 calon, cal off spectra
• a robust linear fit to the rms vs freq was done to help exclude rfi from the total power computation.
•  6% of the spectra on each edge was ignored for the first iteration of the fit.
  
• freq chans with 3 sigma outliers were ignored.
• this was done to the calOn and caloff spectra separately and then the masks were anded  to give a single mask for the on,off.
• This was repeated for each pol,band
• The mask was then used to compute the total power.
  
• We usually ended up with 165 Mhz out of the original 172 Mhz band (excluding about 3% of the bandpass).
  
• The  gain correction is done by dividing the tpCalOff/CalDeflection.
• If the cal outpt is stable then this should cancel gain variations  slower than 40 milliseconds.
• The cal is about 40K, Tsys is about 120K.
• The noise on the calDeflection is sqrt(2) less than the tsys noise (since we subtracted calon,off)
• The fractional noise on the calDeflection is 120/40/sqrt(2) times the tpOff noise or about 2.12.
• This will blow up the noise on the tpCalOffCorrected signal (probably by less than sqrt(1+2.12^2) since the caldeflecttion noise will be correlated with the calOff).

---

Plotting the data

    The first plots shows the total power vs time for the 120 seconds of data (.ps) (.pdf)

• Page 1: Normalize total power vs time.
  
• Black in the calOn total power, red is the caloff total power
• each of these has been normalized to their median value (to show variations).
• The left column is polA the right column is polB
• The 7 rows are the 7 frequency bands.
• All of polA has a negative going slope of about .6% in 120 seconds
• PolB has and increasing level with 1% in 120 seconds.
• The calOn and calOff power track one another.
• Page 2 over plot caloff  total power
• top: polA
• bottom: polB
• The frequencies track one another.
• must be a variation prior to our mixing the separate bands.

    The 2nd plots show the total power before and after correction  (.ps) (.pdf)

•  The tpCalDif=tpCalOn - TpCalOff was computed for each cal cycle.
• The tpOffCorrected=tpOff /tpCalDif was computed for each cal cycle.
• dividing by the calDif blows up the noise.
• each of the total power time series have been normalized to their median value
• Black is the tpCalOff with no correction.
• You can see the larger curvature of the polB signal.
  
• red has been corrected by dividing by the cal deflection
• green is the red signal smoothed to .4 seconds.
• There are 4 frames per page.
• the top 2 frames are polA,B  of a freq band,
• the bottom 2 frames are polA,B of the next freq band.
  
• the rms for each set of data is printed at the top of each frame.
• The expected rms would be 1./sqrt(165e6*.016secs). =.00055

    The final plot reduces the corrected TotPwr noise by smoothing and fitting to the calDeflection (.ps)  (.pdf)

•  Page 1 rms of Corrected totalPower using different polynomial  fits to the cal deflection
• The 120 secs of CalOff total power was corrected by fitting polynomials of order 1 thru 5 to the cal deflection.
  
• horizontal axis: order of polynomial fit
• Vertical axis rms [tsys units] of calOfftotalpwr after cal correction.
• Colors: the 7 frequency bands.. Purples is the expected rms for 165Mhz, 16milliseconds.
  
• Top: polA
• there is little difference between a 1st and 5th order fit for polAl
  
• Bottom: polB
• The rms decreases up to a 3rd order fit.
• this is fitting the curvature seen in the polB  gain variation.
• Page 2: Compare the totPwrCalOff rms noise for the different methods:
• + solid lines are polA
• *---- are polB
• Each point is separate 172 Mhz band.
  
• Top plot: 120 seconds of 16 ms total power data.
  
• Black is the original data with no cal correction
• red: correct with calDeflection (no smoothing or fitting).
• The rms is larger than the raw data since the calDeflection  division blows up the noise.
• Green: smooth the CalDeflection  by 25 calcycles (1 second of wall time)
• the rms is better than the uncorrected data by factors of 2 to 3.
• Blue: fit a 3rd order polynomial to the calDeflection before correction.
• The rms smaller than the green 1 second  averaging.
• Bottom: Look at data averaged to 25 calcycles (1 sec wall time, .4 sec integration)
• The vertical scale is blown up from the top frame
• Black no correction.
• the tp rms for 1 sec is the same as 16 milliseconds because of the large gain variation across 120 seconds.
• red, green correct with 16ms calDeflection then average to 1 second of wall time.
• the red,green are the same since they both end up averaging the same time.
  
• blue,: correct with 3rd order caldeflection fit then average to 1 second of wall time
• the rms is better than the other methods
• purple: expected rms for 165Mhz, .4 sec integration. (.00012 Tsys)

---

Summary

• Blank sky was tracked for 120 seconds with the 25Hz hardware winking cal running.
• The measured total power varied by about 1%
• polA was linear
• polB was curved.
• Dividing by the  25Hz caldeflection
  
• can correct for this gain variation.
• It will also blowup the noise  (since cal deflection is much smaller than Tsys)
• The increase in total power noise can be compensated by:
• average the caldeflection (in this case to 1 second wall time)
  
• fit a polynomial to the cal deflection (using the 120 secs of   data).
• For this example a 3rd order polynomial gave the best results.
• The order of the fit will probably be a function of :
• how long the dataset to process
• polA,B
• the current gain variations.
• For 165MHz, .4 sec integration, 1 sec all time
• we got within a factor of 2-3 of the expected rms.
• Part of the rms will be determined by how well we removed any rfi  from the total power samples.
• Drawbacks for this technique:
• we are ignoring 50% of the integration time by only looking at the calOff data
• if the caldeflection was smaller (currently 40K) we might avg the calon,off and take a hit of 1/2 the cal value in tsys.
• But a smaller Caldeflection will blow up the noise more (hopefully the fitting will do good enough job)
• We are ignoring samples adjacent to the cal deflection.
  
• This reduces the integration time by   2*spectralTm/20ms
• we could sample faster (say 1ms) but this would increase the disk usage.
• The data rate for 1freqbandx2048chanx2pol*16bit/sample
  
• 2ms sampling:
  
• 1 band: 4 MBytes/sec
• 7 bands: 28MBytes/sec
• 1ms sampling:
• 1 band: 8MBytes/sec
• 7 bands: 56MBytes/sec
• You could increase the sampling rate while decreasing the freq channels (depending on how narrow the rfi is).

processing:x101/220218/wcal_stability.pro