Test the new Pdev spectrometer boxes

The pdev spectrometer boxes were checked to make sure that they all worked. The downstairs IFLO noise source was used for testing (all the receivers were down during the platform painting). A similar set of tests was done on b103 (i think) back in 26jan07. A short set of integrations (with birdies of known frequencies) were used followed by longer integrations of just noise.
Some things to look for are:

Do the spectra and cross spectra look reasonable(short integrations)
Are there birdies in the spectra that are coming from the boxes. Are these birdies stable in time.
Is the spectrum of the the total power time series flat? If not do the birdies come from gain variations in the if/lo, mixer or from the pdev spectrometer?
Does the noise decrease as we integrate for longer periods.

The hardware setup: top

The IF2 noise diode was split and sent to polA and polB in the IF2 racks. It leaves the IF2 racks (distribution amplifier 4 front panel spigot) with a 500 MHz bandwidth centered at 250 MHz.

Looking at the small values in the cross spectra, there must be two diodes one feeding polA and the other feeding polB.

The lo's used for the high and low spectrometers were 310 and 190 MHz.
An external clock at 170MHz was used to drive the boxes.
Short Integrations (120 seconds of 1 second dumps, 1 second of 1 ms dumps):

A birdie from an hp Synth (locked to the station clock) was split and then added to polA and polB of each band (at IF) using directional couplers.

Low band: birdie at 180 MHz. This should come out 10 MHz below the center of the band (190MHz).
Hi band: birdie at 330 MHz. This is should come out 20 MHz above the center of the band.

Long integrations 3600 seconds of 1 second dumps, 60 seconds of 1 millisecond dumps:

No birdies were added to the noise signal.

The signals were mixed to baseband using a temporary mixer chassis (the final versions were not yet ready).
The table below shows the boxes that were tested. One of the boxes that we have, was not tested because it was in use for the radar development.

we have received 19 boxes:

103-111, 113-120,122-123

Short Integration Tests:
boxes	date tested
103,104	31may07
105,106,107,108,109,110	01jun07
111,112,113	not tested
114,115,116,117,118	04jun07
119,120,121,122,123	not tested

Long Integration Tests
Boxes	date tested
103,104,114,115	07jun07
105,106,107,108,109,116,117	08jun07
110,118	09jun07
113,119,120,122	13jun07
123	14jun07
111 (in use by tamara)	not tested

Steps for testing: top

Boxes were tested one at a time (since we only had 2 mixers chassis's .. one for the low band and one for the high band).
The 8 signal cables, clock, 1pps, and Ethernet cable were hooked up to a box.
The -sigstat option was run to measure the rms values of the signal in digitizer units. The levels were set to have 1 sigma=35 to 40 counts. There was no overflow in the signal path.
The -sigcal option was run to measure the digitizer offsets and create the cal.conf file to remove these offsets.
The built in diagnostics were run for 30 seconds to a minute with an i/o rate of 40 Mb/sec.

ltest: Pseudo random sequence through data path testing values on spectrometer CPU.
rtest: Pseudo random sequence through data path testing values on fileserver.
rcomp: Pseudo random sequence through data path merged and compared on remote server.
The only failure was box 116 the first time --ltest was run. It stopped with too many errors (data value not expected value). I reran the test a number of times and it never failed again.

Two complete data sets were taken:

Full stokes data was taken for 120 seconds using 1 second sampling and 32 bits. Pola,polB 1 millisecond sampling for 1 second using 16 bits. This data had two birdie injected at known frequencies.
Full stokes data was taken for 3600 seconds using 1 second sampling and 32 bits. Pola,polB 1 millisecond sampling for 60 seconds using 16 bits.

8192 channels were used for all data taking.

Looking at the data (short integrations) top

The average spectrum. top

The 120 seconds of data was averaged and then plotted (.ps 5.7mb) (.pdf 3.8mb):

Each box is color coded. The first page is the low band while the second page is the high band.

The spectra from each box has been offset for display purposes.
The top two plots are the spectral density for polA and polB
The bottom two plots are the two cross spectra.
On the first page (low band) the birdie comes out at + 10MHz. On the second page (high band) the birdie is at -20 MHz. This means that I've probably got the i/q cables from the mixer into the a/d chassis plugged in backwards. The birdies should have been at -10 and +20 MHz
The bandpass and birdie location look ok.

PolA,B phase difference. top

The 1 seconds data took full stokes data. The last two cross spectra are stokes parameters U and V. The ratio V/U = sin(deltaPh)/cos(deltPh) where deltaPh is the phase difference for the polA, polB electric fields (kraus radio astronomy page 4-10 2nd edition). Taking the arctangent shows the change in phase versus frequency.
The plots show the phase change (polA-polB) for each box (.ps) (.pdf):

The top plot is the low band. The bottom plot is the high band.
The different boxes are color coded. Each plot is offset for display purposes.
There is a linear ramp with frequency (which is expected for a constant phase path).
There are jumps at the injected frequencies. This is because they were injected downstream from the noise diode. The phase difference for polA-B will then be different for this signal path.
The curvature around +/- 60 MHz is probably coming from the analog filters where they start to fall off (giving a non linear phase ramp that differs for PolA and polB).

The Bandpass shape

Average spectra with bandpass included: top

The plots show the average spectra for the 3600 seconds of data with no bandpass removal (.pdf):

The 18 spectra are overplotted.
The 4 sections are: polA loband, polA hiband, polB loband, polB hiband.

Birdies in the bandpass

Average spectra with bandpass removed (birdies in band): top

A bandpass correction was generated by low pass filtering the 3600 second averaged bandpass keeping 10% of the lowest frequencies. This was then divided into the average spectra. The bandpass removal lets you see low level birdies that are in the system. There will be birdies from the iflo system, the baseband mixers, and the pdev spectrometer. The birdies from the iflo and mixers should be in all of the boxes. If a birdie only shows up in a few boxes it is probably coming from the boxes. The pdev device can also generate birdies in all of the boxes (eg the power pc bus frequency). If a birdie is added after the mixer then it should appear in both sides of the spectrum (since the i/q birdie paths do not have the correct phase to cancel the image of the birdie). The units are fraction of Tsys (since we divided by an average bandpass).
The first plots show the 3600 second averaged spectra after bandpass removal with all boxes over plotted (.pdf):

Each box is color coded. The spectra over plotted with an offset for display
The top 2 plots are polA lowband,high band. The bottom two plots are polB lowband,highband.
You can see which birdies are common to which boxes.

The 2nd set of plots show the 3600 sec average spectra after bandpass removal with one box per plot (.pdf):

each page contains 1 box. PolA low/high on top and polBlow/high on the bottom.
dashed colored lines are plotted at some frequencies that appear. For those birdies with positive and negative frequencies, only the positive frequency is flagged.
If a birdie is added in the i or q signal, you would expect to see it in both sides of the spectra since the image rejection should not work. Birdies that are only seen in 1 side of the band are probably coming from before the mixer.
If a birdie is in the iflo then you would not expect it to be in both bands at the same frequency (since they have different IF frequencies).

Some frequencies that are present:

Birdies (MHz)	locations
+/- 20	In most boxes. high band and low band. Appears at both pos and neg frequencies.
+/-24.9	In most boxes but not +/- together. So probably coming from before the mixer.
+/- 35	B114 strong, b110 weak. This is in both bands so it is probably not in the iflo.
+/- 42.5	all boxes. This is a step in the spectra (see jump in spectra below).
+/- 50	strongest in box 116
-65	in a few boxes but only on 1 side. So probably before mixer.
+/-70	in most boxes
+/-80	almost all bands
+/-83.3	almost all bands.

The major standouts are the +/- 35 MHz birdie in b114 and box 120 polA low band which has lots of birdies.

Rms/Mean by spectral channel. Looking at spectral birdie stability. top

If the spectra birdies are not constant in time, then they can not be removed by a bandpass correction. The rms/mean along frequency channels was computed for the 1 second sampled data (3600 samples). Any channels with large rms/mean will not cancel with bandpass corrections. The mean value of the rms/Mean should be 1./sqrt(bw*time) where the bw is now 170Mhz/8192channels and the time is 1 second.
The plots show the rms/mean by spectral channel for the 1 second data (.pdf):

There is one page for each box. Each page has the 4 rms/mean for that box: polA Lo,polA hi, polB lo,polB hi.
The dashed horizontal line is the expected rms for the bandwidth and integration time.
Channels that stick up are spectral birdies that change with time.
Some boxes have average values of the rms/Mean that is much larger than the expected value. This is do to the gain varying during the 1 hour.
The birdies that show time variability (at 1 second integration levels) are: 24.9,35,70,80,83.3 Mhz.

Comparing pdev spectral birdies with Interim cor spectra top

The interim correlator can take 4*50MHz bands with 3 level sampling. On 12jun07 3600 seconds of interim cor data was taken using the if/lo noise source. The bands were centered at: 185,235,285, and 335 MHz in the IF. The freq resolution was 50 KHz after hanning smoothing. This is twice the 20KHz widths of the pdev spectra. The bandpass was flattened using a low pass filter that kept 30% of the band (vs 10% for the wider pdev spectra).
The plot shows the box 103 average spectra vs the IF frequency with the interim correlator data over plotted (.pdf).

I flipped the pdev frequencies to line up with the actual IF frequencies (from the injected birdie tests).
White is polA, red is polB, and green is pola,polb of the interim correlator.
The bandpass removal did not work as well for the interim correlator (since it uses a narrower filter).
You don't see any of the pdev birdies in the interim correlator data (you don't see any birdies at all in the interim correlator data).
So the birdies in the pdev spectra must be coming from the pdev device, the mixers, or the signal path out of the if/lo that differs from that of the interim correlator (we're using the front panel outputs for the pdev signal).

Jumps in the spectra top

When flattening the spectra, glitches were seen at +/- 42.5 MHz in all spectra at a level of about .001 Tsys. The birdie was positive and negative. This +/- is an artifact of the band pass removal. When looking at the raw averaged spectra, there is a positive jump at +/- 42.5 MHz in all spectra. This is channel number 6144 which is 2048 channels above the center of the band (chan 4096). The dump increases when moving to larger freq/channel number (for both negative and positive frequencies). Differentiating each spectra (shifting the spectra by 1 channel and then subtracting it ) shows the amplitude of the jump.
The plots show the step in the spectra at 42.5 MHz (.pdf):

Page 1: Normalize each spectra to the average value in 41.5 to 43.5 MHz and then over plot each box. Each box jumps up at 42.5 MHz. The jump is about the same for all boxes (even those with different spectral slope).
Page 2: The spectra were differentiated by shifting by 1 channel and subtracting. The horizontal units are channel number (count from 0) covering the same 41.5 to 43.5 MHz. There are 8192 channels in total with dc at 4096. The jump at 6144 is 2048 channels above the center of the band.
Page 3: The jumps at +/-42.5 MHz using b103 polA, lowband. The horizontal axis are channel number (counting from 0).

Top jump at 42.5 MHz
2nd: jump at -42.5 MHz
3rd: differentiate spectra at 42.5 MHz
4th : differentiate spectra at -42.5 MHz.
Both jumps are about 32052 counts (since the baselines aren't exactly at 0). The jump is the same even though the power level at the different jumps is different: 2.68e7 vs 2.17e7. So the size of the jump is not proportional to the signal level.

Where do the jumps come from;

The jumps occur at +/- 2048 channels from the center of the spectra. This hints that it is a digital and not analog problem.
The jump is in all boxes at about the same level: .001 Tsys or about 32000 counts (in an integration of 21000 fftaccums times 3600 seconds).
The size of the jump at +/- freq is the same even though the signal levels are not the same. So it could be tied to the number of operations rather than the value of the signal.
I do the bit reversal in idl rather than jeff's code. To verify that this was being done correctly, i used spcvt to average 3600 seconds of one spectra and then looked at the data (this uses jeff's code for bit reversal). The jumps were still there.
The jumps do not occur in the cross spectra.

Total power measurements:

The total power was computed for each 1 second sample. +/- 200 Khz about Dc and 2 Mhz at the edge of each band were note used in the computation.

The total power time series:

The total power time series for the 1 second data (.pdf) is plotted for each box.

PolA (low,high band) are the top two plots. PolB (low,high band) are the bottom two plots.
The data has been normalized to the median total power (Tsys).
The slope in the spectra is probably the gain changing over time.

Spectra of the total power: top

The spectra of the total power time series was computed. The 1 millisecond and 1 second sampled data were processed separately.

Total power spectra of 1 second data. 0 to .5Hz(.pdf)

Each box is over plotted (with an offset) in color. The maximum frequency is .5 Hz.
PolA (high and low band) has a lot more birdies than polB (which is pretty clean).
The polA birdies move in frequency with time/box.
This variation is probably coming from the if/lo and not the mixer/boxes since they are common to PolA.

Comparing the totPwr 1 sec spectra with the interim correlator (.pdf)

3600 seconds of noise source data was taken with the interim correlator in 4*50 MHz bands. The total power was computed and then the transform taken.
Top plot: The interim correlator spectra of the total power time series for polA. There are 4 50 MHz bands..
2ndPlot: total power spectra of box 103 polA.
3rd Plot: The interim correlator spectra of the total power time series for polB
4th plot: total power time series of box 103 polB.
The interim correlator shows tp birdies in polA in the 4th (blue) band. This is the same band that is being fed to the pdev spectrometers. The 0 to .5 Hz birdies (gain variation) are coming from the if/lo rack.

Total power spectra of 1 millisecond data with boxes over plotted: (boxes 103-113 .pdf) (boxes 114-123 .pdf)

There are two sets of data to cover all the boxes.
Page 1 has a blowup of 0 to 10 hz. All boxes are over plotted (with an offset for display).

The top 2 plots are polA (low/high bands). The bottom 2 plots are polB (low/high bands).
PolA is relatively clean. The birdies seen in the polA 1 second are on the far left.
PolB has birdies around 1hz, 1.8 hz, 3.7Hz.. These are probably associated with the if/lo rather than the mixers,pdev spectrometer.

Page 2 has the entire 0 to 500 Hz with all boxes over plotted.

There are birdies at 60,180, and 240 hz.
The 60 hz frequency is the same for all the boxes. It looks stronger in the upper (band1) band).
The 180 hz birdies are at the same frequency for each box. They are in band 1 (upper band) but not the lower band.
The 240 Hz birdie is closer to 241-242 hz. It wanders around by about a hz box to box (or maybe or time). It is stronger in band 1 (the upper band).

Total power spectra of 1 millisecond data. Each box plotted separately:(boxes 103-113) (.pdf), (boxes 114-123) (.pdf):

There are two sets of plots (to keep the file size down).
There are two pages for each box. The first page is the 0 to 500 Hz data, the second page is a blowup of 0 to 10 hz.
It is easier to see the frequencies and amplitudes on these set of plots.
Dashed vertical lines have been made at 1.04,1.87,3.71, 7.41, 60, and 241.3 hz to cross reference various plots.

Rms noise vs integration time.

Rms vs integration time using total power: top

Rms of the total power time series vs integration time

The total power time series was computed for the 1 millisecond and 1 second data sets. Spectral channels with large rms/mean were not included in the total power computation. The rms/mean and the square root of the allen variance was then computed using time averages of : (1,2,4,8,16,32,64,128,256,512) milliseconds and (1,2,4,8,16,32,64,128) seconds.

The plots show the rms/Mean and allen variance for 1 ms through 128 second total power averages (.pdf):

Page 1: rms/mean. The solid black line is the expected rms (1/sqrt(bw*time)). This data diverges from the expected curve after a few milliseconds.
Page 2: The sqrt of the allen variance.

Pol A follows the expected rms until 100 milliseconds
PolB diverges closer to 16 milliseconds (60 hz). The polB spectra of the total power time series had birdies around 1-2 hz while polA was clean.
Gt 1 second: some polA boxes show a difference around 8 seconds. For polB this difference starts at 1 second.
The total power time series 0-.5 Hz was a lot messier for polA than polB

The changes in the allen variance correlate with polarization and not with high/low bands. The variation is probably occurring in the if/lo.
I used the total power rather than the spectral channels. The expected rms is 1/sqrt(8192) =90 times smaller for the total power.

Rms of the total power time differences vs integration time.

The 3600 seconds of total power data was first differenced giving 1800 difference samples. This data set was then integrated at [2,4,8,16,32,64,128] seconds and the rms's were computed.
The plots show the Rms/Mean vs integration time for these total power differences (.pdf):

The first page overplots all of the pdev boxes. The expected rms is plotted with diamonds. The rms/mean now falls off with integration time although it is larger than the expected value.
The 2nd page does the same computation using the interim correlator data. It now falls off with the integration time. It is also much larger than the expected value.
The total power data set has jumps, gain variations that are probably in the noise source/amps.

Rms vs integration time using On/Off spectra: top

Typical observing modes use on/off position switching of spectra. To simulate this, the 3600 seconds of 1 second dumps was used in an on,off mode. The gain variations will not affect the rms across the spectra as long as the gain change is not a function of frquency (which it probably is).
The processing was:

Avg N 1 second spectra spectra. Call this the ON.
Avg the next N spectra and call this the OFF.
Compute ON/Off and then take the rms/mean for this spectra (ignoring channels at the edges of the spectra).
Continue this sequence till all of the 3600 1 second records have been processed.
Repeat the computation using ON integration times of 1,2,4,8,16,32,64, and 128 seconds.
To try and separate out the pdev spectrometer response from the rest of the system, also do the above using the interim correlator.

Interim correlator:

The interim correlator spectra were taken with 50 Mhz over 2048 channels and 3 level sampling using 1 second dumps.
The plots show the on/off rms vs integration time using the interim correlator(.pdf).

Page 1 has the individual measurements of rms/mean across each on/off spectra.

The top plot is polA, the bottom plot is polB
The 8 integration times are shown.
The dotted color lines show the expected rms for each integration time.
At each integration time there are 4 measurements using sbc 1,2,3,4 of the interim correlator. These correspond to the 4 mixing stages, buffer amplifiers of the downstairs if/lo. The pdev data was taken with buffer amp 4 (the blue traces).
At 1 second avg there are 3600/2=1800 rms's measured. At 128 seconds there are 3600/(2*128)=14 rms's measured.
At the beginning there were a few measurements that had large rms's.
The measured rms's are always a little larger than the expected rms. The expected rms was computed as:

expRms=sqrt(2)*1.23/(bw*time).. bw=50e6/2048*1.208. The sqrt(2) comes from the division. the 1.23 is from the 3 level sampling. The 1.208 is the actual filters widths.

Page 2 plots the avgRms/Mean vs integration time.

The median value for the rms's at each integration time was computed. This excluded the outliers at the start.

The top plot is pola, the bottom plot is polB
The measured rms is 8 to 10% larger than the expected rms and is pretty constant for the integration times. So maybe i've messed up somewhere on the exprms computation

Pdev spectrometer:

The pdev spectra were taken with 170Mhz over 8192 channels and 1 second sampling. When computing the rms across the band, +/- 2 Mhz about the center and 4 Mhz at each edge were excluded. This was to say away from the filter edges and Dc.

On/offs rms vs integration time (by box) (.pdf)

The plots show the rms/Mean for various integration times.
There is one page per box. Top: polA low band, 2nd polA hiband, 3rd polB lowBand, bottom: polB hiband.
The 8 integration times are shown.
The dotted color lines show the expected rms for each integration time.
At 1 second avgerages, there are 3600/2=1800 rms's measured. At 128 seconds there are 3600/(2*128)=14 rms's measured.
The expected rms was computed as: expRms=sqrt(2)/(bw*time).. bw=170e6/8192, The sqrt(2) comes from the division.
Some of the boxes had short jumps in the rms. This is seen in boxes:

b103 polA low. b106 all bands.
Since the boxes were not done at the same time, these may be glitches/birdies in the if/lo or the pdev boxes.

Some boxes had rms's with lots of scatter:

B114 polALo, polB hi,lo. Box 119 polA lo, polB lo,hi.

On/offs avg rms vs integration time (.pdf) . The median rms value at each integration time was take for each box and then plotted vs the integration time.

The 4 plots are polA lo,hi then polB lo,hi. All boxes are overplotted on the log, log scale.
The expected rms is plotted with diamonds. It was computed as sqrt(2)/sqrt(170e6/8192*integTime)
The rms noise falls off with the expected 1/sqrt(integTime). The ratio of measured/exprms = 1.02 and is constant with integration time.

Conclusions: top

The boxes work. They generated spectra with no glaring anomalies. The cross spectra gave reasonable phase variation. The rms vs integration time for simulated on/off spectra integrate down correctly.
Spectral birdies:

Box 114 has larger than normal birdies at 35 MHz.
Box 120 polA low band has a bunch of extra birdies.
The spectral birdies seen in the pdev spectra were not seen in a 200 MHz band was taken with the interim correlator and the noise source. This implies that the spectral birdies are not in the if/lo system.
the birdies: 24.9,35,70,80,83.3 Mhz show temporal variations (using a 1 second integration time).

Computation of the spectra:

The jump in the spectra at +/- 2048 channels from the center may be coming from the spectra computation. They do not occur in the cross spectra.

Total power time series.
- The total power time series shows changes in the gain. These are probably coming from the analog amps.
Total power spectra:

0 to .5 Hz from 1 second data:

polA has lots of birdies polB is clean
The polA birdies were also seen in the interim correlator data (in sbc 4). These variations are in band 4 of the if/lo chassis.

0 to 10 Hz 1 ms data.

PolB has birdies around 1.05,1.9,3.7,7.5 Hz. They vary a little box to box. They are probably in the if/lo.
PolA is clean.

0 to 500 Hz 1 ms data.

Pol A and B has 60 hz.
PolA and B High band has 180 and 241 Hz. These wander box to box by about a hz. They are probably in the temporary mixer chassis.

Rms noise vs integration time using the total power.

When using the individual samples, the rms noise vs integration time did not decrease as 1/sqrt(integtime). The variation/jumps in the gain could have caused this.
Taking the total power differences and then computing the rms/mean vs integration time, we see a falloff with the integration time (although offset from the expected value).
This is seen in the pdev boxes and the interim correlator so it is probably in the analog if/lo.
The allen variance followed the expected value until 10 ms (polB) and 100 milliseconds (polA) of integration time.

Rms noise vs integration using on/off spectra.

The interim correlator rmsNoise vs integration time fell off as 1/sqrt(integTime). It was 8 to 10% larger than the expected value.
The Pdev boxes rms Noise vs Integration were 2% higher than the expected value.
A few of the pdev boxes showed short glitches in the rms/mean. We need to see if these glitches are repeatable by box or whether they were in the if/lo.

processing: x101/pdev/tbox/tstshort/chkall.pro plotall.pro
x101/pdev/tbos/tstlong/cmpall.pro plotall.pro

<- page up
home_~phil