dec05 pulsar 9 level folding problem with the wapps



        Project P1979 (snezana and joel)  were doing HI absorption on pulsars using 9 level sampling and folding mode with the wapps dumping acfs. They reported a problem where a pulsar that was known to have no absorption  showed  large absorption with 9 level sampling and no absorption with 3 level sampling. It looked like the level correction was the culprit.


The steps in going from acf to spectrum in pulsar mode using the wapps  are:

  1. accumulate the acf in the correlator chip then read it out. For pulsars this is typically every 64 to 1024 usecs.
  2. Combine multiple chips if 9 level  or interleaved sampling is used.
  3. correct for any bias/offsets of the 3/9 level correlator chips (e.g. -1,0,1 is recorded as 0,1,2 in the 3 level chips).
  4. If pulsar folding mode then accumulate the acf in the correct phase bin.
  5. Compute the digitizerThreshold/rmsVoltage using the zero lag.
  6. 3 or 9 level correct the acf
  7. fourier transform the acf to get the spectrum (after symmeterizing or zero extending the acf).

Plots of the problem (and solution).

    The pulsar 0950+08 was observed for about 600 seconds. Acf's were folded and written out  every 20 seconds (giving 30 sets of 100 phase bins). Wapp1 was configured for 9 level sampling and wapp 3 was configured for 3 level sampling. I analyzed this data in 7 different modes. For each "mode", I averaged the 30 folded sets of phase bins. I then found the range of pulsar on, off bins and computed (pulsOn(frq)-pulsoff(frq))/median(pulsOn(frq)-pulsoff(frq)) to see the absorption. The divide scales the values to the pulsar continuum emission.  Note that there is no divide by the bandpass so the spectra for the 9 level and 3 level results will be a little different (since they have different IF 50 Mhz filters). The 7 modes of analyzing the data were:     The plots show the results of processing the data with the 7 different methods (.ps) (.pdf) :     So the two methods of doing the 3 level correction are identical. The 9 level correction are the same to within an additive and multiplicative  constant. It turns out that these methods use slightly different values for the optimum digitizerThreshold/rmsvoltage. For this comparison I used the values from fred schwab. Maybe if murray redid his fit with the new optimum alfa/sigma the correction would be a little closer.  You can see that the 9 level spectral correction does not make a big difference (since 9 levels is pretty close to multi bit). Only at high correlation values (those close to the 0 lag) does the correction do much.

What is causing the problem:

    Snez was running in 9 level folding mode and writing uncorrected acf's to disc. She then ran the program  foldfft (/share/wappsrc/utility/foldfft.c). This routine read in the folded data, called fred schwab's routines to 9 level correct the data and then fft it.

    One of these routines in the level correction: thresh() (in /share/wappsrc/lib/vanvleck.c) is used to compute the digitizer threshold/rmsVoltage. It expects the 0 lag to be input without the factor of 16 removed. The online folding routine had removed this factor (see /share/wappsrc/wapp/fold.c line 109:

     if( c->mode->level == NINELEV )

         c->fold_count /= 16.0;

The thresh() thought that there was 16 times less power than actually present. This caused an incorrect threshold to be used in the later 9 level correction which made a big difference in the corrected spectrum.

The routine thresh() is called from: vanvleck_z(), pow9lev(), attndb9lev() all in /share/wappsrc/lib/vanvleck.c . These routines are then called from a bunch of other places.

How to fix the problem(s):

You should not put factors of 16 in the vanvleck.c file since these routines are also used by the non-folding modes were the correct lag0 is being input.

What is not broken:

    It was the folding accumulation that divided by 16. Pulsar 9 level search and spectral line 9 level do not have this division so they should be ok (or at least not have this problem).
processing: x101/051212/, foldfft.c,,