Glitches in the atm digital receiver: test data 11aug05
The echotek digital receiver was used to take some test
data on 11aug05. The setup was:
The data taking program has two input buffers (100 ipps
each). These get transferred to a set of 2 buffers (100 ipps each) which
are then written out to disc. The data recording is continuous across files.
The transmitter was cycled between power profiles (10 seconds) and coded
long pulse (10 seconds).
The RF was centered close to 434 Mhz.
The ipp was set to 10 milliseconds.
A single channel data (complex 16 bit I/Q) sampled at 10 Mhz
bw were recorded.
Data was written in the .shs (simple header system) data format. Each file
An ascii primary header
multiple records containing an ascii data record header followed by 100
ipps of binary (short) data.
Each recorded ipp included:
560 useconds of transmitter samples (5600*2 shorts)
4000 usecs of height data (40000*2 shorts)
5 usecs of noise data (50*2 shorts)
36 2gb files were recorded. Most files contained 117 records (11700 ipps).
This was 117 seconds of data (since each record was 1 second). Occasionally
there was a shortened file.
The data set had some records that were out of order.
This was evident when you looked at the transmitter samples. The power
program and clp programs have different transmitter samples:
looking at the data:
To automatically differentiate between power and clp ipps, the rms was
computed (using the I sample) for tx samples 1000 thru 1500. This region
was close to zero for the power ipps (because of the blanking) while the
clp ipps had the code here (so the rms would be large).
Power: 52 usecs (520 samples) of transmitter sample followed by 200 useconds
(2000 samples) of no data (since the blanking interval was set to 200 usecs
for the clutter).
Clp: 500 usecs (5000 samples) of transmitter sample followed by 60
usecs (600 samples) of blanking.
The first plot shows the
spectra of the clp height data (.ps) (.pdf).
The following plots show when
the out of order records (glitches) occurred (.ps) (.pdf):
8192 sampled points (complex sampled at 10 Mhz) were used. 100 spectra
The echotek card was configured to only provide a 5 Mhz output bandwidth.
The glitches will only be seen near a program transition.
If a power ipp is switched with a power ipp, then the rms's will be the
same. It is only near the program transitions where the the difference
is obvious. Since it is not locked to the transitions, the glitches are
probably occurring at the start of each 100 second record.
Fig 1 top: This shows the power (black) and clp (red) tx samples
for the I value. This is what the ipps should look like (except for the
offset used for plotting). The gap between 560 and 2600 in the power
profile is from the IF blanking. This starts at sample 5000 for the
red (clp) data. The data comes from file.000 ipps 1600 and ipp 1800.
Fig 1 bottom: This had ipps 1699, 1700, and 1701. Ipp 1700
is a coded long pulse ipp. It should have been a power ipp (the program
switched to clp on ipp 1800). The dashed purple lines show
the portion of the ipp that was used to compute the rms over each ipp.
The clp ipps had tx samples (so the rms would be large) while the power
ipp was blanked (so the rms should be small).
Fig 2 top: This shows the rms computed over samples 1000 to 1500
for each ipp in file.000 . There are 11700 ipps in the file. The file started
with power (where the rms was small). At ipp 1800 it switched to clp. The
dashed red lines show single ipp glitches. In these cases the rms showed
that the wrong kind of record was recorded. The dashed green lines
showed where the program transitions occurred. These were generated automatically
using the algorithms described below.
Fig 2 middle.. the glitches: To find the glitches, a 7 sample median
filter was run across the 11700 ipps to create a median filtered array.
Since the glitches were less than 4 ipps, they where not included in the
filtered array. The filtered array was then subtracted from the initial
data. This removed the program transitions and left the glitches. The result
is plotted here. Anything that was greater than 30 a/d counts was considered
a glitch. You can see 10 of them in this file. They occur about 100 ipps
before a transition.
Fig 2 bottom.. the program transitions: The rms was median filtered
(7 points long). The derivative of the filtered data was then taken (shift
by 1 and subtract). The absolute of this is plotted. Anything
larger than 30 a/d counts is considered a program transition.
Fig3 top.. glitch to next transition: The distance from a glitch
to the next program transition was computed for every glitch in the 36
files (445 glitches). This distance was then plotted versus the cumulative
ipp (for the 36 files). You can see that the glitches are drifting in time
relative to when the program transitions occur. This means that the problem
is not related to the program transition itself. The drifting is 200 ipps.
This is the length of the two buffers that are used for input.
Fig 3 2nd.. ipps per program transition: The ipps for the
program transitions are plotted modulo 1005.365 ipps vs the cumulative
ipp. It shows that the time for the power or clp fluctuated between
1005 and 1006 ipps. This is because the switching is done in software after
the sps has run for 10 seconds and the ri has transferred all of the data
to the vme computer.
Fig 3 3rd.. ipps between glitches. This plots the cumulative ipp
for each glitch modulo 100 ipps. You see that the ipp number (within 1..1000)
remains constant for awhile and the jumps by 200 ipps. This corresponds
to when the top plot has drifted through 200 ipps.
Fig 4 bottom.. the glitches occur on the 1st ipp of each 1 sec rec:
plots the cumulative ipp for each glitch modulo 100 ipps. It is exactly
zero. So the glitches occur at the start of the 1 second (100 ipp) records.
There are some points that don't line up. The clipping
levels were picked using the first file. If the levels changed a little,
or a different glitch (rfi) occurred, then the algorithm could mess up.
Around ipp 180000 it looks like the cycling of the programs (power, clp)
was stopped for awhile (no glitches) and then restarted (or maybe the receiver
was switched to load for awhile).
processing: x101/050811/findtransitions.pro, plttransitions.pro