• File layout
• header
• main pdev header
• sp1 header
• ao hdr
• free space hdr
  
• spectral data
• time domain data
• file naming conventions
  
• Outstanding issues:
• History:
• Other pdev Info
  

Terminology

• Pols: The number of polarizations used. It can be 1, 2:polA, polB, or 4: PolA,polB,U,V (stokes).
  
• band: Each spectrometer box can compute two 170 Mhz frequency bands. Each band can have two spectra (polA,polB) or 4 spectra A,B,U,V if stokes.
  
• box: We have 14 spectrometer boxes. Each box can compute two 170 Mhz bands (high and low frequency bands). See specrometer specs for more info.
  
• dump: When taking spectra  this is the integration of a single spectra (or group of 4 spectra if stokes).
• field: A fits row contains multiple fields. Each field contains information for the time of this row. Some fields are: datapoint, statuspointer, raPos,decPos, etc..
  
• row: Data is written to the fits file 1 row at a time. A row has about 137 fields. One of these fields is a pointer to the spectral data. A second field is a pointer to the status data.  When taking data at higher rates each row will contain multiple  dumps of spectral/status data. The other fields in the row usually contain values for the first dump of the row.
  

Intro

File layout: (top)

• A scan:
•  is a contiguous data acquistion that may span more than one data file.
  
• The first file of a scan:
•  contains a 1024 byte header followed by data.
  
• The first file of a scan always starts on a filenumber that is a multiple of 100
  
• files after 1st of scan:
• just contain data.
• The filenumbers get incremeneted by one on each file of a scan.
• The header and data is little endian.
  

header layout  (top)

• 1024 bytes long starting at the beginning of the first file of the scan. It consists of:
  

• hdr	length in bytes	start at byte	Description
  pdevMainHdr	32*4=128	0	general info
  sp1Hdr	56*2=112	128	fpga spectral line register setup
  aoHdr		240	ao hdr holding pointing,time,freq info
  freespace	696	328	unused space at end of 1024 byte header block

  
  

• pdev main header. 128 bytes (8 * 32bit integers) holding generic info.  (top)

• name	value	description
  magic_num	0xfeffbeef	magic number identifying filetype
  magic_sp	0x2e83fb01	magic number for fpga sp section that follows main header. 0x2e83fb01 is the standard  sp1 header. Other values can be used for different fpga code (eg aeronomy filter code)
  adcf	17203200	measured adc clock frequency (it will be in error by a few %).
  byteswapCode	2	The spectrometer uses a power pc (bigendian) while the fileservers are intel little endian. The byteswap code is used by the spectrometer to put the power pc data in the little endian format: 0  4,8 bit data 1  16 bit data 3   32 bit data
  blkSize		block, record length on ouput. spectral output: The record length is computed to be an integral number of spectra (including the 8 byte status block for each spectral integration). It is set to be the smaller of 20 Mbytes or 1 second of data. TimeDomainOutput: The blkSize is taken from the packet keyword in the [dump] section of the spectrometer configuration file. A file will contain an integral number of blkSize bytes. Each output record does not contain an 8byte status block. Cima makes this value 2^20 bytes (1048576)
  nblksdumped		Number of requested blocks to dump. This is not the actual number written since the datataking could stop prematurely
  beam	0 - 6	The alfa beam or single pixel band. The
  subband	0,1	Which of the pdevbox subands this data comes from. The value is also reflected in the filename bNs0g0 Single pixel will always have a 1.
  lo1mix	0	lo1 mixer value used to mix from sky to first IF.. Unfotunately it doesn't always fit in a 32 bit integer. The value is normally set to 0.
  lo2mix0	175000000	(hz) 2nd LO used to complex mix IF1  to baseband (s0 lower IF1 BAND)
  lo2mix1	325000000	(hz) 2nd LO used to complex mix IF1 to baseband (s1 Upper IF1 band) Single pixel observing only uses this band.
  adcclk	172032000	(hz) requested adc clock frequency. This should be exact.
  time	1300879274	(secs) Start time (unix time seconds from 1970) for start of this scan. Will be time at the 1 second tick start.
  resv1	1
  pdevAoMagic	0x12345678	if 0x12345678 in this location, then pdevAo header follows the sp1 header.
  if1	250000000	(hz) center of IF1 frequency.
  fill(16) longs

  
  

• sp1 hdr: holds reg config  for std fpga sp1 code (56*2byte shorts)=112 bytes (top)

• These are all 2byte short integers
  

• name	description
  fmtWid	0 8 bit,1-16 bit, 2-32bit data
  fmtType	0-stokes I 1: polA, polB 2: pola, polB, stokes U, stokes V
  len	fftlen used
  dumpstrt	1st  channel dumped count from 0
  dumpstop	last channel dumped count from 0
  FCNT	number of ffts to accumulate 4<=fcnt<=65535
  DCNT	number ffts to drop between accumulated spectra normally 0 giving 100% integration stokes 32bit data requires this be set to 1 putting fftlen*smprate dead time between accumulations.
  arsel	dig sel 0=adc0,1=adc1,2=adc2,3=adc3,4-test,5=0
  aisel
  brsel
  bisel
  arneg	1-> negate voltage
  aineg
  brneg
  bineg
  pfbby	bypass low pass filter of polyphase filter bank
  pshift	bit telling whether or not to downshift before each butterfly stage (1--> downshfit) B0 (least significant bit) is the final butterfly stage.
  vshift	bits to upshift after fft, power (before accumulation)
  Dshift_S0	bits to downshift data prior to integration.
  Dshift_S1
  Dshift_S2
  Dshift_S3
  Ashift_S0	upshift accumulated data before packing. 0..7 bits.
  Ashift_S1
  Ashift_S2
  Ashift_S3
  Ashift_SI
  fftDropSt	fft to drop at the start.
  tsPhase	test signal info (if xxsel=3)
  tsFreqH
  tsFreqL
  tsCwA
  tsCwB
  tsNoiseA
  tsNoiseB
  dLo	digital mixing lo
  dLoPhase	for pos a,b phase difference
  hrMode	hires mode 0 hires mode off (just 172 Mhz bandwidths) All other hr keyword below are ignored. 1 hires mode on. digital lowpass filter used
  hrDec	hires decimation 1 ..1024 amount to decimate the data
  hrShift	hires upshift: the Dlpf uses a 26 bit accumulator The upper 12 bits are passed to the next stage of the processing hrshift specified how many bits to upshift the output of the dlpf before grabbing the uppermose 12 bits.
  hrOffset	dlpf offset
  hrLpf	Not 0 --> time domain sampling (and hr_mode=1) 1 4 bits PolA only 2 8 bits polA 8 only 3 16 bits polA only 4 not used 5 4 bits polA and polB 6 8 bits polA and polB 7 16 bits polA and polB
  hrDwell
  hrInc
  blanksel	gpio pin for extern blank 0xffff=none
  blankper	0-use extBlnkTime.>0# ticks to blank afte
  ovfadc_thr	adcOvf before blank. 0xffff=off
  ovfadc_dwell	ticks to blank after ovfadc occurs
  calsel	gpio pin for cal input. 0xf--> no cal
  calphase	0 to FCNT-1 when cal transitions in dump
  calctl	B0=1 cal on, B1=1 --> winking cal
  calon	dumps for cal on
  caloff	dumps for cal off

  
  

• ao header (88 bytes): pointing, freq, etc info.  (top)

• name	type	description
  hdrVer	char*4	'1.00' no null termination.  000 --> not valid header
  bandIncrFreq	uint*4	1--> sampled band increasing freq 0 --> sampled band decreasing freq
  cfrHz	double*8	freq center (Hz) of band (topocentric)
  bandWdHz	double*8	bandwidth in hz (1/samplefreq)
  object	char*16	source name from observing file . no null termination
  frontEnd	char*8	frontend name (receiver name) no null termination ch,327,430,800,lbw,sbn,sbw,sbh,sb,cb,cbh,xb,alfa
  raJDeg	double*8	ra  J2000 start of scan in deg
  decJDeg	double*8	dec J2000 start of scan in deg
  azDeg	double*8	azimuth (deg) near start of scan
  zaDeg	double*8	zenith angle (deg) near start of scan
  imjd	int*4	integral mjd day (utc) at  start of scan
  isec	int*4	integral seconds midnite form imjd at start of scan. Since hardware tick start, the start second is always integral.

  
  

• Free space: (top)

• 1024 - (128 + 112 + 88) = 696 bytes free.

Spectral data layout: (top)

• The number of pols dumped is determined by FMTYPE in the sp1hdr
• The output data is integer (stokes I is unsigned, the others are signed).
• The number of bits in the output data is set by sphdr:FMTWID. 8,16, or 32 bits.
  
• For each integration the order of the spectral data is:
• all chan polA, all chan polB, all chan stokes U, all chan stokesV, then  8 byte integration hdr.
• The 8 byte integration header holds info for an accumulation.

• byte 0,1	byte 2,3	byte 4	byte 4	byte 5	byte 5	byte 6	byte 6	byte 7	byte 7
  16 bits	16	4	4	4	4	4	4	4	4
  seqnum	fftaccum	calon	adcovr	pfbovr	vshft	accs2s3	accs0s1	ashfts2s3	ashfts0s1

  
  

• name	type	description
  seqnum	ushort	counts spectra modulo 65536
  fftaccum	ushort	number off fft's actually accumulated this integration. Can be less then requested if blanking is in on.
  calon	4bits	cal on =1, caloff =0
  the overflow,saturation counts (that follow) are coded as the log base 2 of the number of errors: 0 0 errors 1 1 error 2 2-3 errors 3 4-7 errors ... 13 >= 4096 errors
  adcovr	4bits	adc blanking overflow count
  pfbovr	4bits	polyphase filter bank overflow count
  vshft	4bits	saturation count on vshift upshift
  accs2s3	4bits	saturation count accumulating  stokes u,v
  accs0s1	4bits	saturation count accumulating polA,polB
  ashfts2s3	4bits	saturation count upshifting stokes U,V for packing (after accumulation)
  ashfts0s1	4bits	saturation count upshifting pola,b for packing (after accumulation)

  
  

TimeDomainData: (top)

• samp1: Ia,Qa,Ib,Qb samp2:Ia,Qa,Ib,Qb ....... sampN:Ia,Qa,Ib,Qb
• The number of pols and bits output is specified by  hdrsp1:hrlpf
• The data is twos complement integer.
  
• There is no integration  header with time domain data.
• Examples: Let Ian be polA sampleN,
• 
  

• bits/pol

  B0

  B1

  B2

  B3

  B4

  B5

  B6

  B7

  B8

  B9

  B10

  B11

  B12

  B13

  B14

  B15

  16 A

  Ia0

  Qa0

  Ia1

  Qa1

  16 AB

  Ia0

  Qa0

  Ib0

  Qb0

  Ia1

  Qa1

  Ib1

  Qb1

  8 A

  Ia0

  Qa0

  Ia1

  Qa1

  8AB

  Ia0

  Qa0

  Ib0

  Qb0

  Ia1

  Qa1

  Ib1

  Qb1

  4A

  Ia0,Qa0

  Ia1,Qa1

  4Ab

  Ia0,Qa0

  Ib0,Qb0

  Ia1,Qa1

  Ib1,Qb1

File naming conventions  (top)

• We  have two spectrometers each with 7 spectrometer boxes. This will handle the primary observer and the piggy back experiment.
• Alfa observing:
• Each pdev spectrometer box will handle 1 alfa beam, 300 Mhz all  pols.
• It will output two files. 1 file for the upper 170 Mhz band  (s1) and 1 file for the lower 170 Mhz band (s0).
  
• Single pixel observing
• Each pdev spectrometer box will handle 1 170 Mhz spectra (all  pols).
• It will ouput a single file
• This uses the upper 170 Mhz IF band (s1)
• Different boxes(beams) can be used to cover different 170 Mhz IF bands of the single pixel reciever.
  

• nnnn..nn is an identifier for the experiment. it is usually the projectid of the observer.
• yyyymmdd . is the date when the file was opened AST.
• bX:
•  for alfa this will be b0,b1,b2...b6 corresponding to the alfa beam.
• For single pixel this will correspond to the different frequency bands.
  
• s1:  It always uses the upper 170 IF band of pdev.
• gZ:group.  g0 is data from the first pdev spectrometer (1st  7 boxes), g1 is data from the 2nd 7 boxes.
• sssss is a 5 digit sequence number the increments during the day. Each scan starts on a multiple of 100. If multiple files are taken during a scan then then increment by 1. eg startscan: .00100.fits,  .00101.fits, .00102.fit, startscan:.00200.fits, .00201.fits.
  
• all files end with .pdev
• The online files are mapped via nfs to the offline computes via:
• group 0
• beam0: /share/pdata1/pdev/
• beam1: /share/pdata2/pdev/
• beam6: /share/pdata7/pdev/
• group 1
  
• beam0: /share/pdata8/pdev/
• beam1: /share/pdata9/pdev/
• beam6: /share/pdata14/pdev/

Outstanding issues:  (top)

History:   (top)

home_~phil