Converting Arecibo Interim Correlator Data Into SDFITS (For Aips++) or CLASS Internal Format

Two nearly-identical programs convert from raw Arecibo correlator data to either CLASS and Single-Dish FITS formats on-the fly, with each record getting dumped as it is written (or within a second, at any rate). The programs that do this conversion are /usr/S2local/bin/cltest and /usr/S2local/bin/sdtest, respectively. They behave very similarly, except where necessary because of the differences between SDFITS and CLASS format data.

Before running them you need to know the following:

Once you feel confident you know these things, login to an Arecibo computer and then go to whatever directory you wish your CLASS files to reside in. Type cltest to run the CLASS or sdtest to run the SDFITS conversion program. The program will will ask a few questions and then start converting your data.


Note that you can also run these programs "on-the-fly" to continuously update a datafile with the online data via: cltest -c or sdtest -c

In that case, when you are finished you must quit out of the conversion program by typing ^C (possibly a few times).

You can specify the file names on the command line too:

cltest -c -A /share/olcor/corfile -F <output_file> -L 30 -R 28 -P c


where:

(The other available options can be found via typing "cltest -h")

Notes


CLASS

Class Data

CLASS format has a very limited space for describing observations. Cltest encodes the observation in the telescope name. So the telescope looks like: AO-BPT: B is Board, a number from 1 to 4. P is polarization, and can be X, Y (linear), L, R (circular), I, Q, U, V (stokes). Note that each board can be set up to take 1, 2, or 4 polarizations. T is type, and can be O, F, C, D, meaning On-source, Off-source, Cal-on, Cal-off (respectively). Thus, AO-1LF means Arecibo, correlator board 1, left circular polarization, Off source, and if you use 4 boards with two polarizations per board, you will get

AO-1XO
AO-1YO
AO-2XO
AO-2YO
AO-3XO
AO-3YO
AO-4XO
AO-4YO

for each dump.

The "scan numbers" for class data are the last six digits of the AO scan number, that is, the least significant digit of the day-of-year, and the 5-digit number for the scan id.

The object names are taken from the header, but the AO format allows longer names than does class, so they can be truncated.

CLASS Example

A data file with only two observations, each of a different galaxy (UGC 3489 and UGC 4131).
Each cycle consisted of ON, OFF, CAL ON, CAL OFF


/home/nolan/src2/fi/cltest -c
Enter Input (AO) filename: /proj/a9999/corfile.01jan00.a9999.1
Enter Output File: 01Jan00a.cls

Channel A system temp? :30

Channel B system temp? :28

Linear or Circular inputs? :C

Total number of scans in a cycle:4

Scan 103700038, UTC=2001-02-06T08:45:01, AST=2001-02-06T04:45:01
Object: U5440
Scan 103700038: U5440-1.  RA 10:05:36.20 Dec +04:16:23.0
Scan 103700039: U5440-2.  Offset 5400.000000 (5400.215304)
Scan 103700040: U5440-3.  Offset 5400.000000 (5400.215304)
Scan 103700041: U5440-4.  Offset 5400.000000 (5400.215304)
Scan 103700090, UTC=2001-02-06T10:01:49, AST=2001-02-06T06:01:49
Object: U5790
Scan 103700090: U5790-1.  RA 10:39:04.70 Dec +04:38:51.0
Scan 103700091: U5790-2.  Offset 5400.000000 (5400.264480)
Scan 103700092: U5790-3.  Offset 5400.000000 (5400.264480)
Scan 103700093: U5790-4.  Offset 5400.000000 (5400.264480)
Scan 103700094, UTC=2001-02-06T10:13:53, AST=2001-02-06T06:13:53
Object: U5790
Scan 103700094: U5790-1.  RA 10:39:04.70 Dec +04:38:51.0
.
.
.

CLASS_WRITER: scan 103700038 written.
CLASS_WRITER: scan 103700039 written.

Commands are (all caps required):
  QUIT to stop converting,
  DRIVE to finish entire file with same object name,
  UNTIL <scan>  to continue for num or until reaching scan,
  SKIP  <num> to skip to scan <num>,
  ASKFREQ to prompt for frequencies each time,
  ASKMAP to prompt for map pos.
  ASK to ask basic questions again.(cr will keep U3489): U3489
Scan 930900009: U3489-1.  RA 11:12:24.40 Dec +31:24:41.0
Scan 930900010: U3489-2.  Offset 5400.000000 (5400.165333)
Scan 930900011: U3489-3.  Offset 5400.000000 (5399.659220)
Scan 930900012: U3489-4.  Offset 5400.000000 (5399.719945)
Object:U4131
Number of scans in an image:4
Number of dumps in a scan:300

Commands are (all caps required):
  QUIT to stop converting,
  DRIVE to finish entire file with same object name,
  UNTIL <scan>  to continue for num or until reaching scan,
  SKIP  <num> to skip to scan <num>,
  ASKFREQ to prompt for frequencies each time,
  ASKMAP to prompt for map pos.
  ASK to ask basic questions again.(cr will keep U4131): U4131
Scan 930900009: U4131-1.  RA 07:59:11.43 Dec +31:48:23.6
Scan 930900010: U4131-2.  Offset 5400.000000 (5400.165333)
Scan 930900011: U4131-3.  Offset 5400.000000 (5399.659220)
Scan 930900012: U4131-4.  Offset 5400.000000 (5399.719945)
9632 records translated



SDFITS

SDFITS and Aips++

Aips++ can read single-dish FITS data. These data are stored as FITS binary tables with a single row for each spectrum: A single dump usually generates 4 or 8 spectra (four subcorrelators with either one or two polarizations per board). If data are taken with different numbers of channels per spectrum, the file size will be as if they all had the largest number of channels. If the number of channels increases during the conversion, then sdtest will resize the entire file, which may take a while.

SDFITS Example #1

A data file with only two observations, each of a different galaxy (UGC 3489 and UGC 4431). Each cycle consisted of ON, OFF, CAL ON, CAL OFF

sdtest
Enter Input (AO) filename: /proj/a9999/corfile.01jan00.a9999.1
Enter Output File: 01Jan00a.fits

Channel A system temp? :35

Channel B system temp? :35

Linear or Circular inputs? :C
Total number of scans in a cycle:4
Scan 930900005, UTC=2000-01-05T12:06:42, AST=2000-01-05T08:06:42
Object: U3489
nter Command (UPPER CASE) or Object Name: (cr will keep U3489
  QUIT to stop converting,
  DRIVE to finish entire file with same object name,
  UNTIL <num|scan>  to continue for num or until reaching scan,
  NEWCYCLE <num> to change the number of scans in a cycle to num,
  SKIP  [num | scan] to skip this scan, or num scans, or until scan
  ASKFREQ to prompt for frequencies each time,
  ASK to ask basic questions again.): U3489
Scan 930900009: U3489-1.  RA 11:12:24.40 Dec +31:24:41.0
Scan 930900010: U3489-2.  Offset 5400.000000 (5400.165333)
Scan 930900011: U3489-3.  Offset 5400.000000 (5399.659220)
Scan 930900012: U3489-4.  Offset 5400.000000 (5399.719945)
Object:U3489
Enter Command (UPPER CASE) or Object Name: (cr will keep U3489
  QUIT to stop converting,
  DRIVE to finish entire file with same object name,
  UNTIL <num|scan>  to continue for num or until reaching scan,
  NEWCYCLE <num> to change the number of scans in a cycle to num,
  SKIP  [num | scan] to skip this scan, or num scans, or until scan 
  ASKFREQ to prompt for frequencies each time,
  ASK to ask basic questions again.): U4431
Scan 930900009: U4131-1.  RA 07:59:11.43 Dec +31:48:23.6
Scan 930900010: U4131-2.  Offset 5400.000000 (5400.165333)
Scan 930900011: U4131-3.  Offset 5400.000000 (5399.659220)
Scan 930900012: U4131-4.  Offset 5400.000000 (5399.719945)
9632 records translated

SDFITS EXAMPLE #2

A data file with three observations, two of one galaxy (U5361) and one of another (U5592). Each cycle consisted of ON, OFF, CAL ON, CAL OFF

sdtest
Enter Input (AO) filename: /proj/a9999/corfile.01jan00.a9999.4 
Enter Output File: 01Jan00d.fits   

Channel A system temp? :9.2

Channel B system temp? :9.2

Linear or Circular inputs? :L
Total number of scans in a cycle:4
Scan 930900061, UTC=2000-01-05T15:30:13, AST=1999-11-05T11:30:13
Object: U5361
Enter Command (UPPER CASE) or Object Name: U5361
Enter Command (UPPER CASE) or Object Name: (cr will keep U5361
  QUIT to stop converting,
  DRIVE to finish entire file with same object name,
  UNTIL <num|scan>  to continue for num or until reaching scan,
  NEWCYCLE <num> to change the number of scans in a cycle to num,
  SKIP  [num | scan] to skip this scan, or num scans, or until scan 
  ASKFREQ to prompt for frequencies each time,
  ASK to ask basic questions again.): U5361
Scan 930900061: U5361-1.  RA 09:55:03.25 Dec +25:12:10.6
Scan 930900062: U5361-2.  Offset 8955.000000 (8948.313686)
Scan 930900063: U5361-3.  Offset 8955.000000 (8948.299047)
Scan 930900064: U5361-4.  Offset 8955.000000 (8948.302798)
Scan 930900065, UTC=2000-01-05T15:42:43, AST=1999-11-05T11:42:43
Object: U5361
Enter Command (UPPER CASE) or Object Name: (cr will keep U5361
  QUIT to stop converting,
  DRIVE to finish entire file with same object name,
  UNTIL <num|scan>  to continue for num or until reaching scan,
  NEWCYCLE <num> to change the number of scans in a cycle to num,
  SKIP  [num | scan] to skip this scan, or num scans, or until scan 
  ASKFREQ to prompt for frequencies each time,
  ASK to ask basic questions again.): 
Scan 930900065: U5361-1.  RA 09:55:03.25 Dec +25:12:10.2
Scan 930900066: U5361-2.  Offset 8955.000000 (8948.189632)
Scan 930900067: U5361-3.  Offset 5400.000000 (5399.836752)
Scan 930900068: U5361-4.  Offset 8955.000000 (8948.473693)
Scan 930900069, UTC=2000-01-05T15:57:01, AST=1999-11-05T11:57:01
Object: U5361
Enter Command (UPPER CASE) or Object Name: (cr will keep U5361
  QUIT to stop converting,
  DRIVE to finish entire file with same object name,
  UNTIL <num|scan>  to continue for num or until reaching scan,
  NEWCYCLE <num> to change the number of scans in a cycle to num,
  SKIP  [num | scan] to skip this scan, or num scans, or until scan 
  ASKFREQ to prompt for frequencies each time,
  ASK to ask basic questions again.): U5592
Scan 930900069: U5592-1.  RA 10:21:31.63 Dec +22:32:51.0
Scan 930900070: U5592-2.  Offset 5400.000000 (5401.432019)
Scan 930900071: U5592-3.  Offset 5400.000000 (5400.222629)
Scan 930900072: U5592-4.  Offset 5400.000000 (5400.127029)
14448 records translated

Last modified: Tue May 18 17:49:17 AST 2004

Contact: Mike Nolan

Valid HTML 4.01!