i. Converting AO Data into SDFITS
AIPS++++ can read singe-dish FITS data. Arecibo's program to convert native AO format into sdfits is called "sdtest", and it is located at /usr/S2local/bin/sdtest.
SDFITS data are stored as FITS binary tables with a single row for each spectrum: A single dump usually generates 4 or 8 spectra (four subcorrelators with either one or two polarizations per board). If data are taken with different numbers of channels per spectrum, the file size will be as if they all had the largest number of channels. If the number of channels increases during the conversion, then sdtest will resize the entire file, which may take a while.
Before running sdtest you need to know the following:
Once you feel confident you know these things, login to an Arecibo computer and then go to whatever directory you wish your SDFITS files to reside in. Type sdtest to run the SDFITS conversion program. The program will will ask a few questions and then start converting your data.
Note that you can also run these programs "on-the-fly" to continuously update a datafile with the online data via: sdtest -c
In that case, when you are finished you must quit out of the conversion program by typing C (possibly a few times).
You can specify the file names on the command line too:
output_file
-L 30 -R 28 -P c
where:
output_file is the name of the sdtest file to which the data is being output. This file must have an extension (.something) for the "ao.import" routine to find it.
Notes
SDFITS Example #1
A data file with only two observations, each of a different galaxy (UGC 3489 and UGC 4431). Each cycle consisted of ON, OFF, CAL ON, CAL OFF
sdtest Enter Input (AO) filename: /proj/a9999/corfile.01jan00.a9999.1 Enter Output File: 01Jan00a.fits Channel A system temp? :35 Channel B system temp? :35 Linear or Circular inputs? :C Total number of scans in a cycle:4 Scan 930900005, UTC=2000-01-05T12:06:42, AST=2000-01-05T08:06:42 Object: U3489 Enter Command (UPPER CASE) or Object Name: (cr will keep U3489 QUIT to stop converting, DRIVE to finish entire file with same object name, UNTIL <num|scan> to continue for num or until reaching scan, NEWCYCLE <num> to change the number of scans in a cycle to num, SKIP [num | scan] to skip this scan, or num scans, or until scan ASKFREQ to prompt for frequencies each time, ASK to ask basic questions again.): U3489 Scan 930900009: U3489-1. RA 11:12:24.40 Dec +31:24:41.0 Scan 930900010: U3489-2. Offset 5400.000000 (5400.165333) Scan 930900011: U3489-3. Offset 5400.000000 (5399.659220) Scan 930900012: U3489-4. Offset 5400.000000 (5399.719945) Object:U3489 Enter Command (UPPER CASE) or Object Name: (cr will keep U3489 QUIT to stop converting, DRIVE to finish entire file with same object name, UNTIL <num|scan> to continue for num or until reaching scan, NEWCYCLE <num> to change the number of scans in a cycle to num, SKIP [num | scan] to skip this scan, or num scans, or until scan ASKFREQ to prompt for frequencies each time, ASK to ask basic questions again.): U4431 Scan 930900009: U4131-1. RA 07:59:11.43 Dec +31:48:23.6 Scan 930900010: U4131-2. Offset 5400.000000 (5400.165333) Scan 930900011: U4131-3. Offset 5400.000000 (5399.659220) Scan 930900012: U4131-4. Offset 5400.000000 (5399.719945) 9632 records translated
SDFITS EXAMPLE #2
A data file with three observations, two of one galaxy (U5361) and one of another (U5592). Each cycle consisted of ON, OFF, CAL ON, CAL OFF
sdtest Enter Input (AO) filename: /proj/a9999/corfile.01jan00.a9999.4 Enter Output File: 01Jan00d.fits Channel A system temp? :9.2 Channel B system temp? :9.2 Linear or Circular inputs? :L Total number of scans in a cycle:4 Scan 930900061, UTC=2000-01-05T15:30:13, AST=1999-11-05T11:30:13 Object: U5361 Enter Command (UPPER CASE) or Object Name: U5361 Enter Command (UPPER CASE) or Object Name: (cr will keep U5361 QUIT to stop converting, DRIVE to finish entire file with same object name, UNTIL <num|scan> to continue for num or until reaching scan, NEWCYCLE <num> to change the number of scans in a cycle to num, SKIP [num | scan] to skip this scan, or num scans, or until scan ASKFREQ to prompt for frequencies each time, ASK to ask basic questions again.): U5361 Scan 930900061: U5361-1. RA 09:55:03.25 Dec +25:12:10.6 Scan 930900062: U5361-2. Offset 8955.000000 (8948.313686) Scan 930900063: U5361-3. Offset 8955.000000 (8948.299047) Scan 930900064: U5361-4. Offset 8955.000000 (8948.302798) Scan 930900065, UTC=2000-01-05T15:42:43, AST=1999-11-05T11:42:43 Object: U5361 Enter Command (UPPER CASE) or Object Name: (cr will keep U5361 QUIT to stop converting, DRIVE to finish entire file with same object name, UNTIL <num|scan> to continue for num or until reaching scan, NEWCYCLE <num> to change the number of scans in a cycle to num, SKIP [num | scan] to skip this scan, or num scans, or until scan ASKFREQ to prompt for frequencies each time, ASK to ask basic questions again.): Scan 930900065: U5361-1. RA 09:55:03.25 Dec +25:12:10.2 Scan 930900066: U5361-2. Offset 8955.000000 (8948.189632) Scan 930900067: U5361-3. Offset 5400.000000 (5399.836752) Scan 930900068: U5361-4. Offset 8955.000000 (8948.473693) Scan 930900069, UTC=2000-01-05T15:57:01, AST=1999-11-05T11:57:01 Object: U5361 Enter Command (UPPER CASE) or Object Name: (cr will keep U5361 QUIT to stop converting, DRIVE to finish entire file with same object name, UNTIL <num|scan> to continue for num or until reaching scan, NEWCYCLE <num> to change the number of scans in a cycle to num, SKIP [num | scan] to skip this scan, or num scans, or until scan ASKFREQ to prompt for frequencies each time, ASK to ask basic questions again.): U5592 Scan 930900069: U5592-1. RA 10:21:31.63 Dec +22:32:51.0 Scan 930900070: U5592-2. Offset 5400.000000 (5401.432019) Scan 930900071: U5592-3. Offset 5400.000000 (5400.222629) Scan 930900072: U5592-4. Offset 5400.000000 (5400.127029) 14448 records translated
ii. AIPS++ Versions Available at Arecibo
Once you have converted your data into sdfits format, you have to decide which AIPS++ package you wish to use. The are a number of different version of AIPS++ available at Arecibo:
Stable: This is the version which should be the most reliable AIPS++ version available at Arecibo. On the other hand, this is often the least recent version available, as it is only updated every month/few months.
Weekly: This version exists for those people who wish to run an up-to-date version of AIPS++, and who are willing to accept a few bugs/problems with the version. As its name indicates, this version is updated once a week and is used mainly be folks working on developing AIPS++ software.
Daily: This version should only be used by AIPS++ software developers. Because it is updated daily, it can be extremely unreliable.
iii. Starting AIPS++ at Arecibo
To start up AIPS++ at Arecibo, you must first load all the AIPS++ environment variables. To do this, type: "source /pkg/aips++/version/aipsinit.csh" where version is "stable","weekly", or "daily".
You can now startup AIPS++ by simply typing "aips++" or, if you wish the naic package to be loaded for you at startup, type "aips++ -l naic.g"
At this point, you need to convert the sdfits dataset into a "measurement set". To do this, (and assuming you have loaded the naic.g package), you just type something along the lines of ao.import("sdfits file") where "sdfits file" is the name of the sdfits file which you have created (i.e. "U12289.sdfit"). Once you have done this, you will have a working measurement set which can be manipulated by all sorts of fun AIPS++ routines.
iv. Available AIPS++ Routines
The naic.g package contains routines which are designed for the import and calibration of data taken at Arecibo Observatory. A typical session for this may go something like:
% aips++ -l naic_start.g
- ao.import('U3564.sdfits')
- aoname:='U3564_ms1'
- ao.gaincorr()
- ao.plot('average1')
After the above commands, the calibrated data is shown in the dish pgplotter screen. Note that the variable "aoname" is assumed to be the name of a measurement set as it appear in the dish results manager (i.e. aoname:='U3564_ms1' and NOT aoname:='U3564_ms')
Known problems:
Planned upgrades:
- Make routines more generic. This means having the routines automatically
recognize the ON, OFF, CAL_ON, CAL_OFF
- No calibration for mapping data is in place yet
The primary files routines are described below. Details on these, and all routines available in the naic.g package, can be found at http://www.naic.edu/~astro/data/aips++/ao_utils.shtml.
(1) ao.import
Description: This program imports sdfits data and converts it into
a measurement set. In the process, the program adds
a couple 'missing' columns to the data.
Syntax: ao.import(sdfitsname)
Where sdfitsname is the full name of the sdfits file,
and which is assumed to be of the form ..
The output measurement set will have the name
\_ms
Subroutines called: ao.fixpnt
(2) ao.gaincorr
Description:
This routine reads in data from a measurement set,
does (ON - OFF)/OFF for the individual data dumps,
and then applies the gain/temp correction to the
individual (ON - OFF)/OFF data dumps. The output
is a measurement set whose data column contains the
calibrated data. The measurement set will be named
msname\_on\_off. Additionally, the calibrated data will
be averaged and the average will be written to the
dish results manager as X records, where X is the
number of boards x the number of polarizations of the scan.
Note that the data is currently assumed to be in the
form of ON, OFF, CAL\_ON, CAL\_OFF. The temperature
is determined from the CAL measurements, and the
gain correction is obtained from the 'standard' AO
gain curves, as described in the subroutines below.
Syntax: ao.gaincorr([name=aoname,onscan=0,calscan=0,tsys=0,gainval=0,offscan=0,weight='none',basel='F',convert='T'])
Here name is the name of the measurement set which contains
the data of interest. As with all these routines, it defaults to th
definition of the aoname variable. The other options are:
onscan = Scan number of the ON source observation
Default is the first scan of the data set
calscan = First scan number of affiliated noise diode observation
Default is onscan+2
tsys = array of tsys values. This must be a X by 6 array, where X
is the number of (polarizations)x(boards).
Default is to calculate this value from the cals.
Note entering a value here will override any entry for the
"calscan"
gainval = array of gain values. This must be a X by 4 array, where X
is the number of (polarizations)x(boards).
Default is to calculate this value from the gain curve.
offscan = Scan number of the OFF source observation
Default is onscan+1
weight = weighting scheme for the averaging of the scans. The
options are "none" (the default), "rms" (rms across each spectra
is used for weighting), and "tsys" (the weight for all channels in
each spectra is the same value).
basel = name of the baseline (an sdrecord in the results manger)
to use as the "off" in the calibration. If 'F' is entered,
the baseline fitting routine is not used, and instead a scan
is used as the off (the default)
convert = convert the data to Janskies (value of 'T', the default), System temperature (value of 'Tsys')
or leave as raw data (value of 'F')
Subroutines Called:
ao.tsys, ao.gain, ao.onoff, ao.avg, get\_scan
(3) ao.calmany
Description:
This routine allows the user to run the above ao.gaincorr function
on a list of objects (scans), provided the scans all lie within
the same measurement set. For a more complete description of the
routine, see the ao.gaincorr description.
Syntax: ao.calmany([name=aoname,numpairs=1,pattern=4,onscan=0,calscan=0,tsys=0,gainval=0,offscan=0,
weight='none', basel='F',convert='T'])
The entries are identical to those of the ao.gaincorr routine, bar:
numpair = number of dataset (objects) to reduce
pattern = the number of scans in a given pattern (i.e. a set which
consists of an ON, OFF, CAL\_ON, CAL\_OFF would have pattern=4).
Subroutines Called: ao.gaincorr,ao.tsys, ao.gain, ao.onoff, ao.avg, get\_scan
v. Useful Resources
There are a number of useful sites to explore to learn about available AIPS++ routines for single dish radio data. These include