ALFALFA Survey - Observations for A2010

ALFALFA Survey - Setup and Observing Cookbook

WARNING: Version date: September 2011

Cookbook date:	10 Sep 2011
CIMA version :	"A `smart' version of CIMA (3.1.08 2010-02-01 17:44 "Smart")"
WAPPdata :	/share/pserverf.sda3/wappdata/ (a2010 alias is "wdata1")

This has not been fully updated for the Fall 2011 season; for now, it has the general details, but please check the checklist for specifics.

Find the 2007 version here, the Spring 2008 version here, the Fall 2008 version here and the Spring 2009 version here. The special version used in March-April 2010 is here. The special zenith strip observing for Fall 2010 instructions are here. The normal transit mode observing instruction for Fall 2010 are here.

IMPORTANT NOTES FOR Sep-Nov 2011

Note: Beam 4a is dead. Therefore, its power will display as low (you may see some spikes in the spectrum for 4a; just ignore them!). You will also see error messages:
- "check_wapp_levels: WAPP power levels are between 0.103 and 1.126 (still bad)"
  As long as the power levels look ok in the other ampliers, all should be ok.
Note: Warnings about WAPP chip count errors appear in the cimalog file (but not in the CIMA log window). Examples are:
- "WAPP=4-chip_count: Counts don't match between chip 0 and chip 4 (99999964 != 99999966 diff = 2)!"
- "WAPPDATA-search_agc: Interpolating agc-data over 2.00 second time gap (65545.0 - 65547.0)!"
- "WARNING WAPPDATA-search_pnt: WARNING extrapolating pnt-data from 55592.243889 to 55592.243900!" As long as the "diff" is small or the interpolation is over only a few secs or the extrapolation is likewise small, there is no real problem. Just check the data, both in the waterfall plots and using "quicklook".
While these CIMA warnings are not fatal, they make the cimalog files large (Mbytes). If size of the cimalog file exceeds the limit of the web service, you may not be able to view it. Be sure to include the link in the loglinks file anyway. And, of course, keep an eye on the WAPPS for more serious **error** messages.
Note: Sept 12, 2010: The TOGS calibration script sometimes hangs at the stage of rotating ALFA into position and times out (after 2 min). Run it again, and it should work. The problem has been reported. We have not seen this for a while, so we hope the problem has been fixed.
You need to start CIMA by typing "cima --X". (Notice the "--X".)
"Help" info is available by right clicking on any widget or parameter
The sources in your catalog which are "visible" are indicated along with their current location, slew time, etc.
It is possible to launch the system monitoring widgets from within CIMA.
"Accept" is the same as "Apply"+"Dismiss"
The scan number convention is "ydddNNNNN.fits" where "y" is the year number, "ddd" is the day number within the year and "NNNNN" is a sequential number.
If you are practicing with CIMA:
1. Use the machine called "fusion00" (or one of the other aolcN machines); Do NOT use "dataview" and do NOT log into "observer2".
2. Be sure that your VNC sessions are really killed on "fusion00" after you are done.
3. Do NOT run the data monitoring programs "alfadatawin", "alfaobswin" etc. Remember that we aren't the only ones who use these machines and that the observer has priority on dataview and observer2 and to the data monitoring.
During Fall sky runs in 2011, run TOGS only at the START of the observing block (and only if time is allocated for it). Use the file command_galfasrc_meridian.cmd or command_galfasrc_zenstrip.cmd.
Note: TOGS is allocated 15 minutes before the nominal start of A2010, so that means you should plan to start up 15 minutes BEFORE the posted started of A2010.

Disclaimer: CIMA changes, conditions change, and we have to adapt to situations as they develop; this cookbook works today but may not tomorrow! In particular, we are using the latest released CIMA version, called the 'The Smart' version of CIMA" 3.1.08 2010-02-01 17:44. Please read the documentation on the CIMA website. If you want to experiment with CIMA offline, be careful not to overwrite or change the observing files!

Before Observing

The ALFA cover must have been removed. ALFA is covered during radar runs, so take a look at the telescope schedule beforehand. For instance, if it looks like ALFA had to be covered the day before your observations, make sure that time is allocated to remove the cover before your run starts (this can be done during maintenance or system check time). In any case, it is a good idea to double check with the telescope operator. Know the schedule a few days ahead of time; but also ASK!
The ALFA motor must be switched on (this should be ok but be aware; if you have any doubts, ask the operator. It can be done from the control room.). If you have any doubts, look for ALFA rotation using the dome video monitors (you should be able to see it move). Also, the operator may need to turn on ALFA in software from the control room.
In 2011, we should be observing only north of zenith. Hence the azimuth should be positioned at 180 degrees, and you should use the a2010_az180_2011.conf configuration file.
Ask the telescope operator to move to the appropriate azimuth and approximate zenith angle for the night and, if necessary, to rotate ALFA into place. Ask him/her to do this before starting GALSPECT. During Fall 2011, we use Az =180 to observe north of zenith. Note that Az=540 is equivalent to Az=180. Note where the previous observer has left the telescope, think about options, and then make the right choice (Az or Az+360) so as to avoid unwrapping the telescope.
Be sure the operator returns telescope control to you. Note that once you start up CIMA, you'll take over control of the telescope motion and it will stop. So wait to start up CIMA. In the meantime, have the operator start up GALSPECT (see below).
Set up the TOGS starting position. Open up the catalog file /share/obs4/usr/a2010/a2010.cat and edit the entry called "galfasrc" to the new right ascension and declination for that night. As exact start times and slew times can change, the observer needs to be ready to adjust the entry by estimating it as the previous observer is finishing, and again during setup if there are delays, when the telescope will be on the meridian. This needs to be done as quickly as possible.)
We suggest also that you take a look at the TOGS setup document (provided by the TOGS group) to understand a bit about their setup.

TOGS GALFA setup - the operator will carry out this step.

Login to any machine in the control room to begin the GALFA startup.
Secure shell to the dataview machine: ssh -l guest dataview
Secure shell to the galfa machine: ssh -i galfa_key galfa@galfa1
Check if ghost galspect processes are running by typing ps. If processes called gdiag are present, kill them by typing the command kill followed by the process number.
Warm up GALSPECT via typing /var/diag
Type the command: /var/levels_togs. If you get the message LO2: Set failed, got back: ERROR setting freq, then you need to start the gpib on the wapp server. To do this, start CIMA and click Restart GPIB under the Utilities menu. Go back to the xterm logged into galfa1 and again type /var/levels_togs. You should not receive an error.
Enter the command /var/togs. This will run for the rest of the night, so minimize the windows.
Note: if the operator is not able to get GALSPECT to run, he/she should call the TOGS contact for the night.

Datataking

Quickly log into OBSERVER2 as "dtusr" (password taped to the monitor).
When ALFA is in place and you are close to the target position, start CIMA either by clicking with the right button of the mouse on the background or by typing "cima --X" in an available window. When you start CIMA, slewing will stop until you issue the observe command, and if you are not close to target position, you will have to wait. The CIMA version selection window will pop up. Select the version we are currently using, the "The Smart version of CIMA (Ver 3.1.08 2010-02-01 17:44 "Smart")". The Start New CIMA window will pop up; read below.
Start New CIMA Session
- Project number: a2010. This is important. Your settings and log files will be saved (if you decide to do so) in the corresponding project directory, which for us is /share/obs4/usr/a2010/. (note: even if you type "A", CIMA will change it to "a" when you click "Line")
- Observer: Your name for entry CIMAlog and FITS headers.
- Select Line observing mode.
- Click Accept and CIMA will start up.
  If you are using a new version, the CIMA observing session preferences set-up window may appear; alternatively, you may need to set the preferences again using the widget under the "Utilities" menu. Please do NOT change the preferences (which determine what goes in the CIMAlog file) without consulation of Martha. Note: they have been saved to the file .CIMA_prefs_1.0, and are currently set to:
  - Write daily log-files: Yes
  - Write log-file also for off-line sessions: No
  - Show the datataking status window: Yes
  - Warn if the telescope comes close to the limits: Yes
  - Keep observations going if a WAPP fails: No
  - Help-windows only stay open as....: No
  - Color scheme for log messages: Full
  - Level of information shown in log display: Info level 5
  - Level of debug messages shown in log file: No debug messages
  - Message format to use in the log window: Long
  - Message format to use in the log file: New
  - Font size to use: Medium
  Now, several windows will pop up: the CIMA Observation Status window, the CIMA Log Display window, the CIMA main menu (observing session), the CIMA executive ( red font) and a widget entitled Receiver selection menu. The CIMA main menu (which should display the correct version number, now "Version 3.1.08 'smart'", is the main window and should always be open. The buttons therein open new windows, which allow you to control the telescope. These windows will be indicated with orange fonts in this document; all the other ones will be red. (Note: we need to check this color coding stuff!)
Receiver selection
Select ALFA, Quick Tsys=No, Rotate Receiver=Yes, Retrack=Yes then click on Accept. Make sure that ALFA shows up in the AOSTATUS monitor (to your left on top if you are in the control room). Keep an eye on the CIMA Log Display and don't move to the next step until the previous has completed; if you click too fast, you run the risk of getting an "executive busy" command.
Click Utilities in the CIMA main menu, Restart the WAPPS. When the output to log window is complete, close the Utilities window.
Note: During spring 2010 runs, we normally run TOGS only at the end of the A2010 block, in which case, you can skip this step and go immediately to Load configuration below.
If running TOGS at the beginning (not in spring 2011!), immediately click on Command File Observing, and click on the Load button. You may have to click to view "All Files". Choose the file command_galfasrc_meridian.cmd. The file should appear in the right hand box. Be sure that "Run Command File" is set to Once and "Start From" is set to First line. Click Run to run the file. The Observing Menu Window will open; it will indicate that the Observing Mode is "smart frequency switching". The telescope will slew to the position of galfasrc in the "a2010.cat" file. As soon as it gets there it will load the togs_11.conf file, their IFLO setup, and begin about 6-7 minutes of frequency switching calibration. One only needs to let it run. During this time, you can use the waterfall dataview application on the dataview machine to see that data are being taken. You may also open the log files if you wish.
Note: if the GALSPECT spectrometer is not set up within ten minutes of their allotted time (so that the ~7 mins of calibration cannot be run before A2010 is scheduled to start), do not run the TOGS calibration if the prime A2010 map LST range is reached.
As soon as "observing menu" appears it can be used to verify settings and then closed. The "command file" window cannot be closed until the run file is finished, at which point click Close .
Restart the WAPPs under Utilities as above.
Click on Load configuration in the CIMA main menu.
- Select a2010_azNNN_2011.conf (where NNN is 180, 360 or another azimuth as appropriate). You will see a summary of its "contents" appearing; check that everything looks ok to you. This file now sets up just about everything you need. It fires the cal at the end of each scan ("loopend-OFF"), and adopts the topocentric reference frame. This will set most of the parameters, but you must still apply the configuration in each of the widgets described below. Select the appropriate file and click Load and apply. A window warning you to "wait" will pop up while the configuration is taking place. The log display should indicate that things are happening and the CIMA observing menu window should pop up with the observing mode set to Fixed azimuth drift map. You should also see the basic observing setup indicated: # loops=99, strip time=600secs, Azimuth=180.0 (360 or other as appropriate) degs, ALFA rot angle=19 (or other as appropriate) degs, startposition=(the first source in the file), drift type=Follow Dec J2000, Calibration type=Hcorcal, Calibration time=1sec, Calibrate when=On@loopend, Add extra cal=No, Adjust power=First loop only, Loops per file=1.
  Note: in general, the configuration file depends on the azimuth: a2010_az360_2011.conf (Azimuth 360; observing south of zenith) or a2010_az180_2011.conf (Azimuth 180/540; observing north of zenith).
When you first start up, the start position may not be correct so be sure to click to select the proper drift from the a2010.cat file. In general, you will need to adjust the RA (so that the drift starts on time). Once done, you can start observing by hitting OBSERVE. However if you need to adjust, or you want to check to be sure you have the right starting coordinates, click on Pointing Control. Here you can set the ALFA parameters and point the telescope. If you have run TOGS, ALFA should be in place and the telescope should be close to the correct azimuth. If you have not run TOGS, double-check that ALFA is in place. The telescope won't move until the receiver floor has finished rotating (to ALFA, from some other receiver).
- The source "catalog" a2010.cat will be already loaded. Check at the top of the window if this file is loaded, and, if not, load with the "Catalog" button (on the top left; find it by clicking on Project catalogs). If you have run TOGS, the "Source" will be listed as "galfasrc". You need to change it.
- In the central box, you will see a list of the Visible sources. If you don't see the one you want, click on All sources Choose the appropriate drift***p* "source" corresponding to the current observing block from the list of drifts. You can also type the coordinates of the source (RA and Dec); if you do so, be sure to use epoch "equatorial - J2000".
- If necessary, update the RA to the current LST. This can be done in a2010.cat before opening the Pointing Control menu, but usually, you'll need to tweak it here.
- Click Accept and continue with the following while the telescope points.
- Check that velocity is set to 0, reference frame to topocentric and type to optical.
- If for some reason (e.g., zenith strip observing) you need to, you can update the ALFA rotation using the ALFA Rotation Control Window. In the menu that pops up, set Rotation angle to +19.0 (degrees) and hit Send. You can actually see if ALFA is rotating by looking at the TV camera in the control room (ask the telescope operator to show the appropriate camera). The blue number next to the "Send" button should change to your entry once ALFA reaches that position. (For drifts at 0 or 180 degrees azimuth, the correct angle to give equal Declination beam spacing is thought to be 19 deg. At azimuth 90 deg, it is thought to be -14 deg.) Once ALFA has rotated into position, click Close . (Note that ALFA should be at 19 if you have run TOGS, but that we do this step to make sure.)
- Check in the Pointing Control window, that the "ALFA center" is set to 0 to choose beam 0 as the ALFA center.
- Note: if there are problems with ALFA not responding (e.g., it doesn't rotate), click Restart ALFA under the Utilities menu.
If necessary (for example, if you restarted the WAPPs) or you want to check, click on Receiver IF/LO selection menu
- Make sure that Receiver=ALFA, Filter choice= Full width, IF/LO path=ALFA, Center frequency =1385.0MHz, Backend=WAPP, WAPP bandwidths=100MHz, Configuration type= Fast, Radar blanker=OFF.
- When ready, click Accept.
- Not that rxview window will not update all IF/LO selection until data taking begins, do not use it as a check. Note: Not needed if you loaded the configuration but is needed anytime you restart the WAPPS!
If necessary (for example, if you restarted the WAPPS), click on Backend Control Menu Note: Not needed if you loaded the configuration but is needed anytime you restart the WAPPS!
- This opens the Configure Wapp ALFA window. Check that all WAPPs are using 2 channels and 3 levels and at 100MHz.
- Accept this configuration.

Beginning Observations:

CIMA Observing Menu This Observer's Widget should still be open since you didn't close the resulting window after completing the Load Configuration. This is the window you'll be using most of the night. Note it also has buttons that allow you to start a "new file" and "adjust power".
- Click on the button to the right of the start position marked drift#p# and select proper drift from the a2010.cat, check the RA is set ahead enough for the starting RA, check declination, and click use now and then click accept. Note: This menu overrides the pointing control menu but the converse is not true. Changes in the pointing control menu do not control those in the observing menu.
- The rest of the values in red should be quickly checked. Correct values:
  - The observing mode is Fixed azimuth drift map
  - Number of loops is set to cover length of run. Easiest to leave it at the default 99.
  - Strip time is 600 secs.
  - Azimuth position should be 180 degs or 360 degs or another value if appropriate (e.g., zenith strip observing). (Note 180 is equivalent to 540 and telescope will know to go to the closer angle, so you do not need to change to 540.)
  - ALFA rot angle is 19 degrees for on-meridian observing or another value according to the azimuth in use.
  - Start position is the drift you want; if not, click on it to load another from the catalog "a2010.cat" or to update the R.A.
  - Drift type is Follow Dec J2000
  - Calibration type is Hcorcal
  - Calibration time is 1 sec
  - Calibrate when is On @ loopend
  - Add extra cal is No
  - Adjust power is First loop only
  - Loops per file is 1 loop
- Click Adjust power Wait until output in log window confirms power adjustment. Once observations have started, check power numbers Observer's Monitor (alfaobswin) to make sure OK (the numbers should be between 0.8 and 1.2)
- Click Observe to begin the observation. Hopefully only a small countdown will remain, and the photon collecting will begin.
- Note: You DO NOT have to start a new fits, CIMA does this for you
- Check that the telescope is actually at the correct declination - look at the "Rslt" line in the monpnt window (the upper right window on the screen below the frequency band monitor). Dec value should stay within a few arcseconds of the drift value.
- Check the FITS files: Once you have actually started recording data, you should take a look at the data to make sure that it can be read in IDL. In another terminal, remotely login and start up an IDL session on one of the fast linux machines (e.g. fusion00, aolc1,2,3,4) so that you can run "quicklook" as a backup to the waterfall plots. Note these instructions make use of the aliases in the a2010 account.
  ssh -X a2010@remote.naic.edu (if you are observing remotely)
  ssh -X a2010@fusion00
  (or aolc1, etc) cd /share/a2010-2/data
  idl
  @wasinit2
  quicklook,'filename'
  where 'filename' is the complete name of the last complete FITS file, e.g. '/share/pserverf.sdb1/wappdata/wapp.20080822.a2010.0003.fits'. (The directory path is specified in the log files - look at the previous night's log.) You have the option to display the bandpasses for each beam, both pols of the first record of the file. Not as nice as Mikael's data monitoring, but quick and convenient for remote observers as an extra check or as backup if the data monitoring programs don't work.
  Or if you want to look at an arbitrary record, see the file "checkfitsnotes.txt" in the /share/obs4/usr/a2010 directory.
During observing, do the following (see Data Monitoring section for useful tools):
- Monitor your data: watch particularly for power dips, GPS, birdies, and gaps in the data. Be vigilant for new sources of RFI; carefully check the waterfall plots and/or quicklook. Know the expected locations in the band of the "usual suspects". The FAA radar is usually at 1350 MHz (although they sometimes use 1330 MHz, out of our bandpass!) For example, strong RFI at 1367 and 1382 MHz is due to the Punta Salinas naval base radar, which should not be on during ALFALFA run. (see this link).
- Watch for WAPP errors and follow procedure to recover from WAPP errors below.
- Maintain a log file, with an entry for each new fits file that is written. The file should be in /share/obs4/usr/a2010/ ; if it's not there, make one using the format logYYMMDD. More instructions below.
- Check the declination on the monpnt window to confirm that it remains near the J2000 value (within a few arcseconds).
- If you abort a scan during the night with the "Abort" command in the Spectral Line Observing widget, you do NOT need to start a new fits file before observing again; CIMA does this for you.
- Sometimes the WAPPS are left in an uncertain state if a scan is aborted abruptly. Be sure to restart the WAPPs after aborting a scan-in-progress (i.e. after data taking starts), and then repeat the start up sequence starting with Load Configuration. Unfortunately this means that you have to readjust the power, so you really want to be careful not to abort needlessly.
- See observational troubleshooting list for other observing issues.)

Ending the Observations

When finished, abort the last scan, after a few seconds of integration time.
If running TOGS (if not, skip to next step); in Spring 2011, we usually run TOGS at the end of our runs.
- Restart the WAPPS using that option under Utilities. Note any error messages; if there are any, try again.
- Click on Command File Observing, and click the Load button. Choose and run the script command_galfacurpos_spr2011.cmd. This will again load the togs_11.conf file and IFLO setup, and begin the TOGS calibration at the current telescope position. Let the calibration finish (about 6 minutes) and then follow the exit procedure as normal.
Select Exit normally to exit CIMA.
Exit data monitoring programs.
Log out of observer2.
The operator will end the galspect process. (The procedure is to pull up the window running GALSPECT, and hit CTRL-C to exit. Logout of the galfa1 machine and then exit dataview.)

Recovering from WAPP errors

The only sign of this happening in CIMA is that messages appear in red in the CIMA log window. WAPP error messages include: WAPPs confused indicates a hardware problem (Note the messages WAPP (3) WARNING - Counts don't match for chip 1 and WAPPDATA-queue_agcdata: WARNING received duplicate AGC-datablock are not WAPP errors. a hardware problem on WAPP3. Occasionally there is no warning message from CIMA, but the countdown time on the aostatus window stops, meaning that the WAPPs are no longer taking data. Sometimes the problems go away, so try to restart the WAPPs and then the observing again.
Procedure:
- Abort Observation in, e.g., the observing menu
- Restart the WAPPS under Utilities
- Repeat start up process, beginning from Load Configuration
If this does not work, try again, up to a few times. If this still does not help, exit CIMA and start again. If the error messages keep appearing, it will be necessary to have the operator call an engineer to look into the problem. Currently, the WAPP experts are Giacomo Comes and Arun.
If you abort a scan during the night with the "Abort" command in the Observing menu widget,you DO NOT need to start a new fits file before observing again; CIMA does this for you.
Sometimes the WAPPS are left in an uncertain state if a scan is aborted abruptly. Be sure to restart the WAPPs after aborting a scan-in-progress (i.e. after data taking starts), and then make sure to reconfigure them. Unfortunately this means that you have to readjust the power, so you really want to be careful not to abort needlessly.

Data Monitoring

Mikael has developed a great package for telescope monitoring and data display. These programs read the data that are being broadcasted to the FITS files, and not the FITS files themselves. The screens are updated every 10 seconds.
On Dataview, click on the background and select from the menu:

Quicklook Data Display -- launches the ALFA on-line display. Click 'ALFA Pattern' and 'plot type - diagrams' to see all 7 beams
Observers Monitor -- opens a widget that gives the power levels. Normal power levels are coded in blue.
Array Rotation -- shows ALFA rotation angle - check at beginning of night.
ALFA Beam Position -- shows approximate positions of each beam; can set to RA,Dec (note: currently, the positions aren't quite perfect but still helpful.) Check at beginning of night.
Frontend Monitor -- launches the ALFA instrument monitor, showing voltages, currents, and temperatures in all beams (same display as to the right of the operator's station)..

You can open the same widgets by typing on an xterm on a linux machine:

alfadatawin -- waterfall plots in your color of choice. Too cool. Enough said.
alfaobswin -- power for each beam
alfamotorwin -- ALFA rotation angle
alfabeamwin -- beam positions
alfamonwin -- instrument monitor

Note: you can now pull up many of these widgets from CIMA buttons directly. See the CIMA website for other cool widgets developed by Mikael.

The program alfadatawin is currently only available on Linux machines. The other commands are available on other machines, too. Notice that you will not be able to start alfadatawin if the process is already running on dataview (started by one of the previous observers). If this happens, ask the telescope operator to contact Mikael, Arun, or Phil to kill the previous process(es).

Notes

After a cal, the WAPP power levels will spike and so the displays will momentarily change scale.
If you reset the WAPPs, you will have to exit and restart alfadatawin to get it to grab the online data again.
To log out of dataview at the end of the observing run, close the windows "politely" by hitting exit in each display window before logging out. This will make sure that the processes terminate.

Log files

It is very important to keep an accurate log of what happens, even to note "an uneventful night". To do so:

ssh to another machine as a2010
open a new file logYYMMDD in /share/obs4/usr/a2010 using your favorite text editor. Use a posted log file as the template.
record **anything** of note, especially
(1) the CIMA version number and the saved state (i.e., configuration file = a2010_az360_2009.conf or a2010_az180_2009.conf)
(2) any bad files,
(3) any power adjustments,
(4) unusual rfi etc. (see observational troubleshooting list to check
To quote Brian: I cannot stress the importance of keeping accurate logs. While the file conversion software works automatically, it is good to have a backup to check against when comparing scan numbers. Sometimes CIMA squeezes in extra scans at the beginning or end of the run, or has some other unpredictable behavior. It is also important to note "empty" files, as IDL will hiccup on these. The folks who calibrate/process the drifts you take will thank you for your carefully note-taking (and will be entirely justified in saying unkind things about you otherwise).

After Observing

After the run has finished, do the following things to keep the rest of the a2010 group informed about how the run is going.

Follow the instructions in the list of tasks to do after each run . Details about IDL portion are given below.
Check with the scheduling team about other mods that they or you should make. If anything important happened, send a message to RG and MH.

Reading Data into IDL and saving to Disk

Once a observing session is complete, all the FITS files can be converted into a set of IDL structures for future use; ask Brian about any details about this. It is probably best to do this at your "office" workstation at AO, and run the job on one of the fusion machines. Below, IDL> means "inside idl".

Login as user a2010. The ".idlstartup" file in ~a2010 is configured with the proper routines, which are located in /share/a2010/idl and its subdirectories.
Change to the data directory /share/a2010/data/{run_directory} (ie /share/a2010/data/07.02.17/
Filecreator needs as input a list of the FITS files with full path names that you wish to process. The best way to do this is
```
 ls /share/pserverf.sdb1/wappdata/wapp.YYYYMMDD.a2010.* > filesYYYYMMDD.txt 
```
This will put all the a2010 files from that date into the text file. Check your log to confirm the directory.
This next step is very, very, VERY important. You MUST consult the log to see what files should be removed from this list. If there are files that have no data (ie the files at the end of the run), they will cause the filecreator program to hiccup, and therefore should be removed from the list. If there are files you KNOW contain junk, remove them from the list. ALSO, if you run crossed over midnight, you may need to add or delete files. You can ls, grep, do whatever you want to make the list, just make sure that the list contains all the viable files you wish to process for a single observing run. Make sure that there is no blank line at the end of the list, or filecreator will crash.
Start up IDL in /share/a2010/data/{run_directory} by typing idl Make sure that you are on a fast machine (fusion00,aolc1,2,3,4,6,g,h,j,k,m,o,p), and that someone else isn't using up all the cpu (do a "top" to check that).
IDL>@wasinit2 (If this doesn't work, check that your idl account is set up properly)
IDL> .com filecreator
You are now ready to process a run's data. There are five different file types that will be created:
- sssssssss.sav - A 600 second d structure *.sav IDL file
- ssssssssCALOFbegin - The first record of a drift used as a cal off
- ssssssssCALOFend - The last record of a drift used as a cal off
- ssssssssCALON - A 1 second CALON
- ssssssssCALOF - A 1 second CAL off
IDL> filecreator, filelist
filelist is a string of the filename you created above, ie 'files20070223.txt'.
The files being created will be reported on screen as the job is done. In addition, the names of the CAL files will be written to the file 2007-MM-DD.CALlist. This list is important, as it will form the calibration structures from the procedures calib1 and calib2.
Example from February 23, 2007: There were 26 files made that night, so I created a list in the /share/a2010/data/07.02.23/ directory with ls /share/pserverf.sdb1/wappdata/wapp.20070223.a2010.*.fits > files20070223.txt and ls /share/pserverf.sdb1/wappdata/wapp.20070224.a2010.*.fits >> files20070223.txt, because the run crossed midnight. However, upon examining the log I see that files 15 and 11 were aborted. I removed these files from the list, as they contain no data. I then start IDL with idl while on one of the fast machines listed above. I compile filecreator as above, and then invoke the command:
IDL> filecreator, 'files20070223.txt'
IDL goes to work. When the job is done there will be many IDL save files, CALON files, and CALOFbegin and CALOFend files, the filesYYYMMDD.txt you created, and a new CALlist file.

Useful Links

This page is maintained by the members of the Cornell ExtraGalactic Group and their ALFALFA team friends
Last updated Sat Sep 10 15:23:41 EDT 2011 by martha