Before Observing

• The ALFA cover must have been removed. ALFA is covered during radar runs, so take a look at the telescope schedule beforehand. For instance, if it looks like ALFA had to be covered the day before your observations, make sure that time is allocated to remove the cover before your run starts (this can be done during maintenance or system check time). In any case, it is a good idea to double check with the telescope operator. Know the schedule a few days ahead of time; but also ASK!
• The ALFA motor must be switched on (this should be ok but be aware; if you have any doubts, ask the operator. It can be done from the control room.). If you have any doubts, look for ALFA rotation using the dome video monitors (you should be able to see it move). Also, the operator may need to turn on ALFA in software from the control room.
• Ask the telescope operator to move to the appropriate azimuth and approximate zenith angle for the night and, if necessary, to rotate ALFA into place. Ask him/her to do this before starting GALSPECT. For the a2010 project, the telescope is fixed at AZ= 360 for Declinations < +16 degrees, and at AZ =180 for Declinations > 20 degrees. The scenario is still be determined for other values. (Note that we use azimuth=360 rather than azimuth=0 to avoid wrap issues. Note also that Az=180 is equivalent to Az=540: choose the value that has the smallest movement from the current position.)
  Starting January 2009, the azimuth should be set to 360 since we are mostly observing south of zenith. But, that may change in February 2009, so be sure to CHECK THE SCHEDULE.
  
• Be sure the operator returns telescope control to you. Note that once you start up CIMA, you'll take over control of the telescope motion and it will stop. So wait to start up CIMA. In the meantime, have the operator start up GALSPECT (see below).
• Set up the TOGS starting position. Open up the catalog file /share/obs4/usr/a2010/a2010.cat and edit the entry called "galfasrc" to the new right ascension and declination for that night. As exact start times and slew times can change, the observer needs to be ready to adjust the entry by estimating it as the previous observer is finishing, and again during setup if there are delays, when the telescope will be on the meridian. This needs to be done as quickly as possible. We suggest also that you take a look at the TOGS setup document (provided by the TOGS group) to understand a bit about their setup.

TOGS GALFA setup - the operator will carry out this step.

• Login to any machine in the control room to begin the GALFA startup.
• Secure shell to the dataview machine: ssh -l guest dataview
• Secure shell to the galfa machine: ssh -i galfa_key galfa@galfa1
• Check if ghost galspect processes are running by typing ps. If processes called gdiag are present, kill them by typing the command kill followed by the process number.
• Warm up GALSPECT via typing /var/diag
• Type the command: /var/levels_togs. If you get the message LO2: Set failed, got back: ERROR setting freq, then you need to start the gpib on the wapp server. To do this, start CIMA and click Restart GPIB under the Utilities menu. Go back to the xterm logged into galfa1 and again type /var/levels_togs. You should not receive an error.
• Enter the command /var/togs. This will run for the rest of the night, so minimize the windows.
  Note: if the operator is not able to get GALSPECT to run, he/she should call the TOGS contact for the night.

Datataking

• Quickly log into OBSERVER2 as "dtusr" (password taped to the monitor).

• When ALFA is in place and you are close to the target position, start CIMA either by clicking with the right button of the mouse on the background or by typing "CIMA --X" in an available window. When you start CIMA, slewing will stop until you issue the observe command, and if you are not close to target position, you will have to wait. The CIMA version selection window will pop up. Select the version we are currently using, the "The Smart version of CIMA (Ver 3.0.06 2008-10-24 20:28)". The Start New CIMA window will pop up; read below.

• Start New CIMA Session
  • Project number: a2010. This is important. Your settings and log files will be saved (if you decide to do so) in the corresponding project directory, which for us is /share/obs4/usr/a2010/. (note: even if you type "A", CIMA will change it to "a" when you click "Line")
  • Observer: Your name for entry CIMAlog and FITS headers.
  • Select Line observing mode.
  • Click Accept and CIMA will start up.
    If you are using a new version, the CIMA observing session preferences set-up window may appear; alternatively, you may need to set the preferences again using the widget under the "Utilities" menu. Please do NOT change the preferences (which determine what goes in the CIMAlog file) without consulation of Martha. Note: they have been saved to the file .CIMA_prefs_1.0, and are currently set to:
    • Write daily log-files:     Yes
    • Write log-file also for off-line sessions:     No
    • Show the datataking status window:     Yes
    • Warn if the telescope comes close to the limits:     Yes
    • Keep observations going if a WAPP fails:     No
    • Help-windows only stay open as....:     No
    • Color scheme for log messages:     Full
    • Level of information shown in log display:     Info level 5
    • Level of debug messages shown in log file:     No debug messages
    • Message format to use in the log window:     Long
    • Message format to use in the log file:     New
    • Font size to use:     Medium

    Now, several windows will pop up: the CIMA Observation Status window, the CIMA Log Display window, the CIMA main menu (observing session), the CIMA executive ( red font) and a widget entitled Receiver selection menu. The CIMA main menu (which should display the correct version number, now "Version 3.1.02   'smart'", is the main window and should always be open. The buttons therein open new windows, which allow you to control the telescope. These windows will be indicated with orange fonts in this document; all the other ones will be red. (Note: we need to check this color coding stuff!)

• Receiver selection
  Select ALFA, Quick Tsys=No, Rotate Receiver=Yes, Retrack=Yes then click on Accept. Make sure that ALFA shows up in the AOSTATUS monitor (to your left on top if you are in the control room). Keep an eye on the CIMA Log Display and don't move to the next step until the previous has completed; if you click too fast, you run the risk of getting an "executive busy" command.

• Click Utilities in the CIMA main menu, Restart the WAPPS. When the output to log window is complete, close the Utilities window.

• If running TOGS, immediately click on Command File Observing, and click on the Load button. You may have to click to view "All Files". Choose the file command_galfasrc09.cmd. The file should appear in the right hand box. Be sure that "Run Command File" is set to Once and "Start From" is set to First line. Click Run to run the file. The Observing Menu Window will open; it will indicate that the Observing Mode is "smart frequency switching". The telescope will slew to the position you just set for 'galfasrc' in the a2010.cat file, which should be very close to the meridian. As soon as it gets there it will load the togs.gui file, their IFLO setup, and begin about 6 minutes of frequency switching calibration. One only needs to let it run. During this time, you can use the waterfall dataview application on the dataview machine to see that data are being taken. You may also open the log files if you wish.
  Note: if the GALSPECT spectrometer is not set up within ten minutes of their allotted time (so that the ~7 mins of calibration cannot be run before A2010 is scheduled to start), do not run the TOGS calibration if the prime A2010 map LST range is reached. However, even if you don't run it at the beginning, run the TOGS calibration at the end, if there is time and GALSPECT is running.
  As soon as "observing menu" appears it can be used to verify settings and then closed. The "command file" window cannot be closed until the run file is finished, at which point click Close .

• Restart the WAPPs under Utilities as above.

• Click on Load configuration in the CIMA main menu.
  • Select a2010_az360_2009.conf (or the "180" equivalent, as appropriate). You will see a summary of its "contents" appearing; check that everything looks ok to you. This file now sets up just about everything you need. It fires the cal at the end of each scan ("loopend-OFF"), and adopts the topocentric reference frame. This will set most of the parameters, but you must still apply the configuration in each of the widgets described below. Select the appropriate file and click Load and apply. A window warning you to "wait" will pop up while the configuration is taking place. The log display should indicate that things are happening and the CIMA observing menu window should pop up with the observing mode set to Fixed azimuth drift map. You should also see the basic observing setup indicated: # loops=99, strip time=600secs, Azimuth=360.0 (or 180) degs, ALFA rot angle=19degs, startposition=(the first source in the file), drift type=Follow Dec J2000, Calibration type=Hcorcal, Calibration time=1sec, Calibrate when=On@loopend, Add extra cal=No, Adjust power=First loop only, Loops per file=1.
    Note: in general, the configuration file depends on the azimuth: a2010_az360_2009.conf (Azimuth 360) or a2010_az180_2009.conf (Azimuth 180/540).

• When you first start up, the start position will be set to galfasrc. Click on it to select the proper drift from the a2010.cat file. If you do not need to adjust the RA (so that the drift starts on time), you should be ready to roll, so hit OBSERVE. However if you need to adjust, or you want to check to be sure you have the right starting coordinates, click on Pointing Control. Here you can set the ALFA parameters and point the telescope. If you have run TOGS, ALFA should be in place and the telescope should be close to the correct azimuth. If you have not run TOGS, double-check that ALFA is in place. The telescope won't move until the receiver floor has finished rotating (to ALFA, from some other receiver).
  • The source "catalog" a2010.cat will be already loaded. Check at the top of the window if this file is loaded, and, if not, load with the "Catalog" button (on the top left; find it by clicking on Project catalogs). If you have run TOGS, the "Source" will be listed as "galfasrc". You need to change it.
  • In the central box, you will see a list of the Visible sources. If you don't see the one you want, click on All sources Choose the appropriate drift***p* "source" corresponding to the current observing block from the list of drifts. You can also type the coordinates of the source (RA and Dec); if you do so, be sure to use epoch "equatorial - J2000".
  • If necessary, update the RA to the current LST. This can be done in a2010.cat before opening the Pointing Control menu.
  • Click Accept and continue with the following while the telescope points.
  • Check that velocity is set to 0, reference frame to topocentric and type to optical.
  • If for some reason you need to, you can update the ALFA rotation using the ALFA Rotation Control Window. In the menu that pops up, set Rotation angle to +19.0 (degrees) and hit Send. You can actually see if ALFA is rotating by looking at the TV camera in the control room (ask the telescope operator to show the appropriate camera). The blue number next to the "Send" button should change to your entry once ALFA reaches that position. (For drifts at 0 or 180 degrees azimuth, the correct angle to give equal Declination beam spacing is thought to be 19 deg. At azimuth 90 deg, it is thought to be -14 deg.) Once ALFA has rotated into position, click Close . (Note that ALFA should be at 19 if you have run TOGS, but that we do this step to make sure.)
  • Check in the Pointing Control window, that the "ALFA center" is set to 0 to choose beam 0 as the ALFA center.
  • Note: if there are problems with ALFA not responding (e.g., it doesn't rotate), click Restart ALFA under the Utilities menu.
  
  
• If necessary (for example, if you restarted the WAPPs) or you want to check, click on Receiver IF/LO selection menu
  • Make sure that Receiver=ALFA, Filter choice= Full width, IF/LO path=ALFA, Center frequency =1385.0MHz, Backend=WAPP, WAPP bandwidths=100MHz, Configuration type= Fast, Radar blanker=OFF.
  • When ready, click Accept.
  • Not that rxview window will not update all IF/LO selection until data taking begins, do not use as a check.
  
  
• If necessary (for example, if you restarted the WAPPS), click on Backend Control Menu   Note: Not needed if you loaded the configuration but is needed anytime you restart the WAPPS!
  • This opens the Configure Wapp ALFA window. Check that all WAPPs are using 2 channels and 3 levels and at 100MHz.
  • Accept this configuration.

Beginning Observations:

• CIMA Observing Menu This Observer's Widget should still be open since you didn't close the resulting window after completing the Load Configuration. This is the window you'll be using most of the night. Note it also has buttons that allow you to start a "new file" and "adjust power".
  • Click on the button to the right of the start position marked drift#p# and select proper drift from the a2010.cat, check the RA is set ahead enough for the starting RA, check declination, and click use now and then click accept. Note: This menu overrides the pointing control menu but the converse is not true. Changes in the pointing control menu do not control those in the observing menu.
  • The rest of the values in red should be quickly checked. Correct values:
    • The observing mode is Fixed azimuth drift map
    • Number of loops is set to cover length of run. Easiest to leave it at the default 99.
    • Strip time is 600 secs.
    • Azimuth position should be either 180 degs or 360 degs. (Note 180 is equivalent to 540 and telescope will know to go to the closer angle, so you do not need to change to 540.)
    • ALFA rot angle is 19 degrees (at least for now).
    • Start position is the drift you want; if not, click on it to load another from the catalog "a2010.cat" or to update the R.A.
    • Drift type is Follow Dec J2000
    • Calibration type is Hcorcal
    • Calibration time is 1 sec
    • Calibrate when is On @ loopend
    • Add extra cal is No
    • Adjust power is First loop only
    • Loops per file is 1 loop
  • Click Adjust power Wait until output in log window confirms power adjustment. Once observations have started, check power numbers Observer's Monitor (alfaobswin) to make sure OK (the numbers should be between 0.8 and 1.2)
  • Click Observe to begin the observation. Hopefully only a small countdown will remain, and the photon collecting will begin.
  • Note: You DO NOT have to start a new fits, CIMA does this for you
  • Check that the telescope is actually at the correct declination - look at the "Rslt" line in the monpnt window (the upper right window on the screen below the frequency band monitor). Dec value should stay within a few arcseconds of the drift value.
  • Check the FITS files:   Once you have actually started recording data, you should take a look at the data to make sure that it can be read in IDL. In another terminal, remotely login and start up an IDL session on one of the fast linux machines (e.g. fusion01,02,aolc1,2,3,4) so that you can run "quicklook" as a backup to the waterfall plots. Note these instructions make use of the aliases in the a2010 account.
    ssh -X a2010@remote.naic.edu (if you are observing remotely)
    ssh -X a2010@fusion01
    (or fusion02) cd /share/a2010/idl/eggidl
    idl
    @wasinit2
    quicklook,'filename'
    where 'filename' is the complete name of the last complete FITS file, e.g. '/share/pserverf.sda3/wappdata/wapp.20080822.a2010.0003.fits'. (The directory path is specified in the log files - look at the previous night's log.) You have the option to display the bandpasses for each beam, both pols of the first record of the file. Not as nice as Mikael's data monitoring, but quick and convenient for remote observers as an extra check or as backup if the data monitoring programs don't work.
    
    Or if you want to look at an arbitrary record, see the file "checkfitsnotes.txt" in the /share/obs4/usr/a2010 directory.

• During observing, do the following (see Data Monitoring section for useful tools):
  • Monitor your data: watch particularly for power dips, GPS, birdies, and gaps in the data. Be vigilant for new sources of RFI; carefully check the waterfall plots and/or quicklook. Know the expected locations in the band of the "usual suspects". The FAA radar is usually at 1350 MHz (although they sometimes use 1330 MHz, out of our bandpass!) For example, strong RFI at 1367 and 1382 MHz is due to the Punta Salinas naval base radar, which should not be on during ALFALFA run. (see this link).
  • Watch for WAPP errors and follow procedure to recover from WAPP errors below.
  • Maintain a log file, with an entry for each new fits file that is written. The file should be in /share/obs4/usr/a2010/   ; if it's not there, make one using the format logYYMMDD. More instructions below.
  • Check the declination on the monpnt window to confirm that it remains near the J2000 value (within a few arcseconds).
  • If you abort a scan during the night with the "Abort" command in the Spectral Line Observing widget, you do NOT need to start a new fits file before observing again; CIMA does this for you.
  • Sometimes the WAPPS are left in an uncertain state if a scan is aborted abruptly. Be sure to restart the WAPPs after aborting a scan-in-progress (i.e. after data taking starts), and then repeat the start up sequence starting with Load Configuration. Unfortunately this means that you have to readjust the power, so you really want to be careful not to abort needlessly.
  • See observational troubleshooting list for other observing issues.)

Recovering from WAPP errors

• The only sign of this happening in CIMA is that messages appear in red in the CIMA log window. WAPP error messages include: WAPPs confused indicates a hardware problem (Note the messages WAPP (3) WARNING - Counts don't match for chip 1 and WAPPDATA-queue_agcdata: WARNING received duplicate AGC-datablock are not WAPP errors. a hardware problem on WAPP3. Occasionally there is no warning message from CIMA, but the countdown time on the aostatus window stops, meaning that the WAPPs are no longer taking data. Sometimes the problems go away, so try to restart the WAPPs and then the observing again.
• Procedure:
  • Abort Observation in, e.g., the observing menu
  • Restart the WAPPS under Utilities
  • Repeat start up process, beginning from Load Configuration
  If this does not work, try again, up to a few times. If this still does not help, exit CIMA and start again. If the error messages keep appearing, it will be necessary to have the operator call an engineer to look into the problem. Currently, the WAPP experts are Giacomo Comes and Arun.

• If you abort a scan during the night with the "Abort" command in the Observing menu widget,you DO NOT need to start a new fits file before observing again; CIMA does this for you.
• Sometimes the WAPPS are left in an uncertain state if a scan is aborted abruptly. Be sure to restart the WAPPs after aborting a scan-in-progress (i.e. after data taking starts), and then make sure to reconfigure them. Unfortunately this means that you have to readjust the power, so you really want to be careful not to abort needlessly.

Data Monitoring

• Quicklook Data Display -- launches the ALFA on-line display. Click 'ALFA Pattern' and 'plot type - diagrams' to see all 7 beams
• Observers Monitor -- opens a widget that gives the power levels. Normal power levels are coded in blue.
• Array Rotation -- shows ALFA rotation angle - check at beginning of night.
• ALFA Beam Position -- shows approximate positions of each beam; can set to RA,Dec (note: currently, the positions aren't quite perfect but still helpful.) Check at beginning of night.
• Frontend Monitor -- launches the ALFA instrument monitor, showing voltages, currents, and temperatures in all beams (same display as to the right of the operator's station)..

• alfadatawin -- waterfall plots in your color of choice. Too cool. Enough said.
• alfaobswin -- power for each beam
• alfamotorwin -- ALFA rotation angle
• alfabeamwin -- beam positions
• alfamonwin -- instrument monitor

Notes

• After a cal, the WAPP power levels will spike and so the displays will momentarily change scale.

• If you reset the WAPPs, you will have to exit and restart alfadatawin to get it to grab the online data again.

• To log out of dataview at the end of the observing run, close the windows "politely" by hitting exit in each display window before logging out. This will make sure that the processes terminate.

Log files

• ssh to another machine as a2010
• open a new file logYYMMDD in /share/obs4/usr/a2010 using your favorite text editor. Use a posted log file as the template.
• record **anything** of note, especially
  (1) the CIMA version number and the saved state (a2010_az360.gui or a2010_az180.gui)
  (2) any bad files,
  (3) any power adjustments,
  (4) unusual rfi etc. (see observational troubleshooting list to check
  To quote Brian: I cannot stress the importance of keeping accurate logs. While the file conversion software works automatically, it is good to have a backup to check against when comparing scan numbers. Sometimes CIMA squeezes in extra scans at the beginning or end of the run, or has some other unpredictable behavior. It is also important to note "empty" files, as IDL will hiccup on these. The folks who calibrate/process the drifts you take will thank you for your carefully note-taking (and will be entirely justified in saying unkind things about you otherwise).

After Observing

• Follow the instructions in the list of tasks to do after each run .   Details about IDL portion are given below.
• Check with the scheduling team about other mods that they or you should make. If anything important happened, send a message to RG and MH.

Reading Data into IDL and saving to Disk

• Login as user a2010. The ".idlstartup" file in ~a2010 is configured with the proper routines, which are located in /share/a2010/idl and its subdirectories.
• Change to the data directory /share/a2010/data/{run_directory} (ie /share/a2010/data/07.02.17/
• Filecreator needs as input a list of the FITS files with full path names that you wish to process. The best way to do this is

   ls /share/pserverf.sda2/wappdata/wapp.YYYYMMDD.a2010.* > filesYYYYMMDD.txt 

  This will put all the a2010 files from that date into the text file. Check your log to confirm the directory.
• This next step is very, very, VERY important. You MUST consult the log to see what files should be removed from this list. If there are files that have no data (ie the files at the end of the run), they will cause the filecreator program to hiccup, and therefore should be removed from the list. If there are files you KNOW contain junk, remove them from the list. ALSO, if you run crossed over midnight, you may need to add or delete files. You can ls, grep, do whatever you want to make the list, just make sure that the list contains all the viable files you wish to process for a single observing run. Make sure that there is no blank line at the end of the list, or filecreator will crash.
• Start up IDL in /share/a2010/data/{run_directory} by typing idl Make sure that you are on a fast machine (fusion01, fusion02, aolc1,2,3,4,6,g,h,j,k,m,o,p), and that someone else isn't using up all the cpu (do a "top" to check that).
• IDL>@wasinit2         (If this doesn't work, check that your idl account is set up properly)
• IDL> .com filecreator
• You are now ready to process a run's data. There are five different file types that will be created:
  • sssssssss.sav - A 600 second d structure *.sav IDL file
  • ssssssssCALOFbegin - The first record of a drift used as a cal off
  • ssssssssCALOFend - The last record of a drift used as a cal off
  • ssssssssCALON - A 1 second CALON
  • ssssssssCALOF - A 1 second CAL off
• IDL> filecreator, filelist
• filelist is a string of the filename you created above, ie 'files20070223.txt'.
• The files being created will be reported on screen as the job is done. In addition, the names of the CAL files will be written to the file 2007-MM-DD.CALlist. This list is important, as it will form the calibration structures from the procedures calib1 and calib2.
• Example from February 23, 2007: There were 26 files made that night, so I created a list in the /share/a2010/data/07.02.23/ directory with ls /share/pserverf.sda2/wappdata/wapp.20070223.a2010.*.fits > files20070223.txt and ls /share/pserverf.sda2/wappdata/wapp.20070224.a2010.*.fits >> files20070223.txt, because the run crossed midnight. However, upon examining the log I see that files 15 and 11 were aborted. I removed these files from the list, as they contain no data. I then start IDL with idl while on one of the fast machines listed above. I compile filecreator as above, and then invoke the command:
• IDL> filecreator, 'files20070223.txt'
• IDL goes to work. When the job is done there will be many IDL save files, CALON files, and CALOFbegin and CALOFend files, the filesYYYMMDD.txt you created, and a new CALlist file.

Useful Links

• The A2010 project website
• The ALFALFA website.
• The Where in the World is...
• Observing checklist
• Remote Observing Checklist