Hints for Level 1 Processing     Feb 2007

This page is a troubleshooting guide for problems or data peculiarities encountered during Level 1 Processing of ALFALFA data. It assumes that you have a basic knowledge of what you are doing. A list of problems encountered and solutions provided by experts is below: hopefully it will help some of us remember the answers to questions that we have already asked. As always, though, ask someone if you are unsure about how to proceed with your data!

Basic Procedure

AO computers:

• Fast AO Computers: fusion01-02,aolc1-4,6,g,h,j,m,o,p. Note that they are firewalled and won't pull up DSS/SDSS images, e.g. via "g" in flagbb; best to use a slower AO machine for this. luquillo, camuy, etc (in visitor's office) have lots of RAM and work fine.

No *list* files in data directory:

• The Problem: You go to copy *list* files from data directory into processing directory, but they don't exist.
• What To Do: Check that you have right path, etc. If you do, then it is possible that Filecreator crashed after processing the last fits file but before making *list*: this happens if there is an extra line at the end of files*.txt in the data directory.
  Check that all the .sav files are there: if they aren't, then filecreator will need to be run again and you probably want to ask someone about this. If all the .sav files are there, the make the *list* files yourself. Example:
  share/a2010-2/data/06.08.14>ls -l *CAL* > cal060814.list
  share/a2010-2/data/06.08.14>ls -l * > drift060814.list
  Edit the last file to remove *CAL* files, and proceed as usual.

Power levels adjusted during run for some reason (WAPP gap, 2 decs in 1 run)

• The Problem: Log file indicates that WAPP power levels adjusted: "WAPP gap" when caused by WAPP confusion, or power levels adjusted between 2 decs in the same run.
• What To Do: Cals will need to be split. See examples.
• Example Processing Logs: WAPP gap: 06.01.07, 2 decs: 06.05.16

CALIB1/CALIB2

Double Spectra

• The Problem: calib1 reveals a double spectrum in one pol. of 1 beam, or both pols. of 1 beam.
• What To Do: Note this in the log and proceed: check the CRAT plots carefully in calib2, to make sure there are no spurious points or jumps. A double spectrum in both pols of a beam usually means that it happened to be sitting on a continuum source when the cal was fired. A double spectrum in only in one pol means that a power fluctuation is the likely culprit.

Jumps in CRAT

• The Problem: calib2 reveals a "jump" in CRAT values in a particular beam or set of drifts.
  - In this example, there is a ~1.5K jump in beam 6.0 only. The bandpass showed a "double spectrum" and the Tsys was okay.
  - In this example, the jump is only ~0.4K in beam 3.1. You might also encounter a a dip of similar amplitude.
• What To Do: it depends on the size of the jump: for the 1.5K jump, you'd want to split the cals into sessions before and after the jump: as a general rule, you can do this as long as there is ~1.5 hours' worth of data (ie. >8-10 cal values) on each side of the jump. For the 0.4K jump and/or the dip, do not bother to split the cals: the calibration is only good to 5-10% so changes of this magnitude are expected. The final scale will be set by NVSS sources, so this will get calibrated out after the Level 1 stage.
• Example Processing Log: 06.08.14 (1.5K jump)

General FLAGBB Hints

AO machines and flagbb:

• Note re. DSS/SDSS feeds: Fast computers are fusion01-02,aolc1-4,6,g,h,j,m,o,p. But they won't pull up DSS/SDSS images with "g" in flagbb; best to use a slower AO machine for this. luquillo, camuy, etc (in visitor's office) have lots of RAM and work fine.

Useful IDL procedures to supplement flagbb

• a2010fv: Converts channel number to frequency for ALFALFA configuration.
• skyd: Generates a web page with links to useful databases near input coordinates. Input format is HHMMSSS+DDMMSS (note there is no decimal between in seconds of time!): you can get this easily from the inspect window.

Continuum plot and "h"

• The Problem: After you use "h" in flagbb to look for potential HVCs in a drift, the continuum plot on the RHS of the flagbb window doesn't display properly. Example: before "h" and after "h" (look at the continuum plot on the right)
• What To Do: type "i" to flag a region of your choice. Pick any lower left corner, upper right corner. Hit "q" at the prompt, and instead of entering the box corners type '0,0,0,0' and hit enter. This effectively erases the box you just drew, and resets the continuum plot.

Which peaks in the record plot to flag?

• The Problem: You see peaks in the record total plot below the RA-freq window in flagbb, but you aren't sure whether or not they are significant enough to flag.
• What To Do: Use "t", which identifies peaks above a reasonable threshold (use "u" to erase peaks identified in a previous use of "t"). In general, if "t" doesn't identify it and you can't see it in the RA-freq window, it isn't worth flagging. If either of these conditions isn't true, add a bad box.

Flagging RFI, etc with FLAGBB: examples

Difference between RFI and HVCs near Galactic HI

• Example: This drift has both RFI (56) and and an HVC (58) near the Galactic HI. They have similar morphologies, but you want to flag the RFI and not the HVC. How do you tell the difference between them? HVC will be in both pols of one beam and often in multiple beams. RFI is usually only in one pol. Using "h" to check for previous HVC detections can also help.

Standing Waves

• Examples: mild, really bad
• What To Do: See examples for appropriate bad boxes. Note "sw" in log. Standing waves will get calibrated out at the gridding/signal detection stage (ie. beyond Level 1), so if they're fairly regular there is no need to flag them. Get rid of anything worse than a simple sinusoidal variation.

GPS

• Examples: mild, worse, worse still, really bad
• What To Do: See examples for appropriate bad boxes. Note "GPS" in log, plus records in which the burst happened. Flag burst and any troughs before or after it. Mild standing waves okay, but flag really bad stuff.

Intermodular RFI

• Examples: bad
• What to do: Note "intermods" in log, but don't try to flag all of the peaks (use "t" instead). If it is really bad, you could downweight the whole board using "s".

Standing waves in the fiber optics of beam 1.0

• Examples: mild, worse (standing waves too)
• What to do: Note in log. If it is really bad (ie. you can see strong peaks in the record total plot), you could downweight the whole board: type "s", "a" for all records (if appropriate) and "2" to set the board weight to 1/2 (typical).

RFI bursts and other weird stuff

• Examples: not so bad
• What to do: See examples for appropriate bad boxes.