routines to access rfi rms data

Last modified: Wed Oct 25 13:19:34 2017.


List of Routines


Routine Descriptions

0README - RFIRMS ROUTINES

[Next Routine] [List of Routines]
NAME:
0Readme - rfirms routines
   These are the idl routines that compute fractional occurence of rfi
by frequency. It does this by computing the rms/mean for each frequency
channel in a scan. Any  rms/mean greater then a threshold value are 
called rfi. 

   Users may want to use rfi plthistloop to look at the data.

Computing the rms by channel from correlator files:
 --------------------------------------------------
       pfrms - compute rms spectra for every scan in a file.
               (corrms() is called to process each scan).
pfrmsinpfiles- take a filelist and call pfrms for every filename in the list.

Making histograms of rfi by frequency channel for rms data:
 ----------------------------------------------------------
rfihistscan    - compute rfi for 1 scan add to histogram. 
pfrfihist      - take a filelist and call rfihistscan for every scan in
                 each file.

Processing the monthly data:
 ---------------------------
rfimkhist      - for each month find all of the archived rms files and make
                 a histogram for each month (of the requested receiver range).
                 Store the histograms in idl save files.
rfihistlist    - called by rfimkhist to make the filelist for a month.
rfihistinp     - after making the monthly histograms this routine will make
                 the year to date histogram including all months.

Plotting histograms:
 -------------------
rfihistinp     - inputs a histogram save file.
rfiplthist     - plot a histogram.
rfiplthistloop - input and plot a range of histograms.

toUse rfiplthistloop:
 -------------------

   idl
   @phil
   @rfiinit
   rfiplthistloop,'lb',1,9 .. to plot jan-sep. see the documention for
                the arguments.

(See /pkg/rsi/local/libao/phil/rfirms/Readme.pro)


PFRFIHIST - COMPUTE HISTOGRAM FOR ALL FILES IN FILELIST

[Previous Routine] [Next Routine] [List of Routines]
NAME:
pfrfihist - compute histogram for all files in filelist
SYNTAX: numscans=pfrfihist(fname,freqmin,freqmax,histinfo,histAr,$
                 rcvList=rcvList,edgefr=edgefr,rejectfr=rejectfr,
             sigma=sigma,binstep=binstep,verbose=verbose,wait=wait,han=han,$
			   alfaBadBms=alfaBadBms)
ARGS:
       fname:  string . file containing list of filenames to process
    freqmin :  float  . min frequency to histogram in Mhz.
    freqmax :  float  . max frequency to histogram in Mhz.

RETURNS:
    histInfo       : {rfihistinfo} holds the histogram parameters used
    histart[nchn,3]:long histogram of [*,tot=0,rfi=1,reject=2]
           

KEYWORDS:   
         han:      if set then processed data was hanning smoothed.
   rcvList[]: int  if supplied then limit data to these receiver numbers.
      edgefr: float. fraction of each band that is discarded on each edge
                     (do to filter rolloff). default: .05
    rejectfr: float. If the fraction of bad channels in a single rms 
                     bandpass exceeds this fraction, then rms bandpass will
                     not be included in either histogram. default=.5
       sigma: float. The number of sigmas above that defines an rfi event.
     binstep: float. The binsize in mhz. default: .1 Mhz.
                    if not provided then log to default log file
alfaBadBms[2,8]: int bad beams for alfa.. [pol, bm] 
                    8 beams since last is duplicated in wapps.
                    putting a 1 in a location will ignore this
                    pol,beam in the histogram. use for bad beams..

DESCRIPTION:
   This routine will create a histogram of rfi versus frequency. fname 
is an ascii file that contains files to process. Each of these files contains
rms spectra that have been processed by corrms (usually via pfrms()). 
An rms spectra has had the rms/Mean computed for each frequency channel
in a scan (typically sampled at 1 or 6 seconds).
Rfi is defined as any rms greater than a default value (set by the sigma
keyword). 

A histogram of the total number of counts, and the total number of rfi's
found is returned. The histogram bins are centered at the start,end bins.
If the data resolution is greater than the histogram resolution, then 
all of these data channels map into 1 total count. Any (and all)data channels
in the group will count as 1 rfi count.

The expected sigma is computed using the time, channel bandwidth, double 
nyquist, and level sampling. The bins are assumed to be rectangular.

A fraction of the bandpass is discarded at each edge because of filter
rolloff. Bandpassed will not be included if the fraction of bad channels
exceeds a predetermined value. 

(See /pkg/rsi/local/libao/phil/rfirms/pfrfihist.pro)


PFRMS - COMPUTE RMS/MEAN BY CHAN FOR EVERY SCAN IN FILE

[Previous Routine] [Next Routine] [List of Routines]
NAME:
pfrms - compute rms/mean by chan for every scan in file
SYNTAX: numproc=pfrms(filein,fileout,rembase=rembase,minrecs=minrecs,
                      excludepat=excludepat,zerorate=zerorate,lunLog=lunLog,
                      han=han,doplot=doplot)
ARGS:
       fileIn  :string     name of input file
       fileOut :string     name of file to store rms/mean by channel
KEYWORDS:
       minrecs : int       minimum number of records needed in a scan
                           to process it. (default=5).
       rembase : int       if set then remove linear baseline before rms
   excludepat[]: strings   array of patterns to exclude.
   zerorate    : int       if set then require zero rate to process scan.
   han         : int       if set then hanning smooth the input data.
   lunLog      : int       lun to append ascii log info to.
   doplot      : int       if set then plot each rms after computing
RETURNS:
   numproc :   int         number of scans output.
                        -1 error opening input file
                        -2 error opening output file
DESCRIPTION:
  For each scan in fileIn compute rms/mean by channel. Output the
rms/mean by channel along with the header to file fileOut. This data
can be read in later with corget(lun,b,/noscale);
The data output will have the same format header,data except that:
1. the freq data is replaced by rms/mean by channel
2. there is  only one record per scan.

(See /pkg/rsi/local/libao/phil/rfirms/pfrms.pro)


PFRMSINPFILES - COMPUTE RMS/MEAN FOR ALL FILES IN FILELIST

[Previous Routine] [Next Routine] [List of Routines]
NAME:
pfrmsinpfiles - compute rms/mean for all files in filelist
SYNTAX: numscans=pfrmsinpfiles(fname,outputdir,rembase=rembase,doplot=doplot)
ARGS:
       fname:  string . file containing list of filenames to process
    ouputdir:  string . directory name for output files.

KEYWORDS:   
   rembase : int   if set then remove baseline before rms computed
   minrecs : int   minimum number of recs in scan need to process a scan.
                   defaults to 5.
   doplot  : int   if set then plot each rms as it is computed
   logfile : string if provided then log ascii status to logfile.
                    if not provided then log to default log file

DESCRIPTION:
   Call pfrms for all filesname in the file fnames.
fnames is a file that holds 1 filename perline with.
Process each of these files by calling pfrms. Use outputdir/basename
as the output filename.
This routine will automatically hanning smooth, set zerorate true,
and excluce the patterns corcrossch

(See /pkg/rsi/local/libao/phil/rfirms/pfrmsinpfiles.pro)


RFIHISTINP - INPUT AND SUM A TIME RANGE OF HISTOGRAMS

[Previous Routine] [Next Routine] [List of Routines]
NAME:
rfihistinp - input and sum a time range of histograms
SYNTAX: rfihistinp,frq1,frq2,mon,prefix,hall,hinfo,freq,dir=dir
ARGS:
	frq1: long - starting frequency in Mhz
	frq2: long - ending   frequency in Mhz
  mon[]: long - array of yymm months to process
 prefix: string - any prefix that was prepended to hist file name
			       (in case you were testing.. default is '')
KEYWORDS:
	 dir: string directory to find the histogram files. default is
			     /share/megs/rfi/rms/histdat/
RETURNS:
  hall[n,3]: long [n,0] total counts in n bins	
                  [n,1] total rfi counts in n bins	
                  [n,2] total bad spectra in n bins	
 histinfo:{rfihistinfo} contains info on data in hall:frqst,end,binwidht,etc..
 freq[n] :    float     frequency for each bin.

fileformat is:
   prefixhsav_f1_f1.yymm

(See /pkg/rsi/local/libao/phil/rfirms/rfihistinp.pro)


RFIHISTLIST - MAKE FILELIST FOR HISTOGRAM

[Previous Routine] [Next Routine] [List of Routines]
NAME:
rfihistlist - make filelist for histogram
SYNTAX: rfihistlist,yymmAr,inpdir=inpdirar,outdir=outdir
ARGS:
       yymmAr[]: int   dates to process. eg [0101,0102]
KEYWORDS:
       inpdir[]: string directories where the rms spectra files are located.
                 default:...
       outdir  : string directory to write the list of files to process.
                 default: /share/megs/rfi/rms/histdat/
DESCRIPTION:
   pfrfihist() computes a histogram of rfi versus frequency using the
rms spectra computed by pfrms(). It needs a list of rmsspectra files
to process. This routine will generate that list of files and write them
to disc. The user supplies an array of yymm values and the program 
searches the rmsspectra directories for all files that fall within each
of the yymm ranges (by checking the monyy part of the filename). For
each yymm it will write a file (hlistinp.yymm) to the output directory
that has a list of all rms spectra for this month.

(See /pkg/rsi/local/libao/phil/rfirms/rfihistlist.pro)


RFIHISTSCAN - COMPUTE RFI FOR RMS SPECTRA. ADD TO HISTOGRA.

[Previous Routine] [Next Routine] [List of Routines]
NAME:
rfihistscan - compute rfi for rms spectra. add to histogra.
SYNTAX: rfihistscan,b,hInfo,htotcnts,hrficnts,hrejcnts,han=han,$
                   verbose=verbose, wait=wait,badSbc=badSbc
ARGS:
       b        : {corget} corget structure containing rms info for a scan.
                   usually computed from routine corrms().
       hInfo    : {rfihistInfo} structure telling routine how to bin the 
                   histogram. User fills in before calling.
   histAr[nchn,3]: long array containing the histogram (user allocates).
                  [*,0=total,1=rfi,2=rejected]

   han          : if set, then the input spectra were hanning smoothed
                  prior to computing the rms.
 badSbc[2,8]    : if a one in a position, then ignore this
                  sbc,pol... Used to get rid of alfa beams that are bad

(See /pkg/rsi/local/libao/phil/rfirms/rfihistscan.pro)


RFIMKHIST - MAKE RFI HISTOGRAMS

[Previous Routine] [Next Routine] [List of Routines]
NAME: 
rfimkhist - make rfi histograms
SYNTAX: rfimkhist,nm,yymm,sigma=sigma,badfr=badfr,prefnm=prefnm,$
               outdir=outdir,mklist=mklist,frange=frange,$
               frqstep=frqstep,verbose=verbose, wait=wait,han=han,$
				alfaBadBms=alfaBadBms
ARGS:
   nm    : string .. receiver name to process
   yymm[]: int array holding mon,year to process

KEYWORDS:
   sigma : float number of sigmas to clip at.
   badfr : float  fraction before we call it bad spectra
 frqstep : float  freq step (def .1 Mhz)
 frange[2]: float  min max freq (mhz) for histogram. default is from table
		           below.
 verbose :        if set then output clipping info
 wait    :        if set then wait after each spectra for keyboard input
 outdir  :string  for inplistfiles and output histogram files.
 mklist  :        if set then make the input filelists.
 prefnm  :string  prepend to each output file name
 han     :        hanning smoothing was used in rms computation.def=yes
                  if not, then set han=0
alfaBadBms[2,8]: int bad beams for alfa.. [pol, bm]
                    8 beams since last is duplicated in wapps.
                    putting a 1 in a location will ignore this
                    pol,beam in the histogram. use for bad beams..

(See /pkg/rsi/local/libao/phil/rfirms/rfimkhist.pro)


RFIPLTHIST - PLOT HISTOGRAMS OF RFI RMS DATA

[Previous Routine] [Next Routine] [List of Routines]
NAME:
rfiplthist - plot histograms of rfi rms data
SYNTAX : rfiplthist,histar,histinfo,ln=ln,title=title,over=over,xp=xp,$
                   sclln=sclln
ARGS   :  
   histAr[nchn,3] : long hold totalcnts,rfi, rejected histograms
   histInfo       :{rfihistinfo} hold histinfo.
KEYWORDS:
  title: string if supplied then title for plot
  over :        if set then overplot on last plot.
   ln  : int    if supplied then line number for labels (colors, sigma,etc).
   xp  : float  0 to 1. xoffset for labels
   sclln:float  to space label lines (default 1.)
   cs   :float  character size scale for label. default:1.
   

(See /pkg/rsi/local/libao/phil/rfirms/rfiplthist.pro)


RFIPLTHISTLOOP - LOOP PLOTTING HISTOGRAMS FOR A GROUP OF MONTHS

[Previous Routine] [List of Routines]
NAME:
rfiplthistloop - loop plotting histograms for a group of months
SYNTAX: rfiplthistloop,rcvr,yr,m1,m2,delay=delay,_extra=e
ARGS:
   rcvr:string     rcvr to plot . 'lb'
     yr:int        year to plot, 2001..
     m1:int        first month to plot (1..12)
     m2:int        last  month to plot (1..12)
KEYWORDS:
   delay:  float   seconds to delay between plots. If < 0 then prompt user
                   to continue after each plot.
   _extra:  keywords=values .. pass these to rfiplthist

DESCRIPTION:
   Call rfiplthist for a group of months of a particular year. You
can set the horizontal scale to zoom in on a frequency region (you may
then have to set the vertical scale so that it is scaled correctly for
all months. The default delay between plots in 1 seconds.

   The data is read from the files in /share/megs/rfi/rms/histdat. If the
histograms have not been made, they can't be plotted.

EXAMPLES:
   rfiplthist,'lb',2001,1,9  .. plot jan-sept with 1 sec between plots.
   hor,1300,1450
   ver,0,1
   rfiplthist,'lb',2001,1,9,delay=-1 .. plot jan-sept 1300-1450 Mhz 
                                     prompt user to continue after each plot.

(See /pkg/rsi/local/libao/phil/rfirms/rfiplthistloop.pro)