Online monitoring of the telescope and system

updated 19sep14

Monitor programs
Logging programs
Dumping monitor and data files.
History

Online monitoring displays of the telescope are now available from Linux and Solaris computers on AOnet. The displays use the online information that is sent out over the local ethernet once a second (multi cast data in computer lingo) or data that is written to a file. Some of the routines are text based (tcl/Tk) while others generate graphics (via idl). There are options that allow you to change how the program appears. Since all of the displays are X based, you can send the output to any x display (via the DISPLAY environment variable).

Note:

table boxes in red are old 305meter data that is no longer actively being monitored

Running the programs:

logon (or slogin) to a Linux or Solaris machine on AO net.

Solaris: It will not work on the "pointing" computer in the control room (since it isn't on aonet).
Linux: The scripts are in /pkg/aosoft/common/bin. The executables have been compiled for the i386 fedora version (i'll get to the 64 bit version one of these days). If the routines are not found on your computer, make sure your path includes:

/pkg/aosoft/common/bin and /pkg/aosoft/fedora4/i386/bin.

Example:
monpnt
montel
montie
montie 8

Available monitor programs:

Text Based Progs	Options	OS	Monitor Function	Notes
monallsm	kill	L,S	start monpnt,monif,montel, and montie. Use smaller fonts.	1.
monif	all	L,S	The upstairs and downstairs IF/LO source:/share/megs/phil/svn/aosoft/src/ifMonGet.c	2.
monif2	all	L	if2 monitor as used by 12meter (excludes udc)
monpnt	all	L,S	The requested pointing positions, errors, etc. source:/share/megs/phil/svn/aosoft/src/pntMonGet.c
monpnt12	all	L	the requested pointing positions, errors, etc for 12meter. source:/share/megs/phil/svn/aosoft/src/pntMon12Get.c
monrcv	pos,fg,bg,font,kill	L,S	The dewar temperatures, voltages, and currents. source: /share/megs/phil/svn/aosoft/src/rcvMonGet.c see also rcvMonProgN.c
monrcv12	all	L	12meter dewar monitoring source: /share/megsphil/svn/aosoft/src/rcvMon12Get.c
monrcvtemp	pos,fg,bg,font,kill	L,S	The dewar temperatures. source: /share/megs/phil/svn/pdev/aosoft/src/rcvMonGet.c
monsitepwr	all	L	monitor site substation voltages, currents,power once a minute source /share/megs/phil/svn/aosoft/gen/sitepwrMonGet.c
montel	all	L,S	The azimuth, gregorian, and carriage house system source:/share/megs/phil/svn/aosoft/src/agcMonGet.c
monp12m	all	L	monitors the 12meter hardware info.. encoder positions,vel,currents, state source: /share/megs/phil/svn/aosoft/src/p12mMonGet.c
montie	all	L,S	A summary of the 3 tiedowns. source: /share/megs/phil/svn/aosoft/src/tieMonSum1.c
montie 12,4,or 8	all	L,S	A more detailed monitoring of an individual tiedown. source: /share/megs/phil/svn/aosoft/src/tieMonGet.c
montur	all	L,S	The turret floor in the gregorian receiver room. source: /share/megs/phil/svn/aosoft/src/turMonGet.c
monturtemp	all	L,S	display turret room temperature.
plattemp	none	L	print platform temperature then exit: dayNum:174 tm:14:40:10 temp:79.35
Graphic Programs
monplthght	kill , yymmdd	L,S	Monitor the platform height using the distomats (idl). The default is to monitor the current day. If a date is entered (eg 050709 ,yymmdd) then that day will be plotted. source:
monmottq	kill,pc, az,gr,ch,yymmdd	L,S	monitor the gregorian motor torques (idl). select az,gr, or ch to monitor (default is gregorian dome) yymmdd specifies the date (default is current day)
monrcvpl monrcvplvm	kill kill	L,S S	Plot 16K dewar temp for the current day. Update every 22 secs Same as monrcvpl but uses idl virtual machine (no license used)
monrcv12pl	see right frame.	L	monitor 12m dewar monitor info by plotting data. Usage:monrcv12pl kill -h -c -l -v minTemp maxTemp -t 16,70,case -b gate,drain,lna,cur -p see monrcv12pl -h for an explanation.
imgui	none	L,S	An idl gui to look at the hilltop monitoring data. Can be run from a Solaris or linux machine.
pnterrrot	--help	L	plot pointing error from platform rotation and platform height. Needs all 6 distomat measurements to work For how to use: pnterrrot --help
cgenstat	-h -n -d -t -c -p	L	Plot the cummings generator status for current month. Includes total power, fuel rate, fuel rate vs power, efficiency, and cumulate fuel usage. For a description and arguments: cgenstat -h source:/share/megs/phil/svn/aosoft/common/bin/cgenstat, ../etc/cgenstat.idl
cgendiag	-h -n -d -t -c -p	L	Plot diagnostic info for the cummings generators. This includes: total power, power by gen, oil pressure, oil and coolant temps, power factor, amps,and volts. source: /share/megs/phil/svn/aosoft/common/bin/cgendiag, ../etc/cgendiag.idl
tdplotsum	-h -d yymmdd -r "8 12" -t td	L	Call idl routine tdplotsum for the requested date and time tdplotsum -h for usage message
azencdif	-h -d yymmdd -n ndays -v v1,v2 -p	L	plot azimuth encoder difference averaged to 1 deg bins for a day. If multiple days requested, then overplot them uses idl routines agcazencdifinpavg and agcanzencdifplot azencdif -h for a help description of the routine

Notes:
For the OS column, L =Linux (32 bit), S=Solaris

monnallsm starts many programs so it does not support any options.
the default update rate for monif is about 4 seconds (since the if/lo changes very slowly).
The source for most of the starting scripts is in /share/megs/phil/svn/aosoft/common/bin

Program options:

The mon programs accept options that allow you to modify certain parameters. They are entered on the same line as the command. The options are:

pos +x+y

fg color

bg color

fl color

Some programs (montie, montur) will flash data values that are outside an allowable range. The default color used for flashing is red. The fl keyword lets you change the flash color to something else.

time secs

The number of seconds to delay between each update. This lets you slow down the i/o rate. It is handy for people logging in over slow network connections.

font fontindex

kill

How the displays work:

For each monitor there is a "get" program that connects to the multi cast data on ethernet, reads the data blocks that are needed, formats the output data, and writes this data as ascii to standard output. A wish script reads this data and displays it in the dedicated display window.

The "get" program binaries are in ~phil/Solaris/bin/ on solaris and /pgk/aosoft/fedora4/i386/bin on Linux . The source codes are in

Solaris: ~phil/vw/shm/Mon/(Agc,If,Pnt,Tie,Tur)/xxxMonGet.c
Linux: ~phil/Linux/filters/

If the "get" programs are run by themselves, then an ascii display will be updated on your terminal once a second with the data. An example is:

~phil/vw/Solaris/pntMonGet
start
trk :  onSrc      master:Gr      wrap:0
ast :  10:39:26.0 dut1Ms: -288   secM:38366
lmst:  08:35:03.9    mjd:52513   tolA:100000

Pos :  05:21:14.4  28:09:50.0 Cs:B Tm:imd
Off :
Rat :
Rslt:  05:24:23.5  28:12:31.8 Cs:J
ha  :  03:10:31.1
          az deg       za deg     totErr
cor :      0.1061      -0.0637
reqP:    112.0028      44.4967
curP:    251.7287       9.9129
errP:   -139.7259      34.5838    77704.3"

vHel:     27.7560 vGeo:-0.2867    km/sec
pend:   none
stat:\  OK
cumO:  0.00000  0.00000 deg
end

There is a single wish script that drives all of the monitor displays (~phil/vw/shm/Mon/monProg.sh). Each display has a configuration file in ~phil/vw/shm/Mon/Setups that customizes the window size, fonts, colors, etc.. Each monxxx above then has a line that looks like:
/usr/S2local/bin/wish -file /home/phil/vw/datatk/shm/Mon/monProg.sh tieMonSum.init $* 2>/dev/null &

The graphics programs run off of datafiles that are being written to disc. Idl is started from a script that contains the idl commands to execute. These scripts use the tcl script /home/phil/Sunos/bin/newProg.tcl to stop any old copies before starting the new one.

Dumping monitor datafiles

A number of routines have been written to dump the contents (or partial contents) of some montor/data files. They all run on Solaris and Linux. The linux programs are in the /pkg/aosoft directory hiearchy which is only available on the newer fedora machines.

program	OS
cordmp -c -d -f firstChar filename -c Only output the 1st scan of each calibrate pattern -d Output debugging info as file is read -f firstChar ..Prepend this char to each line output. filename .. Filename to output. Dump a summary of interim correlator hdr records to stdOut. It calls the program cordmprg.	S,L
dmphdrs -h -b -f filename -h .. print help msg and quit -b .. if you hit a bad hdr, keep on searching. Use for atm files where the start of the file is not a hdr. -f filename .. filename to dump Dump the headers from an interim correlator or ri datafile. Works for cordata, some atm data, and maybe the radar files. Calls the programs: stripVme and then printHdr.	S,L
masdmp filename Dump some information about a pdev fits file... still being developed. (uses idl).
tiedump -d devNum yymmdd blkstoSkip -d devNum .. tds to include in dump (1=td12,2=td4,4=td8). You can or them together. The default is 7 (all tiedowns). yymmdd .. date to dump blksToSkip .. number of seconds to before dumping. Negative numbers count from the end of the the file.
wasdmp filename Dump a summary of was (wapp) fits files. Uses idl.	S,L

Logging programs:

The table below lists some of the logging programs that write the logfiles containing telescope and site info.

DSHM_Mon=/home/phil/vw/datatk/shm/Mon
DSVN=/share/megs/phil/svn
DAOSOFT=/share/megs/phil/svn/aosoft/src
VWSTART=/home/online/vw/boot

For a more up to date list, checkout crrontab -l -u phil on megs3.. usually a copy is in ~phil/admin/Cron.megs3..

Function	Dir	program	cpuRunsOn	startup	discOutPut	Notes
vertex Data	DSHM_MON/Agc	agcLogD.c	observer	/etc/rc3/S99.local	/shar/obs1/pnt/log	agcLogDTmp.c is now running using /mnt from /mnt/obs2 instead of /share/obs2
vertex/pnt data	DSVN/pdev/pdev/datatk	scramLog.c	pdevs1	/etc/rc3.d/S99local	agcyymmdd.dat pntyymmdd.dat	dumps the scramblocks from pntMProg and agcProg (pnt, pnt1 vxWorks cpus). This holds a copy of the vertex info as well as all of the program info..
tiedown dat	DSHM_MON/Tie	tieLogD.c	observer	/etc/rc3/S99.local	/share/obs2/tie/log	tieLogDTmp.c is currently running. It writes to /mnt/obs2/tie/log
laser ranging	?		lprc dos machine		/share/obs5/lrpc/lr_data	pc logs to files once every 2 minutes this is alsocalled /share/lrpc, /home/lrpc
hilltop rfi	~phil/vw/intfM/Progs DAOSOFT/vwconvert/intfmon	intfMon.c	galfas2	/etc/rc.local	/share/rfidat/data/ IMYYMMDD.dat	this gets backed up to /share/rfi/data/ynn/imMM/
turret	DAOSOFT/gen	turLogD.c	aserv12	rc3.d/S99local	/share/phildat/turret	logs turret scramnet info to disc using the multicast packets.
sitepower	DAOSOFT/sitepwr	sitepwrlog	aserv12	rc3.d/S99local	/share/phildat/sitepwr	logs site power from substation once a minute.
cummings gen	DAOSOFT/cummings	cgenlog	galfas2	rc3.d/S99local	/share/phildat/cummings	logs generator info once a minute when they are running
12meter	daosoft/p12m/prog	p12mProg	galfas2	manual	/share/phildat/p12m/ logdata_yyyymmdd.dat	output 12m status once a sec. Just for debugging
dewar monitoring	daosoft/vwconvert/rcvmon	rcvMNProg.c	galfas2	rc3.d/S99local	/share/obs4/rcvm/rcvnN	this is the single pixel dewar monitoring
12m dewar monitoring	DSVN/aosoft/p12m/wbrcvr/monctl	p12mrcvm.c	galfas2	/etc/rc.local	/share/phildat/rcvm12m
pdevBufPool	DSVN/pdev/pdev/datatk/	bpMon.c	pdevsN	/etc/rc3.d/S99local/pdevctl	/data/pdevfiles	dumps state of pdev bufpools every 5 secs. Use to check for i/o write backups. Use script bpmonclr (pdev/pdev/scripts) to copy and then clear.. Need to then use pdevcmdall restart when
windmeter (on platform)	/share/wind	newindmeter.tcl	obsdisplay	/etc/rc3.d/S99local	/share/wind	monthly save files /share/phildat/wind idl routines @windinit yrly rawdat backup /share/phil/dat/wind/rawdat
weatherstation	runs on weather station. downloads via ftp.		weatherstation	on weathstat	/share/orion/	orion weather station on lidar lab. data results

History (top)

22jun08: monrcvpl. added options for dana...

-h print the options description message and exit
-l Make vertical scale logarithmic.
-t 16,70,omt which stage to monitor. (note that the 70K stage is currently not monitored).
-v minValue maxValue vertical scale for the display. (note: -l does not like minValue <=0).
(logarithmic scale) -v (min max values) -t (16,70,omt) to display.

 home_~phil