Vertex drive systems (az,ch, gregorian).
Daily monitoring of az,dome, ch status.
Average motor torques by month.
A brief introduction. (top)
The vertex drive systems control the
azimuth, gregorian, and carriage house. The control hierarchy
- The motors are controlled by the DCS . It is an analog control
- The DCS is commanded by the Siemens cpu928 PLC controller. The
PLC controller sits in a Siemens crate that also contains the
LCU, PCU interface, and i/o boards. The PLC is programmed
with the STEP 5 programming language from Siemens.
Internally the PLC is a 186 computer. The software runs on
a 10 ms update cycle.
- The PLC communicates with the LCU (a 486 computer running MS
DOS. also called a Siemens cp581) via a shared memory that sits
on the LCU. This interface is proprietary to the Siemens crate.
- The LCU runs MS DOS 6.2. It talks to the PLC via the shared
memory. The LCU's task is to take external requests,
process them, and then pass them on to the PLC. External
requests can come from:
The LCU contains a ISA bus that has an irig card, hard/floppy
discs, and an ethernet card. Onboard the LCU is a 1 Mb silicon
disc. Normal operation of the LCU uses the silicon disc. The
system can also be run from the hard disc (switch able from the
prom monitor setup).
- A keyboard / monitor connected to the LCU
- The OCU (located in the control room). It interfaces
to the LCU via a serial port.
- The AO computers interface to the LCU via
a dedicated ethernet link.
The table below contains the most recent
electronic version of the manuals from klaus (vertex). I've
included the original .doc file as well as an .odt version
(created by libre office). I need to check if the paper versions
have more recent updates.
The original location of the manuals was /share/megs/vertex
|operation and maintenance manual
|LAN interface description
(i have a corrupted version of this..LAN5034.TXT
siemens simatic s5-135u
Description of Internal interfaces
|single line diagram and
control circuit diagram
(the electrical schematics)
|list of parts
list of plc routines
device hardware interface pinouts
Each morning summary plots of the previous
days status are made (via a cronjob on megs2).
How the status bits respond to
The plots show how
vertex status bits respond to various commands (.ps) (.pdf):
- The system started out under vme control with the motors on.
- The command sequence was:
- pwr on (even though it was already on)
- Program track 3 (dome and azimuth)
- These commands are the pntpwrdip sequence.
- Plot 1 General status. These commands did not change
any of these bits. In particular, the stop command does not turn
off drive power (the main contactors remain closed).
- Plot 2 Dome axis status. The only affect is the stop
command. It affects:
- BrkRel: The brake release bit goes from released to brakes
- drvEnable: The amplifier drive enable bit goes from enabled
to not enabled. This occurs because the plc drives the drive
enable request to false once the brakes are on.
- Plot 3 Dome axis mode, equipment status. The only
affect is that the system goes from program track mode to stop
mode when the stop command is sent. The dome contactors do not
open on a stop request.
- Plot 4 Dome amplifier status: The amplifier status
does not change because of these commands. The amplifiers remain
ready and there are no faults generated.
On 10apr 17 the following sequence occurred:
- 16:xx az encoder 1 fails. system switches to using azimuth
encoder 2 for pointing.
- The az encoder 1 value jumped down by about 900 degrees.. It
stayed with this large offset
- There was a close by lightning strike (< 2 secs flash to
thunder) around this time..
- system continued to run using az encoder 2 for pointing
- I don't know if the operator issued a reset or not..
- 20:00 (about) site loses power, generators come on.
- 20:01:48 azimth encoder 2 fault arrives.
- This was probably caused by the loss of power.
- 20:02:24 az encoder 1 fault is reset
- operator probably entered pntpwrdip.. this does a reset
- The system switches back to using az encoder 1. Az encoder 1
still has the value around -600 degrees.
- az encoder 2 fault remains on.
- 20:11:04 az encoder 2 fault is reset.
- the az encoder 2 value did not jump
- 22:xx osvaldo, willy working on encoders.
- they reset the encoder 1 value and start moving it.
- encoder 2 start to jump around during their testing.
- 23:00 -> 08:00 telescope left stationary
- 08:00 next day
- move telescope to az=270 with the pointer, and they reset az
The plots show what happened (.ps)
- page 1: plots for entire day
- top frame: az encoder vs hour of day
- black: az encoder 1 (normally used for pointing).
- red: az encoder 2
- The blue dashed lines show where encoder failures
(or recoveries ) occurred.
- az encoder 1 jumped around 16:00. az encoder 2 was stable
- bottom frame: az encoder2 - az encoder 2 (in deg)
- the difference jumped at 16:00 (because encoder 1 jumped).
- the stuff around 22:00 is when willie and osvaldo were
moving the telescope.
- page 2: the azimuth status for the day
- 16:0x .. enc1 flt occurs and stays on till 20:00
- telescope was now using encoder 2 to do the pointing
(probably 10s' of arc sec pointing error).
- 20:00 encoder 2 flt, encoder 1 reset.. probably from
the power dip.
- page 3: az status blowup 20:00 to 20:12
- 20.03 servo failure.
- enc1flt was already active, encoder 2 had a fault. this
was probably from the power dip
- 20.04 enc1 flt reset
- must have been the operator entering pntpwrdip after the
power dip. this reset the enc1 fault
- It did not bring the encoder 1 value back from 1500+
- page 4: blowup of encoder positions around 20:00
- top frame: az encoder values vs hour of day
- black enc1, red enc2.
- green line.. power dip.. enc1 goes 1600-> 0 probably
since system has no power
- blue line, enc1 reset. system using encoder 1.
- purple line: enc2 flt is reset.. (not sure what did this).
- middle frame azimuth velocity from amplifier
- at blue line , velocity goes to -slew rate (-.41deg/sec).
- The position in frame 1 is decreasing (though it is
- So the telescope is moving.
- At 20.14 the drive is disabled (see previous status
page), motion stoppped. This was probably the
operator stopping the motion because of the strange
- bottom frame: size of steps made by encoder.
- This shows the 1 second differences between encoder values
(we read them once a second).
- The units are now encoder values
- I've overplotted green dashed lines showing power of
2 changes. a few of the steps are close to a power of 2 (a
bad bit?). I think the values read back from the
encoders are grey codes..
- 16:00 encoder 1 jumps by about 900 degrees. this may have been
from a lightning strike
- 16:00 to 20:00.
- The system switched to use encoder 2 from 16:00 to 20:00 .
- This will introduce pointing errors up to 1 arc minute
(depending on azimuth).
- The value from encoder 1 remained way off. it later
jumped up by about 2500 deg.
- at 20:00 there was a power dip.
- enc2 had a fault ( which is normal for a power dip.
- the operator must have done a reset to clear the power dip
- enc1 fault reset, but the large offset value remained in the
- enc2 fault reset about 10 minutes later.
- enc2 did not have a large jump in value.
- 22:00 willie and osvaldo move the telescope
manually. The azimuth encoder value finally jumped back to 49
- 08:00 willie and osvaldo moved the telescope to
270degrees (with the pointer) and reset the enc1 to 270 degrees:
- both the dome and the carriage house were at stow when this
- az 1 encoder read:
- 1872693[5-8] counts at 270. [5-8] means this digit
jumped around between 5 and 8.
- The photograph shows the position of the az pointer when the
reset to 270 degrees was done (.jpeg)
- It looks like some offset was put in the enc1 . I wonder if
the encoder has a command that can cause it to reset to a
default value? Could the lightning have triggered this?. enc2
seemed to behave a lot better..(although it did jump while
willie and osvaldo moved it.
11oct16:when system returns to enc1 after
We did some tests to see when the azimuth drive
system returns to enc1 after an enc1 fault. The normal sequence is:
What we did:
- enc1 fault appears. yellow on the display
- system switches to using enc2.
- You can tell if enc1 is working or not by looking at the
encoder difference printed out.
The result is that after an encoder 1 failure, the system continues
to use encoder 2 until the operator resets the encoder1 fault:
- move to az 700 since large encoder difference
- trkinit lbw
- pnt tr 700 10.9999 -cx
- pos output: 700
- Test with failure during system stopped
- stop system
- remove enc 1
- enc1 failure
- motion failure (since large jump in position moving enc1 to
- error is red.
- position now 700.0463 (coming from enc2 values)
- encoder difference is garbage (since no enc 1)
- pnt reset 15
- gets rid of motion failure
- enc1 failure now yellow ... tells us it is using enc2.
- re insert enc 1
- encoder dif returns
- position still 700.0463 .. so still using enc2
- pnt mode pt 1
- re enable tracking mode az
- telescope moves.. since last prog track position was 700 and
enc2 was at 700.0463
- pos output now 700.. this is coming from enc2..
- pnt mode reset 15
- clears enc1 flt
- telescope position jumps then comes back to 700.
- it is now using enc1
- Test with failure occurring during system active.
- both encoder connected
- trkinit lbw
- pnt tr 700 10.9999 -cx
- pull out encoder 1
- causes enc1 fault and motion failure. error is red
- pos reads 700.0462
- pnt reset 15
- resets motion failure
- enc1 fault now yellow
- pnt pt 1
- go back to tracking mode
- telescope moves to position 700..so still using enc2
- pnt reset 15
- removes enc1 fault
- enc difference now valid. enc1 working
- telescope position jumps and then returns to 700 so now
using encoder 1.
- pnt tr 15
- Or pntpwrdip (will do the same).
10oct16: az 1 encoder fault (top)
Azimuth encoder 1 had encoder faults on:
When encoder 1 fails, the system switches to
encoder 2 for the pointing. The pointing model is made with
azimuth encoder 1. The difference encoder 2 - encoder 1 changes as
the azimuth goes 0 to 360 degrees. The maximum error is:
- 10oct15 12:22 to 12:27
- 10oct15 12:27:54 to 12:28:24
- 10oct15 14:50 to 11oct15 07:00 (when the
operator did a reset because of a power dip.
- za=10: .06deg*sin(10) = 37 arc seconds great circle
- za=20: .06deg*sin(20) = 74 arc seconds great circle.
We also record the difference between the 2
encoders (for the azimuth bending). If encoder 1 fails, then the
difference is fixed (since the value of encoder 1 is not
During the azencoder 1 fault, the az
encoder difference was still being recorded. So the encoder 1
faults looks like it was a glitch. The error message gets latched
until the operator resets it (so we can see that the error
On 11oct16 we verified that after an encoder 1
fault, the system continues to use encoder 2 until the fault is
reset. So we had degraded pointing 14:54 10oct16 till 07:00
The plots show the pointing error
from using az encoder 2 (.ps) (.pdf)
01oct16: az 1 encoder failure. (top)
Azimuth 1 encoder failed at 08:16:31 on 01oct16.
Willie replaced the encoder and then used the pointer at 270
degrees to align the encoder. The steps were:
- ch a stow, gr at 11 deg.
- moved ccw into 270 degrees.
- positioned with track mode: pnt tr 270 11 -cx
- Used tracking offsets till the pointer was lined up with the
- Willie then adjusted the encoder offset will we had 270
degrees on the readout
- Note: when he updated the encoder position, there would be
an encoder fault (because it jumped). You needed to do a reset
to get the updated value output.
- the final values were:
- there was a little jumping around in the last digit.
- Then moved to za=6.5 (jan16 swap) and still had 18726920
with occasional jumps 910->923 (wind?).
the encoder and moving it to the stripe, willie took some pictures
of the alignment:
- pict 1: looking down the
- pict_2: looking down the stripe
on the az ring girder. You see the pointer at the stripe.
- pict_3: another look
along the pointer.
We didn't have a ruler against the line so not
sure exactly how far it is off. The previous pictures showed that
the scribe mark is about 1 mm wide.
- The radius to the scribe mark is about 60 ft.
- a 1 mm error on the pointer/scribe mark is :( 1/25.4)/12/60
*radeg*3600=11.3 Asecs /mm (little circle)
- a 1 mm error at za=10deg = 2asecs
- a 1 mm error at za=20deg=4 asecs great circle
- We really need to align the pointer/scribe to the position
that was used when the model was made.
Down in the lab we opened the encoder. The
glass disc with the position info was broken.
This was the encoder that was installed 10jan16.
22jun15: az 1 encoder failure (top)
Az encoder 1 failed atround 18:00 22jun15. The
telescope continued to track (since it switches to use enc2
for the pointing).
22jun15 (.ps) (.pdf)
- the data covers 17:00 to 24:00. The failure occurred around
- the dashed vertical lines show when the enc1 failure occurred,
and when the telescope stopped moving.
- When encoder 1 fails, the system automatically switches to
use encoder 2 for pointing.
- page 1, bottom is the encoder difference enc1-enc2,. These are
actually narrow jumps. so the bad values were intermittent.
- page 3: the motion failure which stopped the telescope motion
occurred about 18 minutes after the encoder 1 failure.
status (blowup when motion stopped) (.ps) (.pdf)
- This is a blowup around when the telescope finally stopped
- page 3: the enc1flt went away (around 18.198). On the next
second we got a motion failure.
- My guess is that the system switched back to using encoder
1.. It probably then tried to use the current and previous
positions on enc1 to check for motion failure. Since the old
values were probably wrong, this probably generated the motion
failure and caused things to stop
status during failure (.ps) (.pdf)
- For reference, this shows the position of the dome
On 23jun15 around 10:00 osvaldo replaced the encoder1. It was
pretty dirty... things then started to work.
Aligning new encoder.
- The dome was at 6.5 degrees
- We moved the azimuth to 270.0000 degrees using program track
mode. then stopped the azimuth. (it was pretty windy).
- The encoder reads at 270 were:
- enc1: 1872690
- enc2: 1872599
- We placed a metric tape measure in above the alignment mark,
and then took a picture of
the alignment arrow (.jpeg)
- from the picture you can see that the arrow is about 1 mm to
the left of the mark on the ring girder.
17jan13:az encoder 2 replaced (top)
On 17 jan 13 aeronomy was doing azimuth spins
when encoder 2 failed. The encoder was replaced.
The plots show the azimuth encoder
difference before and after the replacement (.ps) (.pdf)
- Page 1: 17jan13 azimuth encoder difference before and after
- Top frame: encoder units
- bottom frame: degrees
- black is CW new encoder
- Red CCW new encoder
- green CW old encoder
- blue CCW old encoder
- The dome was at za=15 and the ch was at za=0
- Page 2: 22jan13 az encoder difference with new encoder
23jan12:az encoder 1 becomes loose. (top)
There was an az encoder 1 fault (the encoder used
for pointing) on 21jan12 during a world day. We continued running
for 2 days using encoder 2. On 23jan12 the coupling holding encoder
1 was found to be loose. The coupling
was tightened. The azimuth position at az=270 was not reset using
the pointer on the azimuth arm.
On 23jan12 around 11am Az swings were then done
to measure the encoder difference.
the plots show the encoder difference
for the swings (.ps) (.pdf):
- The motion was:
- dome=10.976 degrees, ch at stow.
- 270 to 0 at .4 degrees/sec
- 0 ->360 at .4 degrees/sec
- 360->270 at .4 deg/sec
- Page 1:The black trace is the clockwise motion. The red trace
is the counter clockwise spin.
- Top: azEnc1 - azEnc2 in encoder units.
- Bottom: azenc1-azEnc2 in degrees. The radius is about 60
feet so one degree is about 1 foot at the encoder radius.
- Page 2: compare az encoder dif with 19jan12 before coupling
- If the azenc1 moves and everything else remains constant,
then the encoder difference will also change. To check this,
an az swing during the world day (19jan12) was used. On
19jan12 the come was at za=15 degrees (ch at stow) so it was
not exactly the same as 23jan12.
- Top: Az difference clockwise spin comparison.
- black 23jan11, green: 19jan11
- Bottom: az difference ccw spin comparison.
- red 23jan11, green 19jan11.
- Comparing the 2 dates, the az encoder diff has shifted about
.01 to .02 degrees. This is most likely from the different za
of the dome.
- Looking back on 25nov11 (see below) the dome za was actually
moving during the measurement so it probably can't be used for
- For reference:
- .01deg change at 60 feet is about .12 inches.. this is
probably the accuracy we can position the pointer at 270.
- .01Deg az error on the rack gear is on the sky:
- 20deg: 12 arc seconds
- 10 deg: 6.2 arc seconds.
- Need to redo an az swing at dome=15 to compare with 19jan12.
- When ever we do the az swings to check the az dif we should
use a fixed dome za (say 11) and a fixed ch za (say stow).
25nov11: az encoder 2 replaced. (top)
On 25nov11 the az2 encoder (on the carriage house
side) failed and was replaced. The azEnc1 is the one used for
pointing. The encoder value for azEnc2 was set with the azimuth at
270. After replacement, azimuth swings were done to check the
azimuth encoder difference (azEnc1 - azEnc2).
The plots show the azimuth encoder
difference for the azimuth swings after the azEnc2 replacement
- The motion was:
- 270 to 0 at .4 degrees/sec. dome at 5 degrees, ch at stow
- 0 ->360 at .4 degrees/sec. dome at 5 till az=110
then dome moved to 10 deg za (got there when az=240)
- The black trace is the clockwise motion. The red trace is the
counter clockwise spin.
- Top: azEnc1 - azEnc2 in encoder units.
- Bottom: azenc1-azEnc2 in degrees. The radius is about 60 feet
so one degree is about 1 foot at the encoder radius.
- The az encoder difference for the swings has a negative offset
of about 140 Encoder counts.
- To center the difference, azenc2 should have it's value
decreased by 140 counts.
- This is probably not critical since i think the az bending
limit is set to a value larger the 400 encoder counts.
- The PI control loop uses the azEnc1 encoder to control the
- The plots show that the azEnc2 (on the opposite side)
leads the azEnc1 a little (probably some offset in voltages sent
to these motors).
12apr09: mot52 fail at az 326
The was a servo failure: motor 52 not ready on
11apr09 and 12apr09. Both of these occurred at an azimuth of 326
degrees. carl heiles was observing on both days.
plots of 11apr09 status (.ps) (.pdf)
plots of 12apr09 status (.ps) (.pdf)
- The failure occurred near the same time on both days.
- This must be because carl was tracking the same
- The dome was moving down from 19 degrees at .002 deg/sec and
the azimuth was increasing at .01 deg/sec.
- It was then commended to slew to a new position. The failure
occurred 5 seconds into the slew.
- To replicate the failure:
- pnt tr 325 18.92 -cx
- wait for the telescope to get there.
- pnt tr 327 17 -cx
- this should move it thru az 326 at slew like the failures.
07mar06 azimuth encoder 2
On 07mar06 azimuth encoder 2 (carriage house side) jumped
around 21.1 arcseconds.
The plots show the
encoder values (.ps) (.pdf):
The azimuth arm had been sitting at az=359.650
for about 18 minutes when the az2 encoder decided to jump. So the
problem was not because the gear jumped a tooth.
- Top: the azimuth position (encoder 1) versus hour of day.
- Bottom: The azimuth encoder difference (azGr - azCh) in
On 08mar06 the encoder 2 jumped again. It was
replaced by a new encoder on 09mar06
The torque on motor 22 was glitching for
about 1 second when the dome slowed down or speeded up. The
amplifier was moved to motor 21 and the problem moved to motor 21
(so it was the amplifier). This amplifier was replaced and the
glitching continued on motor 21. When the spare amplifier was
replaced with yet another amplifier, the glitching went away (so
both amps had the problem). The
show the torques while the dome was moved between 13 and 15
degrees za at .02 deg/sec (half slew rate). The hydraulic
brakes were not connected during this test. The motor numbering is
from uphill to downhill 11,12,21,22,31,32,41,42.
motor amp glitching when changing direction. (top)
Fig 1 top. Torques versus hour of day. The zenith angle is
plotted in black at the top. The green spikes are the torques
from motor 21.
Fig 1 bottom. This is a blowup of the top plots. The sampling
is once per second so the glitch is on the order of 1
Fig 2. The top is torque versus velocity (degrees/second),
middle is torque versus acceleration (deg/sec^2), and the bottom
plot is torque versus za. The green crosses are motor 21. The
glitches only occur at positive acceleration.
The azimuth encoders started to fail on 11nov05.
The links show the daily summary of the azimuth for. The timeline
11nov05 azimuth encoder
(.ps): 16:00 enc2 fault. 16:20 enc1 fault. replaced
encoder 1 with a spare encoder. Used the az index to align the
(.ps): 11:07 encoder 1 fault. Kept running
(.ps): early morning . encoder 1 (the one that was
installed on friday) failed. Moved enc2 -> enc1. put bad
encoder on encoder 2.
- 16nov05: replaced encoder 2 with a new encoder.
At 11:45 am on 12aug04 some tests were being
conducted with the dome, azimuth while the azimuth was at a low
zenith angle. Two member of the electronics department who were in
the dome at the time reported large vibrations/shaking of the dome
structure while this movement was taking place.
dome vibrations while moving at slew rate at low za. (top)
I looked at the az/za positions, velocities,
torques that are recorded once a second. The plots
the actual motion of the dome and azimuth (not the requested
motion) during the shaking period.
After 46 minutes (after 11 am) the motion of the dome was at a low
zenith angle (about 2 degrees za). The velocities that were being
requested were plus/minus slew rate for both axis (probably because
they were moving large distances).
- Fig 1. The top two plot are the azimuth position (black) and
azimuth velocity (red). For the azimuth .4 degrees/second is the
slew rate. The bottom two plots are the dome position (black)
and the dome velocity(red). The dome slew rate is .04
- Fig 2. The top plot are the azimuth motor torques for the 8
motors. The red graph at the top is the azimuth velocity. The
solid black line is a scaled version of the azimuth position.
The bottom plot is the torques for the 8 dome motors. The solid
black line is the dome zenith angle while the red lines show the
The torques for the dome are large (20 ft-lbs per motor) when moving
downhill at low zenith angle. This is caused by the hydraulic
braking system installed on the dome. When moving downhill the fluid
creates a resistance that is proportional to the velocity squared.
At normal tracking positions (above say 6 degrees za) the fluid
resistance is reacting against the component of gravity that is
pushing the dome downhill. At 2 degrees za, there is not
gravitational component to counteract. I wonder if the shaking
that was felt is an interaction between the PI loop and the
hydraulic brakes that is not normally seen with gravity present. We
need to check this out further.
servo failures. Motor torques build up while stopped.
There have been a number of servo failures with
the azimuth amplifiers recently. The error message is (amp xx not
ready). When the amplifier leds are inspected, an over speed
condition is present. The problem occurred at more than 1 az, za
position but it was repeatable whenever we used az=334.6 and za=2.
The sequence that caused the problem was:
- moving in increasing azimuth (at say .1 deg/sec) close to
334.6, dome at low za so most of the weight is on the wheels
opposite the dome side.
- user switches receivers. the trkinit routine is called and
People were placed near each azimuth wheel when this
sequence was executed. We were thinking that maybe the wheels were
spinning in this area. They reported that:
- pnt x vme 3. This command transfers control to the vme
system (even though it already had it). A side affect is
that the motors go to stop mode.
- A second or two after this command is executed, a
pnt mode tr 3 command is sent. This requests the az and
gr to move to program track mode.
show the motor torques and i/o bits during this sequence. The
x-axis is the time in seconds from midnight. The data is sampled
once a second so transitions can be occurring that are not being
- The wheels would slow down and they could hear the brake
- While the brake was on, they could hear a noise in the
motor. It sounded like the motor was fighting the brake.
- After about 5 seconds there was a large bang and then the
wheel would spin fast and then stop.
- Figure 1 shows the 8 motor torques versus time (about 11
seconds). The torques are offset for plotting. They take about
4 seconds to rise to a maximum value of 20 amps. This is
happening with the brakes on. On the fifth second they plunge
- Figure 2 shows some of the i/o bits during this time period.
The most interesting bits are:
Figure 3 shows the transition from azimuth moving at .1
degrees/sec to stop without a restart request. From bottom to
top the bits plotted are: drvsEnabled, brksReleased, then
output bytes: Q10,Q8,Q9. These output bytes control the relays
that cause different portions of the velocity request to be
included in the value sent to the amplifiers. The average
velocity is also over plotted in green (*). [note: it bothered
me the first time i saw the drvenable going low before the
brkrelease had gone low (the drives should never be disabled
without the brakes on). After searching the code it looked
like the drvenable bit high implied that the drives had
finished the drive up sequence (or they had not yet started
the drive down sequence)].
- IvelAz. This controls the PI error integration. It remains
on for the entire sequence. Bit 4 of digital output byte 10
(sum_n controller) also stays on so the output of the PI
loop is being sent to the motors.
- BrkG48Rel,BrkG15Rel show that the brakes were on at 60982
- horn. This went off at 60981 and back on at 60982. This
implies that a new request to move was being activated.
- The green line with * over plots the motor torques for az
motor 11 during this period.
The time sequence for the failure (figs 1,2) was:
The likely sequence of events was then:
- 60978 - The transfer control command was probably sent. This
started the slowdown/stop sequence.
- 60981 - request brakes M11,M12 to be on. The horn was turned
off here (this is on when in motion). The DCS az 0 speed
signal has become true.
- 60982 - request brakes M51,52,41,42,81,82 to be on. The read
back shows that all brakes are now on. The program track
request was probably sent here too since the horn has been
re-activated (this occurs pending new motions). The torques
begin to rise at this point.
- 60986 - the torques reach a maximum of about 20 amps.
- 60987 - a servo failure occurs (over speed). The horn is
- 60988 - the DCS az I portion of PI loop is turned off.
- The azimuth was asked to stop so it started slowing down.
- motors m11,m12 stopped first. The other motors stopped 1
second later. In between this time a new request to move
arrived. The differential stop times may have been because
some of the wheels were slipping.
- The brakes are on and dcs spd0 is true. This disconnects the
input velocity request in dcs brd 4a1 via relay K7.
- There must have been a non-zero difference between the
grounded velocity request and the actual speed value read back
from the motors. This value integrated up for the 5 seconds
that it took to startup. With the brakes on, this caused
the noise and vibrations that the people heard. For some
reason, the velocity error integrator was not turned off
during the 5 seconds.
- When the startup time had completed, the brakes were
released. The 20 amps of torque now had 0 load. It caused the
loud noise and the motor wheels to spin rapidly. The rapid
spinning made the amplifiers go over speed and shut the system
The az,gr,ch information is recorded once a second. I
searched the files from april through august and found more
occurrences of this problem: 2 in april, 5 in august, 11 in
june, 8 in july, 5 in august. I never saw it occur in the gr or
ch. This is why i think that a wheel slippage may be
related to this.
I've changed the trkinit code so that it doesn't try to take
control of the axis if it already has it. This will cut down on
the number of times that this condition should occur. It should
still be fixed since users can request the azimuth to stop and
start without going through the vme system (via the lcu or ocu).
This problem will stress the motors/amplifiers (removing the
load so quickly). The brakes should also be inspected. Jose
maldonado pointed out that wheels spinning on the track will
remove the grease and could cause damage to the rail.
01sep01 update. Motor/amp 12 on the azimuth had an
output voltage of 300 millivolts when the azimuth was not
moving. All of the other motors/amps were under 10 millivolts.
This value was averaged with the other 7 motors and caused the
velocity error to be non zero and integrate up while the brakes
were on. Fixing this will decrease the chances of this happening
again. The code should still be changed so that the integrator
is disabled whenever the brakes are on (at least for the
- 02oct16: az encoder one fails. replaced and realigned use
index at az=270.
- 10jan16: az encoder one fails, replaced and realigned..
- look at elec page for failures here
- 12jul13: drag link comes loose (pig motors 22,21), connection
plate bent on dome.
- 18jun13: gregorian mot22 replaced after problems with 22
continued (more info)
- 17jun13: gregorian amp22 replaced after multiple failures
- 17jan13: az encoder 2 bearing froze up. replaced encoder. enc
dif is a little off but probably ok.
- 18jun12: azStpCC, servoFail
- These 2 error occurred together. No other errors were
visible. It was caused by the 120 volt breaker (behind the
computer monitor rack) tripping. This breaker gives the 120
the motor amps and to the cable car az limit
switch. The power is pd2208 See pd.22 in
- 26jan12: az encoder 1
- 25nov11: encoder 2 failed.. replaced.
- 23oct11: power dip, amp 81 not ready. lcu parmeter menu
encoder offset has 281 deg. Reset to 0.
- 09dec08: klaus comes for drive tuneup.
- 11jun08: lcu azimuth encoder offset went 1 degree -> 0.
- 21feb08: lcu azimuth encoder offset went from 0 to 1 degree.
(returns to 0 11jun08)
- 05feb08: reset pots on az motor amplifiers to standard
position. The cmdScale, current limit of some were not set
- 10aug07: azbending error 9:06am. corner 8 inside painting
containment. Grit got into mount spring mechanism that pushes
encoder into the rack gear. encoder had come away from rack gear
causing bending problem. Manually reset encoder 2 position to
same encoder difference as before the failure. cleaned encoder
azimuth encoder 1 fault.
encoder 1 and 2 reset.
21/22 goes offline on dome. Amplifier 21 and later 22 of
the dome have been going offline when starting to move downhill
at high za. This has occurred on 13,15, and 16 feb07.
- 14nov06: az encoder1 fails. replaced. Had to reposition az
- 14oct06: encoder board port 1 (az enc1 ) fails. replaced board
- ??sep06: az encoder 1 failed and was replaced (not sure about
- 16aug06: aux encoder replaced with newer version.
- 06apr06: ch brakes changed.
- 05apr06: greg brakes m11,m12 replaced.
- 04apr06: greg brakes m42,m41,m32,m31,m22,m21 replaced.
- 10oct05: telescope only moves to smaller az when put in
tracking mode: The azimuth 2 encoder was changed in the
morning (because it was reporting encoder failure). In the
afternoon when the telescope was put in tracking mode, the
telescope would only moved toward azimuth of zero. Preset and
rate mode worked ok. This problem has occurred before (but i
don't have any notes on it). During the morning maintenance, the
plc was powered off then back on. After the problem jorge tried
a reset and then an overall reset of the plc. The problem
continued. He then tried replacing the plc. The azimuth started
- 26jan05 cam limit switch dome: found that the cam
limit switch for gregorian is bad. May have a loose tooth or
something inside. This generates the hardware version of the pre
and final limits (there is also a software version that comes
from the encoder). The lower limit was creeping up from 1.06
-> 2.5 -> 3.6 degrees over the last week or so. Readjusted
the upper limit to be 19.5 degrees. Should order a new unit.
- 19jan05 vertex shelter overheats: air conditioners
vertex shelter tripped off. Trouble with pointing. Only moves in
negative az in tracking mode. Turned out that the plc wasn't
working (replaced). Other problem was that the negative
direction seemed to come from a 1 degree az offset that
installed. This comes from f8 params menu in ocu. Gets stored in
the file params.dat.. Looking back at old data , this offset has
been there since last jun04. With it in now, it is causing the
tracking to fail. May be that the old version of the software
was ignoring this??. Reset to 0 via f8 ocu. Every time lcu
rebooted need to do this. Looks like the new value is not
getting set in the file params.dat (we are running on the
silicon disc...). Need to figure this out.