Turret not arriving on position
to change the turret dac zero velocity offset
160112: turret not
arriving on position
170219: turret not arriving on position
180529: may18 turret data shows zeroVelErr
comes from dac/signal conditioning board.
We have had problems with the turret timing
out waiting to be on position. The turret positioning sequences
When cima sets up a receiver:
- it sends the turret to the position and then calls
turwaitpos() to wait for the turret to be in
- The routine is called from within setupall().proc
- see /home/online/Tcl/Proc/pnt/turwaitpos.proc
- By default this routine declares the turret to be on position
when it is within .1 turret degrees of the requested position.
- The turret plate scale is 45 asecs great circles on the sky
for each turret degree.
- turwaitpos() times out in 200 seconds..
- I'm not sure if cima times out before this.
If the turret times out, cima log file has:
2017-Feb-19 03:16:12 LOG4 got_cormsg: From
DATATAKING: waiting for turret position:26.64
2017-Feb-19 03:16:53 ERROR end_wait_seconds:
ERROR timed out while waiting for configuring IF/LO path!
2017-Feb-19 03:16:53 LOG1 task_failed:
Task 'waiting on configuring IF/LO path' failed!
When timeouts occur, they have been traced to the zero velocity
offset of the turret dac, It drifts over time until it is outside
the allowable region. This drift can take days.. So we need
to monitor this value daily.. and update the dac before it gets
too far out..
not arriving on position.
On 12jan16 the turret failed to arrive on
position. The final position was greater than .1 degrees from
the requested position.
This is a similar to problems we had back in
aug14. At that time i thought the problem was from the
backlash board being misadjusted.. but it turned out to be a
problem with the zero velocity offset of the dac that drives the
motor amplifier (in velocity mode).
To find out when this problem started i went
back through the turret log files and plotted the turret position
and the different (actual - requested turret position):
- The plots show:
- top frame turret position vs hour of day.
- 2nd frame: Actual - Requested turret position vs hour of
- The red lines show the limit of +/- .1 degrees when cima
thinks the turret is on position.
- 3rd,bottom frames: same info as 1,2.. except a different
- The errors should be small when the turret is sitting at one
position (while it is moving, the error will be large).
- plots of turret position and position
offset for 1st day of each month for 2015 (.ps) (.pdf)
- each day has info from the first day of two months.
- You can see that the errors were small on 01Nov15 and then
large on 01dec15.
- The daily plots for Nov15 show when the errors increased.
- plots of turret position and
position offset for each day of Nov15 (.pdf)
- each page has the position and position error for two days.
- the turret was not moving 151109 to 151115
- this was when we had problems with the main contactor
being driven incorrectly by some digital outputs.
- when the turret started moving again on 151116, the turret
error had increased to up to .05 degrees.
- They probably put a new dac in the turret during the
- The error continued to degrade through Nov and dec15.
- It's interesting to note that when moving to a new turret
- the turret would move at 3deg/sec and then arrive at a
position (that would be off by some error)
- It would then creep toward the correct position over a few
- The PID loop integrator must be working very slowly...
- On 12jan16 the error had gotten so large that it no longer
arrived at the .1 degree threshold of turwaitpos().
- I used the procedure (found
here) to update the zero velocity offset for the
- Once this was done, the turret came rapidly to the requested
position .. to within .005 degrees.
- If the turret doesn't come within .1 degrees of the requested
position, you need to update the zero velocity offset (see procedure)
- The PID loop does a very slow (hours) approach to the correct
position when there is a zero velocity error in the dac.
- This is probably a bug in the code...
processing: x101/160112/turerryr.pro, turerryrmon.pro
170219 turret not arriving on position.
Timeline of the problem:
- 23:30 a power dip occurred. The system was reset and the
- 01:38 observation finished. Tsysall run and completes
- 01:40 next observer tries to run. Has trouble when
turret does not get within .1 degree of the requested
- 09:00 luis and phil change the zero velocity offset
from 57 to 80 counts, and the turret now reaches the correct
The plots shows the turret
position and the (Actual - requested) position for Feb 1 to 19
2017. (.ps) (.pdf)
- Each page has 2 days of data.
- upper frame: turret position
- 2nd frame (actual - requested) turret position (in deg),.
- The red horizontal lines are the +/- .1 deg turret on
- We first thought that the power dip caused a problem with the
turret. Looking at the data for the month of Feb17, you can see
that the error was slowly increasing for about 11 days.
- things were ok until 08Feb17 when the actual -req error
started to increase
- on 18Feb17 it was getting close to -.1 deg.
We should monitor this error. When it starts to increase, the
zero velocity offset should be recomputed.
180529: may18 data shows
zeroVelErr from dac/signalCond board
We've had a problem with the turret not
arriving at the final requested position (for the last few years).
Debugging showed that this occurred when the zero velocity offset
for the dac had drifted. This has been occurring more regularly.
During 2018 the zero velocity offset had to be updated on:
- 09Feb18 zeroVelcnt: 2038
- 09Apr18 zeroVelCnt: 2094
- 04may18 zeroVelCnt:2143
- 16may18 zeroVelCnt:2000
The turret data for 01may18 - 29may18 was used
to look into this problem.
What happens when commanding the turret to move:
- the PI loop generates a digital commanded velocity to use.
- This is a number from 0 to 4095 with 0
turret velocity around 4095/2
- the zero velocity offset is added to this value and becomes
the Vel_command sent to the dac board
- the zero velocithy offset is measured by commanding the
turret to a position, and seeing what commanded value is
needed to hold it stationary (once it gets in position).
- This is needed when we cross one of the turret encoder
pre-limits. In this case, the zero velocity value is sent to
the dac (rather than using the PI loop)
- The vel_command is sent to the dac.
- commanded values 0..4095 are mapped to 0 to 10 volts dac
- The dac output is sent to the signal conditioning board.
- This offsets the signal by -5 volts and then multiplies it
by 2 (the amplifier wants +/- 10 volts)
- the signal conditioning output is sent to the torque bias
board and then the motor amplifiers.
- there are actually two motors (used for backlash
- between the signal conditioning board and the torque bias
board, the voltage is measured by the a/d converter. This is
called the velocity feedback signal.
- The velocity of the turret floor is measured in two ways:
- Master amp speed readback
- This is a voltage output from the motor amplifier that is
sent to the a/d. It is "proportional" to the motor
- Encoder position. An absolute encoder is read once
a second to get the floor position
- Floor velocity, and acceleration can be computed by
differencing the position values.
Plotting the may18 turret data
(Warning.. the .pdf files are 1-2MBytes. they may take 10-20 secs
Velocity command vs velocity
feed back, Master amp speed vs velocity feedback (.pdf)
- Top frame: velocity command vs velocity feedback.
- The vel_command is sent to the dac board
- the velocity feedback is the output of the signal
conditioning board (input to the torque bias board),.
- The velCmd,velFdback relationship changed during
- changing the zero velocity offset should not change the
vel_command, velocity feedback ratio.
- Bottom frame:
- Master amp speed vs velocity feedback.
- This remains a linear function for the entire month.
- So the jumps are not caused by the amplifier, torque
bias board, or motors.
- So changes in the vel_command vs motor speed are occurring in
the dac or signal conditioning board.
Velocity command for 0 vel
vs date (.pdf)
Checking the +24Volts power
- Top frame: vel_command (for zero velocity) vs date
- data points with velocity feedback within 2 counts of zero
velocity were plotted vs date.
- Black: before 04may18 zeroVelocity change
- Red: between 04may18 and 16may18 zeroVelOffset changes
- green: after 16may18 zeroVelOffset change
- The hour for the changes were not recorded, so i
guesstimated the time.
- vel_command vs velocity feedback.
- The different colors show that the different relationships
go with the 3 sections of time.
Plotting values vs encoder
- the 24 volt power supply value is plotted vs date.
- This is sampled by the a/d converter
- The red points are when the motor is energized.
- The voltage drops about .3 volts when the load is applied.
- You can also see a slight day,night variation of voltage
(probably from temperature changes).
- There is no systematic power supply voltage change that could
have caused the jumps.
- the encoder position was used to compute the encVelocity and
- encVel(t+.5) =( encPos[i+1] - encPos[i])/dt[i]
- then encVel was then interpolated from (t+.5) back to (t)
- The encAccel was computed the same way (but using encVel)
- top: cmd_velocity vs encoder velocity
- there are multiple patterns.
- 2nd: velocity feedback vs encoder velocity
- there is a single pattern here.
- the figure 8 is probably from acceleration vs
- The smaller pattern is limited by the maxVel of 3deg/sec we
normally run at
- the larger pattern is when the sband transmitter is used.
- Looks like they are upping the maxTurVel to 12deg/sec.
- bottom: encoder acceleration vs encoder velocity
- these are both computed from the encoder position
- you probably need to flip the bottom plot to compare it with
the 2nd plot .
- the constant velocity feedback (around 200 counts)
probably occurs when de-accelerating.
- The turret will occasionally not arrive at the final requested
- This is caused by the zero velocity changing.
- There is a jump in the command velocity vs output to amp
- The changes are occurring in the dac or signal conditioning
- We should probably change the dac and signal conditioning
- It would also be a good idea to write a diagnostic
program to see the dac output vs input (this should be done on