2bit quantization levels for rdbe

Other links:
using the 12m,fs,rdbe at AO: an overview

---

    The rdbe samples the data with an  8 bit digitizer. The signal is processed inside the fpga and then it is quantized to 2 bits. The rdbe quantize command will return a histogram of the 2bit data after the quantization:

• dbe_ddc_quantize?0:0:5119372:10874810:10885028:5120790:1939:-1924:7
• the boldface numbers are the counts in the levels:
  
• Val< -lev
• val>= -lev  and val< 0
• val >= 0  and val< clip
• val >= clip
• The optimum value for lev (in units of sigma) is .995  (quantize_effic.pdf)

When lev is set optimally then the ratio of the  middle counts to the outside counts should be .465

To test this i did the following:

•  using the randomn noise routine in idl (guassian noise)
• generate 10,000,000 points with rms of 1.
• scale the noise to .995/1.
• verify that the rms is .995 levels
  
• generate a histogram:
  
• h[0] = points <  -.995
• h[1]=  points >= -.995   and < 0
  
• h[2]=  points >=0 and < .995
• h[3]=  points >= .995

The table below shows the values from the noise simulation as well as the values used by the program on vlbis1 to adjust the levels:

histogram of 2bit noise

	h[0]	h[1]	h[2]	h[3]
raw counts	1587004	3415464	3411209	1586323
scl to mean h[1:2]	0.4649	1.0006	0.9994	0.4647
program to set levels uses:
ddc_quantize_read_adj_set.py	.47	1.	1.	.47

Note:

• computing the rms on the clipped integer data  will return an rms of .940.
  
• the clipping has set all of the values above/below the +/-clipping  level to the +/- clipping level.
  
• So you would expect the rms to be less than .995 of the floating point data.
  

processing:x101/211019/2bitlevels.pro