[mus422] HW 2 number table

[Craig Sapp]

Hi Music 422 Class,

Here are the quantization codes and dequantized values for the
table in HW 2 (extra credit to the first person who finds anything
wrong with them).

There were two ways of interpreting the answer for the Block Floating
Point (3 scale/5 mantissa):
   (1) Treat all input numbers in the table as a single block
        (such that the maximum |number| value is 1 for the block).
   (2) Treat each number as a block of one number
         (such that the maximum value for the block is |number|).
Both interpretations are equally valid; however, I would suggest
that people who did interpretation #2 to also try interpretation #1.
There are cases in #1 which can uncover errors in your dequantizer
which case #2 cannot.

To be explicit on that last point, Marina mentioned in the second class
that the block floating point adds a one in the binary digit after the
mantissa value in the intermediate uniform quantization code, EXCEPT
if the mantissa value is zero (excluding the sign bit, of course). (You can
correct me if I misheard).  So the dequantization value for 0.008 using
interpretation #1 is 0.0, while the dequantization value for 0.008 is
0.00830281 using interpretation #2 listed above.

In the following list for each quantization type, the input number is
listed first, then the intermediate code in binary notation, followed by
the dequantized output value.

The "_" character in the quantization codes is just to separate
the binary digits into quartets for easier reading of the values.
For float codes, the scale is in parenthesis in front of the mantissa bits.
For uniform codes and the float mantissa, the most significant bit
is the sign bit (0=positive, 1=negative).  The uniform code following the
floating point codes (after "u:") are the intermediate codes which are unpacked
from the floating-point codes, just before final dequantization. (The temporary
uniform quantization for input into fp quant is the same as the 12-bit midtread
values on the last row of the HW2 table).

I have attached a PDF of the following numbers so that those
without fixed-width fonts in your email can see the numbers aligned
better.

I am still going through your quantization code (especially if your answers
do not match mine), but the values below will give you a hint as to whether your
quantizer functions are working properly.  Don't worry (too much) if they
are not correct for HW 3 (hopefully I will spot where the errors are before
I give you back HW2), but you will need to get quantization working perfectly
starting in HW 4... And if you are still stuck with a buggy quantization
function after HW3, you should just use the solution code for HW2 which
should be ready by Friday (although you should try to fix your own code first).

-=+Craig

================================================

8-bit midtread:

-1      1111_1111       -0.996078
-0.97   1111_1100       -0.972549
-0.41   1011_0100       -0.407843
0       0000_0000       0
0.008   0000_0001       0.00784314
0.51    0100_0001       0.509804
0.82    0110_1001       0.823529
0.999   0111_1111       0.996078
1       0111_1111       0.996078

3,5 FP:

-1      (000)1_1111 u:1111_1110_0000    -0.984615
-0.97   (000)1_1111 u:1111_1110_0000    -0.984615
-0.41   (001)1_1010 u:1011_0101_0000    -0.414164
0       (111)0_0000 u:0000_0000_0000    0
0.008   (110)0_0000 u:0000_0001_0000    0.00781441
0.51    (000)0_0000 u:0100_0010_0000    0.515751
0.82    (000)0_1010 u:0110_1010_0000    0.828327
0.999   (000)0_1111 u:0111_1110_0000    0.984615
1       (000)0_1111 u:0111_1110_0000    0.984615

3,5 BFP: (1 number per block)

-1      (000)1_1111 u:1111_1100_0000    -0.968987
-0.97   (000)1_1111 u:1111_1100_0000    -0.968987
-0.41   (001)1_1101 u:1011_0110_0000    -0.421978
0       (111)0_0000 u:0000_0000_0000    0
0.008   (110)0_1000 u:0000_0001_0001    0.00830281
0.51    (000)0_1000 u:0100_0100_0000    0.53138
0.82    (000)0_1101 u:0110_1100_0000    0.843956
0.999   (000)0_1111 u:0111_1100_0000    0.968987
1       (000)0_1111 u:0111_1100_0000    0.968987

3,5 BFP: (all numbers in block [max is 1.0])

-1      (000)1_1111 u:1111_1100_0000    -0.968987
-0.97   (000)1_1111 u:1111_1100_0000    -0.968987
-0.41   (000)1_0110 u:1011_0100_0000    -0.406349
0       (000)0_0000 u:0000_0000_0000    0
0.008   (000)0_0000 u:0000_0000_0000    0
0.51    (000)0_1000 u:0100_0100_0000    0.53138
0.82    (000)0_1101 u:0110_1100_0000    0.843956
0.999   (000)0_1111 u:0111_1100_0000    0.968987
1       (000)0_1111 u:0111_1100_0000    0.968987

12-bit midtread:

-1      1111_1111_1111  -0.999756
-0.97   1111_1100_0010  -0.969963
-0.41   1011_0100_0111  -0.409768
0       0000_0000_0000  0
0.008   0000_0001_0000  0.00781441
0.51    0100_0001_0100  0.50989
0.82    0110_1000_1111  0.820024
0.999   0111_1111_1101  0.998779
1       0111_1111_1111  0.999756

-=+Craig
-------------- next part --------------
A non-text attachment was scrubbed...
Name: quantization_values.pdf
Type: application/pdf
Size: 7570 bytes
Desc: not available
Url : http://cm-mail.stanford.edu/pipermail/422/attachments/20100126/37eb6f3f/attachment.pdf