Make clear that this isn't the official version

[dirac-spec-errata.git] / mc.tex

\label{motioncompensate}

This section defines the operation of the process
$motion\_compensate(ref1, ref2,  pic, c)$ for motion-compensating a
picture component array  $pic$ of type $c=Y, U$ or $V$ from reference 
component arrays $ref1$ and $ref2$ of the same type.

This process shall be invoked for each component in a picture, subsequent to the 
decoding of coefficient data, specified in Section \ref{transformdec}, and the Inverse Wavelet Transform (IWT), specified in Section \ref{idwt}. 

Motion compensation shall use the motion block data $\BlockData$ and optionally may use the
global motion parameters $\GlobalParams$.

\begin{informative*}
\subsubsection{Overlapped Block Motion Compensation (OBMC) (Informative)}

Motion compensated prediction methods provide methods for determining 
predictions for pixels in the current picture by using motion vectors to 
define offsets from those pixels to pixels in previously decoded
pictures. Motion compensation techniques vary in how those pixels are grouped
together, and how a prediction is formed for pixels in a given group. In 
conventional  block motion compensation, as used in MPEG2, H.264 and many other
codecs, the picture is divided into {\em disjoint} rectangular blocks and the
motion vector or vectors associated with that block defines the offset(s) into
the reference pictures.

In OBMC, by contrast, the predicted picture is divided into a regular overlapping 
blocks of dimensions $xblen$ by $yblen$ that cover at least the entire picture 
area as shown in figure \ref{fig:blockcoverage}.  Overlapping is ensured by starting
each block at a horizontal separation $xbsep$ and a vertical separation $ybsep$ 
from its neighbours, where these values are less than the corresponding block dimensions.
\end{informative*}

\begin{figure}[!ht]
\centering
\includegraphics[width=0.7\textwidth]{figs/block-coverage}
\caption{Block coverage of the predicted picture}
\label{fig:blockcoverage}
\end{figure}

\begin{informative*}
The overlap between blocks horizontally is $xoffset=(xblen - xbsep)/2$ both on the left
and on the right, and vertically is $yoffset=(yblen - ybsep)/2$ both on the top and the
bottom. As a result pixels in the overlapping areas lie in more than
one block, and so more than one motion vector set (and set of associated predictions)
applies to them. Indeed, a pixel may have up to eight predictions, as it may belong to
up to four blocks, each of which may have up to two motion vectors. These are combined
into a single prediction by using weights, which are so constructed so as to sum to 1. In
 the  Dirac integer implementation, fractional weights are achieved by insisting that weights
 sum to a power of 2, which is then shifted out once all contributions have been summed.

In Dirac blocks are positioned so that blocks will overspill the left and top edges by 
($xoffset$) and ($yoffset$) pixels.  The number of blocks has been
determined (Section \ref{motiondatadimensions}) so that the picture area is wholly
 covered, and the overspill
 on the right hand and bottom edges will be at least the amount on the left and top edges. 
Indeed, the number of blocks has been set so that the blocks divide into whole superblocks
(sets of 4x4 blocks), which mean that some blocks may fall entirely out of the picture area. 
 Any predictions for pixels outside the actual picture area are discarded.
\end{informative*}

\subsubsection{Overall motion compensation process}
\label{mcprocess}

The motion compensation process shall form an integer prediction for each pixel in 
the predicted picture component $pic$, which shall be added to the pixel value, and
 then clipped to keep it in range.

The $motion\_compensate()$ process is defined by means of a temporary data
array $mc\_tmp$ for storing the motion-compensated prediction for the 
current picture. 

The $motion\_compensate()$ process shall be defined as follows:

\begin{pseudo}{motion\_compensate}{ref1, ref2,  pic, c}
\bsIF{c==Y}
    \bsCODE{bit\_depth=\LumaDepth}
\bsELSE
    \bsCODE{bit\_depth=\ChromaDepth}
\bsEND
\bsCODE{init\_dimensions(c)}{\ref{mcdimensions}}
\bsCODE{mc\_tmp=init\_temp\_array()}{\ref{mctemparray}}
\bsFOR{j=0}{\BlocksY-1}
    \bsFOR{i=0}{\BlocksX-1}
        \bsCODE{block\_mc(mc\_tmp,i,j,ref1,ref2,c)}{\ref{blockmc}}
    \bsEND
\bsEND
\bsFOR{y=0}{\LenY-1}
    \bsFOR{x=0}{\LenX-1}
       \bsCODE{pic[y][x] += (mc\_tmp[y][x]+32)\gg 6}
        \bsCODE{pic[y][x] = \clip(pic[y][x], -2^{bit\_depth-1}, 2^{bit\_depth-1}-1)}
    \bsEND
\bsEND
\end{pseudo}

\begin{informative}
Six bits are used for the overlapped-block weighting matrix. This ensures that 10-bit
data may normally be motion compensated using 16-bit words as per Section \ref{blockmc}.
\end{informative}

\subsubsection{Dimensions}
\label{mcdimensions}
Since motion compensation shall apply to both luma and (potentially subsampled)
chroma data, for simplicity a number of variables are defined by the 
$init\_dimensions()$ function, which is as follows:

\begin{pseudo}{init\_dimensions}{c}
\bsIF{c==Y}
   \bsCODE{\LenX=\LumaWidth}
   \bsCODE{\LenY=\LumaHeight}
   \bsCODE{\XBlen=\LumaXBlen}
   \bsCODE{\YBlen=\LumaYBlen}
   \bsCODE{\XBsep=\LumaXBsep}
   \bsCODE{\YBsep=\LumaYBsep}
\bsELSE
   \bsCODE{\LenX=\ChromaWidth}
   \bsCODE{\LenY=\ChromaHeight}
   \bsCODE{\XBlen=\ChromaXBlen}
   \bsCODE{\YBlen=\ChromaYBlen}
   \bsCODE{\XBsep=\ChromaXBsep}
   \bsCODE{\YBsep=\ChromaYBsep}
\bsEND
\bsCODE{\XOffset = (\XBlen-\XBsep)//2}
\bsCODE{\YOffset = (\YBlen-\YBsep)//2}
\end{pseudo}

\begin{informative}
The subband data that makes up the IWT coefficients is padded in order that the IWT
may function correctly. For simplicity, in this specification, padding data is removed
after the IWT has been performed so that the picture data and reference data arrays have
the same dimensions for motion compensation. However, it may be more efficient to 
perform all operations prior to the output of pictures using padded data, i.e. to discard
 padding values subsequent to motion compensation. Such a course of action is equivalent,
 so long as it is realised that blocks must be regarded as edge blocks if they overlap the
 actual picture area, not the larger area produced by padding.
\end{informative}

\subsubsection{Initialising the motion compensated data array}
\label{mctemparray}

The $init\_temp\_array()$ function shall return a two-dimensional data array with
horizontal size $\LenX$ and vertical size $\LenY$, such that each element of the two dimensional array shall be set to zero.


\subsubsection{Motion compensation of a block}
\label{blockmc}

This section defines the $block\_mc()$ process for motion-compensating a single
block.

Each block shall be motion-compensated by applying a weighting matrix to a block prediction and adding the weighted prediction into the motion-compensated 
prediction array. 

The $block\_mc()$ process shall be defined as follows:

\begin{pseudo}{block\_mc}{mc\_pred,i,j,ref1,ref2,c}
\bsCODE{xstart = i*\XBsep-\XOffset}
\bsCODE{ystart = j*\XBsep-\XOffset}
\bsCODE{xstop = (i+1)*\XBsep+\XOffset}
\bsCODE{ystop = (j+1)*\YBsep+\YOffset}
\bsCODE{mode=\BlockData[j][i][\RMode]}
\bsCODE{W=spatial\_wt(i,j)}{\ref{pixelprediction}}
\bsFOR{y=\max(ystart,0)}{\min(ystop,\LenY)-1}
    \bsFOR{x=\max(xstart,0)}{\min(xstop,\LenX)-1}
        \bsCODE{p=x-xstart}
        \bsCODE{q=y-ystart}
        \bsIF{mode==\Intra}
            \bsCODE{val=\BlockData[j][i][dc][c]}
        \bsELSEIF{mode==\RefOneOnly}
            \bsCODE{val=pixel\_pred(ref1, 1, i, j, x, y, c)}{\ref{pixelprediction}}
            \bsCODE{val*=\RefOneWeight+\RefTwoWeight}
            \bsCODE{val=(val+2^{\RefsWeightPrecision-1})\gg\RefsWeightPrecision}
        \bsELSEIF{mode==\RefTwoOnly}
            \bsCODE{val=pixel\_pred(ref2, 2, i, j, x, y, c)}{\ref{pixelprediction}}
            \bsCODE{val*=\RefOneWeight+\RefTwoWeight}
            \bsCODE{val=(val+2^{\RefsWeightPrecision-1})\gg\RefsWeightPrecision}          
        \bsELSEIF{mode==\RefOneAndTwo}
            \bsCODE{val1=pixel\_pred(ref1, 1, i, j, x, y, c)}{\ref{pixelprediction}}
            \bsCODE{val1*=\RefOneWeight}
            \bsCODE{val2=pixel\_pred(ref2, 2, i, j, x, y, c)}{\ref{pixelprediction}}
            \bsCODE{val2*=\RefTwoWeight}
            \bsCODE{val=(val1+val2+2^{\RefsWeightPrecision-1})\gg\RefsWeightPrecision}
        \bsEND
        \bsCODE{val *= W[q][p]}
        \bsCODE{mc\_tmp[y][x]+=val}
    \bsEND
\bsEND
\end{pseudo}

\begin{informative}
Note that if the two reference weights are 1 and $\RefsWeightPrecision$ is 1, then
reference weighting is transparent and

\begin{pseudo*}
\bsCODE{val=pixel\_pred(ref1, 1, i, j, x, y, c)}
\bsCODE{val*=\RefOneWeight+\RefTwoWeight}
\bsCODE{val=(val+2^{\RefsWeightPrecision-1})\gg\RefsWeightPrecision}
\bsCODE{\ldots}
\end{pseudo*}

reduces to

\begin{pseudo*}
\bsCODE{val=pixel\_pred(ref1, 1, i, j, x, y, c)}
\bsCODE{\ldots}
\end{pseudo*}

In this case, therefore, the normal reference weighting produces no additional dynamic
range for internal processing and 10 bit video can be motion compensated with 16 bit
unsigned internal values.

In general, however, the worst case internal bit widths consist of the video bit depth plus the maximum of: 6 (the spatial matrix bit width) and the value of $\RefsWeightPrecision$. 6 bits
should be sufficient for most fading compensation applications, and so 16 bit internals will
suffice for all practical motion compensation scenarios for 8 and 10 bit video.
\end{informative}


\subsubsection{Spatial weighting matrix}

\label{mcspatialweights}

This section specifies the function $spatial\_wt(i,j)$ for deriving the 6-bit spatial weighting 
matrix that shall be applied to the block with coordinates  $(i,j)$. 

Note that other weights shall be applied to the prediction as a result of the 
weights applied to each reference.

The same weighting matrix shall be returned for all blocks within the interior
of the picture component array. Suitably modified weighting matrices shall
be returned for blocks at the edges of the picture component data array.

The function shall return a two-dimensional spatial weighting matrix. This
shall apply a linear roll-off in both horizontal and vertical directions.

The spatial matrix returned shall be the product of a horizontal and a vertical
weighting matrix. It shall be defined as follows:

\begin{pseudo}{spatial\_wt}{i,j}
\bsFOR{y=0}{\YBlen-1}
    \bsFOR{x=0}{\XBlen-1}
        \bsCODE{W[y][x]=h\_wt(i)[x]*v\_wt(j)[y]}
    \bsEND
\bsEND
\bsRET{W}
\end{pseudo}

The horizontal weighting function shall be defined as follows:

\begin{pseudo}{h\_wt}{i}
\bsIF{\XOffset!=1}
    \bsFOR{x=0}{2*\XOffset-1}
        \bsCODE{hwt[x]=1+(6*x+\XOffset-1)//(2*\XOffset-1)}
        \bsCODE{hwt[x+\XBsep]=8-hwt[x]}
    \bsEND
\bsELSE
    \bsCODE{hwt[0]=3}
    \bsCODE{hwt[1]=5}
    \bsCODE{hwt[\XBsep]=5}
    \bsCODE{hwt[\XBsep+1]=3}
\bsEND
\bsFOR{x=2*\XOffset}{\XBsep-1}
    \bsCODE{hwt[x]=8}
\bsEND
\bsIF{i==0}
    \bsFOR{x=0}{2*\XOffset-1}
        \bsCODE{hwt[x]=8}
    \bsEND
\bsELSEIF{i==\BlocksX-1}
    \bsFOR{x=0}{2*\XOffset-1}
        \bsCODE{hwt[x+\XBsep]=8}
    \bsEND
\bsEND
\bsRET{hwt}
\end{pseudo}

The vertical weighting function  shall be defined as follows:

\begin{pseudo}{v\_wt}{j}
\bsIF{\YOffset!=1}
    \bsFOR{y=0}{2*\YOffset-1}
        \bsCODE{vwt[y]=1+(6*y+\YOffset-1)//(2*\YOffset-1)}
        \bsCODE{vwt[y+\YBsep]=8-vwt[y]}
    \bsEND
\bsELSE
    \bsCODE{vwt[0]=3}
    \bsCODE{vwt[1]=5}
    \bsCODE{vwt[\YBsep]=5}
    \bsCODE{vwt[\YBsep+1]=3}
\bsEND
\bsFOR{y=2*\YOffset}{\YBsep-1}
    \bsCODE{vwt[y]=8}
\bsEND
\bsIF{j==0}
    \bsFOR{y=0}{2*\YOffset-1}
        \bsCODE{vwt[y]=8}
    \bsEND
\bsELSEIF{j==\BlocksY-1}
    \bsFOR{y=0}{2*\YOffset-1}
        \bsCODE{vwt[y+\YBsep]=8}
    \bsEND
\bsEND
\bsRET{vwt}
\end{pseudo}

\begin{informative}
The horizontal and vertical weighting arrays satisfy the perfect reconstruction property across block overlaps by construction:
\begin{eqnarray*}
hwt[x+\XBsep] & = & 8 - hwt[x] \\ 
vwt[y+\YBsep] & = & 8 - vwt[y]  
\end{eqnarray*}

In addition, it can be shown they are always symmetric (except at picture edges), or
equivalently the leading edges have skew-symmetry about the half-way point:
\begin{eqnarray*}
hwt[\XBlen-1-x] & =  & hwt[x] \\
vwt[\YBlen-1-y] & = & vwt[y] 
\end{eqnarray*}

The horizontal and vertical weighting matrix components for various block
 overlaps are shown in Table \ref{table:leadingedges}. 
These encompass all the default values listed 
in Table \ref{blockparamsvalues} for both luma and chroma.
\end{informative}
\begin{table}[!ht]
\centering
\begin{tabular}{|c|c|c|}
\hline
\rowcolor[gray]{0.75}\bf{Overlap}  & \bf{Offset} & \bf{Leading edge} \\
\rowcolor[gray]{0.75}\bf{(length-separation)} & & \\
\hline
2 & 1 & 3,5\\
\hline
4 & 2 & 1,3,5,7\\
\hline
8 & 4 & 1,2,3,4,4,5,6,7\\
\hline
16 & 8 & 1,1,2,2,3,3,3,4,4,5,5,5,6,6,7,7 \\
\hline
\end{tabular}
\caption{Leading and trailing edge values for different block overlaps}
\label{table:leadingedges}
\end{table}

\begin{comment}
The profile of the matrix 
for interior blocks is illustrated in Figure \ref{fig:weightprofile}.

\begin{figure}[!ht]
\centering
\includegraphics[width=0.7\textwidth]{figs/obmc-profile}
\caption{Profile of overlapped-block motion compensation matrix}
\label{fig:weightprofile}
\end{figure}

\begin{informative*}
\subsubsection{Reference weights and fade prediction (Informative)}

The reference prediction weights used for each prediction mode for 
block prediction (Section \ref{blockmc}) may appear 
confusing. It is helpful
to think of two cases for using reference picture weighting. The first is interpolative 
prediction, where the picture being predicted is, for example, a cross-fade and is
closely approximated by some mixture of the reference pictures:
 $P\backsimeq\delta R_1+(1-\delta)R_2$. Here the weights we'd like to
use for each frame prediction add up to 1 (or $2^\RefsWeightPrecision$ 
for integer weights). 
The second case is scaling prediction, where 
the weights we'd like to use for the frame predictions don't add up to 1: for example,
a fade to or from black
$P\backsimeq\delta_1 R_1$ and $P\backsimeq\delta_2 R_2$. It is not possible to choose 
weights for each prediction mode which will be optimal both cases. The weighting
factors chosen will give work with interpolative prediction (which is more common) 
but are not perfect for scaling prediction. It would have been possible to create a variety of
prediction modes to cover all cases, however the potential savings do not justify the
additional complexity.

For interpolative prediction, all data in the current picture will be of commensurate scale to
that of the references. In forming the bi-directional prediction, a value 
$W_1 p_1 + W_2 p2_2$ is 
formed, so the prediction has "scale" $W_1+W_2$. $W_1+W_2$ is 
therefore the weighting value used to scale unidirectional prediction, in order to provide
predictions of commensurate order. The unity weighting value 
$2^\RefsWeightPrecision$ is used
for DC blocks as this gives the best prediction, and in the interpolative case 
this equals $W_1+W_2$
so all predictions are of the same order.

The weighting factors we would like to use for unidirectionally 
redicted blocks in the scaling case
are $2W_1$ and $2W_2$ - the factor 2 takes into account that 
we're only adding in one prediction
value as against two for bidirectional prediction. These factors differ f
rom $W_1+W_2$, and hence
unidirectional prediction is incorrect when there are two references. 
Note, however, that we can
still perform prediction with the correct scaling values when we 
only have a single reference. Note
also that the value of $W_1+W_2$ was selected instead of 
$2^\RefsWeightPrecision$, which
would be equivalent in the interpolative case, as it gives a 
better approximation when the
weights do not sum to $2^\RefsWeightPrecision$.
\end{informative*}
\end{comment}

\subsubsection{Pixel prediction}
\label{pixelprediction}

This section defines the operation of the $pixel\_pred(ref, ref\_num, i, j, x, y, c)$ 
process which shall be used for forming the prediction for a pixel 
with coordinates $(x,y)$ in component $c$, belonging to the block with coordinates $(i,j)$.

The pixel prediction process shall consist of two stages. In the first stage, a motion vector
 to be applied to pixel $(x,y)$ shall be derived. For block motion, this shall be a block
 motionvector that shall apply to all pixels in a block. For global motion the motion
vector shall be computed from the global motion parameters and may vary pixel-by-pixel.

In the second stage, the motion vector shall be used to derive coordinates in an reference picture.

\begin{pseudo}{pixel\_pred}{ref,ref\_num,i,j,x,y,c}
\bsIF{\BlockData[j][i][\GMode]==\false}
  \bsCODE{mv= \BlockData[j][i][\Vect][ref\_num]}
\bsELSE
  \bsCODE{mv=global\_mv(ref\_num, x, y)}{\ref{globalmv}}
\bsEND
\bsIF{c!=Y}
  \bsCODE{mv = chroma\_mv\_scale(mv)}{\ref{chromamvscale}}
\bsEND
\bsCODE{px = (x\ll \MotionVectorPrecision)+mv[0]}
\bsCODE{py = (y\ll \MotionVectorPrecision)+mv[1]}
\bsIF{\MotionVectorPrecision>0}
  \bsRET{subpel\_predict(ref, c, px, py))}{\ref{upconvert}}
\bsELSE
  \bsRET{ref[\clip(py,0,\height(ref)-1)][\clip(px,0,\width(ref)-1)]}
\bsEND
\end{pseudo}

\subsubsection{Global motion vector field generation}
\label{globalmv}

This section specifies the operation of the $global\_mv(ref\_num, x,y)$ process
for deriving a global motion vector for a pixel at location $(x,y)$.

The function shall be defined as follows:

\begin{pseudo}{global\_mv}{ref\_num, x,y}
\bsCODE{ez  =  \GlobalParams[ref\_num][\ZRSexponent]}
\bsCODE{ep  =  \GlobalParams[ref\_num][\PerspectiveExponent]}
\bsCODE{b=\GlobalParams[ref\_num][\PanTilt]}
\bsCODE{A=\GlobalParams[ref\_num][\ZRS]}
\bsCODE{c=\GlobalParams[ref\_num][\Perspective]}
\bsCODE{m=2^{ep}-(c[0]*x+c[1]*y)}
\bsCODE{v[0]=m*((A[0][0]*x+A[0][1]*y)+2^{ez}*b[0])}
\bsCODE{v[1]=m*((A[1][0]*x+A[1][1]*y)+2^{ez}*b[1])}
\bsCODE{v[0] = (v[0]+(1\ll(ez+ep)) )\gg (ez+ep)}
\bsCODE{v[1] = (v[1]+(1\ll(ez+ep)) )\gg (ez+ep)}
\bsRET{v}
\end{pseudo}

\begin{informative}
Write ${\bf x}=\left( \begin{array}{c} x\\y \end{array}\right)$. 
Mathematically, we wish the global motion vector ${\bf v}$ to be defined by:
\[{\bf v}=\dfrac{{\bf Ax}+{\bf b}}{1+{\bf c}^T{\bf x}}\]
where: ${\bf A}$ is a matrix describing the degree of zoom, rotation or shear; ${\bf b}$
is a translation vector; and ${\bf c}$ is a perspective vector which expresses the
degree to which the global motion is not orthogonal to the axis of view.

In Dirac, this formula is adjusted in two ways in order to get an implementable result.
Firstly, the perspective element is adjusted to remove a division, changing the 
formula to:
\[{\bf v}=(1-{\bf c}^T{\bf x})({\bf Ax}+{\bf b})\]
which is valid for small ${\bf c}$. Secondly, the formula is re-cast in terms of integer
arithmetic by giving the matrix element an accuracy factor $\alpha$ and the perspective
element an accuracy factor $\beta$:
\[{\bf v}=(1-2^{-\beta}{\bf c}^T{\bf x})(2^{-\alpha}{\bf Ax}+{\bf b})\]
where the parameters ${\bf A}, {\bf b},{\bf c}$ are now integral. (No accuracy bits are required for the translation, since it must be an integral number of sub-pixels.) 

This reduces to
\[2^{\alpha+\beta}{\bf v}=(2^\beta-{\bf c}^T{\bf x})({\bf Ax}+2^\alpha{\bf b})\]
and this formula is used for the computation of values.
\end{informative}

\subsubsection{Chroma subsampling}
\label{chromamvscale}

When motion compensating chroma components, motion vectors shall be scaled by the
$chroma\_mv\_scale()$ function. This produces chroma vectors in units of 
$\MotionVectorPrecision$ with respect to the chroma samples, as follows:

\begin{pseudo}{chroma\_mv\_scale}{v}
\bsCODE{sv[0] = v[0]//chroma\_h\_ratio()}{\ref{picturedimensions}}
\bsCODE{sv[1] = v[1]//chroma\_v\_ratio()}{\ref{picturedimensions}}
\bsRET{sv}
\end{pseudo}.

\begin{informative}
Recall that division in this specification rounds towards -infinity. This division can be achieved by a bit-shift in C/C++ as chroma dimension ratios are 1 or 2.
\end{informative}


\subsubsection{Sub-pixel prediction}
\label{upconvert}

This section defines the operation of the $subpel\_predict(ref, c, u, v)$ function
for producing a sub-pixel accurate value at location $(u,v)$ from an upconverted picture reference component of type $c$ (Y, C1 or C2). 

Upconversion shall be defined by means of a half-pixel interpolated reference array
$upref$.  $upref$ shall have dimensions $(2W-1)$x$(2H-1)$ where the original reference 
picture component array has dimensions $W$x$H$, as per Section \ref{halfpel}. 

Motion vectors shall be permitted to extend beyond the edges of reference picture data,
 where values lying outside shall be determined by edge extension. 

If $\MotionVectorPrecision==1$, upconverted values shall be derived directly from the
the half-pixel interpolated array $upref$, which shall be calculated as per Section \ref{halfpel}.

If $\MotionVectorPrecision==2$ or $\MotionVectorPrecision==3$, upconverted values shall be
derived by linear interpolation from the half-pixel interpolated array.

The sub-pixel prediction process shall be defined as follows:

\begin{pseudo}{subpel\_predict}{ref,c,u,v}
\bsCODE{upref=interp2by2(ref,c)}{\ref{halfpel}}
\bsCODE{hu = u \gg (\MotionVectorPrecision-1)}
\bsCODE{hv = v \gg (\MotionVectorPrecision-1)} 
\bsCODE{ru = u-(hu\ll (\MotionVectorPrecision-1))}
\bsCODE{rv = v-(hv\ll (\MotionVectorPrecision-1))}
\bsCODE{w00 = (2^{\MotionVectorPrecision-1}-rv)*(2^{\MotionVectorPrecision-1}-ru)}
\bsCODE{w01 = (2^{\MotionVectorPrecision-1}-rv)*ru}
\bsCODE{w10 = rv*(2^{\MotionVectorPrecision-1}-ru)}
\bsCODE{w11 = rv*ru}
\bsCODE{xpos = \clip(hu, 0, \width(upref)-1)}
\bsCODE{xpos1 = \clip(hu+1, 0,\width(upref)-1)}
\bsCODE{ypos = \clip(hv, 0, \height(upref)-1)}
\bsCODE{ypos1 = \clip(hv+1, 0, \height(upref)-1)}
\bsCODE{\begin{array}{ll} val = & w00*upref[ypos][xpos]+w01*upref[ypos][xpos1]+ \\
                            & w10*upref[ypos1][xpos]+w11*upref[ypos1][xpos1]
        \end{array}}
\bsIF{\MotionVectorPrecision>1}
    \bsRET{(val+2^{2*\MotionVectorPrecision-3})\gg(2*\MotionVectorPrecision-2)}
\bsELSE
    \bsRET{val}
\bsEND
\end{pseudo} 

\begin{informative}
$hu$ and $hv$ represent the half-pixel part of the sub-pixel position $(u,v)$.

$ru$ and $rv$ represent the remaining sub-pixel component of the position.
$ru$ and $rv$ satisfy \[0\leq ru,rv <2^{\MotionVectorPrecision-1}\] 

The four weights $w00,w01,w10$ and $w11$ sum to $2^{2*\MotionVectorPrecision-2}$, and
hence the upconverted value is returned to the initial pixel ranges in the pseudocode 
above.

Note that the remainder values $ru$ and $rv$, and hence the four weight values, 
only depend on the motion vectors. This is because
$u$ and $v$ have been computed by scaling the picture coordinates by
$2^{\MotionVectorPrecision}$ and adding the motion vector.

In particular constant linear interpolation weights are applied throughout a 
block when block motion is used. Likewise, the necessity of clipping the ranges of
$xpos$, $ypos$ etc can be determined in advance for each block by checking whether any 
corner of the reference block will fall outside of the reference picture area. In most
cases it will not and clipping will not be required for motion compensating most blocks. 

For half-pixel motion vectors ($\MotionVectorPrecision$ is 1), the majority of the 
pseudocode is redundant, and the return value $val$ will merely be the value at 
position $(u,v)$, clipped to the ranges of the upconverted reference. 

\end{informative}

\subsubsection{Half-pixel interpolation}
\label{halfpel}

This section defines the $interp2by2(ref,c)$ process for generating
an upconverted reference array $upref$ representing a half-pixel interpolation of 
the reference array $ref$ for component $c$ (Y, C1, or C2). 

$upref$ shall be created in two stages. The first stage shall upconvert vertically. The second stage shall upconvert horizontally. 

$upref$ shall have width $2*\width(ref)-1$ and height $2*\height(ref)-1$, so that all
edge values shall be copied from the original array and not interpolated.

The interpolation filter shall be the 8-tap symmetric filter with taps as defined in Figure \ref{upfilter}.

\begin{figure}[h!]
\begin{centering}
\begin{tabular}{l|ccccc}
Tap & $t[0]$ & $t[1]$ & $t[2]$ & $t[3]$\\
\hline
Value & 21 & -7 & 3 & -1
\end{tabular}
\caption{Interpolation filter coefficients \label{upfilter}}
\end{centering}
\end{figure}

Where coefficients used in the filtering process fall outside the bounds of the 
reference array, values shall be supplied by edge extension. 

The overall process shall be defined as follows:

\begin{pseudo}{interp2by2}{ref,c}
\bsIF{c==Y}
    \bsCODE{bit\_depth=\LumaDepth}
\bsELSE
    \bsCODE{bit\_depth=\ChromaDepth}
\bsEND
\bsFOR{q=0}{2*\height(ref)-2}
    \bsIF{q\%2==0}
        \bsFOR{p=0}{\width(ref)-1}
            \bsCODE{ref2[q][p]=ref[q//2][p]}
        \bsEND
    \bsELSE
        \bsFOR{p=0}{\width(ref)-1}
            \bsCODE{ref2[q][p]=16}
            \bsFOR{i=0}{3}
                \bsCODE{ypos=(q-1)//2-i}
                \bsCODE{ref2[q][p]+=t[i]*ref[\clip(ypos,0,\height(ref)-1)][p]}
                \bsCODE{ypos=(q+1)//2+i}
                \bsCODE{ref2[q][p]+=t[i]*ref[\clip(ypos,0,\height(ref)-1)][p]}
            \bsEND
            \bsCODE{ref2[q][p] \gg=5}
            \bsCODE{ref2[q][p] = \clip(ref2[q][p], -2^{bit\_depth-1}, 2^{bit\_depth-1}-1)}
        \bsEND    
    \bsEND
\bsEND
\bsFOR{q=0}{2*\height(ref)-2}
    \bsFOR{p=0}{2*\width(ref)-2}
        \bsIF{p\%2==0}
            \bsCODE{upref[q][p]=ref2[q][p//2]}
        \bsELSE
            \bsCODE{upref[q][p]=16}
            \bsFOR{i=0}{3}
                \bsCODE{xpos=(p-1)//2-i}
                \bsCODE{upref[q][p]+=t[i]*ref2[q][\clip(xpos,0,\width(ref)-1)]}
                \bsCODE{xpos=(p+1)//2+i}
                \bsCODE{upref[q][p]+=t[i]*ref2[q][\clip(xpos,0,\width(ref)-1)]}
            \bsEND
            \bsCODE{upref[q][p] \gg=5}
            \bsCODE{upref[q][p] = \clip(upref[q][p], -2^{bit\_depth-1}, 2^{bit\_depth-1}-1)}
        \bsEND
    \bsEND
\bsEND
\end{pseudo}

\begin{informative}
While this filter may appear to be variable separable, the integer rounding and 
clipping processes prevent this being so. Note also that the clipping process for
filtering terms implies that the upconversion uses edge-extension at the array
edges, consistent with the edge-extension used in motion-compensation itself.
\end{informative}