CN113950838B - Sub-block-based intra-block copying - Google Patents

Abstract

A method of video processing includes determining that a current block of video is divided into a plurality of sub-blocks for a transition between the current block and a bitstream representation of the video. At least one of the plurality of blocks is encoded using a modified Intra Block Copy (IBC) codec technique that uses reference samples from one or more video regions of a current picture of the current block. The method also includes performing a conversion based on the determination.

Description

Sub-block based intra block replication

Cross Reference to Related Applications

The present application aims to claim in time the priority and benefit of international patent application No. pct/CN2019/090409 filed on 6 th month 6 of 2019, in accordance with applicable patent laws and/or in accordance with rules of paris convention. The entire disclosure of the foregoing application is incorporated by reference as part of the disclosure of this application for all purposes in accordance with the united states law.

Technical Field

This document relates to video and image encoding and decoding techniques.

Background

Digital video occupies the largest bandwidth usage on the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth requirements of digital video usage are expected to continue to increase.

Disclosure of Invention

The disclosed techniques may be used by video or image decoder or encoder embodiments to perform encoding or decoding of video bitstream intra block copy segmentation techniques at the sub-block level.

In one example aspect, a method of video processing is disclosed. The method includes determining that a current block of video is divided into a plurality of sub-blocks for a transition between the current block and a bitstream representation of the video. At least one of the plurality of blocks is encoded using a modified Intra Block Copy (IBC) codec technique that uses reference samples from one or more video regions of a current picture of the current block. The method also includes performing a conversion based on the determination.

In another example aspect, a method of video processing is disclosed. The method includes determining that a current block of video is divided into a plurality of sub-blocks for a transition between the current block and a bitstream representation of the video. According to the mode, each of the plurality of sub-blocks is encoded in the encoded representation using a corresponding encoding and decoding technique. The method also includes performing a conversion based on the determination.

In another example aspect, a method of video processing is disclosed. The method includes determining an operation associated with a motion candidate list for a transition between a current block of video and a bitstream representation of video based on a condition related to a characteristic of the current block. The motion candidate list is constructed for the codec technique or based on information from previously processed blocks of the video. The method also includes performing a conversion based on the determination.

In another example aspect, a method of video processing is disclosed. The method includes determining, for a transition between a current block of video and a bitstream representation of video, that the current block encoded using an inter-frame encoding and decoding technique based on time domain information is divided into a plurality of sub-blocks. At least one of the plurality of blocks is encoded using a modified Intra Block Copy (IBC) codec technique that uses reference samples from one or more video regions of a current picture that includes the current block. The method also includes performing a conversion based on the determination.

In another example aspect, a method of video processing is disclosed. The method includes determining a sub-block intra block copy (sbIBC) codec mode to be used in a transition between a current video block and a bitstream representation of the current video block in a video region, wherein the current video block is divided into a plurality of sub-blocks and each sub-block is encoded based on reference samples from the video region, wherein a size of the sub-block is based on a division rule, and performing the transition using sbIBC codec mode for the plurality of sub-blocks.

In another example aspect, a method of video processing is disclosed. The method includes determining a coding mode using intra-sub-block copying (sbIBC) in a transition between a current video block and a bitstream representation of the current video block in a video region, wherein the current video block is divided into a plurality of sub-blocks and each sub-block is coded based on reference samples from the video region, and performing the transition using sbIBC coding mode for the plurality of sub-blocks, wherein the transition includes determining an initialized motion vector (initMV) for a given sub-block, identifying a reference block from initMV, and deriving Motion Vector (MV) information for the given sub-block using MV information for the reference block.

In another example aspect, a method of video processing is disclosed. The method includes determining to use a sub-block intra block copy (sbIBC) codec mode in a transition between a current video block and a bitstream representation of the current video block in a video region, wherein the current video block is divided into a plurality of sub-blocks and each sub-block is encoded based on reference samples from the video region, and performing the transition using the sbIBC codec mode for the plurality of sub-blocks, wherein the transition includes generating sub-block IBC candidates.

In another example aspect, a method of video processing is disclosed. The method includes performing a transition between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the transition includes processing a first sub-block of the plurality of sub-blocks using an intra-sub-block codec (sbIBC) mode and processing a second sub-block of the plurality of sub-blocks using the intra-block codec mode.

In another example aspect, a method of video processing is disclosed. The method includes performing a conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the conversion includes processing all of the plurality of sub-blocks using an intra-coding mode.

In another example aspect, a method of video processing is disclosed. The method includes performing a conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the conversion includes processing all of the plurality of sub-blocks using a palette of representative pixel values for a palette codec mode that encodes each sub-block.

In another example aspect, a method of video processing is disclosed. The method includes performing a conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the conversion includes processing a first sub-block of the plurality of sub-blocks using a palette of representative pixel values for encoding and decoding the first sub-block, and processing a second sub-block of the plurality of sub-blocks using an intra block copy encoding and decoding mode.

In another example aspect, a method of video processing is disclosed. The method includes performing a conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the conversion includes processing a first sub-block of the plurality of sub-blocks using a palette of representative pixel values for encoding and decoding the first sub-block, and processing a second sub-block of the plurality of sub-blocks using an intra-coding mode.

In another example aspect, a method of video processing is disclosed. The method includes performing a transition between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the transition includes processing a first sub-block of the plurality of sub-blocks using an intra-sub-block codec (sbIBC) mode and processing a second sub-block of the plurality of sub-blocks using an inter-block codec mode.

In another example aspect, a method of video processing is disclosed. The method includes performing a conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the conversion includes processing a first sub-block of the plurality of sub-blocks using an intra-sub-block codec mode and processing a second sub-block of the plurality of sub-blocks using an inter-block codec mode.

In another example aspect, a method of video processing is disclosed. The method comprising deciding to encode a current video block into a bitstream representation using the method of any of the preceding claims, and including information indicative of the decision in the bitstream representation at a decoder parameter set level or sequence parameter set level or video parameter set level or picture header level or slice header level or maximum codec unit level or codec unit row level or LCU group level or transform unit level or prediction unit level or video codec unit level.

In another example aspect, a method of video processing is disclosed. The method comprising deciding to encode a current video block into a bitstream representation based on encoding conditions using the method of any of the preceding claims, and performing encoding using the method of any of the preceding claims, wherein the conditions are based on one or more of a codec unit, a prediction unit, a transform unit, a current video block, or a position of a video codec unit of the current video block.

In another example aspect, a method of video processing is disclosed. The method includes determining to use an intra block copy mode and an inter prediction mode for a transition between blocks in a video region and a bitstream representation of the video region, and performing the transition using the intra block copy mode and the inter prediction mode for blocks in the video region.

In another example aspect, a method of video processing is disclosed. The method includes performing a motion candidate list construction process and/or a table update process for updating a history-based motion vector predictor table based on codec conditions during a transition between a current video block and a bitstream representation of the current video block, and performing the transition based on the motion candidate list construction process and/or the table update process.

In another example aspect, the above-described method may be implemented by a video decoder apparatus including a processor.

In another example aspect, the above-described method may be implemented by a video encoder apparatus comprising a processor.

In yet another example aspect, the methods may be embodied in the form of processor-executable instructions and stored on a computer-readable program medium.

These and other aspects are further described in this document.

Drawings

Fig. 1 shows the derivation process of the Merge candidate list construction.

Fig. 2 shows an example of the location of the spatial Merge candidate.

Fig. 3 shows an example of candidate pairs that consider redundancy checks for spatial Merge candidates.

Fig. 4 shows example locations of the second PU for nx2n and 2nxn partitions.

Fig. 5 shows an example of a graphical representation of motion vector scaling of a temporal Merge candidate.

Fig. 6 shows an example of candidate positions of the time domain Merge candidates C0 and C1.

Fig. 7 shows an example of combining bi-prediction Merge candidates.

Fig. 8 shows an example of a derivation process of motion vector prediction candidates.

Fig. 9 shows an example illustration of motion vector scaling of spatial motion vector candidates.

Fig. 10 shows an example reduced affine motion model of a 4-parameter affine model (left) and a 6-parameter affine model (right).

Fig. 11 shows an example of an affine motion vector field for each sub-block.

FIG. 12 illustrates example candidate locations for an affine Merge schema.

Fig. 13 shows an example of a modified Merge list construction process.

Fig. 14 shows an example of inter prediction based on triangle division.

Fig. 15 shows an example of a CU to which the first weighting factor group is applied.

Fig. 16 shows an example of motion vector storage.

Fig. 17 shows an example of a final motion vector expression (UMVE) search process.

Fig. 18 shows an example of UMVE search points.

FIG. 19 shows an example of MVDs (0, 1) mirrored between List 0 and List 1 in DMVR

Fig. 20 shows MVs that can be checked in one iteration.

Fig. 21 is an example of intra block copy.

Fig. 22 is a block diagram of an example of a video processing apparatus.

Fig. 23 is a flowchart of an example of a video processing method.

FIG. 24 is a block diagram of an example video processing system in which the disclosed techniques may be implemented.

Fig. 25 is a flowchart representation of a method for video processing in accordance with the present technique.

Fig. 26 is a flow chart representation of another method for video processing in accordance with the present technique.

Fig. 27 is a flow chart representation of another method for video processing in accordance with the present technique.

Fig. 28 is a flow chart representation of yet another method for video processing in accordance with the present technique.

Detailed Description

The present document provides various techniques that may be used by decoders of image or video bitstreams to improve the quality of decompressed or decoded digital video or images. For brevity, the term "video" is used herein to include sequences of pictures (conventionally referred to as video) and individual images. Furthermore, the video encoder may also implement these techniques during the encoding process in order to reconstruct the decoded frames for further encoding.

For ease of understanding, chapter titles are used in this document and the embodiments and techniques are not limited to corresponding chapters. Thus, embodiments from one section may be combined with embodiments from other sections.

1. Summary

This document relates to video codec technology. In particular, it relates to intra block copy (also known as current picture reference, CPR) codec. It may be applied to existing video codec standards (e.g., HEVC), or standards to be finalized (multi-function video codec). It may also be applicable to future video codec standards or video codecs.

2. Background

Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO/IEC standards. ITU-T generates h.261 and h.263, ISO/IEC generates MPEG-1 and MPEG-4Visual, and these two organizations combine to generate the h.262/MPEG-2 video and 264/MPEG-4 advanced video codec (Advanced Video Coding, AVC) standard and the h.265/HEVC standard. Since h.262, video codec standards have been based on hybrid video codec structures, where temporal prediction plus transform coding is utilized. To explore future video codec technologies beyond HEVC, VCEG and MPEG have combined in 2015 to form a joint video exploration team (Joint Video Exploration Team, JVET). Thereafter, JVET employed a number of new approaches and placed them into reference software named joint exploration model (Joint Exploration Model, JEM). A joint video expert group (Joint Video Expert Team, JVET) between VCEG (Q6/16) and ISO/IEC JTC1 SC29/WG11 (MPEG) was established to address the VVC standard with the goal of 50% bit rate reduction compared to HEVC, month 4 of 2018.

2.1 Inter prediction in HEVC/H.265

For an inter-Codec Unit (CU), it can codec with one Prediction Unit (PU), 2 PUs, according to the partition mode. Each inter prediction PU has the motion parameters of one or two reference picture lists. The motion parameters include a motion vector and a reference picture index. The use of one of the two reference picture lists may also be signaled using inter predidc. The motion vector may be coded and decoded as an increment relative to the predicted value.

When a CU is encoded in skip mode, one PU is associated with the CU and there are no significant residual coefficients, no motion vector delta or reference picture index for the encoding and decoding. The Merge mode is specified whereby the motion parameters of the current PU are obtained from neighboring PUs that include spatial and temporal candidates. The Merge mode may be applied to any inter prediction PU, not just for skip mode. An alternative to the Merge mode is the explicit transmission of motion parameters, wherein motion vectors, more precisely motion vector differences (Motion Vector Difference, MVD) compared to motion vector predictors, corresponding reference picture indices and reference picture list usage per reference picture list are explicitly signaled per PU. Such a mode is named advanced motion vector prediction (Advanced Motion Vector Prediction, AMVP) in this disclosure.

When the signaling indicates that one of the two reference picture lists is to be used, a PU is generated from a block of one sample. This is called "unidirectional prediction". Unidirectional prediction applies to both P-stripes and B-stripes.

When the signaling indicates that two reference picture lists are to be used, a PU is generated from two sample blocks. This is called "bi-prediction". Bi-directional prediction is only applicable to B-stripes.

2.1.1 Reference Picture List

In HEVC, the term inter prediction is used to refer to predictions derived from data elements (e.g., sample values or motion vectors) of reference pictures other than the currently decoded picture. As in h.264/AVC, pictures may be predicted from multiple reference pictures. The reference pictures for inter prediction are organized in one or more reference picture lists. The reference index identifies which reference picture in the list should be used to create the prediction signal.

A single reference picture list (list 0) is used for P slices and two reference picture lists (list 0 and list 1) are used for B slices. It should be noted that the reference pictures included in list 0/1 may be from past and future pictures in terms of capture/display order.

2.1.2Merge mode

2.1.2.1 Derivation of candidates for Merge mode

When predicting a PU using the Merge mode, an index to an entry in the Merge candidate list is parsed from the bitstream and used to retrieve motion information. The construction of this list is specified in the HEVC standard and can be summarized according to the following sequence of steps:

step 1 initial candidate derivation

Step 1.1 spatial candidate derivation

Step 1.2 redundant checking of airspace candidates

Step 1.3 time Domain candidate derivation

Step 2, appending candidate inserts

Step 2.1 creation of bi-prediction candidates

Step 2.2 inserting zero motion candidates

These steps are also schematically depicted in fig. 1. For spatial-domain Merge candidate derivation, up to four Merge candidates are selected from among the candidates located at five different positions. For time domain Merge candidate derivation, a maximum of one Merge candidate is selected among the two candidates. Since the number of candidates per PU is assumed to be constant at the decoder, additional candidates are generated when the number of candidates obtained from step 1 does not reach the maximum number of Merge candidates signaled in the slice header (MaxNumMergeCand). Since the number of candidates is constant, the index of the best Merge candidate is encoded using truncated unary binarization (Truncated Unary, TU). If the size of the CU is equal to 8, then all PUs of the current CU share a single Merge candidate list, which is the same as the Merge candidate list of the 2Nx2N prediction unit.

Hereinafter, the operations associated with the foregoing steps are described in detail.

2.1.2.2 Spatial candidate derivation

In the derivation of the spatial-domain Merge candidates, up to four Merge candidates are selected from among candidates located at the positions depicted in fig. 2. The deduced sequences are a ₁、B₁、B₀、A₀ and B ₂. Position B ₂ is only considered when either PU of position a ₁、B₁、B₀、A₀ is not available (e.g., because it belongs to another slice or slice) or is intra-coded. After the candidates at the position a ₁ are added, redundancy check is performed on the addition of the remaining candidates, which ensures that candidates having the same motion information are excluded from the list, thereby improving the codec efficiency. In order to reduce the computational complexity, all possible candidate pairs are not considered in the redundancy check mentioned. Instead, only pairs linked by arrows in fig. 3 are considered, and corresponding candidates are added to the list only when the candidates for redundancy check do not have the same motion information. Another source of duplicate motion information is a "second PU" associated with a partition other than 2n×2n. As an example, fig. 4 depicts a second PU for the case of nx2n and 2nxn. When the current PU is partitioned into nx2n, the candidate at position a ₁ is not considered for list construction. In fact, by adding this candidate will result in two prediction units having the same motion information, which is redundant for having only one PU in the coding unit. Similarly, when the current PU is partitioned into 2nxn, position B ₁ is not considered.

2.1.2.3 Time domain candidate derivation

In this step, only one candidate is added to the list. Specifically, in the derivation of the temporal Merge candidate, a scaled motion vector is derived based on the collocated PU in the collocated picture. The derived reference picture list to be used for the collocated PU is signaled explicitly in the slice header. A scaled motion vector of the temporal Merge candidate is obtained as shown by the dashed line in fig. 5, which is scaled from the motion vector of the collocated PU using POC distances tb and td, where tb is defined as the POC difference between the reference picture of the current picture and td is defined as the POC difference between the reference picture of the collocated picture and the collocated picture. The reference picture index of the temporal Merge candidate is set to zero. The actual implementation of the scaling procedure is described in the HEVC specification. For the B slices, two motion vectors are obtained, one for reference picture list 0 and the other for reference picture list 1, and combined to form bi-prediction Merge candidates.

2.1.2.4 Juxtaposition pictures and juxtaposition PUs

When TMVP is enabled (e.g., slice_temporal_mvp_enabled_flag is equal to 1), the variable ColPic representing the collocated picture is derived as follows:

if the current stripe is a B stripe and the signaled localized_from_l0_flag is equal to 0, colPic is set equal to RefPicList1[ localized_ref_idx ].

Otherwise (slice_type equals B and allocated_from_l0_flag equals 1, or slice_type equals P), colPic is set equal to RefPicList0[ allocated_ref_idx ].

Here, the allocated_ref_idx and the allocated_from_l0_flag are two syntax elements that can be signaled in the slice header.

As depicted in fig. 6, in the collocated PU (Y) belonging to the reference frame, the location of the time domain candidate is selected between candidates C ₀ and C ₁. If the PU at position C ₀ is not available, is intra-coded or is outside the current codec tree unit (CTU, also known as LCU, largest codec unit) row, position C ₁ is used. Otherwise, position C ₀ is used in the derivation of the time domain Merge candidate.

The relevant syntax elements are described as follows:

7.3.6.1 generic stripe segment header syntax

2.1.2.5 Derivation of TMVP candidate MVs

More specifically, the following steps are performed to derive TMVP candidates:

(1) A reference picture list x=0 is set, and the target reference picture is a reference picture (for example, currref) in list X, which has an index equal to 0. The derivation of the collocated motion vector is invoked to get the MV with list X pointing to currref.

(2) If the current slice is a B slice, a reference picture list x=1 is set, and the target reference picture is a reference picture (e.g., currref) in list X with an index equal to 0. The derivation of the collocated motion vector is invoked to get the MV with list X pointing to currref.

The derivation of the collocated motion vector is described in the next subsection.

2.1.2.5.1 Derivation of collocated motion vectors

For collocated blocks, they may be intra-or inter-frame coded with unidirectional prediction or bi-prediction. If it is intra-coded, the TMVP candidate is set to unavailable.

If it is a unidirectional prediction from list a, then the motion vector of list a is scaled to the target reference picture list X.

If bi-prediction is performed and the target reference picture list is X, the motion vector of list a is scaled to target reference picture list X and a is determined according to the following rules:

-a is set equal to X if no reference picture has a larger POC value than the current picture.

Otherwise, a is set equal to the allocated_from_l0_flag.

Some related descriptions are included as follows:

8.5.3.2.9 derivation of collocated motion vectors

The inputs of this process are:

a variable currPb, specifying the current prediction block,

A variable colPb specifying the collocated prediction block within the collocated picture specified by ColPic,

Luminance location (xColPb, yColPb), designating a left top sample of the collocated luminance prediction block designated by colPb relative to a left top luminance sample of the collocated picture designated by ColPic,

-Reference index refIdxLX, wherein X is 0 or 1.

The output of this process is:

-a motion vector prediction mvLXCol, which,

Availability flag availableFlagLXCol.

The variable currPic specifies the current picture.

The arrays predflag l0Col [ x ] [ y ], mvL0Col [ x ] [ y ] and refIdxL0Col [ x ] [ y ] are set equal to PREDFLAGL [ x ] [ y ], mvL [ x ] [ y ] and RefIdxL [ x ] [ y ] of the juxtaposed pictures designated by ColPic, respectively, and the arrays predflag l1Col [ x ] [ y ], mvL1Col [ x ] [ y ] and refIdxL1Col [ x ] [ y ] are set equal to PREDFLAGL [ x ] [ y ], mvL [ x ] [ y ] and RefIdxL [ x ] [ y ] of the juxtaposed pictures designated by ColPic, respectively.

Variables mvLXCol and availableFlagLXCol were derived as follows:

if colPb is coded in intra prediction mode, both components of mvLXCol are set equal to 0 and availableFlagLXCol is set equal to 0.

Otherwise, the motion vector mvCol, the reference index refIdxCol and the reference list identifier listCol are derived as follows:

If predFlagL0Col [ xColPb ] [ yColPb ] is equal to 0, mvCol, refIdxCol and listCol are set equal to mvL1Col [ xColPb ] [ yColPb ], refIdxL1Col [ xColPb ] [ yColPb ] and L1, respectively.

Otherwise, if predflag L0Col [ xColPb ] [ yColPb ] is equal to 1 and predflag L1Col [ xColPb ] [ yColPb ] is equal to 0, mvCol, refIdxCol and listCol are set equal to mvL0Col [ xColPb ] [ yColPb ], refIdxL0Col [ xColPb ] [ yColPb ] and L0, respectively.

Otherwise (predflag l0Col [ xColPb ] [ yColPb ] equals 1, and predflag l1Col [ xColPb ] [ yColPb ] equals 1), the following assignments are made:

If NoBackwardPredFlag is equal to 1, mvCol, refIdxCol and listCol are set equal to mvLXCol [ xColPb ] [ yColPb ], refIdxLXCol [ xColPb ] [ yColPb ] and LX, respectively.

Otherwise mvCol, refIdxCol and listCol are set equal to mvLNCol [ xColPb ] [ yColPb ], refIdxLNCol [ xColPb ] [ yColPb ], and LN, respectively, where N is the value of the registered_from_l0_flag.

And mvLXCol and availableFlagLXCol are derived as follows:

-if LongTermRefPic (currPic, currPb, refIdxLX LX, LX) is not equal to LongTermRefPic (ColPic, colPb, refIdxCol, listCol), both components of mvLXCol are set equal to 0 and availableFlagLXCol is set equal to 0.

Otherwise, variable availableFlagLXCol is set equal to 1, refpiclistcol [ refidxcl ] is set to the picture with reference index refIdxCol in reference picture list listCol containing the slice of predicted block colPb in the collocated picture specified by ColPic, and the following applies:

colPocDiff=DiffPicOrderCnt(ColPic,refPicListCol[refIdxCol]) (2-1)

currPocDiff=DiffPicOrderCnt(currPic,RefPicListX[refIdxLX]) (2-2)

-if RefPicListX [ refIdxLX ] is a long-term reference picture, or colPocDiff is equal to currPocDiff, mvLXCol is derived as follows:

mvLXCol=mvCol (2-3)

otherwise, mvLXCol is derived as a scaled version of the motion vector mvCol, as follows:

tx=(16384+(Abs(td)>>1))/td (2-4)

distScaleFactor=Clip3(-4096,4095,(tb*tx+32)>>6) (2-5)

mvLXCol=Clip3(-32768,32767,Sign(distScaleFactor*mvCol)*

((Abs(distScaleFactor*mvCol)+127)>>8)) (2-6)

Wherein td and tb are derived as follows:

td=Clip3(-128,127,colPocDiff) (2-7)

tb=Clip3(-128,127,currPocDiff) (2-8)

NoBackwardPredFlag is defined as:

The variables NoBackwardPredFlag are derived as follows:

-NoBackwardPredFlag is set equal to 1 if DiffPicOrderCnt (aPic, currPic) of each picture aPic in RefPicList0 or RefPicList1 of the current slice is less than or equal to 0.

Otherwise NoBackwardPredFlag is set equal to 0.

2.1.2.6 Additional candidate inserts

In addition to the space-time Merge candidates, there are two additional types of Merge candidates, the combined bi-predictive Merge candidate and the zero Merge candidate. The combined bi-predictive Merge candidate is generated by using the space-time Merge candidate. The combined bi-predictive Merge candidate is only for the B stripe. The combined bi-prediction candidate is generated by combining the first reference picture list motion parameter of the initial candidate with the second reference picture list motion parameter of the other. If the two tuples provide different motion hypotheses they will form new bi-prediction candidates. As an example, fig. 7 depicts the case where two candidates in the original list (on the left) with mvL0 and refIdxL0 or mvL1 and refIdxL1 are used to create a combined bi-prediction Merge candidate that is added to the final list (on the right). There are many rules regarding combinations that are considered to generate these additional Merge candidates.

The zero motion candidate is inserted to fill the remaining entries in the Merge candidate list and thus reach MaxNumMergeCand capacity. These candidates have zero spatial displacement and a reference picture index that starts from zero and increases each time a new zero motion candidate is added to the list. Finally, no redundancy check is performed on these candidates.

2.1.3 AMVP

AMVP exploits the spatial-temporal correlation of motion vectors with neighboring PUs, which is used for explicit transmission of motion parameters. For each reference picture list, a motion vector candidate list is constructed by first checking the availability of left, upper temporal neighboring PU locations, removing redundant candidates, and adding zero vectors to make the candidate list a constant length. The encoder may then select the best predictor from the candidate list and send a corresponding index indicating the selected candidate. Similar to the Merge index signaling, the index of the best motion vector candidate uses truncated unary coding. In this case, the maximum value to be encoded is 2 (see fig. 8). In the following section, details are provided regarding the derivation process of motion vector prediction candidates.

2.1.3.1 Derivation of AMVP candidates

Fig. 8 summarizes the derivation of motion vector prediction candidates.

In motion vector prediction, two types of motion vector candidates are considered, spatial motion vector candidates and temporal motion vector candidates. For spatial domain motion vector candidate derivation, two motion vector candidates are ultimately derived based on the motion vector of each PU located in five different locations as depicted in fig. 2.

For temporal motion vector candidate derivation, one motion vector candidate is selected from two candidates, which are derived based on two different collocated positions. After the first space-time selection list is generated, the repeated motion vector candidates in the list are removed. If the number of potential candidates is greater than 2, motion vector candidates having a reference picture index greater than 1 within the list are removed from the associated reference picture list. If the number of space-time motion vector candidates is less than two, additional zero motion vector candidates are added to the list.

2.1.3.2 Spatial motion vector candidates

In the derivation of spatial motion vector candidates, a maximum of two candidates are considered among five potential candidates derived from PUs located at the same locations as the motion Merge as depicted in fig. 2. The derivation order to the left of the current PU is defined as a ₀、A₁ and scaled a ₀, scaled a ₁. The derivation order above the current PU is defined as B ₀、B₁、B₂, scaled B ₀, scaled B ₁, scaled B ₂. Thus, for each side, there are four cases that can be used as motion vector candidates, two of which do not require the use of spatial scaling and two of which use spatial scaling. These four different cases are summarized as follows:

* Non-spatial scaling

(1) Identical reference picture list and identical reference picture index (identical POC)

(2) Different reference picture lists but the same reference picture (same POC)

* Spatial scaling

(3) Identical reference picture list but different reference pictures (different POCs)

(4) Different reference picture lists and different reference pictures (different POCs)

First check for no spatial scaling, then check for spatial scaling. Regardless of the reference picture list, spatial scaling is considered when POC between the reference picture of the neighboring PU and the reference picture of the current PU is different. If all PUs of the left candidate are not available or are intra-coded, then scaling of the upper motion vector is allowed to assist in the parallel derivation of the left and upper MV candidates. Otherwise, spatial scaling of the upper motion vector is not allowed.

As depicted in fig. 9, in the spatial scaling process, the motion vectors of neighboring PUs are scaled in a similar manner as the temporal scaling. The main difference is that the reference picture list and the index of the current PU are given as inputs, the actual scaling procedure is the same as the scaling procedure of the temporal scaling.

2.1.3.3 Temporal motion vector candidates

All procedures for deriving temporal Merge candidates are the same as those for deriving spatial motion vector candidates except for reference picture index derivation (see fig. 6). The reference picture index is signaled to the decoder.

2.2 Inter prediction method in VVC

There are several new codec tools for inter prediction improvement, such as adaptive motion vector difference resolution (AMVR) for signaling MVD, merge with motion vector difference (MMVD), triangle Prediction Mode (TPM), combined intra-inter prediction (CIIP), advanced TMVP (ATMVP, also known as SbTMVP), affine prediction mode, generalized bi-prediction (GBI), decoder side motion vector refinement (DMVR) and bi-directional optical flow (BIO, also known as BDOF).

Three different Merge list building processes are supported in VVC:

(1) A sub-block Merge candidate list comprising ATMVP and affine Merge candidates. One Merge list construction process is shared for affine and ATMVP modes. Here, ATMVP and affine Merge candidates may be added sequentially. The sub-block Merge list size is signaled in the header and has a maximum value of 5.

(2) Conventional Merge list-for inter-codec blocks, one Merge list construction process is shared. Here, the spatial/temporal Merge candidates, HMVP, the paired Merge candidates, and the zero motion candidates may be sequentially inserted. The regular Merge list size is signaled in the header and has a maximum of 6.MMVD, TPM, CIIP rely on a conventional Merge list.

(3) IBC Merge list, which is done in a similar manner to the conventional Merge list.

Similarly, three AMVP lists are supported in VVC:

(1) Affine AMVP candidate list

(2) Conventional AMVP candidate list

(3) Ibc.amvp candidate list the same construction procedure as IBC Merge list.

Coding and decoding block structure in 2.2.1VVC

In VVC, a quadtree/binary tree/trigeminal tree (QT/BT/TT) structure is used to divide a picture into square or rectangular blocks.

In addition to QT/BT/TT, a separate tree (also known as a double-coded tree) is used in VVC for I frames. In the case of a separate tree, the codec block structure is signaled separately for the luma and chroma components.

Furthermore, a CU is set equal to a PU and a TU, except for blocks that are coded with several specific coding methods (such as intra sub-partition prediction, where PU is equal to a TU but smaller than a CU, and sub-block transforms of inter coding blocks, where PU is equal to a CU but smaller than a PU).

2.2.2 Affine prediction modes

In HEVC, only translational motion models are applied to motion compensated prediction (Motion Compensation Prediction, MCP). Although in the real world there are many kinds of movements, such as zoom in/out, rotation, perspective movement and other irregular movements. In VVC, a reduced affine transformation motion compensated prediction is applied using a 4-parameter affine model and a 6-parameter affine model. As shown in fig. 10, the affine motion field of a block is described by two Control Point Motion Vectors (CPMV) of a 4-parameter affine model and 3 CPMV of a 6-parameter affine model.

The Motion Vector Field (MVF) of a block is described by the following equations with a 4-parameter affine model in equation (1) where 4 parameters are defined as variables a, b, e and f, and a 6-parameter affine model in equation (2) where 4 parameters are defined as variables a, b, c, d, e and f, respectively:

Where (mv ^h ₀,mv^h ₀) is the motion vector of the upper left corner control point, and (mv ^h ₁,mv^h ₁) is the motion vector of the upper right corner control point, and (mv ^h ₂,mv^h ₂) is the motion vector of the lower left corner control point, all three motion vectors are referred to as Control Point Motion Vectors (CPMV), (x, y) represent coordinates of the representative point with respect to the upper left sample point within the current block, and (mv ^h(x,y),mv^v (x, y)) is the motion vector derived for the sample point located at (x, y). The CP motion vector may be signaled (as in affine AMVP mode) or derived on the fly (as in affine Merge mode). w and h are the width and height of the current block. In practice, division is performed by right-shifting and rounding operations. In the VTM, the representative point is defined as the center position of the sub-block, for example, when the coordinates of the upper left corner of the sub-block with respect to the upper left sample point within the current block are (xs, ys), the coordinates of the representative point are defined as (xs+2, ys+2). For each sub-block (e.g., 4 x 4 in VTM), the representative points are utilized to derive the motion vector for the entire sub-block.

To further simplify motion compensated prediction, sub-block based affine transformation prediction is applied. In order to derive the motion vector of each m×n (in the current VVC, M and N are both set to 4) sub-block, the motion vector of the center sample of each sub-block (as shown in fig. 11) may be calculated according to equations (1) and (2) and rounded to 1/16 of the fractional precision. A 1/16 pixel motion compensated interpolation filter may then be applied to generate a prediction for each sub-block with the derived motion vector. Affine mode introduces a 1/16 pixel interpolation filter.

After MCP, the high precision motion vector for each sub-block is rounded and saved to the same precision as the standard motion vector.

2.2.3 Merge of whole block

2.2.3.1 Translation of Merge table construction in conventional Merge mode

2.2.3.1.1 History-based motion vector prediction (HMVP)

Unlike the Merge list design, in VVC, a history-based motion vector prediction (HMVP) method is employed.

In HMVP, previously encoded motion information is stored. The motion information of the previous codec block is defined as HMVP candidates. The plurality of candidates HMVP are stored in a table named HMVP table, and the table is dynamically maintained during the encoding/decoding process. When the encoding/decoding of a new slice/LCU row/stripe is started, the HMVP table is emptied. Whenever there is an inter codec block and a non-sub-block, non-TPM mode, the associated motion information is added to the last entry of the table as a new HMVP candidate. The entire codec flow is depicted in fig. 12.

2.2.3.1.2 Conventional Merge list construction procedure

The construction of a conventional Merge list (for translational movement) can be summarized according to the following sequence of steps:

Step 1, deriving airspace candidates

Step 2, inserting HMVP candidates

Step 3, inserting pairwise average candidates

Step4, default motion candidate

HMVP candidates can be used in the AMVP and Merge candidate list construction process. Fig. 13 depicts a modified Merge candidate list construction process. When the Merge candidate list is not full after TMVP candidate insertion, HMVP candidates stored in the HMVP table may be used to populate the Merge candidate list. Considering that one block typically has a higher correlation with the nearest neighbor block in terms of motion information, HMVP candidates in the table are inserted in descending order of index. The last entry in the table is added to the list first and the first entry is added to the last. Similarly, redundancy removal is also applied to HMVP candidates. Once the total number of available Merge candidates reaches the maximum number of Merge candidates that are allowed to be signaled, the Merge candidate list construction process terminates.

Note that all spatial/temporal/HMVP candidates should be coded in non-IBC mode. Otherwise, it is not allowed to be added to the regular Merge candidate list.

The HMVP table contains up to 5 conventional motion candidates and each of them is unique.

2.2.3.1.2.1 Pruning Process

The candidates are only added to the list if the corresponding candidates for redundancy check do not have the same motion information. Such a comparison process is referred to as a pruning process.

The pruning process among the spatial candidates depends on the use of the TPM for the current block.

When the current block is not being encoded in TPM mode (e.g., conventional Merge, MMVD, CIIP), the HEVC pruning process (e.g., five prunes) of spatial domain Merge candidates is utilized.

2.2.4 Triangle prediction modes

In VVC, a triangle division mode is supported for inter prediction. The triangle splitting mode is only applied to CUs that are 8x8 or larger and are coded in the Merge mode but not in the MMVD or CIIP modes. For CUs that meet these conditions, a CU level flag is signaled to indicate whether or not to apply the triangle splitting mode.

When this mode is used, CU is uniformly divided into two triangle divisions using diagonal divisions or anti-diagonal divisions, as depicted in fig. 14. Each triangle partition in the CU is inter predicted using its own motion, only one-way prediction is allowed for each partition, that is, each partition has one motion vector and one reference index. Unidirectional prediction motion constraints are applied to ensure that, as with conventional bi-prediction, only two motion compensated predictions are required for each CU.

If the CU level flag indicates that the current CU is coded using the triangle partition mode, a flag indicating the triangle partition direction (diagonal or anti-diagonal) and two Merge indexes (one for each partition) are further signaled. After predicting each triangle partition, a fusion (blend) process with adaptive weights is used to adjust the sample values along the diagonal or anti-diagonal edges. This is the prediction signal for the entire CU, and the transform and quantization process will be applied to the entire CU as in other prediction modes. Finally, the motion field of the CU predicted using the triangle division mode is stored in 4x4 units.

The conventional Merge candidate list is reused for triangle segmentation Merge prediction without additional motion vector pruning. For each Merge candidate in the regular Merge candidate list, one and only one of its L0 or L1 motion vectors is used for triangle prediction. Furthermore, the order in which L0 versus L1 motion vectors are selected is based on their Merge index parity. With this scheme, the conventional Merge list can be used directly.

2.2.4.1 Merge list construction process of TPM

In some embodiments, the conventional Merge list construction process may include the following modifications:

(1) How the pruning process is performed depends on the use of the TPM for the current block

Invoking HEVC5 pruning applied to spatial Merge candidates if the current block is not encoded with TPM

Otherwise (if the current block is encoded with TPM), a full pruning will be applied when adding a new spatial Merge candidate. That is, B1 compares with A1, B0 compares with A1 and B1, A0 compares with A1, B1 and B0, and B2 compares with A1, B1, A0 and B0.

(2) Whether the condition for checking the motion information from B2 depends on the use of TPM of the current block

If the current block is not encoded with TPM, then B2 is accessed and checked only if there are less than 4 spatial Merge candidates before B2 is checked.

Otherwise (if the current block is encoded with TPM), B2 is always accessed and checked before B2 is added, no matter how many airspace Merge candidates are available.

2.2.4.2 Adaptive weighting procedure

After predicting each triangle prediction unit, an adaptive weighting process is applied to the diagonal edges between the two triangle prediction units to derive the final prediction of the entire CU. Two sets of weighting factors are defined as follows:

the 1 st weighting factor group is {7/8,6/8,4/8,2/8,1/8} and {7/8,4/8,1/8} are used for luminance and chrominance samples, respectively;

the 2 nd weighting factor group is {7/8,6/8,5/8,4/8,3/8,2/8,1/8} and {6/8,4/8,2/8} for luminance and chrominance samples, respectively.

A set of weighting factors is selected based on a comparison of the motion vectors of the two triangular prediction units. The second set of weighting factors is used when any of the following conditions is true:

the reference pictures of the two triangular prediction units are different from each other

The absolute value of the difference between the horizontal values of the two motion vectors is greater than 16 pixels.

The absolute value of the difference between the vertical values of the two motion vectors is greater than 16 pixels.

Otherwise, the first set of weighting factors is used. An example is shown in fig. 15.

2.2.4.3 Motion vector storage

The motion vectors of the triangle prediction units (Mv 1 and Mv2 in fig. 16) are stored in a 4×4 grid. For each 4 x 4 grid, a uni-directional predicted motion vector or a bi-directional predicted motion vector is stored depending on the position of the 4 x 4 grid in the CU. As shown in fig. 16, unidirectional prediction motion vectors Mv1 or Mv2 are stored for a 4×4 grid located in an unweighted region (i.e., not located at a diagonal edge). On the other hand, bi-predictive motion vectors are stored for 4 x 4 grids located in the weighted region. The bi-predictive motion vector is derived from Mv1 and Mv2 according to the following rules:

(1) In the case where Mv1 and Mv2 have motion vectors from different directions (L0 or L1), mv1 and Mv2 are simply combined to form a bi-predictive motion vector.

(2) In the case where both Mv1 and Mv2 are from the same L0 (or L1) direction,

If the reference picture of Mv2 is the same as the picture in the L1 (or L0) reference picture list, then Mv2 is scaled to that picture. Mv1 and scaled Mv2 are combined to form bi-predictive motion vectors.

If the reference picture of Mv1 is the same as the picture in the L1 (or L0) reference picture list, then Mv1 is scaled to that picture. The scaled Mv1 and Mv2 are combined to form a bi-predictive motion vector.

Otherwise, only Mv1 is stored for the weighted region.

2.2.4.4 Syntax table, semantics and decoding procedure for Merge mode

The added changes are highlighted in bold and italics underlined. The deleted portion is marked with [ ].

7.3.5.1 Generic header syntax

7.3.7.5 Coding unit syntax

7.3.7.7 Merge data syntax

7.4.6.1 Generic slice header semantics

Six_minus_max_num_merge_cand specifies the maximum number of Merge Motion Vector Prediction (MVP) candidates supported in the bin subtracted from 6. The maximum number MaxNumMergeCand of Merge MVP candidates is derived as follows:

MaxNumMergeCand=6-six_minus_max_num_merge_cand (7-57)

MaxNumMergeCand should have a value in the range of 1 to 6 inclusive.

Five minus max num subblock Merge cand specifies the maximum number of sub-block based Merge Motion Vector Prediction (MVP) candidates supported in the stripe subtracted from 5. When five minus max num subblock merge cand does not exist, it is inferred to be equal to 5-sps sbtmvp enabled flag. The maximum number MaxNumSubblockMergeCand of sub-block based Merge MVP candidates is derived as follows:

MaxNumSubblockMergeCand=5-five_minus_max_num_subblock_merge_cand (7-58)

MaxNumSubblockMergeCand should have a value in the range of 0 to 5 inclusive.

7.4.8.5 Coding unit semantics

The pred_mode_flag equal to 0 specifies that the current codec unit is to be coded in the inter prediction mode. The pred_mode_flag equal to 1 specifies that the current codec unit is to be coded in intra prediction mode.

When pred_mode_flag does not exist, it is inferred as follows:

-if cbWidth is equal to 4 and cbHeight is equal to 4, pred_mode_flag is inferred to be equal to 1.

Otherwise, pred_mode_flag is inferred to be equal to 1 when decoding an I-slice, and pred_mode_flag is inferred to be equal to 0 when decoding a P-or B-slice, respectively.

For x= x0.. X0+ cbWidth-1 and y= y0.. Y0+ cbHeight-1, the variable CuPredMode [ x ] [ y ] is derived as follows:

-CuPredMode x y is set equal to MODE INTER if predmode flag is equal to 0.

Otherwise (pred_mode_flag equals 1), cuPredMode [ x ] [ y ] is set equal to mode_intra.

The pred_mode_ IBC _flag equal to 1 specifies that the current codec unit is codec in IBC prediction mode. The pred_mode_ IBC _flag equal to 0 specifies that the current coding unit is not coded in IBC prediction mode.

When pred_mode_ ibc _flag is not present, it is inferred as follows:

-if cu_skip_flag [ x0] [ y0] is equal to 1, cbwidth is equal to 4, and cbHeight is equal to 4, pred_mode_ ibc _flag is inferred to be equal to 1.

Otherwise, if cbWidth and cbHeight are both equal to 128, then pred_mode_ ibc _flag is inferred to be equal to 0.

Otherwise, pred_mode_ ibc _flag is inferred to be equal to the value of sps_ ibc _enabled_flag when decoding an I-slice, and to be equal to 0 when decoding a P or B-slice, respectively.

When pred_mode_ IBC _flag is equal to 1, the variable CuPredMode [ x ] [ y ] is set equal to MODE_IBC for x= x0.. X0+ cbWidth-1 and y= y0.. Y0+ cbHeight-1.

General_merge_flag [ x0] [ y0] specifies whether the inter prediction parameters of the current codec unit are inferred from neighboring inter prediction partitions. The array indices x0, y0 specify the position (x 0, y 0) of the left top luma sample of the considered codec block relative to the left top luma sample of the picture.

When the general_merge_flag [ x0] [ y0] is not present, it is inferred as follows:

-if cu_skip_flag [ x0] [ y0] is equal to 1, then general_merge_flag [ x0] [ y0] is inferred to be equal to 1.

Otherwise, general_merge_flag [ x0] [ y0] is inferred to be equal to 0.

Mvp_l0_flag [ x0] [ y0] specifies the motion vector predictor index of list 0, where x0, y0 specifies the position (x 0, y 0) of the left top luma sample of the considered codec block relative to the left top luma sample of the picture.

When mvp_l0_flag [ x0] [ y0] is not present, it is inferred to be equal to 0.

Mvp_l1_flag [ x0] [ y0] has the same semantics as mvp_l0_flag, where l0 and list 0 are replaced by l1 and list 1, respectively.

Inter _ pred _ idc x0 y0 specifies whether list 0, list 1 or bi-prediction is used for the current codec unit according to tables 7-10. The array indices x0, y0 specify the position (x 0, y 0) of the left top luma sample of the considered codec block relative to the left top luma sample of the picture.

TABLE 7-10 name association of inter prediction modes

When inter_pred_idc [ x0] [ y0] is not present, it is inferred to be equal to pred_l0.

7.4.8.7Merge data semantics

The regular_merge_flag [ x0] [ y0] equal to 1 specifies a regular Merge mode for generating inter prediction parameters of the current codec unit. The array indices x0, y0 specify the position (x 0, y 0) of the left top luma sample of the considered codec block relative to the left top luma sample of the picture.

When the regular_merge_flag [ x0] [ y0] is not present, it is inferred as follows:

-the regular_merge_flag [ x0] [ y0] is inferred to be equal to 1 if all of the following conditions are true:

-sps_ mmvd _enabled_flag is equal to 0.

-General_merge_flag [ x0] [ y0] is equal to 1.

-CbWidth x cbHeight is equal to 32.

Otherwise, the regular_merge_flag [ x0] [ y0] is inferred to be equal to 0.

Mmvd _merge_flag [ x0] [ y0] equal to 1 specifies that the Merge mode with motion vector difference is used to generate the inter prediction parameters of the current codec unit. The array indices x0, y0 specify the position (x 0, y 0) of the left top luma sample of the considered codec block relative to the left top luma sample of the picture.

When mmvd _merge_flag [ x0] [ y0] is not present, it is inferred as follows:

-deducing mmvd _merge_flag [ x0] [ y0] equal to 1 if all the following conditions are true:

-sps_ mmvd _enabled_flag is equal to 1.

-General_merge_flag [ x0] [ y0] is equal to 1.

-CbWidth x cbHeight is equal to 32.

-Regular_merge_flag [ x0] [ y0] is equal to 0.

Otherwise, mmvd _merge_flag [ x0] [ y0] is inferred to be equal to 0.

Mmvd _cand_flag [ x0] [ y0] specifies whether the first (0) or second (1) candidate in the Merge candidate list is used with the motion vector differences derived from mmvd _distance_idx [ x0] [ y0] and mmvd _direction_idx [ x0] [ y0 ]. The array indices x0, y0 specify the position (x 0, y 0) of the left top luma sample of the considered codec block relative to the left top luma sample of the picture.

When mmvd _cand_flag [ x0] [ y0] is not present, it is inferred to be equal to 0.

Mmvd _distance_idx [ x0] [ y0] specifies the index used to derive MMVDDISTANCE [ x0] [ y0] as specified in tables 7-12. The array indices x0, y0 specify the position (x 0, y 0) of the left top luma sample of the considered codec block relative to the left top luma sample of the picture.

TABLE 7-12 Specification of MMVDDISTANCE [ x0] [ y0] based on mmvd _distance_idx [ x0] [ y0]

Mmvd _direction_idx [ x0] [ y0] specifies an index used to derive MmvdSign [ x0] [ y0] as specified in tables 7-13. The array indices x0, y0 specify the position (x 0, y 0) of the left top luma sample of the considered codec block relative to the left top luma sample of the picture.

TABLE 7-13-Specification of MmvdSign [ x0] [ y0] based on mmvd _direction_idx [ x0] [ y0]

mmvd_direction_idx[x0][y0]	MmvdSign[x0][y0][0]	MmvdSign[x0][y0][1]
			0	+1	0
1	-1	0
			2	0	+1
3	0	-1

The two components of Merge plus MVD offset MmvdOffset [ x0] [ y0] are derived as follows:

MmvdOffset[x0][y0][0]=(MmvdDistance[x0][y0]<<2)*MmvdSign[x0][y0][0]

(7-124)

MmvdOffset[x0][y0][1]=(MmvdDistance[x0][y0]<<2)*MmvdSign[x0][y0][1]

(7-125)

merge_ subblock _flag [ x0] [ y0] specifies whether the sub-block based inter prediction parameters of the current coding unit are inferred from neighboring blocks. The array indices x0, y0 specify the position (x 0, y 0) of the left top luma sample of the considered codec block relative to the left top luma sample of the picture. When the merge_ subblock _flag [ x0] [ y0] is not present, it is inferred to be equal to 0.

Merge_ subblock _idx [ x0] [ y0] specifies the Merge candidate index of the sub-block-based Merge candidate list, where x0, y0 specifies the position (x 0, y 0) of the left top luma sample of the considered codec block relative to the left top luma sample of the picture.

When merge_ subblock _idx [ x0] [ y0] is not present, it is inferred to be equal to 0.

Ciip _flag [ x0] [ y0] specifies whether the combined inter picture Merge and intra picture prediction is applied to the current codec unit. The array indices x0, y0 specify the position (x 0, y 0) of the left top luma sample of the considered codec block relative to the left top luma sample of the picture.

When ciip _flag [ x0] [ y0] is not present, it is inferred to be equal to 0.

When ciip _flag [ x0] [ y0] is equal to 1, the variable IntraPredModeY [ x ] [ y ] (where x=xcb.. xCb + cbWidth-1 and y=ycb.. yCb + cbHeight-1) is set equal to intra_PLANAR.

The variable MERGETRIANGLEFLAG [ x0] [ y0] (which specifies whether triangle-shaped based motion compensation was used to generate the predicted samples for the current codec unit when decoding the B-slice) is derived as follows:

MERGETRIANGLEFLAG [ x0] [ y0] is set equal to 1 if all of the following conditions are true:

-sps_triangle_enabled_flag is equal to 1.

-Slice_type is equal to B.

-General_merge_flag [ x0] [ y0] is equal to 1.

-MaxNumTriangleMergeCand is greater than or equal to 2.

-CbWidth x cbHeight is greater than or equal to 64.

-Regular_merge_flag [ x0] [ y0] is equal to 0.

-Mmvd _merge_flag [ x0] [ y0] equals 0.

-Merge_ subblock _flag [ x0] [ y0] equals 0.

-Ciip _flag [ x0] [ y0] is equal to 0.

Otherwise MERGETRIANGLEFLAG [ x0] [ y0] is set equal to 0.

The merge_triangle_split_dir [ x0] [ y0] specifies the partition direction of the Merge triangle pattern. The array indices x0, y0 specify the position (x 0, y 0) of the left top luma sample of the considered codec block relative to the left top luma sample of the picture.

When merge_triple_split_dir [ x0] [ y0] is not present, it is inferred to be equal to 0.

The merge_triange_idx0 [ x0] [ y0] specifies the first Merge candidate index of the triangle-shape based motion compensation candidate list, where x0, y0 specifies the position (x 0, y 0) of the left top luma sample of the considered codec block relative to the left top luma sample of the picture.

When merge_triange_idx0 [ x0] [ y0] is not present, it is inferred to be equal to 0.

The merge_triange_idx1 [ x0] [ y0] specifies the second Merge candidate index of the triangle-shape based motion compensation candidate list, where x0, y0 specifies the position (x 0, y 0) of the left top luma sample of the considered codec block relative to the left top luma sample of the picture.

When merge_triange_idx1 [ x0] [ y0] is absent, it is inferred to be equal to 0.

Merge_idx [ x0] [ y0] specifies the Merge candidate index of the Merge candidate list, where x0, y0 specifies the position (x 0, y 0) of the left top luma sample of the considered codec block relative to the left top luma sample of the picture.

When merge_idx [ x0] [ y0] is not present, it is inferred as follows:

-if mmvd _merge_flag [ x0] [ y0] is equal to 1, then merge_idx [ x0] [ y0] is inferred to be equal to mmvd _cand_flag [ x0] [ y0].

-Else (mmvd _merge_flag [ x0] [ y0] equals 0), merge_idx [ x0] [ y0] being inferred to be equal to 0.

2.2.4.4.1 Decoding procedure

In some embodiments, the decoding process is defined as follows:

8.5.2.2 luminance motion vector derivation procedure for Merge mode

This procedure is invoked only when the general_merge_flag [ xCb ] [ yCb ] is equal to 1, where (xCb, yCb) specifies the left top sample of the current luma codec block relative to the left top luma sample of the current picture.

The inputs of this process are:

luminance position (xCb, yCb) of the left top luma sample of the current luma codec block relative to the left top luma sample of the current picture,

A variable cbWidth specifying the width of the current codec block in the luma samples,

A variable cbHeight specifying the height of the current codec block in the luma samples.

The output of this process is:

Luminance motion vectors mvL0[0] [0] and mvL1[0] [0] with 1/16 fractional sample precision,

Reference indices refIdxL0 and refIdxL1,

The prediction list uses the flags predFlagL0[0] [0] and predFlagL1[0] [0],

The bi-directional prediction weight index bcwIdx,

-A Merge candidate list MERGECANDLIST.

The bi-prediction weight index bcwIdx is set equal to 0.

The motion vectors mvL0[0] [0] and mvL1[0] [0], the reference indices refIdxL0 and refIdxL1, and the prediction utility flags predflag l0[0] [0] and predflag l1[0] [0] are derived by the following sequential steps:

1. The derivation process of the spatial domain Merge candidates from neighboring codec units as specified in clause 8.5.2.4 is invoked with luma codec block position (xCb, yCb), luma codec block width cbWidth and luma codec block height cbHeight as inputs and outputs as availability flags availableFlagA ₀、availableFlagA₁、availableFlagB₀、availableFlagB₁ and availableFlagB ₂, reference indices refIdxLXA ₀、refIdxLXA₁、refIdxLXB₀、refIdxLXB₁ and refIdxLXB ₂, prediction list utilization flags predFlagLXA ₀、predFlagLXA₁、predFlagLXB₀、predFlagLXB₁ and predFlagLXB ₂, motion vectors mvLXA ₀、mvLXA₁、mvLXB₀、mvLXB₁ and mvLXB ₂ (where X is 0 or 1), and bi-prediction weight index bcwIdxA ₀、bcwIdxA₁、bcwIdxB₀、bcwIdxB₁、bcwIdxB₂.

2. The reference index refIdxLXCol (where X is 0 or 1) and the bi-prediction weight index bcwIdxCol of the temporal Merge candidate Col are set equal to 0.

3. The derivation process of temporal luma motion vector prediction as specified in clause 8.5.2.11 is invoked, with luma position (xCb, yCb), luma codec block width cbWidth, luma codec block height cbHeight, and the variable refIdxL0Col as inputs, and output as an availability flag availableFlagL0Col and a temporal motion vector mvL0Col. Variables availableFlagCol, predFlagL Col and predFlagl1Col were derived as follows:

availableFlagCol=availableFlagL0Col (8-263)

predFlagL0Col=availableFlagL0Col (8-264)

predFlagL1Col=0 (8-265)

4. When slice_type is equal to B, the derivation process of temporal luminance motion vector prediction as specified in clause 8.5.2.11 is invoked, with luminance position (xCb, yCb), luminance codec block width cbWidth, luminance codec block height cbHeight, and variable refIdxL1Col as inputs, and outputs as availability flag availableFlagL1Col and temporal motion vector mvL1Col. Variables availableFlagCol and predflag l1Col were derived as follows:

availableFlagCol=availableFlagL0Col||availableFlagL1Col (8-266)

predFlagL1Col=availableFlagL1Col (8-267)

The merge candidate list MERGECANDLIST is constructed as follows:

6. variables numCurrMergeCand and numOrigMergeCand are set equal to the number of Merge candidates in MERGECANDLIST.

7. When numCurrMergeCand is less than (MaxNumMergeCand-1) and NumHmvpCand is greater than 0, the following applies:

Invoking a history-based Merge candidate derivation process as specified in 8.5.2.6, with MERGECANDLIST and numCurrMergeCand as inputs and modified MERGECANDLIST and numCurrMergeCand as outputs.

-NumOrigMergeCand is set equal to numCurrMergeCand.

8. When numCurrMergeCand is less than MaxNumMergeCand and greater than 1, the following applies:

Invoking a derivation procedure of the pairwise average Merge candidates as specified in clause 8.5.2.4, with MERGECANDLIST, reference indices refIdxL0N and refIdxL1N, prediction list utilization flags predflag l0N and predflag l1N, mergeCandList, motion vectors mvL0N and mvL1N, and numCurrMergeCand of each candidate N as input, and outputting motion vectors mvL0avgCand and mvL1avgCand assigned to MERGECANDLIST, NUMCURRMERGECAND, reference indices refIdxL0avgCand and refIdxL1avgCand, prediction list utilization flags predflag l0avgCand and predflag l1avgCand, and candidates avgCand added to MERGECANDLIST. The bi-prediction weight index bcwIdx of the candidate avgCand added to MERGECANDLIST is set equal to 0.

-NumOrigMergeCand is set equal to numCurrMergeCand.

9. The derivation process of the zero motion vector Merge candidate specified in clause 8.5.2.5 is invoked, with MERGECANDLIST, reference indices refIdxL0N and refIdxL1N, motion vectors mvL0N and mvL1N of each candidate N in the prediction list utilization flags predflag l0N and predflag l1N, mergeCandList, and numCurrMergeCand as inputs, and output motion vectors mvL0zeroCand _m and mvL1zeroCand _m assigned to MERGECANDLIST, NUMCURRMERGECAND, reference indices refIdxL0zeroCand _m and refIdxL1zeroCand _m, prediction list utilization flags predflag l0zeroCand _m and predflag l1zeroCand _m, and each new candidate zeroCand _m added to MERGECANDLIST. The bi-prediction weight index bcwIdx of each new candidate zeroCand _m added to MERGECANDLIST is set equal to 0. The number numZeroMergeCand of candidates to be added is set equal to (numCurrMergeCand-numOrigMergeCand). When numZeroMergeCand is greater than 0, m ranges from 0 to numZeroMergeCand-1, including 0 and numZeroMergeCand-1.

10. The following assignment is made, where N is the candidate at position merge_idx [ xCb ] [ yCb ] (n= MERGECANDLIST [ merge_idx [ xCb ] [ yCb ] ]) in the Merge candidate list MERGECANDLIST, and X is replaced with 0 or 1:

refIdxLX=refIdxLXN (8-269)

predFlagLX[0][0]=predFlagLXN (8-270)

mvLX[0][0][0]=mvLXN[0] (8-271)

mvLX[0][0][1]=mvLXN[1] (8-272)

bcwIdx=bcwIdxN (8-273)

11. When mmvd _merge_flag [ xCb ] [ yCb ] is equal to 1, the following applies:

-invoking a derivation procedure of the Merge motion vector difference as specified in 8.5.2.7, with luminance position (xCb, yCb), reference indices refIdxL0, refIdxL1, and prediction list using flags predflag l0[0] [0] and predflag l1[0] [0] as input, and motion vector differences mMvdL and mMvdL1 as output.

The motion vector difference mMvdLX is added to the Merge motion vector mvLX, for X0 and 1, as follows:

mvLX[0][0][0]+=mMvdLX[0] (8-274)

mvLX[0][0][1]+=mMvdLX[1] (8-275)

mvLX[0][0][0]=Clip3(-2¹⁷,2¹⁷-1,mvLX[0][0][0]) (8-276)

mvLX[0][0][1]=Clip3(-2¹⁷,2¹⁷-1,mvLX[0][0][1]) (8-277)

2.2.5 MMVD

In some embodiments, the final motion vector representation (UMVE, also referred to as MMVD) is presented. UMVE is used for skip or Merge modes by a motion vector expression method.

In VVC UMVE reuses the same Merge candidates as those included in the regular Merge candidate list. Among these Merge candidates, a basic candidate may be selected and further extended by a motion vector expression method.

UMVE provides a new Motion Vector Difference (MVD) representation method in which MVDs are represented by a starting point, a motion magnitude, and a motion direction.

In some embodiments, the Merge candidate list is used as is. But only candidates of the DEFAULT Merge TYPE (MRG TYPE DEFAULT N) are considered for the UMVE extensions.

The base candidate index defines a starting point. The basic candidate index indicates the best candidate among the candidates in the list, as follows.

TABLE 4 basic candidate IDX

Basic candidate IDX	0	1	2	3
					Nth MVP	1 St MVP	2 Nd MVP	3 Rd MVP	4 Th MVP

If the number of basic candidates is equal to 1, the basic candidate IDX is not signaled.

The distance index is motion amplitude information. The distance index indicates a predefined distance from the start point information. The predefined distance is as follows:

TABLE 5 distance IDX

The direction index indicates the direction of the MVD relative to the starting point. The direction index may represent four directions as shown below.

TABLE 6 Direction IDX

Direction IDX	00	01	10	11
					X-axis	+	–	N/A	N/A
Y-axis	N/A	N/A	+	–

The UMVE flag is signaled immediately after the skip flag or Merge flag is transmitted. If the skip or Merge flag is true (true), then the UMVE flag is parsed. If UMVE flag is equal to 1, parse UMVE syntax. But if not 1, then parse AFFINE (affine) flags. If AFFINE flag is equal to 1, AFFINE mode, but if not 1, the skip/Merge index is parsed into skip/Merge mode of the VTM.

No additional line buffers due to UMVE candidates are needed. Because the skip/Merge candidate of the software is directly used as a base candidate. Using the input UMVE index, the MV supplementation is determined immediately before motion compensation. It is not necessary to reserve a long line buffer for this purpose.

Under current universal test conditions, either the first or second Merge candidate in the Merge candidate list may be selected as a base candidate.

UMVE is also known as Merge (MMVD) with MV difference.

2.2.6 Combined intra-inter prediction (CIIP)

In some embodiments, multi-hypothesis prediction is proposed, where combining intra and inter prediction is one way to generate multiple hypotheses.

When multi-hypothesis prediction is applied to improve intra mode, the multi-hypothesis prediction combines one intra prediction and one Merge index prediction. In the Merge CU, when the flag is true, a flag is signaled for the Merge mode to select the intra mode from the intra candidate list. For the luma component, the intra candidate list is derived from only one intra prediction mode (e.g., a planar mode). The weights applied to the predicted blocks from intra and inter predictions are determined by the codec modes (intra or non-intra) of the two neighboring blocks (A1 and B1).

2.2.7 Merge of sub-block based techniques

It is suggested that all sub-block related motion candidates are put in a separate Merge list, except for the regular Merge list of non-sub-block Merge candidates.

The sub-block related motion candidates are put into a separate Merge list, which is named "sub-block Merge candidate list".

In one example, the sub-block Merge candidate list includes an ATMVP candidate and an affine Merge candidate.

The sub-block Merge candidate list fills the candidates in the following order:

atmvp candidates (possibly available or unavailable);

2. affine Merge list (including inherited affine candidates; and construction of affine candidates)

3. Zero-filled MV 4 parameter affine model

2.2.7.1ATMVP (also called sub-block temporal motion vector predictor, sbTMVP)

The basic idea of ATMVP is to derive multiple sets of temporal motion vector predictors for a block. Each sub-block is assigned a set of motion information. When ATMVP MERGE candidates are generated, motion compensation is performed at the 8 x 8 level instead of the entire block level.

In the current design, the ATMVP predicts the motion vectors of the sub-CUs within the CU in two steps, which will be described in the following two sub-sections, respectively.

2.2.7.1.1 Initiating derivation of motion vectors

An initialization motion vector is denoted tempMv. When block A1 is available and is not intra-coded (e.g., coded in inter-frame or IBC mode), the following applies to derive the initialization motion vector.

TempMv is set equal to the motion vector of block A1 from list 1, denoted mvL1a ₁, if all the following conditions are true:

The reference picture index of list 1 is available (not equal to-1) and it has the same POC value as the collocated picture (e.g., diffPicOrderCnt (ColPic, refPicList [1] [ refIdxL1a ₁ ]) is equal to 0),

All reference pictures have no larger POC than the current picture (e.g., diffPicOrderCnt (aPic, currPic) is less than or equal to 0 for each picture aPic, diffPicOrderCnt in each reference picture list of the current slice),

The current stripe is equal to the B stripe,

-Localized_from_l0_flag is equal to 0.

Otherwise, tempMv is set equal to the motion vector of block A1 from list 0, denoted mvL0a ₁, if all the following conditions are true:

The reference picture index of list 0 is available (not equal to-1),

It has the same POC value as the collocated picture (e.g., diffPicOrderCnt (ColPic, refPicList [0] [ refIdxL0a ₁ ]) is equal to 0).

Otherwise, the zero motion vector is used as the initialization MV.

The corresponding block (with the center position of the current block plus the rounded MV, clipped to be within a certain range if necessary) is identified in the collocated picture signaled in the slice header along with the initialized motion vector.

If the block is inter-coded, the second step is entered. Otherwise, the ATMVP candidate is set to unavailable.

2.2.7.1.2 Sub-CU motion derivation

The second step is to divide the current CU into sub-CUs and obtain motion information for each sub-CU from the blocks corresponding to each sub-CU in the collocated picture.

If the corresponding block of the sub-CU is encoded and decoded in inter mode, the final motion information of the current sub-CU is derived using the motion information by invoking a derivation process of the collocated motion vector which is not different from that of the conventional TMVP process. Basically, if a corresponding block is predicted from a target list X for unidirectional prediction or bidirectional prediction, a motion vector is utilized, otherwise if it is predicted from a list Y (y=1-X) for unidirectional prediction or bidirectional prediction, and NoBackwardPredFlag is equal to 1, an MV of the list Y is utilized. Otherwise, no motion candidate can be found.

If a block in the collocated picture identified by initializing the MV and the current sub-CU's position is intra-coded or IBC-coded, or motion candidates cannot be found as described above, the following further applies:

The motion vector used to extract the motion field in the collocated picture R _col is denoted MV _col. In order to minimize the impact due to MV scaling, MVs in the spatial candidate list used to derive MVs _col are selected in such a way that if the reference picture of the candidate MV is a collocated picture, that MV is selected and used as MV _col without any scaling. Otherwise, the MV with the reference picture closest to the collocated picture is selected to derive MV _col with scaling.

An example decoding process of the collocated motion vector derivation process is described below:

8.5.2.12 derivation of collocated motion vectors

The inputs of this process are:

A variable currCb, specifying the current codec block,

A variable colCb specifying a concatenated codec block within the concatenated picture specified by ColPic,

Luminance location (xColCb, yColCb), a left top sample specifying the collocated luminance codec block specified by ColCb relative to the left top luminance sample of the collocated picture specified by ColPic,

A reference index refIdxLX, where X is 0 or 1,

-A flag indicating a sub-block time domain Merge candidate sbFlag.

The output of this process is:

Motion vector prediction with 1/16 fractional sample precision,

Availability flag availableFlagLXCol.

The variable currPic specifies the current picture.

The arrays predflag L0Col [ x ] [ y ], mvL0Col [ x ] [ y ] and refIdxL0Col [ x ] [ y ] are set equal to PREDFLAGL [ x ] [ y ], mvDmvrL [ x ] [ y ] and RefIdxL [ x ] [ y ] of the juxtaposed pictures designated by ColPic, respectively, and the arrays predflag L1Col [ x ] [ y ], mvL1Col [ x ] [ y ] and refIdxL1Col [ x ] [ y ] are set equal to PREDFLAGL [ x ] [ y ], mvDmvrL [ x ] [ y ] and RefIdxL [ x ] [ y ] of the juxtaposed pictures designated by ColPic, respectively.

Variables mvLXCol and availableFlagLXCol were derived as follows:

if colCb is coded in intra or IBC prediction mode, both components of mvLXCol are set to 0 and availableFlagLXCol is set equal to 0.

Otherwise, the motion vector mvCol, the reference index refIdxCol and the reference list identifier listCol are derived as follows:

-availableFlagLXCol is set to 1 if sbFlag is equal to 0, and the following applies:

If predFlagL0Col [ xColCb ] [ yColCb ] is equal to 0, mvCol, refIdxCol and listCol are set equal to mvL1Col [ xColCb ] [ yColCb ], refIdxL1Col [ xColCb ] [ yColCb ] and L1, respectively.

Otherwise, if predflag L0Col [ xColCb ] [ yColCb ] is equal to 1 and predflag L1Col [ xColCb ] [ yColCb ] is equal to 0, mvCol, refIdxCol and listCol are set equal to mvL0Col [ xColCb ] [ yColCb ], refIdxL0Col [ xColCb ] [ yColCb ] and L0, respectively.

Otherwise (predflag l0Col [ xColCb ] [ yColCb ] equals 1, and predflag l1Col [ xColCb ] [ yColCb ] equals 1), the following assignments are made:

If NoBackwardPredFlag is equal to 1, mvCol, refIdxCol and listCol are set equal to mvLXCol [ xColCb ] [ yColCb ], refIdxLXCol [ xColCb ] [ yColCb ] and LX, respectively.

Otherwise mvCol, refIdxCol and listCol are set equal to mvLNCol [ xColCb ] [ yColCb ], refIdxLNCol [ xColCb ] [ yColCb ], and LN, respectively, where N is the value of the registered_from_l0_flag.

-Otherwise (sbFlag equals 1), the following applies:

If PredFlagLXCol [ xColCb ] [ yColCb ] is equal to 1, mvCol, refIdxCol and listCol are set equal to mvLXCol [ xColCb ] [ yColCb ], refIdxLXCol [ xColCb ] [ yColCb ] and LX, availableFlagLXCol are set to 1, respectively.

-Otherwise (PredFlagLXCol [ xColCb ] [ yColCb ] equal to 0), the following applies:

-if for each picture aPic, diffPicOrderCnt (aPic, currPic) in each reference picture list of the current slice is less than or equal to 0 and PredFlagLYCol [ xColCb ] [ yColCb ] is equal to 1, mvCol, refIdxCol and listCol are set to mvLYCol [ xColCb ] [ yColCb ], refIdxLYCol [ xColCb ] [ yColCb ] and LY, respectively, wherein Y is equal to | X, wherein X is the value of X for which the procedure is invoked. availableFlagLXCol is set to 1.

Both components of-mvLXCol are set to 0 and availableFlagLXCol is set to 0.

When availableFlagLXCol is equal to TRUE, mvLXCol and availableFlagLXCol are derived as follows:

-if LongTermRefPic (currPic, currCb, refIdxLX LX, LX) is not equal to LongTermRefPic (ColPic, colCb, refIdxCol, listCol), both components of mvLXCol are set equal to 0 and availableFlagLXCol is set equal to 0.

Otherwise, variable availableFlagLXCol is set equal to 1, refpiclist [ listcol ] [ refIdxCol ] is set to the picture with reference index refIdxCol in reference picture list listCol containing slices of codec block colCb in the collocated picture specified by ColPic, and the following applies:

colPocDiff=DiffPicOrderCnt(ColPic,refPicList[listCol][refIdxCol]) (8-402)

currPocDiff=DiffPicOrderCnt(currPic,RefPicList[X][refIdxLX]) (8-403)

Invoking a temporal motion buffer compression procedure of the collocated motion vector as specified in clause 8.5.2.15, with mvCol as input and modified mvCol as output.

-If RefPicList [ X ] [ refIdxLX ] is a long-term reference picture, or colPocDiff is equal to currPocDiff, mvLXCol is derived as follows:

mvLXCol=mvCol (8-404)

otherwise, mvLXCol is derived as a scaled version of the motion vector mvCol, as follows:

tx=(16384+(Abs(td)>>1))/td (8-405)

distScaleFactor＝Clip3(-4096,4095,(tb*tx+32)>>6)(8-406)mvLXCol＝Clip3(-131072,131071,(distScaleFactor*mvCol+128-(distScaleFactor*mvCol>＝0))>>8)) (8-407)

Wherein td and tb are derived as follows:

td=Clip3(-128,127,colPocDiff) (8-408)

tb=Clip3(-128,127,currPocDiff) (8-409)

2.2.8 refinement of motion information

2.2.8.1 Decoder side motion vector refinement (DMVR)

In the bi-prediction operation, for prediction of one block region, two prediction blocks formed using a Motion Vector (MV) of list 0 and a MV of list 1, respectively, are combined to form a single prediction signal. In the decoder-side motion vector refinement (DMVR) method, the two motion vectors of bi-prediction are further refined.

For DMVR in VVC, assume the MVDs mirrored between list 0 and list 1, as shown in fig. 19, and perform bilateral matching to refine the MVs, e.g., find the best MVD among several MVD candidates. MVs of the two reference picture lists are represented by MVL0 (L0X, L0Y) and MVL1 (L1X, L1Y). The MVD of list 0, represented by (MvdX, mvdY), that is capable of minimizing the cost function (e.g., SAD) is defined as the optimal MVD. For the SAD function, it is defined as the SAD between the list 0 reference block and the list 1 reference block, where the list 0 reference block is derived from the motion vector (L0X+ MvdX, L0Y+ MvdY) in the list 0 reference picture and the list 1 reference block is derived from the motion vector (L1X-MvdX, L1Y-MvdY) in the list 1 reference picture.

The motion vector refinement process may be iterated twice. In each iteration, up to 6 MVDs (with integer pixel precision) can be checked in two steps, as shown in fig. 20. In a first step, the MVD (0, 0), (-1, 0), (0, -1), (0, 1) is checked. In the second step, one of MVDs (-1, -1), (-1, 1), (1, -1) or (1, 1) may be selected and further checked. Let the function Sad (x, y) return the Sad value of MVD (x, y). The MVD, denoted by (MvdX, mvdY), checked in the second step is determined as follows:

MvdX=-1;

MvdY=-1;

If(Sad(1,0)<Sad(-1,0))

MvdX=1;

If(Sad(0,1)<Sad(0,-1))

MvdY=1;

In the first iteration, the starting point is the signaled MV, and in the second iteration, the starting point is the signaled MV plus the selected best MVD in the first iteration. DMVR applies only when one reference picture is the previous picture and the other reference picture is the next picture and both reference pictures have the same picture order count distance to the current picture.

To further simplify the DMVR process, in some embodiments, the DMVR design employed has the following main features:

early termination when the (0, 0) position SAD between list 0 and list 1 is less than the threshold.

Early termination when the SAD between list 0 and list 1 is zero for a certain position.

-DMVR block sizes W x H > =64 & & H > =8, where W and H are the width and height of the block.

-Partitioning the CU into a plurality of 16x16 sub-blocks for a CU size >16 x16 DMVR. If only the width or height of the CU is greater than 16, it is divided only in the vertical or horizontal direction.

Reference block size (w+7) × (h+7) (for luminance).

25-Point SAD-based integer-pixel search (e.g., (+ -) 2 refinement search range, single-stage)

DMVR based on bilinear interpolation.

Subpixel refinement based on the "parametric error surface equation". This process is only performed when the minimum SAD cost is not equal to zero and the optimal MVD in the last MV refinement iteration is (0, 0).

Luminance/chrominance MC w/reference block filling (if required).

Refinement MV for MC and TMVP only.

2.2.8.1.1DMVR use

DMVR may be enabled when all of the following conditions are true:

DMVR Enable flag (e.g., SPS dmvr enabled flag) in SPS is equal to 1

TPM flag, inter affine flag and sub-block Merge flag (ATMVP or affine Merge), MMVD flag are all equal to 0

-Merge flag equal to 1

The current block is bi-predictive and the POC distance between the current picture and the reference picture in list 1 is equal to the POC distance between the reference picture in list 0 and the current picture

-The current CU height is greater than or equal to 8

The number of luminance samples (CU width x height) is greater than or equal to 64

2.2.8.1.2 Subpixel refinement based on parametric error surface equation

The method is summarized as follows:

1. the parametric error surface fit is calculated only if the center position is the best cost position in a given iteration.

2. Center position cost and cost at (-1, 0), (0, -1), (1, 0) and (0, 1) positions from the center are used to fit the 2-D parabolic error surface equation of the shape

E(x,y)=A(x-x₀)²+B(y-y₀)²+C

Where (x ₀,y₀) corresponds to the location of the least cost and C corresponds to the least cost value. By solving 5 equations out of 5 unknowns, (x ₀,y₀) is calculated as:

x₀=(E(-1,0)-E(1,0))/(2(E(-1,0)+E(1,0)-2E(0,0)))

y₀=(E(0,-1)-E(0,1))/(2((E(0,-1)+E(0,1)-2E(0,0)))

(x ₀,y₀) any required sub-pixel precision can be calculated by adjusting the precision with which the division is performed (e.g., calculating the quotient of how many bits). For a 1/16 ^th pixel precision, only 4 bits in the absolute value of the quotient need be calculated, which makes it suitable for the implementation of fast shift-based subtraction of the 2 divisions required for each CU.

3. The calculated (x ₀,y₀) is added to the integer distance refinement MV to obtain a subpixel accurate refinement increment MV.

2.3 Intra block copy

Intra Block Copy (IBC), also known as current picture reference, has been employed in HEVC screen content codec extensions (HEVC-SCC) and current VVC test model (VTM-4.0). IBC extends the concept of motion compensation from inter-frame coding to intra-frame coding. As shown in fig. 21, when IBC is applied, the current block is predicted by the reference block in the same picture. Samples in the reference block must have been reconstructed before the current block is encoded or decoded. While IBC is not as effective for most camera captured sequences, it shows significant codec gain for screen content. The reason is that there are many repeated patterns, such as icons and text characters in the screen content picture. IBC can effectively remove redundancy between these repeated patterns. In HEVC-SCC, an inter Coding Unit (CU) may apply IBC if it selects the current picture as its reference picture. In this case, the MV is renamed to a Block Vector (BV), and the BV always has integer pixel precision. For compatibility with the main profile (HEVC), the current picture is marked as a "long-term" reference picture in the Decoded Picture Buffer (DPB). It should be noted that inter-view reference pictures are also labeled as "long-term" reference pictures in the multiview/3D video codec standard similarly.

After the BV finds its reference block, the prediction may be generated by copying the reference block. The residual may be obtained by subtracting the reference pixel from the original signal. The transform and quantization may then be applied as in other codec modes.

However, when the reference block is outside the picture, or overlaps with the current block, or is outside the reconstructed region, or is outside the active region limited by some constraints, some or all of the pixel values are not defined. Basically, there are two solutions to deal with such problems. One is not allowed, for example in bitstream conformance. Another is to apply padding to those undefined pixel values. The following subsections describe the solution in detail.

IBC in 2.3.1VVC test model (VTM 4.0)

In the current VVC test model (e.g., VTM-4.0 design), the entire reference block should be along with the current Codec Tree Unit (CTU) and not overlap with the current block. Thus, no reference or prediction block needs to be filled. The IBC flag is encoded and decoded as a prediction mode of the current CU. Thus, for each CU there are three prediction MODEs in total, mode_intra, mode_inter, and mode_ibc.

2.3.1.1 IBC Merge mode

In IBC Merge mode, the index pointing to an entry in the IBC Merge candidate list is parsed from the bitstream. The construction of the IBC Merge list may be summarized according to the following sequence of steps:

Step 1, deriving airspace candidates

Step 2, inserting HMVP candidates

Step 3, inserting pairwise average candidates

In the derivation of the spatial-domain Merge candidates, a maximum of four Merge candidates are selected among candidates located in positions a ₁、B₁、B₀、A₀ and B ₂ as depicted in fig. 2. The deduced sequences were a ₁、B₁、B₀、A₀ and B ₂. Position B ₂ is only considered when any PU of position a ₁、B₁、B₀、A₀ is not available (e.g., because it belongs to another slice or tile) or is not encoded in IBC mode. After the candidates at position a ₁ are added, the insertion of the remaining candidates is subjected to a redundancy check that ensures that candidates with the same motion information are excluded from the list in order to improve the codec efficiency.

After inserting the spatial candidates, if the IBC Merge list size is still smaller than the maximum IBC Merge list size, IBC candidates from the HMVP table may be inserted. Redundancy checks are performed when HMVP candidates are inserted.

Finally, the pairwise average candidates are inserted into the IBC Merge list.

When the reference block identified by the Merge candidate is outside the picture, or overlaps with the current block, or outside the reconstructed region, or outside the active region limited by some constraint, the Merge candidate is referred to as an invalid Merge candidate.

It should be noted that the invalid Merge candidate may be inserted into the IBC Merge list.

2.3.1.2 IBC AMVP mode

In IBC AMVP mode, an AMVP index is parsed from the bitstream that points to an entry in the IBC AMVP list. The construction of the IBC AMVP list may be summarized according to the following sequence of steps:

Step 1, deriving airspace candidates

-Checking a ₀、A₁ until an available candidate is found.

-Checking B ₀、B₁、B₂ until an available candidate is found.

Step 2, inserting HMVP candidates

Step 3, inserting zero candidates

After inserting the spatial candidates, IBC candidates in the HMVP table may be inserted if the IBC AMVP list size is still smaller than the maximum IBC AMVP list size.

Finally, the zero candidates are inserted into the IBC AMVP list.

2.3.1.3 Chroma IBC mode

In current VVCs, motion compensation in chroma IBC mode is performed at the sub-block level. The chroma block will be divided into several sub-blocks. Each sub-block determines whether the corresponding luminance block has a block vector and, if so, its validity. The current VTM has encoder constraints and if all sub-blocks in the current chroma CU have valid luma block vectors, the chroma IBC mode will be tested. For example, on YUV420 video, the chroma block is nxm and then the co-located luma region is 2nx2m. The subblock size of the chroma block is 2×2. Performing the chrominance mv derivation and block copy processes requires several steps.

(1) The chroma block will first be partitioned into (N > > 1) x (M > > 1) sub-blocks.

(2) Each sub-block with (x, y) upsampling coordinates acquires a corresponding luminance block with (2 x,2 y) covering the same upsampling coordinates.

(3) The encoder checks the block vector (bv) of the acquired luminance block. The bv will be considered invalid if one of the following conditions is met.

A. Bv of the corresponding luminance block does not exist.

B. the prediction block identified by bv has not been reconstructed.

C. The prediction block identified by bv partially or completely overlaps the current block.

(4) The chrominance motion vector of the sub-block is set to the motion vector of the corresponding luminance sub-block.

When all sub-blocks find a valid bv, IBC mode is enabled at the encoder.

2.3.2 Recent advances in IBC

2.3.2.1 Single BV List

In some embodiments, the BV predictors of Merge mode and AMVP mode in IBC share a common predictor list consisting of the following elements:

(1) 2 spatial proximity locations (e.g. A1, B1 in FIG. 2)

(2) 5 HMVP entries

(3) Defaulting to zero vector

The number of candidates in the list is controlled by variables derived from the slice header. For the Merge mode, the top 6 entries of the list may be used at most, and for the AMVP mode, the top 2 entries of the list may be used. And this list meets the shared Merge list area requirement (sharing the same list within the SMR).

In addition to the BV predictor candidate list described above, pruning operations between HMVP candidates and existing Merge candidates (A1, B1) can be simplified. In a simplification, there will be a maximum of 2 pruning operations, as it only compares the first candidate HMVP to the spatial Merge candidate(s).

2.3.2.2 Size limitation of IBC

In some embodiments, the syntax constraint for disabling 128x128 IBC mode may be explicitly used on top of the current bitstream constraint in the previous VTM and VVC versions, which makes the presence of IBC flags dependent on CU size <128x128.

2.3.2.3 Shared Merge list of IBCs

To reduce decoder complexity and support parallel encoding, in some embodiments, the same Merge candidate list for all leaf Codec Units (CUs) of one ancestor node in a CU partition tree may be shared for enabling parallel processing of small skip/Merge codec units. Ancestor nodes are named Merge sharing nodes. The shared Merge candidate list is generated at a Merge sharing node, which is a leaf CU.

More specifically, the following may apply:

If a block has a luminance sample of not more than 32 and is divided into 24 x4 sub-blocks, a shared Merge list between very small blocks (e.g. two adjacent 4x4 blocks) is used.

-If a block has a luminance sample greater than 32, but after the division at least one sub-block is smaller than the threshold (32), all sub-blocks of the division share the same Merge list (e.g. 16x4 or 4x16 division ternary or 8x8 with quaternary division).

Such restrictions are only applied to IBC Merge mode.

3. Problems to be solved by the embodiments

One block may be encoded and decoded in IBC mode. However, different sub-regions within a block may have different content. It is necessary to study how to further explore the correlation with the previous codec block within the current frame.

4. Examples of the embodiments

In this document, intra Block Copy (IBC) may not be limited to current IBC techniques, but may be interpreted as techniques that exclude the use of reference samples within current slices/tiles/pictures/other video units (e.g., CTU lines) of conventional intra prediction methods.

In order to solve the above problems, a subblock-based IBC (sbIBC) codec method is proposed. In sbIBC, the current IBC codec video block (e.g., CU/PU/CB/PB) is partitioned into a plurality of sub-blocks. Each of the sub-blocks may have a size smaller than that of the video block. For each respective sub-block of the plurality of sub-blocks, the video codec may identify a reference block for the respective sub-block in the current picture/slice/tile group. The video codec may determine the motion parameters of the corresponding sub-block using the motion parameters of the identified reference block of the corresponding sub-block.

Furthermore, the IBC is not limited to be applied only to unidirectional predictive codec blocks. Bi-prediction may also be supported, where both reference pictures are current pictures. Alternatively, bi-prediction may also be supported, one from the current picture and the other from a different picture. In yet another example, multiple hypotheses may also be applied.

The following list should be considered as an example to explain the general concept. These techniques should not be interpreted narrowly. Furthermore, these techniques may be combined in any manner. Adjacent blocks A0, A1, B0, B1, and B2 are shown in fig. 2.

1. In sbIBC, a block of size equal to mxn may be divided into more than one sub-block.

A. in one example, the sub-block size is fixed to l×k, e.g., l=k=4.

B. In one example, the sub-block size is fixed to a minimum codec unit/prediction unit/transform unit/unit for motion information storage.

C. in one example, one block may be divided into a plurality of sub-blocks having different sizes or equal sizes.

D. in one example, an indication of the sub-block size may be signaled.

E. in one example, the indication of the sub-block size may change from block to block, e.g., according to the block dimension.

F. In one example, the sub-block size must be in the form of (n1×minw) × (n2× minH), where minw× minH denotes a minimum codec unit/prediction unit/transform unit/unit for motion information storage, and N1 and N2 are positive integers.

G. in one example, the sub-block dimension may depend on the color format and/or the color components.

I. For example, the sub-block sizes of the different color components may be different.

1) Alternatively, the sub-block sizes of the different color components may be the same.

For example, when the color format is 4:2:0, a2 lx2k sub-block of the luma component may correspond to an lxk sub-block of the chroma component.

1) Alternatively, when the color format is 4:2:0, the four 2l×2K sub-blocks of the luminance component may correspond to the 2l×2K sub-blocks of the chrominance component.

For example, when the color format is 4:2:2, a 2L x 2K sub-block of the luma component may correspond to a 2L x K sub-block of the chroma component.

1) Alternatively, when the color format is 4:2:2, the two 2l×2K sub-blocks of the luminance component may correspond to 2l×2K sub-blocks of the chrominance component.

For example, when the color format is 4:4:4, a 2L×2K sub-block of the luma component may correspond to a 2L×2K sub-block of the chroma component.

H. In one example, the MVs of the sub-blocks of the first color component may be derived from a corresponding sub-block or a plurality of corresponding sub-blocks of the second color component.

I. For example, the MVs of the sub-blocks of the first color component may be derived as the average MVs of the multiple corresponding sub-blocks of the second color component.

Furthermore, alternatively, the above method may be applied when using a single tree.

Furthermore, alternatively, the above method may be applied when for a specific block size (such as a 4 x 4 chroma block).

I. in one example, the sub-block size may depend on a codec mode, such as IBC Merge/AMVP mode.

J. In one example, the sub-blocks may be non-rectangular, such as triangular/wedge (wedge). 2. The motion information of the sub-CU is obtained using two phases, including identifying the corresponding reference block with an initialization motion vector (denoted initMV) and deriving one or more motion vectors of the sub-CU from the reference block, at least one of which is equal to the current picture.

A. In one example, the reference block may be in the current picture.

B. In one example, the reference block may be in a reference picture.

I. For example, it may be in a collocated reference picture.

For example, it may be in a reference picture identified by using motion information of the collocated block or neighboring blocks of the collocated block.

Stage 1.A setting with respect to initMV (vx, vy)

C. In one example, initMV may be derived from one or more neighboring blocks (adjacent or non-adjacent) of the current block or current sub-block.

I. the neighboring block may be one block in the same picture.

1) Alternatively, it may be one block in the reference picture.

A. for example, it may be in a collocated reference picture.

B. for example, it may be identified by using motion information of the juxtaposed block or neighboring blocks of the juxtaposed block.

In one example, it may be derived from the neighboring block Z

1) For example, initMV may be set equal to the MV stored in the neighboring block Z. For example, the neighboring block Z may be the block A1.

In one example, it may be derived from a plurality of blocks that are checked in sequence.

1) In one example, a first identified motion vector associated with the current picture as a reference picture from the inspection block may be set to initMV.

D. in one example, initMV may be derived from the motion candidate list.

I. in one example, it may be derived from the kth (e.g., 1 st) candidate in the IBC candidate list.

1) In one example, the IBC candidate list is a Merge/AMVP candidate list.

2) In one example, an IBC candidate list may be utilized that is different from existing IBC Merge candidate list construction processes, such as using different spatial neighboring blocks.

In one example, it may be derived from the kth (e.g., 1 st) candidate in the IBC HMVP table.

E. in one example, it may be derived based on the location of the current block.

F. in one example, it may be derived from the dimension of the current block.

G. In one example, it may be set to a default value.

H. in one example, the indication of initMV may be signaled at the video unit level (such as slice/picture/tile/CTU row/CTU/CTB/CU/PU/TU, etc.).

I. The initial MV may be different for two different sub-blocks within the current block.

J. How the initial MV is derived may vary from block to block, slice to slice, etc.

Stage 1.B identifying corresponding reference blocks of a sub-CU with respect to use initMV

K. In one example, initMV may be first converted to 1-pixel integer precision, and the converted MVs may be utilized to identify the corresponding blocks of the sub-blocks. The transformed MVs are denoted by (vx ', vy').

I. In one example, if (vx, vy) is at F-pixel integer precision, the transformed MV represented by (vx ', vy') may be set to (vx x F, vy x F) (e.g., f=2 or 4).

Alternatively, (vx ', vy') is set directly equal to (vx, vy).

Let the left top position of one sub-block be (x, y) and the sub-block size be kxl. The corresponding block of sub-blocks is set to CU/CB/PU/PB covering coordinates (x+ offsetX +vx ', y+ offsetY +vy'), where offsetX and offsetY are used to indicate the selected coordinates relative to the current sub-block.

I. in one example, offsetX and/or offsetY are set to 0.

In one example offsetX may be set to (L/2) or (L/2+1) or (L/2-1), where L may be the width of the sub-block.

In one example offsetY may be set to (K/2) or (K/2+1) or (K/2-1), where K may be the height of the sub-block.

Alternatively, the horizontal and/or vertical offsets may be further cropped to a range, such as within a picture/strip/tile boundary/IBC reference area, etc.

Stage 2 relates to deriving motion vectors (represented by subMV (subMVx, subMVy)) for sub-blocks using motion information of the identified corresponding reference block

And m. subMV of the sub-blocks are derived from the motion information of the corresponding block.

I. In one example, subMV is set equal to MV if the corresponding block has a motion vector pointing to the current picture.

In one example, subMV is set equal to MV plus initMV if the corresponding block has a motion vector pointing to the current picture.

The n, derived subMV may be further clipped to a given range or clipped to ensure that it points to the IBC reference region.

In a coherent bit stream, the derived subMV must be the effective MV of the IBC of the sub-block.

3. One or more IBC candidates with sub-block motion vectors may be generated, which may be represented as sub-block IBC candidates.

4. The sub-block IBC candidates may be inserted into sub-block Merge candidates including ATMVP, affine Merge candidates.

A. in one example, it may be added before all other sub-block Merge candidates.

B. In one example, it may be added after the ATMVP candidate.

C. in one example, it may be added after inherited affine candidates or constructed affine candidates.

D. In one example, it may be added to the IBC Merge/AMVP candidate list

I. Alternatively, whether it is added may depend on mode information of the current block.

For example, if it is IBC AMVP mode, it may not be added.

E. which candidate list is to be added may depend on the segmentation structure, e.g. dual tree or single tree.

F. Alternatively, a plurality of sub-block IBC candidates may be inserted into the sub-block Merge candidate.

The IBC sub-block motion (e.g., AMVP/Merge) candidate list may be constructed with at least one sub-block IBC candidate.

A. alternatively, one or more sub-block IBC candidates may be inserted into the IBC sub-block Merge candidate, e.g. using a different initialization MV.

B. further, alternatively, whether the IBC sub-block motion candidate list is constructed or an existing IBC AMVP/Merge candidate list may be signaled by an indicator or dynamically (on-the-fly) derived.

C. further, alternatively, if the current block is encoded in IBC Merge mode, the index of the IBC sub-block Merge candidate list may be signaled.

D. Further, alternatively, if the current block is encoded in IBC AMVP mode, the index of the IBC sub-block AMVP candidate list may be signaled.

I. Further, alternatively, the signaled/derived MVD of the IBC AMVP mode may be applied to one or more sub-blocks.

6. The reference block and the sub-block of the sub-block may belong to the same color component.

Expansion of sbIBC by hybrid use of other tools applied to different sub-blocks in the same block

7. One block may be divided into a plurality of sub-blocks, at least one of which is encoded and decoded with IBC and at least one of which is encoded and decoded in an intra mode.

A. In one example, for a sub-block, a motion vector may not be derived. Instead, one or more intra prediction modes may be derived for the sub-blocks.

B. alternatively, palette modes or/and palette tables may be derived.

C. In one example, one intra prediction mode may be derived for an entire block.

8. One block may be divided into a plurality of sub-blocks, wherein all the sub-blocks are encoded and decoded in an intra mode.

9. One block may be divided into a plurality of sub-blocks, wherein all the sub-blocks are encoded and decoded in a palette mode.

10. One block may be divided into a plurality of sub-blocks, wherein at least one sub-block is encoded and decoded in IBC mode and at least one sub-block is encoded and decoded in palette mode.

11. One block may be divided into a plurality of sub-blocks, wherein at least one sub-block is encoded and decoded in an intra mode and at least one sub-block is encoded and decoded in a palette mode.

12. One block may be divided into a plurality of sub-blocks, wherein at least one sub-block is encoded and decoded in IBC mode and at least one sub-block is encoded and decoded in inter mode.

13. One block may be divided into a plurality of sub-blocks, wherein at least one sub-block is encoded in an intra mode and at least one sub-block is encoded in an inter mode.

Interaction with other tools

14. The IBC HMVP table may not be updated when one or more of the above methods are applied.

A. alternatively, one or more of the motion vectors of the IBC codec sub-region may be used to update IBC HMVP tables.

15. The non-IBC HMVP table may not be updated when one or more of the above methods are applied.

B. Alternatively, one or more of the motion vectors of the inter codec sub-region may be used to update the non-IBC HMVP table.

16. The loop filtering process (e.g., deblocking process) may depend on the use of the above-described method.

A. in one example, the sub-block boundaries may be filtered when one or more of the above methods are applied.

A. Alternatively, the sub-block boundaries may be filtered when one or more of the above methods are applied.

B. In one example, blocks encoded using the above method may be processed in a similar manner as conventional IBC codec blocks.

17. Specific codec methods (e.g., sub-block transforms, affine motion prediction, multi-reference line intra prediction, matrix-based intra prediction, symmetric MVD codec, merge with MVD decoder side motion derivation/refinement, bi-directional optical flow, simplified quadratic transforms, multiple transform sets, etc.) may be disabled for blocks that are encoded with one or more of the above methods.

18. The indication of the use of the above-described method and/or sub-block size may be signaled or dynamically derived at the sequence/picture/slice group/slice/tile/CTU/CTB/CU/PU/TU/other video unit level.

A. In one example, one or more of the above methods may be considered a special IBC mode.

I. Further, alternatively, if one block is encoded into IBC mode, further indications using conventional whole block based IBC methods or sbIBC may be signaled or derived.

In one example, a subsequent IBC codec block may utilize the motion information of the current sbIBC codec block as MV prediction.

1. Alternatively, the subsequent IBC codec block may not be allowed to utilize the motion information of the current sbIBC codec block as the MV prediction value.

B. in one example sbIBC may be indicated by a candidate index of the motion candidate list.

I. in one example, a particular candidate index is assigned to sbIBC codec blocks.

C. In one example, the IBC candidates may be classified into two categories, one for the entire block codec and the other for the sub-block codec. Whether a block is coded in sbIBC modes may depend on the class of IBC candidates.

Use of tools

19. Whether and/or how the above method is applied may depend on the following information:

a. Signaling messages in DPS/SPS/VPS/PPS/APS/slice header/slice group header/maximum codec unit (LCU)/Codec Unit (CU)/LCU row/LCU group/TU/PU block/video codec unit

Position of cu/PU/TU/block/video codec unit

C. Block dimension of current block and/or its neighboring blocks

D. Block shape of current block and/or its neighboring blocks

E. Intra mode for current block and/or its neighboring blocks

F. motion/block vector of its neighboring blocks

G. Indication of color format (such as 4:2:0, 4:4:4)

H. Coding and decoding tree structure

I. slice/slice group type and/or picture type

J. color components (e.g., may be applied only to chrominance components or luminance components)

K. Time domain layer ID

Standard profile/level/hierarchy (tier)

Ideas related to Merge list construction procedure and IBC usage

20. For blocks in inter-coded pictures/slices/groups/slices, IBC mode may be used with inter prediction mode.

A. in one example, for IBC AMVP mode, a syntax element may be signaled to indicate whether the current block is predicted from the current picture and a reference picture (denoted as temporal reference picture) that is different from the current picture.

I. Further, alternatively, if the current block is also predicted from a temporal reference picture, a syntax element may be signaled to indicate which temporal reference picture and its associated MVP index, MVD, MV precision, etc. to use.

In one example, for IBC AMVP mode, one reference picture list may include only the current picture and another reference picture list may include only the temporal reference picture.

B. In one example, for IBC Merge mode, the motion vectors and reference pictures may be derived from neighboring blocks.

I. For example, if a neighboring block is predicted from only the current picture, the derived motion information from the neighboring block may refer to only the current picture.

For example, if a neighboring block is predicted from a current picture and a temporal reference picture, the derived motion information may refer to the current picture and the temporal reference picture.

1) Alternatively, the derived motion information may refer only to the current picture.

For example, if a neighboring block is predicted from only temporal reference pictures, it may be considered "invalid" or "unavailable" when constructing an IBC Merge candidate.

C. in one example, fixed weighting factors may be assigned to a reference block from a current picture and a reference block from a temporal reference picture for bi-prediction.

I. Further, alternatively, the weighting factor may be signaled.

21. The motion candidate list construction process (e.g., regular Merge list, IBC Merge/AMVP list, sub-block Merge list, IBC sub-block candidate list) and/or whether/how to update HMVP the table may depend on the block dimensions and/or Merge sharing conditions. The width and height of the block are denoted W and H, respectively. Condition C may depend on the block dimension and/or the codec information.

A. The motion candidate list construction process (e.g., regular Merge list, IBC Merge/AMVP list, sub-block Merge list, IBC sub-block candidate list) and/or whether/how to update HMVP the table may depend on condition C.

B. In one example, condition C may depend on the codec information of the current block and/or its neighboring (neighboring or non-neighboring) blocks.

C. in one example, condition C may depend on a Merge sharing condition.

D. In one example, condition C may depend on a block dimension of the current block, and/or a block dimension of a neighboring (adjacent or non-adjacent) block, and/or a codec mode of the current and/or neighboring block.

E. in one example, if condition C is satisfied, the derivation of the spatial Merge candidate is skipped.

F. In one example, deriving candidates from spatially adjacent (adjacent or non-adjacent) blocks is skipped if condition C is satisfied.

G. in one example, deriving candidates from particular spatial neighboring (adjacent or non-adjacent) blocks (e.g., block B2) is skipped if condition C is satisfied.

H. In one example, if condition C is satisfied, derivation of HMVP candidates is skipped.

I. In one example, if condition C is satisfied, the derivation of the Merge candidate is skipped.

J. in one example, if condition C is satisfied, the number of maximum trimming operations is reduced or set to 0.

I. further, alternatively, pruning operations among the spatial Merge candidates may be reduced or removed.

Further, alternatively, pruning operations among HMVP candidates and other Merge candidates may be reduced or removed.

K. in one example, if condition C is satisfied, the update of HMVP candidates is skipped. i. In one example, HMVP candidates may be added directly to the sports list without pruning.

In one example, if condition C is met, no default motion candidates (e.g., zero motion candidates in the IBC Merge/AVMP list) are added.

M. in one example, when condition C is met, a different order of examination (e.g., from first to last but not from last to first) and/or a different number HMVP of candidates to be examined/added.

N. in one example, condition C may be satisfied when W x H is greater than or not less than a threshold (e.g., 1024).

In one example, condition C may be satisfied when W and/or H is greater than or not less than a threshold (e.g., 32).

In one example, condition C may be satisfied when W is greater than or not less than a threshold (e.g., 32).

In one example, condition C may be satisfied when H is greater than or not less than a threshold (e.g., 32).

In one example, condition C may be satisfied when w×h is greater than or not less than a threshold (e.g., 1024) and the current block is encoded in IBC AMVP and/or Merge mode.

In one example, condition C may be satisfied when W x H is less than or not greater than a threshold (e.g., 16 or 32 or 64) and the current block is encoded in IBC AMVP and/or Merge mode.

I. Further, alternatively, the IBC motion list construction process may include candidates from spatial neighboring blocks (e.g., A1, B1) as well as default candidates when condition C is satisfied. That is, the insertion of HMVP candidates is skipped.

Additionally, alternatively, the IBC motion list construction process may include candidates for HMVP candidates from the IBC HMVP table as well as default candidates when condition C is satisfied. That is, the insertion of candidates from spatial neighboring blocks is skipped.

Further alternatively, after decoding the block if condition C is met, the updating of IBC HMVP tables is skipped.

Alternatively, condition C may be satisfied when one/some/all of the following is true:

1) When W x H is equal to or not greater than T1 (e.g., 16) and the current block is encoded and decoded in IBC AMVP and/or Merge mode

2) When W is equal to T2 and H is equal to T3 (e.g., t2=4, t3=8), the upper block is available and the size is equal to AxB, and the current block and the upper block are both encoded and decoded in a particular mode

A. alternatively, when W is equal to T2 and H is equal to T3 (e.g., t2=4, t3=8), the upper block is available, in the same CTU, and the size is equal to AxB, and the current block and the upper block are both encoded and decoded in the same mode

B. Alternatively, when W is equal to T2 and H is equal to T3 (e.g., t2=4, t3=8), the upper block is available and the size is equal to AxB, and the current block and the upper block are both encoded and decoded in the same mode

C. Alternatively, when W is equal to T2 and H is equal to T3 (e.g., t2=4, t3=8), the upper block is not available

D. Alternatively, when W is equal to T2 and H is equal to T3 (e.g., t2=4, t3=8), the upper block is not available or is outside the current CTU

3) When W is equal to T4 and H is equal to T5 (e.g., t4=8, t5=4), its left block is available and the size is equal to AxB, both the current block and its left block are encoded and decoded in a particular mode

A. alternatively, when W is equal to T4 and H is equal to T5 (e.g., t4=8, t5=4), the left block thereof is not available

4) When W x H is not greater than T1 (e.g., 32), the current block is encoded in IBC AMVP and/or Merge mode, and both the upper and left neighboring blocks are available and equal in size to AxB, and encoded in a particular mode.

A. when W x H is not greater than T1 (e.g., 32), the current block is encoded in a particular mode, its left neighbor block is available, equal in size to AxB, and is IBC encoded, and its upper neighbor block is available, within the same CTU, equal in size to AxB, and encoded in the same mode.

B. When w×h is not greater than T1 (e.g., 32), the current block is coded in a specific mode, its left neighboring block is not available, and its upper neighboring block is available, within the same CTU, and is equal in size to AxB, and is coded in the same mode.

C. when w×h is not greater than T1 (e.g., 32), the current block is coded in a specific mode, its left neighboring block is not available, and its upper neighboring block is not available.

D. When w×h is not greater than T1 (e.g., 32), the current block is encoded in a specific mode, its left neighboring block is available, the size is equal to AxB, and the encoding is performed in the same mode, and its upper neighboring block is not available.

E. when w×h is not greater than T1 (e.g., 32), the current block is coded in a specific mode, its left neighboring block is not available, and its upper neighboring block is not available or outside the current CTU.

F. When w×h is not greater than T1 (e.g., 32), the current block is encoded in a specific mode, its left neighboring block is available, has a size equal to AxB, and is encoded in the same mode, and its upper neighboring block is not available or outside the current CTU.

5) In the above example, the "specific mode" is the IBC mode.

6) In the above example, the "specific mode" is the inter mode.

7) In the above example, "AxB" may be set to 4x4.

8) In the above example, "the adjacent block size is equal to AxB" may be replaced with "the adjacent block size is not greater than or not less than AxB".

9) In the above example, the upper and left neighboring blocks are two blocks that are accessed for spatial Merge candidate derivation.

A. In one example, assuming the coordinates of the top left sample in the current block are (x, y), the left block is one block covered by (x-1, y+H-1).

B. In one example, assuming the coordinates of the top left sample in the current block are (x, y), the left block is one block that covers (x+W-1, y-1).

The above threshold may be predefined or signaled.

I. further, alternatively, the threshold may depend on the codec information of the block, such as the codec mode.

In one example, condition C is satisfied when the current block is under a shared node and the current block is encoded in IBC AMVP and/or Merge mode.

I. Further, alternatively, the IBC motion list construction process may include candidates from spatial neighboring blocks (e.g., A1, B1) as well as default candidates when condition C is satisfied. That is, the insertion of HMVP candidates is skipped.

Additionally, alternatively, the IBC motion list construction process may include candidates for HMVP candidates from the IBC HMVP table as well as default candidates when condition C is satisfied. That is, the insertion of candidates from spatial neighboring blocks is skipped.

Further alternatively, after decoding the block if condition C is met, the updating of IBC HMVP tables is skipped.

In one example, condition C may be adaptively changed, such as according to the codec information of the block.

I. In one example, condition C may be defined based on a codec mode (IBC or non-IBC mode), a block dimension.

Whether the above method is applied may depend on the codec information of the block, such as whether it is an IBC codec block.

I. In one example, the above method may be applied when a block is encoded by IBC.

IBC sports list

22. It is proposed that the motion candidates in IBC HMVP tables are stored with integer pixel precision instead of 1/16 pixel precision.

A. in one example, all motion candidates are stored with a 1-pixel precision.

B. In one example, the rounding process for MVs is skipped when motion information from spatial neighboring (neighboring or non-neighboring) blocks and/or from IBC HMVP tables is used.

23. It is proposed that the IBC proposal list can only contain proposal candidates from one or more IBC HMVP tables.

A. Further, alternatively, the signaling of candidates in the IBC motion list may depend on the number of available HMVP candidates in the HMVP table.

B. further, alternatively, the signaling of candidates in the IBC motion list may depend on the maximum number of HMVP candidates in the HMVP table.

C. Alternatively, HMVP candidates in the HMVP table are added to the list in order, without pruning.

I. in one example, the order is based on an ascending order of the table's entry index.

In one example, the order is based on a descending order of the table's entry index.

In one example, the first N entries in the table may be skipped.

In one example, the last N entries in the table may be skipped.

In one example, entries with invalid BV(s) may be skipped.

vi.

D. Further, alternatively, motion candidates derived from HMVP candidates from one or more HMVP tables may be further modified, such as by adding an offset sum to the horizontal vector

/Or add an offset to the vertical vector.

I. HMVP candidates with invalid BV(s) may be modified to provide valid BV(s).

E. further, alternatively, the default motion candidate may be added after or before one or more HMVP candidates.

F. How/whether HMVP candidates are added to the IBC motion list may depend on the dimensions of the block.

I. for example, when the block dimensions (W and H representing width and height) satisfy condition C, the IBC motion list may contain only motion candidates from one or more HMVP tables.

1) In one example, condition C is W < =t1 and H < =t2, e.g., t1=t2=4.

2) In one example, condition C is W < =t1 or H < =t2, e.g., t1=t2=4.

3) In one example, condition C is w×h < =t, e.g., t=16.

5. Examples

The added changes are highlighted in bold and italics underlined. The deleted portion is marked with [ ].

5.1 Example #1

When the current block is under the shared node, the HMVP table is not updated. Only a single IBC HMVP table is used for the blocks under the shared node.

7.4.8.5 Coding unit semantics

[ [ When all of the following conditions are true, the history-based motion vector predictor list of the shared Merge candidate list region is updated by setting NumHmvpSmrIbcCand to be equal to NumHmvpIbcCand and HmvpSmrIbcCandList [ i ] to be equal to HmvpIbcCandList [ i ] (for i=0.. NumHmvpIbcCand-1):

-IsInSmr [ x0] [ y0] is equal to TRUE.

-SmrX [ x0] [ y0] is equal to x0.

-SmrY [ x0] [ y0] is equal to y0.]]

8.6.2 Derivation of motion vector components for IBC blocks

8.6.2.1 General purpose

The inputs of this process are:

luminance position (xCb, yCb) of the left top luma sample of the current luma codec block relative to the left top luma sample of the current picture,

A variable cbWidth specifying the width of the current codec block in the luma samples,

A variable cbHeight specifying the height of the current codec block in the luma samples.

The output of this process is:

-a luminance motion vector mvL with a 1/16 fractional sample precision.

The luminance motion vector mvL is derived as follows:

-invoking a derivation procedure of IBC luminance motion vector prediction as specified in clause 8.6.2.2, with luminance location (xCb, yCb), variables cbWidth and cbHeight as input, and output as luminance motion vector mvL.

When general_merge_flag [ xCb ] [ yCb ] is equal to 0, the following applies:

1. The variable mvd is derived as follows:

mvd[0]=MvdL0[xCb][yCb][0] (8-883)

mvd[1]=MvdL0[xCb][yCb][1] (8-884)

2. The rounding procedure for the motion vector as specified in clause 8.5.2.14 is invoked with mvX set equal to mvL, RIGHTSHIFT set equal to MvShift +2, and LEFTSHIFT set equal to MvShift +2 as inputs and rounded mvL as outputs.

3. The luminance motion vector mvL is modified as follows:

u[0]=(mvL[0]+mvd[0]+2¹⁸)％2¹⁸ (8-885)

mvL[0]=(u[0]>=2¹⁷)?(u[0]-2¹⁸):u[0] (8-886)

u[1]=(mvL[1]+mvd[1]+2¹⁸)％2¹⁸ (8-887)

mvL[1]=(u[1]>=2¹⁷)?(u[1]-2¹⁸):u[1] (8-888)

Note that the resulting values for 1-mvL [0] and mvL [1] as specified above will always be in the range of-2 ¹⁷ to 2 ¹⁷ -1 inclusive.

When IsInSmr [ xCb ] [ yCb ] is false, the update procedure of the history-based motion vector predictor list as specified in clause 8.6.2.6 is invoked with the luma motion vector mvL.

The left top position inside the reference block (xRefTL, yRefTL) and the right bottom position inside the reference block (xRefBR, yRefBR) are derived as follows:

(xRefTL,yRefTL)=(xCb+(mvL[0]>>4),yCb+(mvL[1]>>4)) (8-889)

(xRefBR,yRefBR)=(xRefTL+cbWidth-1,yRefTL+cbHeight-1) (8-890)

The requirement for bitstream consistency is that the luma motion vector mvL should obey the following constraints:

– ...

8.6.2.4IBC derivation of history-based motion vector candidates

The inputs of this process are:

a list of motion vector candidates MVCANDLIST,

-Number of available motion vector candidates numCurrCand in the list.

The output of this process is:

a modified motion vector candidate list MVCANDLIST,

- [ [ Variable isInSmr, specify whether the current codec unit is inside the shared Merge candidate ] ]

-Number of modified motion vector candidates numCurrCand in the list.

Variables isPrunedA ₁ and isPrunedB ₁ are both set equal to FALSE.

Array smrHmvpIbcCandList and variable smrNumHmvpIbcCand are derived as follows:

[[smr]]HmvpIbcCandList=[[isInSmrHmvpSmrIbcCandList:]]HmvpIbcCandList (8-906)

[[smr]]NumHmvpIbcCand=[[isInSmrNumHmvpSmrIbcCand:]]NumHmvpIbcCand

(8-907)

For each candidate in smrHmvpIbcCandList [ hMvpIdx ] (where index hMvpIdx =1.[ [ smr ] ] NumHmvpIbcCand), the following ordered steps are repeated until numCurrCand equals MaxNumMergeCand:

1. The variables sameMotion are derived as follows:

sameMotion and isPrunedN are both set equal to TRUE if all of the following conditions are TRUE for any motion vector candidate N (where N is a ₁ or B ₁):

-hMvpIdx is less than or equal to 1.

-Candidate [ [ smr ] HmvpIbcCandList [ [ smr ] NumHmvpIbcCand-hMvpIdx ] is equal to motion vector candidate N.

-IsPrunedN is equal to FALSE.

Otherwise sameMotion is set equal to FALSE.

2. When sameMotion is equal to FALSE, the candidate [ [ smr ] ] HmvpIbcCandList [ [ smr ] ] NumHmvpIbcCand-hMvpIdx ] is added to the motion vector candidate list as follows:

mvCandList[numCurrCand++]=[[smr]]HmvpIbcCandList[[[smr]]NumHmvpIbcCand-hMvpIdx] (8-908)

5.2 example #2

When the block size meets certain conditions (such as width x height < K), the checking of the spatial domain Merge/AMVP candidates in the IBC motion list construction process is removed. In the following description, the threshold K may be predefined, such as 16.

7.4.8.2 Codec tree unit semantics

CTUs are root nodes of the codec tree structure.

The array IsInSmr [ x ] [ y ] specifying whether the sample at (x, y) is inside the shared Merge candidate list area is initialized as follows for x=0.. CtbSizeY-1 and y=0.. CtbSizeY-1:

IsInSmr[x][y]=FALSE (7-96)]]

7.4.8.4 codec tree semantics

[ [ For x= x0.. X0+ cbWidth-1 and y= y0.. Y0+ cbHeight-1, isInSmr [ x ] [ y ] is set equal to TRUE when all the following conditions are TRUE:

-IsInSmr [ x0] [ y0] is equal to FALSE

-CbWidth x cbHeight/4 is less than 32

-TreeType is not equal to DUAL_TREE_CHROMA

When IsInSmr [ x0] [ y0] is equal to TRUE. For x= x0..x0+ cbWidth-1 and y= y0..y0+ cbHeight-1, arrays SmrX [ x ] [ y ], smrY [ x ] [ y ], smrW [ x ] [ y ] and SmrH [ x ] [ y ] are derived as follows:

SmrX[x][y]=x0 (7-98)

SmrY[x][y]=y0 (7-99)

SmrW[x][y]=cbWidth (7-100)

SmrH[x][y]=cbHeight (7-101)

when all of the following conditions are TRUE, for x= x0.. X0+ cbWidth-1 and y= y0.. Y0+ cbHeight-1, isInSmr [ x ] [ y ] is set equal to TRUE:

-IsInSmr [ x0] [ y0] is equal to FALSE

-One of the following conditions is true:

-mtt _split_cu_binary_flag is equal to 1 and cbWidth x cbHeight/2 is less than 32

-Mtt _split_cu_binary_flag is equal to 0 and cbWidth x cbHeight/4 is less than 32

-TreeType is not equal to DUAL_TREE_CHROMA

When IsInSmr [ x0] [ y0] is equal to TRUE. For x= x0..x0+ cbWidth-1 and y= y0..y0+ cbHeight-1, arrays SmrX [ x ] [ y ], smrY [ x ] [ y ], smrW [ x ] [ y ] and SmrH [ x ] [ y ] are derived as follows:

SmrX[x][y]=x0 (7-102)

SmrY[x][y]=y0 (7-103)

SmrW[x][y]=cbWidth (7-104)

SmrH[x][y]=cbHeight (7-105)]]

7.4.8.5 coding unit semantics

[ [ When all of the following conditions are true, the history-based motion vector predictor list of the shared Merge candidate list region is updated by setting NumHmvpSmrIbcCand to be equal to NumHmvpIbcCand and HmvpSmrIbcCandList [ i ] to be equal to HmvpIbcCandList [ i ] (for i=0.. NumHmvpIbcCand-1):

-IsInSmr [ x0] [ y0] is equal to TRUE.

-SmrX [ x0] [ y0] is equal to x0.

-SmrY [ x0] [ y0] is equal to y0.]]

For x= x0.. X0+ cbWidth-1 and y= y0.. Y0+ cbHeight-1, the following assignments are made:

CbPosX[x][y]=x0 (7-106)

CbPosY[x][y]=y0 (7-107)

CbWidth[x][y]=cbWidth (7-108)

CbHeight[x][y]=cbHeight (7-109)

Derivation of motion vector component for 8.6.2IBC blocks

8.6.2.1 General inputs to the process are:

luminance position (xCb, yCb) of the left top luma sample of the current luma codec block relative to the left top luma sample of the current picture,

A variable cbWidth specifying the width of the current codec block in the luma samples,

A variable cbHeight specifying the height of the current codec block in the luma samples.

The output of this process is:

-a luminance motion vector mvL with a 1/16 fractional sample precision.

The luminance motion vector mvL is derived as follows:

-invoking a derivation procedure of IBC luminance motion vector prediction as specified in clause 8.6.2.2, with luminance location (xCb, yCb), variables cbWidth and cbHeight as input, and output as luminance motion vector mvL.

When general_merge_flag [ xCb ] [ yCb ] is equal to 0, the following applies:

4. The variable mvd is derived as follows:

mvd[0]=MvdL0[xCb][yCb][0] (8-883)

mvd[1]=MvdL0[xCb][yCb][1] (8-884)

5. The rounding procedure for the motion vector as specified in clause 8.5.2.14 is invoked with mvX set equal to mvL, RIGHTSHIFT set equal to MvShift +2, and LEFTSHIFT set equal to MvShift +2 as inputs and rounded mvL as outputs.

6. The luminance motion vector mvL is modified as follows:

u[0]=(mvL[0]+mvd[0]+2¹⁸)％2¹⁸ (8-885)

mvL[0]=(u[0]>=2¹⁷)?(u[0]-2¹⁸):u[0] (8-886)

u[1]=(mvL[1]+mvd[1]+2¹⁸)％2¹⁸ (8-887)

mvL[1]=(u[1]>=2¹⁷)?(u[1]-2¹⁸):u[1] (8-888)

Note that the resulting values for 1-mvL [0] and mvL [1] as specified above will always be in the range of-2 ¹⁷ to 2 ¹⁷ -1 inclusive.

When smrWidth x SMRHEIGHT is greater than K, the update procedure of the history-based motion vector predictor list as specified in clause 8.6.2.6 is invoked with the luminance motion vector mvL.

The left top position inside the reference block (xRefTL, yRefTL) and the right bottom position inside the reference block (xRefBR, yRefBR) are derived as follows:

(xRefTL,yRefTL)=(xCb+(mvL[0]>>4),yCb+(mvL[1]>>4)) (8-889)

(xRefBR,yRefBR)=(xRefTL+cbWidth-1,yRefTL+cbHeight-1) (8-890)

The requirement for bitstream consistency is that the luma motion vector mvL should obey the following constraints:

– ...

derivation of 8.6.2.2IBC luma motion vector prediction

This procedure is invoked only when CuPredMode [ xCb ] [ yCb ] is equal to mode_ibc, where (xCb, yCb) specifies the left top sample of the current luma codec block relative to the left top luma sample of the current picture.

The inputs of this process are:

luminance position (xCb, yCb) of the left top luma sample of the current luma codec block relative to the left top luma sample of the current picture,

A variable cbWidth specifying the width of the current codec block in the luma samples,

A variable cbHeight specifying the height of the current codec block in the luma samples.

The output of this process is:

-a luminance motion vector mvL with a 1/16 fractional sample precision.

Variables xSmr, ySmr, smrWidth, smrHeight and smrNumHmvpIbcCand were derived as follows:

xSmr=[[IsInSmr[xCb][yCb]?SmrX[xCb][yCb]:]]xCb (8-895)

ySmr=[[IsInSmr[xCb][yCb]?SmrY[xCb][yCb]:]]yCb (8-896)

smrWidth=[[IsInSmr[xCb][yCb]?SmrW[xCb][yCb]:]]cbWidth (8-897)

smrHeight=[[IsInSmr[xCb][yCb]?SmrH[xCb][yCb]:]]cbHeight (8-898)

smrNumHmvpIbcCand=[[IsInSmr[xCb][yCb]?NumHmvpSmrIbcCand:]]NumHmvpIbcCand (8-899)

the luminance motion vector mvL is derived by the following sequential steps:

1. When smrWidth x SMRHEIGHT is greater than K, a derivation procedure of spatial motion vector candidates from neighboring codec units as specified in clause 8.6.2.3 is invoked, with the luma codec block positions (xCb, yCb) set equal to (xSmr, ySmr), the luma codec block widths cbWidth and the luma codec block heights cbHeight set equal to smrWidth and SMRHEIGHT as inputs, and the outputs are the availability flag availableFlagA ₁、availableFlagB₁ and the motion vectors mvA ₁ and mvB ₁.

2. When smrWidth x SMRHEIGHT is greater than K, the motion vector candidate list MVCANDLIST is constructed as follows:

i=0

if(availableFlagA₁)

mvCandList[i++]=mvA₁ (8-900)

if(availableFlagB₁)

mvCandList[i++]=mvB₁

3. when smrWidth x SMRHEIGHT is greater than K, variable numCurrCand is set equal to the number of Merge candidates in MVCANDLIST.

4. When numCurrCand is less than MaxNumMergeCand and smrNumHmvpIbcCand is greater than 0, the derivation process of the IBC history-based motion vector candidates as specified by 8.6.2.4 is invoked with MVCANDLIST, isInSmr and numCurrCand set equal to IsInSmr [ xCb ] [ yCb ] as inputs, and modified MVCANDLIST and numCurrCand as outputs.

5. When numCurrCand is less than MaxNumMergeCand, the following applies until numCurrCand equals MaxNumMergeCand:

Mvcandlist [ numcurrcand ] [0] is set equal to 0.

Mvcandlist [ numcurrcand ] [1] is set equal to 0.

Numcurrcand increases by 1.

6. The variables mvIdx are derived as follows:

mvIdx=general_merge_flag[xCb][yCb]?merge_idx[xCb][yCb]:mvp_l0_flag[xCb][yCb] (8-901)

7. The following assignments were made:

mvL[0]=mergeCandList[mvIdx][0] (8-902)

mvL[1]=mergeCandList[mvIdx][1] (8-903)

8.6.2.4 Derivation of IBC history-based motion vector candidates

The inputs of this process are:

a list of motion vector candidates MVCANDLIST,

-Number of available motion vector candidates numCurrCand in the list.

The output of this process is:

a modified motion vector candidate list MVCANDLIST,

- [ [ Variable isInSmr, specify whether the current codec unit is inside the shared Merge candidate ] ]

-Number of modified motion vector candidates numCurrCand in the list.

Variables isPrunedA ₁ and isPrunedB ₁ are both set equal to FALSE.

Array smrHmvpIbcCandList and variable smrNumHmvpIbcCand are derived as follows:

[[smr]]HmvpIbcCandList=[[isInSmrHmvpSmrIbcCandList:]]HmvpIbcCandList (8-906)

[[smr]]NumHmvpIbcCand=[[isInSmrNumHmvpSmrIbcCand:]]NumHmvpIbcCand

(8-907)

For each candidate in [ [ smr ] ] HmvpIbcCandList [ hMvpIdx ] (where index hMvpIdx =1.. smrNumHmvpIbcCand), the following ordered steps are repeated until numCurrCand equals MaxNumMergeCand:

1. The variables sameMotion are derived as follows:

If smrWidth x SMRHEIGHT is greater than K, all of the following conditions are TRUE for any motion vector candidate N (where N is a ₁ or B ₁), then both sameMotion and isPrunedN are set equal to TRUE:

-hMvpIdx is less than or equal to 1.

-Candidate [ [ smr ] HmvpIbcCandList [ [ smr ] NumHmvpIbcCand-hMvpIdx ] is equal to motion vector candidate N.

-IsPrunedN is equal to FALSE.

Otherwise sameMotion is set equal to FALSE.

2. When sameMotion is equal to FALSE, the candidate [ [ smr ] ] HmvpIbcCandList [ smrNumHmvpIbcCand-hMvpIdx ] is added to the motion vector candidate list as follows:

mvCandList[numCurrCand++]=[[smr]]HmvpIbcCandList[[[smr]]NumHmvpIbcCand-hMvpIdx] (8-908)

5.3 example #3

When the block size meets certain conditions, such as the current block is under the shared node, then the check of the spatial domain Merge/AMVP candidates in the IBC motion list construction process is removed and the HMVP table is not updated.

8.6.2.2 Derivation of IBC luma motion vector prediction

This procedure is invoked only when CuPredMode [ xCb ] [ yCb ] is equal to mode_ibc, where (xCb, yCb) specifies the left top sample of the current luma codec block relative to the left top luma sample of the current picture.

The inputs of this process are:

luminance position (xCb, yCb) of the left top luma sample of the current luma codec block relative to the left top luma sample of the current picture,

A variable cbWidth specifying the width of the current codec block in the luma samples,

A variable cbHeight specifying the height of the current codec block in the luma samples.

The output of this process is:

-a luminance motion vector mvL with a 1/16 fractional sample precision.

Variables xSmr, ySmr, smrWidth, smrHeight and smrNumHmvpIbcCand were derived as follows:

Variables xSmr, ySmr, smrWidth, smrHeight and smrNumHmvpIbcCand were derived as follows:

xSmr=[[IsInSmr[xCb][yCb]?SmrX[xCb][yCb]:]]xCb (8-895)

ySmr=[[IsInSmr[xCb][yCb]?SmrY[xCb][yCb]:]]yCb (8-896)

smrWidth=[[IsInSmr[xCb][yCb]?SmrW[xCb][yCb]:]]cbWidth (8-897)

smrHeight=[[IsInSmr[xCb][yCb]?SmrH[xCb][yCb]:]]cbHeight (8-898)

smrNumHmvpIbcCand=[[IsInSmr[xCb][yCb]?NumHmvpSmrIbcCand:]]NumHmvpIbcCand (8-899)

the luminance motion vector mvL is derived by the following sequential steps:

1. When IsInSmr [ xCb ] [ yCb ] is false, the derivation process of spatial motion vector candidates from neighboring codec units as specified in clause 8.6.2.3 is invoked with the luma codec block position (xCb, yCb) set equal to (xSmr, ySmr), the luma codec block width cbWidth and the luma codec block height cbHeight set equal to smrWidth and SMRHEIGHT as inputs, and the outputs are the availability flag availableFlagA ₁、availableFlagB₁ and the motion vectors mvA ₁ and mvB ₁.

2. When IsInSmr [ xCb ] [ yCb ] is false, the motion vector candidate list MVCANDLIST is constructed as follows:

i=0

if(availableFlagA₁)

mvCandList[i++]=mvA₁ (8-900)

if(availableFlagB₁)

mvCandList[i++]=mvB₁

3. When IsInSmr [ xCb ] [ yCb ] is false, the variable numCurrCand is set equal to the number of Merge candidates in MVCANDLIST.

4. When numCurrCand is less than MaxNumMergeCand and smrNumHmvpIbcCand is greater than 0, the derivation process of the IBC history-based motion vector candidates as specified by 8.6.2.4 is invoked with MVCANDLIST, isInSmr and numCurrCand set equal to IsInSmr [ xCb ] [ yCb ] as inputs, and modified MVCANDLIST and numCurrCand as outputs.

5. When numCurrCand is less than MaxNumMergeCand, the following applies until numCurrCand equals MaxNumMergeCand:

Mvcandlist [ numcurrcand ] [0] is set equal to 0.

Mvcandlist [ numcurrcand ] [1] is set equal to 0.

Numcurrcand increases by 1.

6. The variables mvIdx are derived as follows:

mvIdx=general_merge_flag[xCb][yCb]?merge_idx[xCb][yCb]:mvp_l0_flag[xCb][yCb] (8-901)

7. The following assignments were made:

mvL[0]=mergeCandList[mvIdx][0] (8-902)

mvL[1]=mergeCandList[mvIdx][1] (8-903)

8.6.2.4 Derivation of IBC history-based motion vector candidates

The inputs of this process are:

a list of motion vector candidates MVCANDLIST,

-Number of available motion vector candidates numCurrCand in the list.

The output of this process is:

a modified motion vector candidate list MVCANDLIST,

- [ [ Variable isInSmr, specify whether the current codec unit is inside the shared Merge candidate ] ]

-Number of modified motion vector candidates numCurrCand in the list.

Variables isPrunedA ₁ and isPrunedB ₁ are both set equal to FALSE.

Array smrHmvpIbcCandList and variable smrNumHmvpIbcCand are derived as follows:

[[smr]]HmvpIbcCandList=[[isInSmrHmvpSmrIbcCandList:]]HmvpIbcCandList (8-906)

[[smr]]NumHmvpIbcCand=[[isInSmrNumHmvpSmrIbcCand:]]NumHmvpIbcCand

(8-907)

For each candidate in [ [ smr ] ] HmvpIbcCandList [ hMvpIdx ] (where index hMvpIdx =1.. smrNumHmvpIbcCand), the following ordered steps are repeated until numCurrCand equals MaxNumMergeCand:

3. the variables sameMotion are derived as follows:

sameMotion and isPrunedN are both set equal to TRUE if isInSmr is false and all the following conditions are TRUE for any motion vector candidate N (where N is a ₁ or B ₁):

-hMvpIdx is less than or equal to 1.

-Candidate [ [ smr ] HmvpIbcCandList [ [ smr ] NumHmvpIbcCand-hMvpIdx ] is equal to motion vector candidate N.

-IsPrunedN is equal to FALSE.

Otherwise sameMotion is set equal to FALSE.

4. When sameMotion is equal to FALSE, the candidate [ [ smr ] ] HmvpIbcCandList [ [ smr ] ] NumHmvpIbcCand-hMvpIdx ] is added to the motion vector candidate list as follows:

mvCandList[numCurrCand++]=[[smr]]HmvpIbcCandList[[[smr]]NumHmvpIbcCand-hMvpIdx] (8-908)

8.6.2 Derivation of motion vector components for IBC blocks

8.6.2.1 General purpose

The inputs of this process are:

luminance position (xCb, yCb) of the left top luma sample of the current luma codec block relative to the left top luma sample of the current picture,

A variable cbWidth specifying the width of the current codec block in the luma samples,

A variable cbHeight specifying the height of the current codec block in the luma samples.

The output of this process is:

-a luminance motion vector mvL with a 1/16 fractional sample precision.

The luminance motion vector mvL is derived as follows:

-invoking a derivation procedure of IBC luminance motion vector prediction as specified in clause 8.6.2.2, with luminance location (xCb, yCb), variables cbWidth and cbHeight as input, and output as luminance motion vector mvL.

When general_merge_flag [ xCb ] [ yCb ] is equal to 0, the following applies:

7. the variable mvd is derived as follows:

mvd[0]=MvdL0[xCb][yCb][0] (8-883)

mvd[1]=MvdL0[xCb][yCb][1] (8-884)

8. The rounding procedure for the motion vector as specified in clause 8.5.2.14 is invoked with mvX set equal to mvL, RIGHTSHIFT set equal to MvShift +2, and LEFTSHIFT set equal to MvShift +2 as inputs and rounded mvL as outputs.

9. The luminance motion vector mvL is modified as follows:

u[0]=(mvL[0]+mvd[0]+2¹⁸)％2¹⁸ (8-885)

mvL[0]=(u[0]>=2¹⁷)?(u[0]-2¹⁸):u[0] (8-886)

u[1]=(mvL[1]+mvd[1]+2¹⁸)％2¹⁸ (8-887)

mvL[1]=(u[1]>=2¹⁷)?(u[1]-2¹⁸):u[1] (8-888)

Note that the resulting values for 1-mvL [0] and mvL [1] as specified above will always be in the range of-2 ¹⁷ to 2 ¹⁷ -1 inclusive.

When IsInSmr [ xCb ] [ yCb ] is false, the update procedure of the history-based motion vector predictor list as specified in clause 8.6.2.6 is invoked with the luma motion vector mvL.

The left top position inside the reference block (xRefTL, yRefTL) and the right bottom position inside the reference block (xRefBR, yRefBR) are derived as follows:

(xRefTL,yRefTL)=(xCb+(mvL[0]>>4),yCb+(mvL[1]>>4)) (8-889)

(xRefBR,yRefBR)=(xRefTL+cbWidth-1,yRefTL+cbHeight-1) (8-890)

The requirement for bitstream consistency is that the luma motion vector mvL should obey the following constraints:

– ...

5.4 example #4

When the block size satisfies a specific condition (such as width x height < = K, or width = N, height = 4 and left neighboring block is 4x4 and is coded in IBC mode, or width = 4, height = N and upper neighboring block is 4x4 and is coded in IBC mode), the check of the spatial domain Merge/AMVP candidates in IBC motion list construction is removed and the HMVP table is not updated. In the following description, the threshold K may be predefined, such as 16, and n may be predefined, such as 8.

7.4.9.2 Codec tree unit semantics

CTUs are root nodes of the codec tree structure.

An array isacable [ cIdx ] [ x ] [ y ] specifying whether a sample at (x, y) is available for the derivation process of neighboring block availability as specified in clause 6.4.4 is initialized as follows, for cidx=0..2, x=0.. CtbSizeY-1, and y=0.. CtbSizeY-1:

IsAvailable[cIdx][x][y]=FALSE (7-123)

the array IsInSmr [ x ] [ y ] specifying whether the sample at (x, y) is inside the shared Merge candidate list area is initialized as follows for x=0.. CtbSizeY-1 and y=0.. CtbSizeY-1:

IsInSmr[x][y]=FALSE (7-124)]]

7.4.9.4 codec tree semantics

[ [ For x= x0.. X0+ cbWidth-1 and y= y0.. Y0+ cbHeight-1, isInSmr [ x ] [ y ] is set equal to TRUE when all the following conditions are TRUE:

-IsInSmr [ x0] [ y0] is equal to FALSE

-CbWidth x cbHeight/4 is less than 32

-TreeType is not equal to DUAL_TREE_CHROMA

When IsInSmr [ x0] [ y0] is equal to TRUE. For x= x0..x0+ cbWidth-1 and y= y0..y0+ cbHeight-1, arrays SmrX [ x ] [ y ], smrY [ x ] [ y ], smrW [ x ] [ y ] and SmrH [ x ] [ y ] are derived as follows:

SmrX[x][y]=x0 (7-126)

SmrY[x][y]=y0 (7-127)

SmrW[x][y]=cbWidth (7-128)

SmrH[x][y]=cbHeight (7-129)]]

8.6.2 Derivation of block vector components for IBC blocks

8.6.2.1 General purpose

The inputs of this process are:

luminance position (xCb, yCb) of the left top luma sample of the current luma codec block relative to the left top luma sample of the current picture,

A variable cbWidth specifying the width of the current codec block in the luma samples,

A variable cbHeight specifying the height of the current codec block in the luma samples.

The output of this process is:

-luminance block vector bvL with 1/16 fractional sample precision.

The luminance block vector mvL is derived as follows:

-invoking a derivation process of IBC luminance block vector prediction as specified in clause 8.6.2.2, with luminance locations (xCb, yCb), variables cbWidth and cbHeight as inputs, and output as luminance block vector bvL.

When general_merge_flag [ xCb ] [ yCb ] is equal to 0, the following applies:

1. the variables bvd are derived as follows:

bvd[0]=MvdL0[xCb][yCb][0] (8-900)

bvd[1]=MvdL0[xCb][yCb][1] (8-901)

2. The rounding procedure for the motion vector as specified in clause 8.5.2.14 is invoked, with mvX set equal to bvL, RIGHTSHIFT set equal to AMVRSHIFT, and LEFTSHIFT set equal to AMVRSHIFT as inputs, and rounded bvL as outputs.

3. The luminance block vector bvL is modified as follows:

u[0]=(bvL[0]+bvd[0]+2¹⁸)％2¹⁸ (8-902)

bvL[0]=(u[0]>=217)?(u[0]-218):u[0] (8-903)

u[1]=(bvL[1]+bvd[1]+218)%218 (8-904)

bvL[1]=(u[1]>=217)?(u[1]218:u[1] (8-905)

Note that 1-the result values of bvL [0] and bvL [1] as specified above will always be in the range of-2 ¹⁷ to 2 ¹⁷ -1

Within the range of (containing).

Variable IslgrBlk is set to (cbWidth x cbHeight is greater than K true: false).

If IsLgrBlk is true, isLgrBlk is set to false when CbWidth is equal to N and CbHeight is equal to 4, and the left neighbor block is 4x4 and is being encoded and decoded in IBC mode.

If IsLgrBlk is true, isLgrBlk is set to false when CbWidth is equal to 4 and CbHeight is equal to N, and the upper neighbor block is 4x4 and is being encoded and decoded in IBC mode.

(Or alternatively:

variable IslgrBlk is set to (cbWidth x cbHeight is greater than K true: false).

If IsLgrBlk is true, isLgrBlk is set to false when CbWidth is equal to N and CbHeight is equal to 4, and the left neighbor block is 4x4 and is being encoded and decoded in IBC mode.

If IsLgrBlk is true, isLgrBlk is set to false when CbWidth is equal to 4 and CbHeight is equal to N, and the upper neighbor block is available, and it is 4x4 and is being encoded in IBC mode. )

When IsLgrBlk is true [ [ IsInSmr [ xCb ] [ yCb ] equals false ] ], the update procedure of the history-based block vector predictor list as specified in clause 8.6.2.6 is invoked with the luma block vector bvL.

The requirement for bitstream consistency is that the luma block vector bvL should adhere to the following constraints:

-CtbSizeY is greater than or equal to ((yCb + (bvL [1] > 4)) & (CtbSizeY-1)) + cbHeight.

For x=xcb.. xCb + cbWidth-1 and y＝yCb..yCb+cbHeight-1,IbcVirBuf[0][(x+(bvL[0]>>4))&(IbcVirBufWidth-1)][(y+(bvL[1]>>4))&(CtbSizeY-1)] should not be equal to-1.

8.6.2.2 Derivation of IBC luma block vector prediction

This procedure is invoked only when CuPredMode [0] [ xCb ] [ yCb ] is equal to mode_ibc, where (xCb, yCb) specifies the left top sample of the current luma codec block relative to the left top luma sample of the current picture.

The inputs of this process are:

luminance position (xCb, yCb) of the left top luma sample of the current luma codec block relative to the left top luma sample of the current picture,

A variable cbWidth specifying the width of the current codec block in the luma samples,

A variable cbHeight specifying the height of the current codec block in the luma samples.

The output of this process is:

-luminance block vector bvL with 1/16 fractional sample precision.

The [ (variables xSmr, ySmr, smrWidth and SMRHEIGHT) are deduced as follows:

xSmr=IsInSmr[xCb][yCb]?SmrX[xCb][yCb]:xCb (8-906)

ySmr=IsInSmr[xCb][yCb]?SmrY[xCb][yCb]:yCb (8-907)

smrWidth=IsInSmr[xCb][yCb]?SmrW[xCb][yCb]:cbWidth (8-908)

smrHeight=IsInSmr[xCb][yCb]?SmrH[xCb][yCb]:cbHeight (8-909)]]

variable IslgrBlk is set to (cbWidth x cbHeight is greater than K true: false).

If IsLgrBlk is true, isLgrBlk is set to false when CbWidth is equal to N and CbHeight is equal to 4, and the left neighbor block is 4x4 and is being encoded and decoded in IBC mode.

If IsLgrBlk is true, isLgrBlk is set to false when CbWidth is equal to 4 and CbHeight is equal to N, and the upper neighbor block is 4x4 and is being encoded and decoded in IBC mode.

(Alternatively:

variable IslgrBlk is set to (cbWidth x cbHeight is greater than K true: false).

If IsLgrBlk is true, isLgrBlk is set to false when CbWidth is equal to N and CbHeight is equal to 4, and the left neighbor block is 4x4 and is being encoded and decoded in IBC mode.

If IsLgrBlk is true, isLgrBlk is set to false when CbWidth is equal to 4 and CbHeight is equal to N, and the upper neighbor block is available, and it is 4x4 and is being encoded in IBC mode. )

The luminance block vector bvL is derived by the following sequential steps:

1. When IslgrBlk is true, the derivation process of spatial block vector candidates from neighboring codec units as specified in clause 8.6.2.3 is invoked with the luma codec block position (xCb, yCb) set equal to (xCb, yCb [ [ xSmr, ySmr ] ]), the luma codec block width cbWidth and the luma codec block height cbHeight set equal to [ [ smr ] ] CbWidth and [ [ smr ] ] CbHeight as inputs, and output as the availability flag availableFlagA ₁、availableFlagB₁ and the block vectors bvA ₁ and bvB ₁.

2. When IslgrBlk is true, the block vector candidate list bvCandList is constructed as follows:

i=0

if(availableFlagA₁)

bvCandList[i++]=bvA₁ (8-910)

if(availableFlagB₁)

bvCandList[i++]=bvB₁

3. when IslgrBlk is true, variable numCurrCand is set equal to the number of Merge candidates in bvCandList.

4. When numCurrCand is less than MaxNumIbcMergeCand and NumHmvpIbcCand is greater than 0, the derivation process of the IBC history-based block vector candidates as specified in 8.6.2.4 is invoked with bvCandList, and IsLgrBlk, and numCurrCand as inputs and modified bvCandList and numCurrCand as outputs.

5. When numCurrCand is less than MaxNumIbcMergeCand, the following applies until numCurrCand equals MaxNumIbcMergeCand:

Bvcandlist [ numcurrcand ] [0] is set equal to 0.

Bvcandlist [ numcurrcand ] [1] is set equal to 0.

Numcurrcand increases by 1.

6. The variables bvIdx are derived as follows:

bvIdx=general_merge_flag[xCb][yCb]?merge_idx[xCb][yCb]:mvp_l0_flag[xCb][yCb] (8-911)

7. The following assignments were made:

bvL[0]=bvCandList[mvIdx][0] (8-912)

bvL[1]=bvCandList[mvIdx][1] (8-913)

8.6.2.3 The IBC airspace block vector candidate is derived by the following steps:

luminance position (xCb, yCb) of the left top luma sample of the current luma codec block relative to the left top luma sample of the current picture,

A variable cbWidth specifying the width of the current codec block in the luma samples,

A variable cbHeight specifying the height of the current codec block in the luma samples.

The output of this process is as follows:

Availability flags availableFlagA ₁ and availableFlagB ₁ of neighboring codec units,

Block vectors bvA ₁ and bvB ₁ of 1/16 fractional sample precision adjacent to the codec unit, for the derivation of availableFlagA ₁ and mvA ₁, the following applies:

-the luminance position (xNbA ₁,yNbA₁) inside the neighboring luminance codec block is set equal to (xCb-1, ycb+cbheight-1).

-Invoking a derivation procedure of neighboring block availability as specified in clause 6.4.4, wherein the current luminance position (xCurr, yCurr) set equal to (xCb, yCb), the neighboring luminance position (xNbA ₁,yNbA₁), checkPredModeY set equal to TRUE, and cIdx set equal to 0 are taken as inputs and the output is assigned to the block availability flag availableA ₁.

Variables availableFlagA ₁ and bvA ₁ are derived as follows:

If availableA ₁ is equal to FALSE, availableFlagA ₁ is set equal to 0 and both components of bvA ₁ are set equal to 0.

Otherwise, availableFlagA ₁ is set equal to 1 and the following assignments are made:

bvA₁＝MvL0[xNbA₁][yNbA₁] (8-914)

for the derivation of availableFlagB ₁ and bvB ₁, the following applies:

-the luminance position (xNbB ₁,yNbB₁) inside the neighboring luminance codec block is set equal to (xCb + cbWidth-1, ycb-1).

-Invoking a derivation procedure of neighboring block availability as specified in clause 6.4.4, wherein the current luminance position (xCurr, yCurr) set equal to (xCb, yCb), the neighboring luminance position (xNbB ₁,yNbB₁), checkPredModeY set equal to TRUE, and cIdx set equal to 0 are taken as inputs and the output is assigned to the block availability flag availableB ₁.

Variables availableFlagB ₁ and bvB ₁ are derived as follows:

-availableFlagB ₁ is set equal to 0 and both components of bvB ₁ are set equal to 0 if one or more of the following conditions are true:

-availableB ₁ is equal to FALSE.

-AvailableA ₁ is equal to TRUE and the luminance positions (xNbA ₁,yNbA₁) and (xNbB ₁,yNbB₁) have the same block vector.

Otherwise, availableFlagB ₁ is set equal to 1 and the following assignments are made:

bvB₁＝MvL0[xNbB₁][yNbB₁] (8-915)

8.6.2.4 Derivation of IBC history-based block vector candidates

The inputs of this process are:

-a block vector candidate list bvCandList,

- [ [ Variable isInSmr, specify whether the current codec unit is inside the shared Merge candidate ] ]

Variables IslgrBlk indicating non-tiles,

-Number of available block vector candidates numCurrCand in the list.

The output of this process is:

a modified block vector candidate list bvCandList,

-Number of modified motion vector candidates numCurrCand in the list.

Variables isPrunedA ₁ and isPrunedB ₁ are both set equal to FALSE.

For each candidate in [ [ smr ] ] HmvpIbcCandList [ hMvpIdx ] (where index hMvpIdx =1 ] [ [ smr ] ] NumHmvpIbcCand), the following ordered steps are repeated until numCurrCand equals MaxNumIbcMergeCand:

1. The variables sameMotion are derived as follows:

-if IsLgrBlk is true, and all of the following conditions are for any candidate block vector N

(Where N is a ₁ or B ₁) are both TRUE, then both sameMotion and isPrunedN are set equal to TRUE:

-hMvpIdx is less than or equal to 1.

-Candidate HmvpIbcCandList [ NumHmvpIbcCand-hMvpIdx ] is equal to block vector candidate N.

-IsPrunedN is equal to FALSE.

Otherwise sameMotion is set equal to FALSE.

2. When sameMotion is equal to FALSE, candidates HmvpIbcCandList [ NumHmvpIbcCand-hMvpIdx ] are added to the block vector candidate list as follows:

bvCandList[numCurrCand++]=HmvpIbcCandList[NumHmvpIbcCand-hMvpIdx]

(8-916)

5.5 example #5

When the block size satisfies a specific condition (such as width=n, height=4 and left neighboring block is 4x4 and is coded in IBC mode, or width=4, height=n and upper neighboring block is 4x4 and is coded in IBC mode), the check of the spatial domain Merge/AMVP candidates in IBC motion list construction is removed and the update of HMVP table is removed. In the following description, N may be predefined, such as 4 or 8.

7.4.9.2 Codec tree unit semantics

CTUs are root nodes of the codec tree structure.

An array isacable [ cIdx ] [ x ] [ y ] specifying whether a sample at (x, y) is available for the derivation process of neighboring block availability as specified in clause 6.4.4 is initialized as follows, for cidx=0..2, x=0.. CtbSizeY-1, and y=0.. CtbSizeY-1:

IsAvailable[cIdx][x][y]=FALSE (7-123)

the array IsInSmr [ x ] [ y ] specifying whether the sample at (x, y) is inside the shared Merge candidate list area is initialized as follows for x=0.. CtbSizeY-1 and y=0.. CtbSizeY-1:

IsInSmr[x][y]=FALSE (7-124)]]

7.4.9.4 codec tree semantics

[ [ For x= x0.. X0+ cbWidth-1 and y= y0.. Y0+ cbHeight-1, isInSmr [ x ] [ y ] is set equal to TRUE when all the following conditions are TRUE:

-IsInSmr [ x0] [ y0] is equal to FALSE

-CbWidth x cbHeight/4 is less than 32

-TreeType is not equal to DUAL_TREE_CHROMA

When IsInSmr [ x0] [ y0] is equal to TRUE. For x= x0..x0+ cbWidth-1 and y= y0..y0+ cbHeight-1, arrays SmrX [ x ] [ y ], smrY [ x ] [ y ], smrW [ x ] [ y ] and SmrH [ x ] [ y ] are derived as follows:

SmrX[x][y]=x0 (7-126)

SmrY[x][y]=y0 (7-127)

SmrW[x][y]=cbWidth (7-128)

SmrH[x][y]=cbHeight (7-129)]]

derivation of block vector component for 8.6.2IBC blocks

8.6.2.1 General purpose

The inputs of this process are:

luminance position (xCb, yCb) of the left top luma sample of the current luma codec block relative to the left top luma sample of the current picture,

A variable cbWidth specifying the width of the current codec block in the luma samples,

A variable cbHeight specifying the height of the current codec block in the luma samples.

The output of this process is:

-luminance block vector bvL with 1/16 fractional sample precision.

The luminance block vector mvL is derived as follows:

-invoking a derivation process of IBC luminance block vector prediction as specified in clause 8.6.2.2, with luminance locations (xCb, yCb), variables cbWidth and cbHeight as inputs, and output as luminance block vector bvL.

When general_merge_flag [ xCb ] [ yCb ] is equal to 0, the following applies:

1. the variables bvd are derived as follows:

bvd[0]=MvdL0[xCb][yCb][0] (8-900)

bvd[1]=MvdL0[xCb][yCb][1] (8-901)

2. The rounding procedure for the motion vector as specified in clause 8.5.2.14 is invoked, with mvX set equal to bvL, RIGHTSHIFT set equal to AMVRSHIFT, and LEFTSHIFT set equal to AMVRSHIFT as inputs, and rounded bvL as outputs.

3. The luminance block vector bvL is modified as follows:

u[0]=(bvL[0]+bvd[0]+2¹⁸)％2¹⁸ (8-902)

bvL[0]=(u[0]>=217)?(u[0]-218):u[0] (8-903)

u[1]=(bvL[1]+bvd[1]+218)%218 (8-904)

bvL[1]=(u[1]>=217)?(u[1]218:u[1] (8-905)

The resulting values for note 1-bvL [0] and bvL [1] as specified above will always be in the range of-2 ¹⁷ to 2 ¹⁷ -1 inclusive.

Variable IslgrBlk is set to true when one of the following conditions is true.

-When CbWidth is equal to N and CbHeight is equal to 4, and the left neighboring block is 4x4 and is codec in IBC mode.

-When CbWidth is equal to 4 and CbHeight is equal to N, and the upper neighboring block is 4x4 and is codec in IBC mode.

Otherwise IsLgrBlk is set to false.

(Or alternatively:

Variable IslgrBlk is set to (CbWidth x CbHeight >16true: false) and the following is further checked:

if IsLgrBlk is true, isLgrBlk is set to false when CbWidth is equal to N and CbHeight is equal to 4, and the left neighbor block is 4x4 and is being encoded and decoded in IBC mode.

If IsLgrBlk is true, isLgrBlk is set to false when CbWidth is equal to 4 and CbHeight is equal to N, and the upper neighbor block is 4x4 and is being encoded and decoded in IBC mode. )

When IsLgrBlk is true [ [ IsInSmr [ xCb ] [ yCb ] equals false ] ], the update procedure of the history-based block vector predictor list as specified in clause 8.6.2.6 is invoked with the luma block vector bvL.

The requirement for bitstream consistency is that the luma block vector bvL should adhere to the following constraints:

-CtbSizeY is greater than or equal to ((yCb + (bvL [1] > 4)) & (CtbSizeY-1)) + cbHeight.

For x=xcb.. xCb + cbWidth-1 and y＝yCb..yCb+cbHeight-1,IbcVirBuf[0][(x+(bvL[0]>>4))&(IbcVirBufWidth-1)][(y+(bvL[1]>>4))&(CtbSizeY-1)] should not be equal to-1.

8.6.2.2 Derivation of IBC luma block vector prediction

This procedure is invoked only when CuPredMode [0] [ xCb ] [ yCb ] is equal to mode_ibc, where (xCb, yCb) specifies the left top sample of the current luma codec block relative to the left top luma sample of the current picture.

The inputs of this process are:

luminance position (xCb, yCb) of the left top luma sample of the current luma codec block relative to the left top luma sample of the current picture,

A variable cbWidth specifying the width of the current codec block in the luma samples,

A variable cbHeight specifying the height of the current codec block in the luma samples.

The output of this process is:

-luminance block vector bvL with 1/16 fractional sample precision.

The [ (variables xSmr, ySmr, smrWidth and SMRHEIGHT) are deduced as follows:

xSmr=IsInSmr[xCb][yCb]?SmrX[xCb][yCb]:xCb (8-906)

ySmr=IsInSmr[xCb][yCb]?SmrY[xCb][yCb]:yCb (8-907)

smrWidth=IsInSmr[xCb][yCb]?SmrW[xCb][yCb]:cbWidth (8-908)

smrHeight=IsInSmr[xCb][yCb]?SmrH[xCb][yCb]:cbHeight (8-909)]]

variable IslgrBlk is set to true when one of the following conditions is true.

-When CbWidth is equal to N and CbHeight is equal to 4, and the left neighboring block is 4x4 and is codec in IBC mode.

-When CbWidth is equal to 4 and CbHeight is equal to N, and the upper neighboring block is 4x4 and is codec in IBC mode.

Otherwise IsLgrBlk is set to false.

(Or alternatively:

Variable IslgrBlk is set to (CbWidth x CbHeight >16true: false) and the following is further checked:

if IsLgrBlk is true, isLgrBlk is set to false when CbWidth is equal to N and CbHeight is equal to 4, and the left neighbor block is 4x4 and is being encoded and decoded in IBC mode.

If IsLgrBlk is true, isLgrBlk is set to false when CbWidth is equal to 4 and CbHeight is equal to N, and the upper neighbor block is 4x4 and is being encoded and decoded in IBC mode. )

The luminance block vector bvL is derived by the following sequential steps:

1. When IslgrBlk is true, the derivation process of spatial block vector candidates from neighboring codec units as specified in clause 8.6.2.3 is invoked with the luma codec block position (xCb, yCb) set equal to (xCb, yCb [ [ xSmr, ySmr ] ]), the luma codec block width cbWidth and the luma codec block height cbHeight set equal to [ [ smr ] ] CbWidth and [ [ smr ] ] CbHeight as inputs, and output as the availability flag availableFlagA ₁、availableFlagB₁ and the block vectors bvA ₁ and bvB ₁.

2. When IslgrBlk is true, the block vector candidate list bvCandList is constructed as follows:

i=0

if(availableFlagA₁)

bvCandList[i++]=bvA₁ (8-910)

if(availableFlagB₁)

bvCandList[i++]=bvB₁

3. when IslgrBlk is true, variable numCurrCand is set equal to the number of Merge candidates in bvCandList.

4. When numCurrCand is less than MaxNumIbcMergeCand and NumHmvpIbcCand is greater than 0, the derivation process of the IBC history-based block vector candidates as specified in 8.6.2.4 is invoked with bvCandList, and IsLgrBlk, and numCurrCand as inputs and modified bvCandList and numCurrCand as outputs.

5. When numCurrCand is less than MaxNumIbcMergeCand, the following applies until numCurrCand equals MaxNumIbcMergeCand:

Bvcandlist [ numcurrcand ] [0] is set equal to 0.

Bvcandlist [ numcurrcand ] [1] is set equal to 0.

Numcurrcand increases by 1.

6. The variables bvIdx are derived as follows:

bvIdx=general_merge_flag[xCb][yCb]?merge_idx[xCb][yCb]:mvp_l0_flag[xCb][yCb] (8-911)

7. The following assignments were made:

bvL[0]=bvCandList[mvIdx][0] (8-912)

bvL[1]=bvCandList[mvIdx][1] (8-913)

8.6.2.3 Derivation process of IBC airspace block vector candidate

The inputs of this process are:

luminance position (xCb, yCb) of the left top luma sample of the current luma codec block relative to the left top luma sample of the current picture,

A variable cbWidth specifying the width of the current codec block in the luma samples,

A variable cbHeight specifying the height of the current codec block in the luma samples.

The output of this process is as follows:

Availability flags availableFlagA ₁ and availableFlagB ₁ of neighboring codec units,

Block vectors bvA ₁ and bvB ₁ of 1/16 fractional sample precision adjacent to the codec unit, for the derivation of availableFlagA ₁ and mvA ₁, the following applies:

-the luminance position (xNbA ₁,yNbA₁) inside the neighboring luminance codec block is set equal to (xCb-1, ycb+cbheight-1).

-Invoking a derivation procedure of neighboring block availability as specified in clause 6.4.4, wherein the current luminance position (xCurr, yCurr) set equal to (xCb, yCb), the neighboring luminance position (xNbA ₁,yNbA₁), checkPredModeY set equal to TRUE, and cIdx set equal to 0 are taken as inputs and the output is assigned to the block availability flag availableA ₁.

Variables availableFlagA ₁ and bvA ₁ are derived as follows:

If availableA ₁ is equal to FALSE, availableFlagA ₁ is set equal to 0 and both components of bvA ₁ are set equal to 0.

Otherwise, availableFlagA ₁ is set equal to 1 and the following assignments are made:

bvA₁＝MvL0[xNbA₁][yNbA₁] (8-914)

for the derivation of availableFlagB ₁ and bvB ₁, the following applies:

-the luminance position (xNbB ₁,yNbB₁) inside the neighboring luminance codec block is set equal to (xCb + cbWidth-1, ycb-1).

-Invoking a derivation procedure of neighboring block availability as specified in clause 6.4.4, wherein the current luminance position (xCurr, yCurr) set equal to (xCb, yCb), the neighboring luminance position (xNbB ₁,yNbB₁), checkPredModeY set equal to TRUE, and cIdx set equal to 0 are taken as inputs and the output is assigned to the block availability flag availableB ₁.

Variables availableFlagB ₁ and bvB ₁ are derived as follows:

-availableFlagB ₁ is set equal to 0 and both components of bvB ₁ are set equal to 0 if one or more of the following conditions are true:

-availableB ₁ is equal to FALSE.

-AvailableA ₁ is equal to TRUE and the luminance positions (xNbA ₁,yNbA₁) and (xNbB ₁,yNbB₁) have the same block vector.

Otherwise, availableFlagB ₁ is set equal to 1 and the following assignments are made:

bvB₁＝MvL0[xNbB₁][yNbB₁] (8-915)

8.6.2.4 Derivation of IBC history-based block vector candidates

The inputs of this process are:

-a block vector candidate list bvCandList,

- [ [ Variable isInSmr, specify whether the current codec unit is inside the shared Merge candidate ] ]

Variables IslgrBlk indicating non-tiles,

-Number of available block vector candidates numCurrCand in the list.

The output of this process is:

a modified block vector candidate list bvCandList,

-Number of modified motion vector candidates numCurrCand in the list.

Variables isPrunedA ₁ and isPrunedB ₁ are both set equal to FALSE.

For each candidate in [ [ smr ] ] HmvpIbcCandList [ hMvpIdx ] (where index hMvpIdx =1 ] [ [ smr ] ] NumHmvpIbcCand), the following ordered steps are repeated until numCurrCand equals MaxNumIbcMergeCand:

1. The variables sameMotion are derived as follows:

-if IsLgrBlk is true, and all of the following conditions are for any candidate block vector N

(Where N is a ₁ or B ₁) are both TRUE, then both sameMotion and isPrunedN are set equal to TRUE:

-hMvpIdx is less than or equal to 1.

-Candidate HmvpIbcCandList [ NumHmvpIbcCand-hMvpIdx ] is equal to block vector candidate N.

-IsPrunedN is equal to FALSE.

Otherwise sameMotion is set equal to FALSE.

2. When sameMotion is equal to FALSE, candidates HmvpIbcCandList [ NumHmvpIbcCand-hMvpIdx ] are added to the block vector candidate list as follows:

bvCandList[numCurrCand++]=HmvpIbcCandList[NumHmvpIbcCand-hMvpIdx]– (8-916)

5.6 example #6

When the block size satisfies a specific condition (such as width x height < = K, and the left or upper neighboring block is 4x4 and is coded in IBC mode), the check of the spatial domain Merge/AMVP candidates in IBC motion list construction is removed, and the update of HMVP table is removed. In the following description, the threshold K may be predefined, such as 16 or 32.

7.4.9.2 Codec tree unit semantics

CTUs are root nodes of the codec tree structure.

An array isacable [ cIdx ] [ x ] [ y ] specifying whether a sample at (x, y) is available for the derivation process of neighboring block availability as specified in clause 6.4.4 is initialized as follows, for cidx=0..2, x=0.. CtbSizeY-1, and y=0.. CtbSizeY-1:

IsAvailable[cIdx][x][y]=FALSE (7-123)

the array IsInSmr [ x ] [ y ] specifying whether the sample at (x, y) is inside the shared Merge candidate list area is initialized as follows for x=0.. CtbSizeY-1 and y=0.. CtbSizeY-1:

IsInSmr[x][y]=FALSE (7-124)]]

7.4.9.4 codec tree semantics

[ [ For x= x0.. X0+ cbWidth-1 and y= y0.. Y0+ cbHeight-1, isInSmr [ x ] [ y ] is set equal to TRUE when all the following conditions are TRUE:

-IsInSmr [ x0] [ y0] is equal to FALSE

-CbWidth x cbHeight/4 is less than 32

-TreeType is not equal to DUAL_TREE_CHROMA

When IsInSmr [ x0] [ y0] is equal to TRUE. For x= x0..x0+ cbWidth-1 and y= y0..y0+ cbHeight-1, arrays SmrX [ x ] [ y ], smrY [ x ] [ y ], smrW [ x ] [ y ] and SmrH [ x ] [ y ] are derived as follows:

SmrX[x][y]=x0 (7-126)

SmrY[x][y]=y0 (7-127)

SmrW[x][y]=cbWidth (7-128)

SmrH[x][y]=cbHeight (7-129)]]

8.6.2 Derivation of block vector components for IBC blocks

8.6.2.1 General purpose

The inputs of this process are:

luminance position (xCb, yCb) of the left top luma sample of the current luma codec block relative to the left top luma sample of the current picture,

A variable cbWidth specifying the width of the current codec block in the luma samples,

A variable cbHeight specifying the height of the current codec block in the luma samples.

The output of this process is:

-luminance block vector bvL with 1/16 fractional sample precision.

The luminance block vector mvL is derived as follows:

-invoking a derivation process of IBC luminance block vector prediction as specified in clause 8.6.2.2, with luminance locations (xCb, yCb), variables cbWidth and cbHeight as inputs, and output as luminance block vector bvL.

When general_merge_flag [ xCb ] [ yCb ] is equal to 0, the following applies:

1. the variables bvd are derived as follows:

bvd[0]=MvdL0[xCb][yCb][0] (8-900)

bvd[1]=MvdL0[xCb][yCb][1] (8-901)

2. The rounding procedure for the motion vector as specified in clause 8.5.2.14 is invoked, with mvX set equal to bvL, RIGHTSHIFT set equal to AMVRSHIFT, and LEFTSHIFT set equal to AMVRSHIFT as inputs, and rounded bvL as outputs.

3. The luminance block vector bvL is modified as follows:

u[0]=(bvL[0]+bvd[0]+2¹⁸)％2¹⁸ (8-902)

bvL[0]=(u[0]>=217)?(u[0]-218):u[0] (8-903)

u[1]=(bvL[1]+bvd[1]+218)%218 (8-904)

bvL[1]=(u[1]>=217)?(u[1]218:u[1] (8-905)

The resulting values for note 1-bvL [0] and bvL [1] as specified above will always be in the range of-2 ¹⁷ to 2 ¹⁷ -1 inclusive.

Variable IslgrBlk is set to (cbWidth x cbHeight is greater than K true: false).

If IsLgrBlk is true, isLgrBlk is set to false when the left neighboring block is 4x4 and is coded in IBC mode.

If IsLgrBlk is true, isLgrBlk is set to false when the upper neighboring block is 4x4 and is coded in IBC mode.

(Or alternatively,

Variable IslgrBlk is set to (cbWidth x cbHeight is greater than K true: false).

If IsLgrBlk is true, isLgrBlk is set to false when the left neighboring block is 4x4 and is coded in IBC mode.

If IsLgrBlk is true, isLgrBlk is set to false when the upper neighboring block is in the same CTU as the current block and is 4x4 and is being coded in IBC mode. )

When IsLgrBlk is true [ [ IsInSmr [ xCb ] [ yCb ] equals false ] ], the update procedure of the history-based block vector predictor list as specified in clause 8.6.2.6 is invoked with the luma block vector bvL.

The requirement for bitstream consistency is that the luma block vector bvL should adhere to the following constraints:

-CtbSizeY is greater than or equal to ((yCb + (bvL [1] > 4)) & (CtbSizeY-1)) + cbHeight.

For x=xcb.. xCb + cbWidth-1 and y＝yCb..yCb+cbHeight-1,IbcVirBuf[0][(x+(bvL[0]>>4))&(IbcVirBufWidth-1)][(y+(bvL[1]>>4))&(CtbSizeY-1)] should not be equal to-1.

8.6.2.2 Derivation of IBC luma block vector prediction

This procedure is invoked only when CuPredMode [0] [ xCb ] [ yCb ] is equal to mode_ibc, where (xCb, yCb) specifies the left top sample of the current luma codec block relative to the left top luma sample of the current picture.

The inputs of this process are:

luminance position (xCb, yCb) of the left top luma sample of the current luma codec block relative to the left top luma sample of the current picture,

A variable cbWidth specifying the width of the current codec block in the luma samples,

A variable cbHeight specifying the height of the current codec block in the luma samples.

The output of this process is:

-luminance block vector bvL with 1/16 fractional sample precision.

The [ (variables xSmr, ySmr, smrWidth and SMRHEIGHT) are deduced as follows:

xSmr=IsInSmr[xCb][yCb]?SmrX[xCb][yCb]:xCb (8-906)

ySmr=IsInSmr[xCb][yCb]?SmrY[xCb][yCb]:yCb (8-907)

smrWidth=IsInSmr[xCb][yCb]?SmrW[xCb][yCb]:cbWidth (8-908)

smrHeight=IsInSmr[xCb][yCb]?SmrH[xCb][yCb]:cbHeight (8-909)]]

variable IslgrBlk is set to (cbWidth x cbHeight is greater than K true: false).

If IsLgrBlk is true, isLgrBlk is set to false when the left neighboring block is 4x4 and is coded in IBC mode.

If IsLgrBlk is true, isLgrBlk is set to false when the upper neighboring block is 4x4 and is coded in IBC mode.

(Or alternatively,

Variable IslgrBlk is set to (cbWidth x cbHeight is greater than K true: false).

If IsLgrBlk is true, isLgrBlk is set to false when the left neighboring block is 4x4 and is coded in IBC mode.

If IsLgrBlk is true, isLgrBlk is set to false when the upper neighboring block is in the same CTU as the current block and is 4x4 and is being coded in IBC mode. )

The luminance block vector bvL is derived by the following sequential steps:

1. When IslgrBlk is true, the derivation process of spatial block vector candidates from neighboring codec units as specified in clause 8.6.2.3 is invoked with the luma codec block position (xCb, yCb) set equal to (xCb, yCb [ [ xSmr, ySmr ] ]), the luma codec block width cbWidth and the luma codec block height cbHeight set equal to [ [ smr ] ] CbWidth and [ [ smr ] ] CbHeight as inputs, and output as the availability flag availableFlagA ₁、availableFlagB₁ and the block vectors bvA ₁ and bvB ₁.

2. When IslgrBlk is true, the block vector candidate list bvCandList is constructed as follows:

i=0

if(availableFlagA₁)

bvCandList[i++]=bvA₁ (8-910)

if(availableFlagB₁)

bvCandList[i++]=bvB₁

3. when IslgrBlk is true, variable numCurrCand is set equal to the number of Merge candidates in bvCandList.

4. When numCurrCand is less than MaxNumIbcMergeCand and NumHmvpIbcCand is greater than 0, the derivation process of the IBC history-based block vector candidates as specified in 8.6.2.4 is invoked with bvCandList, and IsLgrBlk, and numCurrCand as inputs and modified bvCandList and numCurrCand as outputs.

5. When numCurrCand is less than MaxNumIbcMergeCand, the following applies until numCurrCand equals MaxNumIbcMergeCand:

Bvcandlist [ numcurrcand ] [0] is set equal to 0.

Bvcandlist [ numcurrcand ] [1] is set equal to 0.

Numcurrcand increases by 1.

6. The variables bvIdx are derived as follows:

bvIdx=general_merge_flag[xCb][yCb]?merge_idx[xCb][yCb]:mvp_l0_flag[xCb][yCb] (8-911)

7. The following assignments were made:

bvL[0]=bvCandList[mvIdx][0] (8-912)

bvL[1]=bvCandList[mvIdx][1] (8-913)

8.6.2.3 The IBC airspace block vector candidate is derived by the following steps:

luminance position (xCb, yCb) of the left top luma sample of the current luma codec block relative to the left top luma sample of the current picture,

A variable cbWidth specifying the width of the current codec block in the luma samples,

A variable cbHeight specifying the height of the current codec block in the luma samples.

The output of this process is as follows:

Availability flags availableFlagA ₁ and availableFlagB ₁ of neighboring codec units,

Block vectors bvA ₁ and bvB ₁ of 1/16 fractional sample precision adjacent to the codec unit, for the derivation of availableFlagA ₁ and mvA ₁, the following applies:

-the luminance position (xNbA ₁,yNbA₁) inside the neighboring luminance codec block is set equal to (xCb-1, ycb+cbheight-1).

-Invoking a derivation procedure of neighboring block availability as specified in clause 6.4.4, wherein the current luminance position (xCurr, yCurr) set equal to (xCb, yCb), the neighboring luminance position (xNbA ₁,yNbA₁), checkPredModeY set equal to TRUE, and cIdx set equal to 0 are taken as inputs and the output is assigned to the block availability flag availableA ₁.

Variables availableFlagA ₁ and bvA ₁ are derived as follows:

If availableA ₁ is equal to FALSE, availableFlagA ₁ is set equal to 0 and both components of bvA ₁ are set equal to 0.

Otherwise, availableFlagA ₁ is set equal to 1 and the following assignments are made:

bvA₁＝MvL0[xNbA₁][yNbA₁] (8-914)

for the derivation of availableFlagB ₁ and bvB ₁, the following applies:

-the luminance position (xNbB ₁,yNbB₁) inside the neighboring luminance codec block is set equal to (xCb + cbWidth-1, ycb-1).

-Invoking a derivation procedure of neighboring block availability as specified in clause 6.4.4, wherein the current luminance position (xCurr, yCurr) set equal to (xCb, yCb), the neighboring luminance position (xNbB ₁,yNbB₁), checkPredModeY set equal to TRUE, and cIdx set equal to 0 are taken as inputs and the output is assigned to the block availability flag availableB ₁.

Variables availableFlagB ₁ and bvB ₁ are derived as follows:

-availableFlagB ₁ is set equal to 0 and both components of bvB ₁ are set equal to 0 if one or more of the following conditions are true:

-availableB ₁ is equal to FALSE.

-AvailableA ₁ is equal to TRUE and the luminance positions (xNbA ₁,yNbA₁) and (xNbB ₁,yNbB₁) have the same block vector.

Otherwise, availableFlagB ₁ is set equal to 1 and the following assignments are made:

bvB₁＝MvL0[xNbB₁][yNbB₁] (8-915)

8.6.2.4 Derivation of IBC history-based block vector candidates

The inputs of this process are:

-a block vector candidate list bvCandList,

- [ [ Variable isInSmr, specify whether the current codec unit is inside the shared Merge candidate ] ]

Variables IslgrBlk indicating non-tiles,

-Number of available block vector candidates numCurrCand in the list.

The output of this process is:

a modified block vector candidate list bvCandList,

-Number of modified motion vector candidates numCurrCand in the list.

Variables isPrunedA ₁ and isPrunedB ₁ are both set equal to FALSE.

For each candidate in [ [ smr ] ] HmvpIbcCandList [ hMvpIdx ] (where index hMvpIdx =1 ] [ [ smr ] ] NumHmvpIbcCand), the following ordered steps are repeated until numCurrCand equals MaxNumIbcMergeCand:

1. The variables sameMotion are derived as follows:

sameMotion and isPrunedN are both set equal to TRUE if IsLgrBlk is TRUE and all the following conditions are TRUE for any candidate block vector N (where N is a ₁ or B ₁):

-hMvpIdx is less than or equal to 1.

-Candidate HmvpIbcCandList [ NumHmvpIbcCand-hMvpIdx ] is equal to block vector candidate N.

-IsPrunedN is equal to FALSE.

Otherwise sameMotion is set equal to FALSE.

2. When sameMotion is equal to FALSE, candidates HmvpIbcCandList [ NumHmvpIbcCand-hMvpIdx ] are added to the block vector candidate list as follows:

bvCandList[numCurrCand++]=HmvpIbcCandList[NumHmvpIbcCand-hMvpIdx]– (8-916)

Fig. 22 is a block diagram of a video processing apparatus 2200. The apparatus 2200 may be used to implement one or more of the methods described herein. The apparatus 2200 may be embodied in a smart phone, tablet, computer, internet of things (IoT) receiver, or the like. The apparatus 2200 may include one or more processors 2202, one or more memories 2204, and video processing hardware 2206. The processor(s) 2202 may be configured to implement one or more methods described in this document. Memory(s) 2204 may be used to store data and code for implementing the methods and techniques described herein. Video processing hardware 2206 may be used to implement some of the techniques described in this document in hardware circuitry. Video processing hardware 2206 may be partially or fully included within the processor(s) 2202 in the form of dedicated hardware or Graphics Processor Units (GPUs) or dedicated signal processing blocks.

Fig. 23 is a flowchart of an example of a video processing method 2300. Method 2300 includes, at operation 2302, determining a codec mode using intra-sub-block copying (sbIBC). The method 2300 further includes performing conversion using sbIBC codec modes at operation 2304.

Some embodiments may be described using the following clause-based description.

The following clauses illustrate example embodiments of the techniques discussed in clause 1 of the previous section.

1.A method of video processing includes determining a sub-block intra block copy (sbIBC) codec mode to be used in a transition between a current video block and a bitstream representation of the current video block in a video region, wherein the current video block is partitioned into a plurality of sub-blocks and each sub-block is encoded based on a reference sample from the video region, wherein a size of the sub-block is based on a partitioning rule, and performing the transition using the sbIBC codec mode for the plurality of sub-blocks.

2. The method of clause 1, wherein the current video block is an MxN block, wherein M and N are integers, and wherein the partitioning rule specifies that each sub-block has the same size.

3. The method of clause 1, wherein the bitstream representation includes a syntax element indicating a partitioning rule or a size of the sub-block.

4. The method of any of clauses 1-3, wherein the partitioning rule specifies a size of the sub-block according to a color component of the current video block.

5. The method of any of clauses 1-4, wherein the sub-blocks of the first color component may derive their motion vector information from the sub-blocks of the second color component corresponding to the current video block.

The following clauses illustrate example embodiments of the techniques discussed in clauses 2 and 6 of the previous section.

1. A method of video processing includes determining a coding mode using intra-sub-block copying (sbIBC) in a transition between a current video block and a bitstream representation of the current video block in a video region, wherein the current video block is divided into a plurality of sub-blocks and each sub-block is coded based on reference samples from the video region, and performing the transition using sbIBC coding mode for the plurality of sub-blocks, wherein the transition includes determining an initialization motion vector (initMV) for a given sub-block, identifying a reference block from initMV, and deriving Motion Vector (MV) information for the given sub-block using MV information for the reference block.

2. The method of clause 1, wherein determining initMV comprises determining initMV from one or more neighboring blocks to the given sub-block.

3. The method of clause 2, wherein one or more neighboring blocks are inspected in order.

4. The method of clause 1, wherein determining initMV comprises deriving initMV from the motion candidate list.

5. The method of any of clauses 1-4, wherein identifying the reference block includes converting initMV to one pixel precision and identifying the reference block based on the converted initMV.

6. The method of any of clauses 1-4, wherein identifying the reference block comprises applying initMV to an offset position within the given block, wherein the offset position is represented as being offset from a predetermined position of the given sub-block (offsetX, offsetY).

7. The method of any of clauses 1-6, wherein deriving the MVs for the given sub-block comprises clipping MV information for the reference block.

8. The method of any of clauses 1-7, wherein the reference block is at a color component different from a color component of the current video block.

The following clauses illustrate example embodiments of the techniques discussed in clauses 3, 4, and 5 of the previous section.

1. A method of video processing includes determining to use a sub-block intra block copy (sbIBC) codec mode in a transition between a current video block and a bitstream representation of the current video block in a video region, wherein the current video block is divided into a plurality of sub-blocks and each sub-block is encoded based on a reference sample from the video region, and performing the transition using the sbIBC codec mode for the plurality of sub-blocks, wherein the transition includes generating sub-block IBC candidates.

2. The method of clause 1, wherein the sub-block IBC candidates are added to a candidate list comprising selectable motion vector predictor candidates.

3. The method of clause 1, wherein the sub-block IBC candidate is added to a list comprising affine Merge candidates.

The following clauses illustrate example embodiments of the techniques discussed in clauses 7, 8, 9, 10, 11, 12, and 13 of the previous section.

1. A method of video processing includes performing a transition between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the transition includes processing a first sub-block of the plurality of sub-blocks using an intra-sub-block intra-block codec (sbIBC) mode and processing a second sub-block of the plurality of sub-blocks using an intra-coding mode.

2. A method of video processing includes performing a transition between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the transition includes processing all of the plurality of sub-blocks using an intra-codec mode.

3. A method of video processing includes performing a conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the conversion includes processing all of the plurality of sub-blocks using a palette of representative pixel values for a palette codec mode that encodes each sub-block.

4. A method of video processing includes performing a conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the conversion includes processing a first sub-block of the plurality of sub-blocks using a palette of representative pixel values for a palette mode of encoding and decoding the first sub-block, and processing a second sub-block of the plurality of sub-blocks using an intra block copy encoding and decoding mode.

5. A method of video processing includes performing a conversion between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the conversion includes processing a first sub-block of the plurality of sub-blocks using a palette of representative pixel values for a palette mode of encoding and decoding the first sub-block, and processing a second sub-block of the plurality of sub-blocks using an intra-coding mode.

6. A method of video processing includes performing a transition between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the transition includes processing a first sub-block of the plurality of sub-blocks using an intra-sub-block codec (sbIBC) mode and processing a second sub-block of the plurality of sub-blocks using an inter-coding mode.

7. A method of video processing includes performing a transition between a bitstream representation of a current video block and the current video block divided into a plurality of sub-blocks, wherein the transition includes processing a first sub-block of the plurality of sub-blocks using an intra-sub-block codec mode and processing a second sub-block of the plurality of sub-blocks using an inter-frame codec mode.

The following clauses illustrate example embodiments of the techniques discussed in clause 14 of the previous section.

8. The method of any of clauses 1-7, wherein the method further comprises avoiding updating the IBC history-based motion vector predictor table after the conversion of the current video block.

The following clauses illustrate example embodiments of the techniques discussed in clause 15 of the previous section.

9. The method of any one or more of clauses 1-7, further comprising avoiding updating the non-IBC history-based motion vector predictor table after the conversion of the current video block.

The following clauses illustrate example embodiments of the techniques discussed in clause 16 of the previous section.

10. The method of any of clauses 1-7, wherein the converting includes selective use of a loop filter based on the processing.

The following clauses illustrate example embodiments of the techniques discussed in clause 1 of the previous section.

17. The method of any of clauses 1-7, wherein performing the conversion comprises performing the conversion by disabling a particular codec mode for the current video block due to use of the method, wherein the particular codec mode comprises one or more of a sub-block transform, affine motion prediction, multi-reference line intra prediction, matrix-based intra prediction, symmetric Motion Vector Difference (MVD) codec, mere derived/refined with MVD decoder side motion, bi-directional optical flow, simplified quadratic transform, or multiple transform set.

The following clauses illustrate example embodiments of the techniques discussed in clause 18 of the previous section.

1. The method of any of the preceding clauses, wherein the indicator in the bitstream representation comprises information on how to apply the method to the current video block.

The following clauses illustrate example embodiments of the techniques discussed in clause 19 of the previous section.

1. A method of video encoding comprising deciding to encode a current video block into a bitstream representation using the method of any of the preceding clauses, and including information in the bitstream representation indicating the decision at a decoder parameter set level or sequence parameter set level or video parameter set level or picture header level or slice header level or maximum codec unit level or codec unit row level or LCU group level or transform unit level or prediction unit level or video codec unit level.

2. A method of video encoding comprising deciding to encode a current video block into a bitstream representation based on encoding conditions using the method of any of the preceding clauses, and performing encoding using the method of any of the preceding clauses, wherein the conditions are based on one or more of a codec unit, a prediction unit, a transform unit, a current video block, or a position of a video codec unit of the current video block,

The block dimensions of the current video block and/or its neighboring blocks,

The block shape of the current video block and/or its neighboring blocks,

Intra mode for the current video block and/or its neighboring blocks,

Motion/block vectors of neighboring blocks of the current video block;

The color format of the current video block;

a codec tree structure;

a slice or group type or picture type of the current video block;

Color components of the current video block;

A temporal layer ID of the current video block;

A profile or level or standard for the bitstream representation.

The following clauses illustrate example embodiments of the techniques discussed in clause 20 of the previous section.

1. A method of video processing includes determining to use an intra block copy mode and an inter prediction mode for conversion between blocks in a video region and bitstream representations of the video region, and performing conversion using the intra block copy mode and the inter prediction mode for blocks in the video region.

2. The method of clause 1, wherein the video region comprises a video picture or video slice or group of video slices or video slices.

3. The method of any of clauses 1-2, wherein the inter prediction mode uses an optional motion vector predictor (AMVP) codec mode.

4. The method of any of clauses 1-3, wherein performing the conversion comprises deriving a Merge candidate for the intra block copy mode from the neighboring blocks.

The following clauses illustrate example embodiments of the techniques discussed in clause 21 of the previous section.

1. A method of video processing includes performing a motion candidate list construction process and/or a table update process for updating a history-based motion vector predictor table based on codec conditions during a transition between a current video block and a bitstream representation of the current video block, and performing the transition based on the motion candidate list construction process and/or the table update process.

2. According to the method of clause 1, different procedures may be applied when the codec condition is met or not met.

3. The method according to clause 1, wherein when the codec condition is satisfied, no history-based update of the motion vector predictor table is applied.

4. The method of clause 1, wherein the derivation of candidates from spatially neighboring (adjacent or non-adjacent) blocks is skipped when the codec condition is satisfied.

5. The method of clause 1, wherein derivation of HMVP candidates is skipped when the codec condition is satisfied.

6. The method of clause 1, wherein the codec condition comprises a block width multiplied by a height of no greater than 16 or 32 or 64.

7. The method of clause 1, wherein the codec condition comprises the block being encoded in IBC mode.

8. The method of clause 1, wherein the codec conditions are as described in clause 21. B.s.iv.

The method of any of the preceding clauses, wherein the converting comprises generating a bitstream representation from the current video block.

The method of any of the preceding clauses, wherein the converting comprises generating samples of the current video block from the bitstream representation.

A video processing apparatus comprising a processor configured to implement the method of any one or more of the preceding clauses.

A computer readable medium having code stored thereon, which when executed causes a processor to implement a method according to any one or more of the preceding clauses.

Fig. 24 is a block diagram illustrating an example video processing system 2400 in which various techniques disclosed herein may be implemented. Various embodiments may include some or all of the components of system 2400. System 2400 can include an input 2402 for receiving video content. The video content may be received in an original or uncompressed format, such as 8 or 10 bit multi-component pixel values, or may be in a compressed or encoded format. Input 2402 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as ethernet, passive Optical Network (PON), etc., and wireless interfaces such as Wi-Fi or cellular interfaces.

System 2400 can include a codec component 2404 that can implement various codec or encoding methods described in this document. The codec component 2404 may reduce the average bit rate of the video from the input 2402 to the output of the codec component 2404 to produce a codec representation of the video. Codec techniques are therefore sometimes referred to as video compression or video transcoding techniques. The output of the codec component 2404 can be stored or transmitted via a communication connection as represented by component 2406. The stored or communicatively transmitted bit stream (or codec) representation of the video received at input 2402 may be used by component 2408 to generate pixel values or displayable video transmitted to display interface 2410. The process of generating user-viewable video from a bitstream representation is sometimes referred to as video decompression. Further, while a particular video processing operation is referred to as a "codec" operation or tool, it will be appreciated that a codec tool or operation is used at the encoder, and that a corresponding decoding tool or operation that inverts the codec results will be performed by the decoder.

Examples of the peripheral bus interface or the display interface may include a Universal Serial Bus (USB), or a High Definition Multimedia Interface (HDMI), or a display port (Displayport), or the like. Examples of storage interfaces include SATA (serial advanced technology attachment), PCI, IDE interfaces, and the like. The techniques described in this document may be embodied in various electronic devices such as mobile phones, laptops, smartphones, or other devices capable of performing digital data processing and/or video display.

Fig. 25 is a flow chart representation of a method 2500 for video processing in accordance with the present technique. Method 2500 includes, at operation 2510, determining that a current block of video is divided into a plurality of sub-blocks for a transition between the current block and a bitstream representation of the video. At least one of the plurality of blocks is encoded using a modified Intra Block Copy (IBC) codec technique that uses reference samples from one or more video regions of a current picture of the current block. Method 2500 includes, at operation 2520, performing a conversion based on the determination.

In some embodiments, the video region includes a current picture, a slice, a tile, or a group of slices. In some embodiments, where the current block has a dimension of mxn, the current block is divided into a plurality of sub-blocks, M and N being integers. In some embodiments, the plurality of sub-blocks have the same size of lxk, where L and K are integers. In some embodiments, l=k. In some embodiments, l=4 or k=4.

In some embodiments, the plurality of sub-blocks have different sizes. In some embodiments, the plurality of sub-blocks have a non-rectangular shape. In some embodiments, the plurality of sub-blocks have a triangular or wedge shape.

In some embodiments, the size of at least one of the plurality of sub-blocks is determined based on the size of a minimum coding unit, a minimum prediction unit, a minimum transform unit, or a minimum unit for motion information storage. In some embodiments, the size of at least one sub-block is represented as (n1×minw) × (n2× minH), where minw× minH represents the size of a minimum codec unit, prediction unit, transform unit, or unit for motion information storage, and where N1 and N2 are positive integers. In some embodiments, the size of at least one of the plurality of sub-blocks is based on a codec mode in which the current block is encoded in the bitstream representation. In some embodiments, the coding mode includes at least an Intra Block Copy (IBC) Merge mode, or a sub-block temporal motion vector prediction mode. In some embodiments, a size of at least one of the plurality of sub-blocks is signaled in the bitstream representation.

In some embodiments, the method includes determining to divide a subsequent block of video into a plurality of sub-blocks for the conversion, wherein a first sub-block in the current block has a different size than a second sub-block in the subsequent block. In some embodiments, the size of the first sub-block is different from the size of the second sub-block depending on the dimensions of the current block and the dimensions of the subsequent blocks. In some embodiments, the size of at least one of the plurality of sub-blocks is based on a color format or color component of the video. In some embodiments, the first sub-block is associated with a first color component of the video and the second sub-block is associated with a second color component of the video, the first sub-block and the second sub-block having different dimensions. In some embodiments, where the color format of the video is 4:2:0, the first sub-block associated with the luma component has a dimension of 2 lx2k and the second sub-block associated with the chroma component has a dimension of lxk. In some embodiments, where the color format of the video is 4:2:2, the first sub-block associated with the luma component has a dimension of 2l×2K and the second sub-block associated with the chroma component has a dimension of 2l×k. In some embodiments, the first sub-block is associated with a first color component of the video and the second sub-block is associated with a second color component of the video, the first sub-block and the second sub-block having the same dimension. In some embodiments, where the color format of the video is 4:2:0 or 4:4:4, a first sub-block associated with the luma component has a dimension of 2l×2K and a second sub-block associated with the chroma component has a dimension of 2l×2K.

In some embodiments, a motion vector of a first sub-block associated with a first color component of a video is determined based on one or more sub-blocks associated with a second color component of the video. In some embodiments, the motion vector of the first sub-block is an average of the motion vectors of one or more sub-blocks associated with the second color component. In some embodiments, the current block is partitioned into a plurality of sub-blocks based on a single tree partition structure. In some embodiments, the current block is associated with a chroma component of the video. In some embodiments, the current block has a size of 4×4.

In some embodiments, the motion information of at least one of the plurality of sub-blocks is determined based on identifying a reference block based on the initial motion vector and determining the motion information of the sub-block based on the reference block. In some embodiments, the reference block is located within the current picture. In some embodiments, the reference block is located within a reference picture of the one or more reference pictures. In some embodiments, at least one of the one or more reference pictures is a current picture. In some embodiments, the reference block is located within a collocated reference picture collocated with one of the one or more reference pictures according to the temporal information. In some embodiments, the reference picture is determined based on motion information of the collocated block or neighboring blocks of the collocated block. The juxtaposition block is juxtaposing with the current block according to the time domain information. In some embodiments, the initial motion vector of the reference block is determined based on one or more neighboring blocks of one or more neighboring blocks or sub-blocks of the current block. In some embodiments, the one or more neighboring blocks include neighboring blocks and/or non-neighboring blocks of the current block or sub-block. In some embodiments, at least one of the one or more neighboring blocks and the reference block are located within the same picture. In some embodiments, at least one of the one or more neighboring blocks is located within a reference picture of the one or more reference pictures. In some embodiments, at least one of the one or more neighboring blocks is located within a collocated reference picture collocated with one of the one or more reference pictures according to the temporal information. In some embodiments, the reference picture is determined based on motion information of a collocated block or neighboring blocks of the collocated block, wherein the collocated block is collocated with the current block according to temporal information.

In some embodiments, the initial motion vector is equal to a motion vector stored in one of the one or more neighboring blocks. In some embodiments, the initial motion vector is determined based on an order in which one or more neighboring blocks are checked for the transition. In some embodiments, the initial motion vector is a first identified motion vector associated with the current picture. In some embodiments, the initial motion vector of the reference block is determined based on a motion candidate list. In some embodiments, the motion candidate list comprises an Intra Block Copy (IBC) candidate list, a Merge candidate list, a sub-block temporal motion vector prediction candidate list, or a history-based motion candidate list determined based on past motion prediction results. In some embodiments, the initial motion vector is determined based on the selected candidates in the list. In some embodiments, the selected candidate is the first candidate in the list. In some embodiments, the candidate list is constructed based on a process that uses spatial neighboring blocks that are different from spatial neighboring blocks in a conventional construction process.

In some embodiments, the initial motion vector of the reference block is determined based on a position of the current block in the current picture. In some embodiments, the initial motion vector of the reference block is determined based on the dimensions of the current block. In some embodiments, the initial motion vector is set to a default value. In some embodiments, the initial motion vector is indicated in the bitstream representation at the video unit level. In some embodiments, a video unit comprises a slice, a picture, a tile, a line of Coding Tree Units (CTUs), a CTU, a Coding Tree Block (CTB), a Coding Unit (CU), a Prediction Unit (PU), or a Transform Unit (TU). In some embodiments, the initial motion vector of a sub-block is different from another initial motion block of a second sub-block of the current block. In some embodiments, the initial motion vectors of the plurality of sub-blocks of the current block are differently determined according to the video unit. In some embodiments, the video unit comprises a block, a slice, or a stripe.

In some embodiments, the initial motion vector is converted to an F-pixel integer precision, F being a positive integer greater than or equal to 1, prior to identifying the reference block. In some embodiments, F is 1,2, or 4. In some embodiments, the initial motion vector is denoted (vx, vy), and wherein the converted motion vector (vx ', vy') is denoted (vxxf, vy x F). In some embodiments, the top left position of the sub-block is denoted (x, y), and the sub-block has a size of l×k, L being the width of the current sub-block, and K being the height of the sub-block. The reference block is identified as covering the region (x+ offsetX +vx ', y+ offsetY +vy'), offsetX and offset are non-negative. In some embodiments offsetX is 0 and/or offsetY is 0. In some embodiments offsetX is equal to L/2, L/2+1, or L/2-1. In some embodiments offsetY is equal to K/2, K/2+1, or K/2-1. In some embodiments offsetX and/or offsetY are clipped to a range that includes a picture, a slice, a tile, or an intra block copy reference area.

In some embodiments, the motion vector of the sub-block is further determined based on motion information of the reference block. In some embodiments, in the case that the motion vector of the reference block points to the current picture, the motion vector of the sub-block is the same as the motion vector of the reference block. In some embodiments, in the case that the motion vector of the reference block points to the current picture, the motion vector of the sub-block is determined based on the motion vector added to the reference block with the initial motion vector. In some embodiments, the motion vectors of the sub-blocks are clipped to a range such that the motion vectors of the sub-blocks point to the intra block copy reference area. In some embodiments, the motion vector of the sub-block is a valid motion vector of an intra block copy candidate for the sub-block.

In some embodiments, one or more intra block copy candidates for the sub-block are determined for use in determining motion information for the sub-block. In some embodiments, one or more intra block copy candidates are added to a motion candidate list, wherein the motion candidate list includes one of a Merge candidate for a sub-block, a sub-block temporal motion vector prediction candidate for a sub-block, or an affine Merge candidate for a sub-block. In some embodiments, one or more intra block copy candidates precede any Merge candidates of the sub-blocks in the list. In some embodiments, one or more intra block copy candidates are located after any sub-block temporal motion vector prediction candidates of a sub-block in the list. In some embodiments, one or more intra block copy candidates follow any inherited or built affine candidates of a sub-block in the list. In some embodiments, whether one or more intra block copy candidates are added to the motion candidate list is based on a codec mode of the current block. In some embodiments, where the current block is encoded using an Intra Block Copy (IBC) sub-block temporal motion vector prediction mode, one or more intra block copy candidates are excluded from the motion candidate list.

In some embodiments, whether one or more intra block copy candidates are added to the motion candidate list is based on the partition structure of the current block. In some embodiments, one or more intra block copy candidates are added as mere candidates for sub-blocks in the list. In some embodiments, one or more intra block copy candidates are added to the motion candidate list based on a different initial motion vector. In some embodiments, whether one or more intra block copy candidates are added to the motion candidate list is indicated in the bitstream representation. In some embodiments, an index indicating the motion candidate list is signaled in the bitstream representation for the current block-based codec mode. In some embodiments, in the case of encoding and decoding the current block using the intra block copy Merge mode, an index indicating a motion candidate list including intra block copy Merge candidates is signaled in the bitstream representation. In some embodiments, in the case of encoding the current block using the intra block copy sub-block temporal motion vector prediction mode, an index indicating a motion candidate list comprising intra block copy sub-block temporal motion vector prediction candidates is signaled in the bitstream representation. In some embodiments, the motion vector difference of the intra block copy sub-block temporal motion vector prediction mode is applied to a plurality of sub-blocks.

In some embodiments, the reference block and the sub-block are associated with the same color component of the video. In some embodiments, whether the current block is divided into a plurality of sub-blocks is based on a codec characteristic associated with the current block. In some embodiments, the codec characteristics include syntax flags in a bitstream representation in a decoder parameter set, a sequence parameter set, a video parameter set, a picture parameter set, an APS, a picture header, a slice group header, a Largest Codec Unit (LCU), a Coding Unit (CU), LCU rows, LCU groups, transform units, prediction unit blocks, or video codec units. In some embodiments, the codec characteristic includes a location of a codec unit, a prediction unit, a transform unit, a block, or a video codec unit. In some embodiments, the codec characteristics include dimensions of the current block or neighboring blocks of the current block. In some embodiments, the codec characteristic includes a shape of the current block or a neighboring block of the current block. In some embodiments, the codec characteristic includes an intra-codec mode of the current block or a neighboring block of the current block. In some embodiments, the codec characteristic includes a motion vector of a neighboring block of the current block. In some embodiments, the codec characteristic includes an indication of a color format of the video. In some embodiments, the codec characteristic includes a codec tree structure of the current block. In some embodiments, the codec characteristics include a slice type, a slice group type, or a picture type associated with the current block. In some embodiments, the codec characteristic includes a color component associated with the current block. In some embodiments, the codec characteristic includes a time domain layer identifier associated with the current block. In some embodiments, the codec characteristics include a profile, level, or hierarchy of standards for the bitstream representation.

Fig. 26 is a flowchart representation of a method 2600 for video processing in accordance with the present technique. The method 2600 includes, at operation 2610, determining that a current block of video is divided into a plurality of sub-blocks for a transition between the current block and a bitstream representation of the video. According to the mode, each of the plurality of sub-blocks is encoded in the encoded representation using a corresponding encoding and decoding technique. The method also includes, at operation 2620, performing a conversion based on the determination.

In some embodiments, the mode specifies that a first sub-block of the plurality of sub-blocks is encoded using a modified Intra Block Copy (IBC) encoding and decoding technique in which reference samples from the video region are used. In some embodiments, the mode specifies that a second sub-block of the plurality of sub-blocks is to be encoded using an intra-prediction encoding and decoding technique in which samples from the same sub-block are used. In some embodiments, the mode designation encodes a second sub-block of the plurality of sub-blocks using a palette coding technique in which a palette of representative pixel values is used. In some embodiments, the mode designation encodes a second sub-block of the plurality of sub-blocks using an inter-frame codec technique in which time domain information is used.

In some embodiments, the mode specifies that a first sub-block of the plurality of sub-blocks is to be encoded using an intra-prediction encoding and decoding technique in which samples from the same sub-block are used. In some embodiments, the mode designation encodes a second sub-block of the plurality of sub-blocks using a palette coding technique in which a palette of representative pixel values is used. In some embodiments, the mode designation encodes a second sub-block of the plurality of sub-blocks using an inter-frame codec technique in which time domain information is used.

In some embodiments, the mode designation uses a single codec technique to codec all of the plurality of sub-blocks. In some embodiments, the single coding technique includes intra-prediction coding techniques for coding multiple sub-blocks from samples of the same sub-block. In some embodiments, the single coding technique includes a palette coding technique for coding a palette of representative pixel values for a plurality of sub-blocks.

In some embodiments, where the mode of one or more codec techniques is applicable to the current block, a history-based table of motion candidates for the sub-block temporal motion vector prediction mode remains the same for the transform, the history-based table of motion candidates being determined based on motion information in past transforms. In some embodiments, history-based tables are used for IBC codec technology or non-IBC codec technology.

In some embodiments, where the mode specifies encoding at least one of the plurality of sub-blocks using IBC codec techniques, one or more motion vectors of the at least one sub-block are used to update a history-based table of motion candidates for the IBC sub-block temporal motion vector prediction mode, the history-based table of motion candidates being determined based on motion information in past transitions. In some embodiments, in the event that the mode specifies to use inter-coding techniques to code at least one of the plurality of sub-blocks, one or more motion vectors of the at least one sub-block are used to update a history-based table of motion candidates for non-IBC sub-block temporal motion vector prediction modes, the history-based table of motion candidates being determined based on motion information in past transitions.

In some embodiments, the use of a filtering process in which boundaries of a plurality of sub-blocks are filtered is based on the use of at least one codec technique according to a pattern. In some embodiments, a filtering process of filtering boundaries of a plurality of sub-blocks is applied in case at least one codec technique is applied. In some embodiments, the filtering process of filtering the boundaries of the plurality of sub-blocks is omitted in the case of applying at least one codec technique.

In some embodiments, the second codec technique is disabled for the current block for the transition according to the mode. In some embodiments, the second coding technique includes at least one of a sub-block transform coding technique, an affine motion prediction coding technique, a multi-reference line intra-prediction coding technique, a matrix-based intra-prediction coding technique, a symmetric Motion Vector Difference (MVD) coding technique, a Merge using MVD decoder side motion derivation or refinement coding technique, a bi-directional optical flow coding technique, a quadratic transform coding technique with dimension reduction based on the dimension of the current block, or a multiple transform set coding technique.

In some embodiments, the use of at least one codec technique according to the mode is signaled in the bitstream representation. In some embodiments, the use is signaled at a sequence level, a picture level, a slice group level, a slice level, a tile level, a Coding Tree Unit (CTU) level, a Coding Tree Block (CTB) level, a Coding Unit (CU) level, a Prediction Unit (PU) level, a Transform Unit (TU) or at another video unit level. In some embodiments, the at least one coding technique comprises a modified IBC coding technique, and the modified IBC coding technique is indicated in the bitstream representation based on an index value indicating a candidate in the motion candidate list. In some embodiments, the predefined value is assigned to the current block that is encoded using a modified IBC codec technique.

In some embodiments, the use of at least one codec technique according to the mode is determined during the transition. In some embodiments, the use of Intra Block Copy (IBC) codec techniques for coding and decoding a current block, where reference samples from the current block are used, is signaled in the bitstream. In some embodiments, the use of Intra Block Copy (IBC) codec techniques for coding the current block, where reference samples from the current block are used, is determined during the conversion.

In some embodiments, motion information of multiple sub-blocks of the current block is used as a motion vector predictor for transitions between subsequent blocks of video and bitstream representations. In some embodiments, the motion information of multiple sub-blocks of the current block is not allowed for conversion between subsequent blocks of video and bitstream representations. In some embodiments, determining that the current block is divided into a plurality of sub-blocks that are encoded using at least one encoding and decoding technique is based on whether the motion candidate is a candidate for a block or a sub-block within a block of video.

Fig. 27 is a flowchart representation of a method 2700 for video processing in accordance with the present technique. The method 2700 includes, at operation 2710, determining an operation associated with a motion candidate list based on a condition related to a characteristic of a current block for a transition between the current block of video and a bitstream representation of the video. The motion candidate list is constructed for the codec technique or based on information from previously processed blocks of the video. The method 2700 further includes, at operation 2720, performing conversion based on the determination.

Among the coding techniques are Merge coding techniques, intra Block Copy (IBC) sub-block temporal motion vector prediction coding techniques, sub-block Merge coding techniques, IBC coding techniques, or modified IBC coding techniques that use reference samples from a video region of a current block for coding and decoding at least one sub-block of the current block.

In some embodiments, the current block has a dimension of w×h, W and H being positive integers. The condition is related to the dimension of the current block. In some embodiments, the condition relates to the codec information of the current block or the codec information of neighboring blocks of the current block. In some embodiments, the condition relates to a Merge sharing condition for sharing the motion candidate list between the current block and another block.

In some embodiments, the operations include deriving spatial Merge candidates of the motion candidate list using Merge codec techniques. In some embodiments, the operations include deriving a motion candidate of the motion candidate list based on spatial neighboring blocks of the current block. In some embodiments, the spatial neighboring blocks include neighboring blocks or non-neighboring blocks of the current block.

In some embodiments, the operations include deriving motion candidates for a motion candidate list constructed based on information from previously processed blocks of the video. In some embodiments, the operations include deriving a pair of Merge candidates of the motion candidate list. In some embodiments, the operations include one or more pruning operations that remove redundant entries in the motion candidate list. In some embodiments, one or more pruning operations are used for spatial domain Merge candidates in the motion candidate list.

In some embodiments, the operations include updating a motion candidate list constructed based on information from previously processed blocks of the video after the conversion. In some embodiments, updating includes adding the derived candidates to the motion candidate list without removing redundant pruning operations in the motion candidate list. In some embodiments, the operations include adding a default motion candidate in the motion candidate list. In some embodiments, the default motion candidates include zero motion candidates using IBC sub-block temporal motion vector prediction coding techniques. In some embodiments, operations are skipped if the condition is met.

In some embodiments, the operations include checking the motion candidates in the motion candidate list in a predefined order. In some embodiments, the operations include checking a predefined number of motion candidates in the motion candidate list. In some embodiments, the condition is satisfied where w×h is greater than or equal to a threshold. In some embodiments, the condition is satisfied where w×h is greater than or equal to a threshold and the current block is encoded using IBC sub-block temporal motion vector prediction encoding or Merge encoding techniques. In some embodiments, the threshold is 1024.

In some embodiments, the condition is satisfied where W and/or H is greater than or equal to a threshold. In some embodiments, the threshold is 32. In some embodiments, the condition is satisfied where w×h is less than or equal to a threshold and the current block is encoded using IBC sub-block temporal motion vector prediction encoding or Merge encoding techniques. In some embodiments, the threshold is 16. In some embodiments, the threshold is 32 or 64. In some embodiments, the operation comprising inserting candidates determined based on spatial neighboring blocks into the motion candidate list is skipped if a condition is met.

In some embodiments, where W is equal to T2, H is equal to T3, and neighboring blocks above the current block are available and are encoded using the same encoding and decoding techniques as the current block, the condition is satisfied, T2 and T3 being positive integers. In some embodiments, the condition is satisfied where the neighboring block and the current block are in the same codec tree unit.

In some embodiments, where W is equal to T2, H is equal to T3, and a neighboring block above the current block is not available or is outside of the current codec tree unit in which the current block is located, the condition is satisfied, T2 and T3 are positive integers. In some embodiments, T2 is 4 and T3 is 8. In some embodiments, where W is equal to T4, H is equal to T5, and a neighboring block to the left of the current block is available and is encoded using the same encoding and decoding technique as the current block, the condition is satisfied, T4 and T5 are positive integers. In some embodiments, where W equals T4, H equals T5, and a neighboring block to the left of the current block is not available, the condition is satisfied, T4 and T5 being positive integers. In some embodiments, T4 is 8 and T5 is 4.

In some embodiments, the condition is satisfied in the case where w×h is less than or equal to a threshold, the current block is encoded using an IBC sub-block temporal motion vector prediction codec technique or a Merge codec technique, and a first neighboring block above the current block and a second neighboring block to the left of the current block are encoded using the same encoding and decoding technique. In some embodiments, the first neighboring block and the second neighboring block are available and are encoded using IBC encoding and decoding techniques, and wherein the second neighboring block is within the same codec tree unit as the current block. In some embodiments, the first neighboring block is not available, and wherein the second neighboring block is available and within the same codec tree unit as the current block. In some embodiments, the first neighboring block and the second neighboring block are not available. In some embodiments, the first neighboring block is available and the second neighboring block is not available. In some embodiments, the first neighboring block is not available and the second neighboring block is outside the codec tree unit where the current block is located. In some embodiments, the first neighboring block is available and the second neighboring block is outside of the codec tree unit where the current block is located. In some embodiments, the threshold is 32. In some embodiments, the first neighboring block and the second neighboring block are used to derive a spatial Merge candidate. In some embodiments, the top left sample of the current block is located at (x, y), and wherein the second neighboring block overlays the sample located at (x-1, y+H-1). In some embodiments, the top left sample of the current block is located at (x, y), and wherein the second neighboring block overlays the sample located at (x+W-1, y-1).

In some embodiments, the same codec technique includes an IBC codec technique. In some embodiments, the same codec technique comprises an inter-frame codec technique. In some embodiments, neighboring blocks of the current block have dimensions equal to a×b. In some embodiments, neighboring blocks of the current block have dimensions greater than a×b. In some embodiments, neighboring blocks of the current block have dimensions less than a×b. In some embodiments, a×b is equal to 4×4. In some embodiments, the threshold is predefined. In some embodiments, the threshold is signaled in the bitstream representation. In some embodiments, the threshold is based on a codec characteristic of the current block, the codec characteristic including a codec mode in which the current block is encoded.

In some embodiments, the condition is satisfied in the case where the current block has a parent node sharing a motion candidate list and is encoded using IBC sub-block temporal motion vector prediction codec or Merge codec. In some embodiments, the conditions are adaptively changed according to the codec characteristics of the current block.

Fig. 28 is a flowchart representation of a method 2800 for video processing in accordance with the present technique. The method 2800 includes, at operation 2810, determining that a current block, which is encoded using an inter-frame encoding and decoding technique based on temporal information, is divided into a plurality of sub-blocks for a transition between the current block of video and a bitstream representation of the video. At least one of the plurality of blocks is encoded using a modified Intra Block Copy (IBC) codec technique that uses reference samples from one or more video regions of a current picture that includes the current block. The method 2800 includes, at operation 2820, performing a conversion based on the determination.

In some embodiments, the video region includes a current picture, a slice, a tile, or a group of slices. In some embodiments, the inter-frame coding technique comprises a sub-block temporal motion vector coding technique, and wherein one or more syntax elements indicating whether the current block is coded based on the current picture and a reference picture different from the current picture are included in the bitstream representation. In some embodiments, in the case of coding a current block based on the current picture and the reference picture, the one or more syntax elements indicate the reference picture used to code the current block. In some embodiments, the one or more syntax elements further indicate motion information associated with the reference picture, the motion information including at least a motion vector prediction index, a motion vector difference, or a motion vector precision. In some embodiments, the first reference picture list includes only the current picture and the second reference picture list includes only the reference picture. In some embodiments, the inter-frame coding technique includes a temporal Merge coding technique, and the motion information is determined based on neighboring blocks of the current block, the motion information including at least a motion vector or a reference picture. In some embodiments, the motion information is applicable only to the current picture in the case where the neighboring block is determined based only on the current picture. In some embodiments, in the case where the neighboring block is determined based on the current picture and the reference picture, the motion information is applicable to the current picture and the reference picture. In some embodiments, the motion information is applicable to the current picture only if neighboring blocks are determined based on the current picture and the reference picture. In some embodiments, where a neighboring block is determined based only on a reference picture, the neighboring block is discarded for determining the Merge candidate.

In some embodiments, a fixed weighting factor is assigned to a reference block from the current picture and a reference block from the reference picture. In some embodiments, the fixed weighting factor is signaled in the bitstream representation.

In some embodiments, performing the conversion includes generating a bitstream representation based on the blocks of video. In some embodiments, performing the conversion includes generating blocks of video from the bitstream representation.

It should be appreciated that techniques for video encoding or video decoding are disclosed. Video encoders or decoders may employ these techniques together to use intra block copying and sub-block based video processing to achieve greater codec efficiency and performance.

Some embodiments of the disclosed technology include making decisions or determinations to enable video processing tools or modes. In an example, when a video processing tool or mode is enabled, the encoder will use or implement the tool or mode in the processing of blocks of video, but may not necessarily modify the generated bitstream based on the use of the tool or mode. That is, when the video processing tool or mode is enabled based on the decision or determination, the conversion of the block of video to the bitstream representation of the video will use the video processing tool or mode. In another example, when a video processing tool or mode is enabled, the decoder will process the bitstream with knowledge that the bitstream has been modified based on the video processing tool or mode. That is, the conversion from the bitstream representation of the video to blocks of the video will be performed using video processing tools or modes that are enabled based on the decision or determination.

Some embodiments of the disclosed technology include making a decision or decision to disable a video processing tool or mode. In an example, when a video processing tool or mode is disabled, the encoder will not use the tool or mode in the conversion of blocks of video into a bitstream representation of video. In another example, when the video processing tool or mode is disabled, the decoder will process the bitstream with the knowledge that the bitstream was not modified using the video processing tool or mode that was enabled based on the decision or determination.

The disclosed and other solutions, examples, embodiments, modules, and functional operations described in this document may be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this document and their structural equivalents, or in combinations of one or more of them. The disclosed and other embodiments may be implemented as one or more computer program products, such as one or more modules of computer program instructions encoded on a computer-readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of materials affecting a machine-readable propagated signal, or a combination of one or more of them. The term "data processing apparatus" encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. In addition to hardware, an apparatus may include code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagated signal is an artificially generated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus.

A computer program (also known as a program, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. The computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

The processes and logic flows described in this document can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer does not require such a device. Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices including by way of example semiconductor memory devices, e.g. EPROM, EEPROM, and flash memory devices, magnetic disks, e.g. internal hard disks or removable disks, magneto-optical disks, and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

Although this patent document contains many specifics, these should not be construed as limitations on the scope of any subject matter or of what may be claimed, but rather as descriptions of features specific to particular embodiments of particular technologies. Certain features that are described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

Similarly, although operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Furthermore, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

Only some embodiments and examples are described and other embodiments, enhancements, and variations may be made based on what is described and shown in this patent document.

Claims (92)

1. A video processing method, comprising: For the conversion between the current block of the video and the video bitstream, it is determined that the current block is divided into multiple sub-blocks, wherein at least one of the multiple sub-blocks is encoded and decoded using a modified intra-block copy (IBC) encoding/decoding technique, wherein the technique uses reference samples from one or more video regions of the current block; and The conversion is performed based on the determination. Wherein, the size of at least one of the plurality of sub-blocks is determined based on the size of the smallest decoding unit, the smallest prediction unit, the smallest transform unit, or the smallest unit for storing motion information, and, The size of at least one of the plurality of sub-blocks is also determined based on the color format or color components of the video. 2. The method according to claim 1, wherein the video area includes the current image, strip, slice, tile, or group of slices. 3. The method according to claim 1 or 2, wherein, when the current block has a dimension of M×N, the current block is divided into multiple sub-blocks, where M and N are integers. 4. The method according to claim 1, wherein the plurality of sub-blocks have the same size of L×K, where L and K are integers. 5. The method according to claim 4, wherein L = K. 6. The method according to claim 4, wherein L = 4 or K = 4. 7. The method of claim 1, wherein the plurality of sub-blocks have different sizes. 8. The method according to claim 1, wherein the plurality of sub-blocks have a non-rectangular shape. 9. The method of claim 8, wherein the plurality of sub-blocks have a triangular or wedge shape. 10. The method according to claim 1, wherein the size of at least one sub-block is expressed as (N1×minW)×(N2×minH), where minW×minH represents the size of the smallest decoding unit, prediction unit, transformation unit, or unit for storing motion information, and where N1 and N2 are positive integers. 11. The method of claim 1, wherein the size of at least one of the plurality of sub-blocks is based on the encoding/decoding mode in which the current block is encoded/decoded in the bitstream. 12. The method according to claim 11, wherein the encoding/decoding mode includes at least an intra-block copy (IBCMerge) mode or a sub-block temporal motion vector prediction mode. 13. The method of claim 1, wherein the size of at least one of the plurality of sub-blocks is signaled in the bit stream. 14. The method of claim 1, comprising: The transformation determines that subsequent blocks of the video are divided into multiple sub-blocks, wherein the first sub-block in the current block has a different size than the second sub-block in the subsequent block. 15. The method of claim 14, wherein the size of the first sub-block is different from the size of the second sub-block based on the dimensions of the current block and the dimensions of the subsequent blocks. 16. The method of claim 1, wherein the first sub-block is associated with a first color component of the video, and the second sub-block is associated with a second color component of the video, and the first sub-block and the second sub-block have different dimensions. 17. The method of claim 16, wherein, in the case of a 4:2:0 color format for the video, the first sub-block associated with the luminance component has a dimension of 2L×2K, and the second sub-block associated with the chrominance component has a dimension of L×K. 18. The method of claim 16, wherein, in the case of a 4:2:2 color format for the video, the first sub-block associated with the luminance component has a dimension of 2L×2K, and the second sub-block associated with the chrominance component has a dimension of 2L×K. 19. The method of claim 1, wherein the first sub-block is associated with a first color component of the video, and the second sub-block is associated with a second color component of the video, and the first and second sub-blocks have the same dimension. 20. The method of claim 19, wherein, in the case that the color format of the video is 4:2:0 or 4:4:4, the first sub-block associated with the luminance component has a dimension of 2L×2K, and the second sub-block associated with the chrominance component has a dimension of 2L×2K. 21. The method of claim 1, wherein the motion vector of the first sub-block associated with the first color component of the video is determined based on one or more sub-blocks associated with the second color component of the video. 22. The method of claim 21, wherein the motion vector of the first sub-block is the average of the motion vectors of one or more sub-blocks associated with the second color component. 23. The method of claim 1, wherein the current block is divided into multiple sub-blocks based on a single-tree partitioning structure. 24. The method of claim 1, wherein the current block is associated with the chroma components of the video. 25. The method of claim 24, wherein the current block has a size of 4×4. 26. The method of claim 1, wherein the motion information of at least one of the plurality of sub-blocks is determined based on the following: Identify reference blocks based on initial motion vectors; and The motion information of the sub-blocks is determined based on the reference block. The reference block is located within the current image. 27. The method of claim 26, wherein the reference block is located within a reference image in one or more reference images. 28. The method of claim 27, wherein at least one of the one or more reference images is the current image. 29. The method of claim 27, wherein the reference block is located within a juxtaposed reference image juxtaposed with one or more reference images according to time-domain information. 30. The method of claim 27, wherein the reference image is determined based on motion information of the juxtaposed block or its neighboring blocks, wherein the juxtaposed block is juxtaposed with the current block according to temporal information. 31. The method of claim 26, wherein the initial motion vector of the reference block is determined based on one or more neighboring blocks of the current block or one or more neighboring blocks of a sub-block. The one or more neighboring blocks include adjacent blocks and/or non-adjacent blocks of the current block or sub-block. 32. The method of claim 31, wherein at least one of the one or more neighboring blocks and the reference block are located within the same image. 33. The method of claim 31, wherein at least one of the one or more neighboring blocks is located within a reference image in one or more reference images. 34. The method of claim 33, wherein at least one of the one or more neighboring blocks is located within a juxtaposed reference image juxtaposed with one or more reference images according to temporal information. 35. The method of claim 33, wherein the reference image is determined based on motion information of the juxtaposed block or its neighboring blocks, wherein the juxtaposed block is juxtaposed with the current block according to temporal information. 36. The method of claim 26, wherein the initial motion vector is equal to the motion vector stored in one of one or more neighboring blocks. 37. The method of claim 26, wherein the initial motion vector is determined based on the order in which one or more neighboring blocks are checked for the transformation. 38. The method of claim 37, wherein the initial motion vector is a first identified motion vector associated with the current image. 39. The method of claim 26, wherein the initial motion vector of the reference block is determined based on a motion candidate list; The motion candidate list includes an intra-block copy (IBC) candidate list, a merge candidate list, a sub-block temporal motion vector prediction candidate list, or a history-based motion candidate list determined based on past motion prediction results. 40. The method of claim 39, wherein the initial motion vector is determined based on a selected candidate from the list. 41. The method of claim 40, wherein the selected candidate is the first candidate in the list. 42. The method of claim 39, wherein the motion candidate list is constructed based on a process of using spatial neighbor blocks that are different from those used in conventional construction processes. 43. The method of claim 26, wherein the initial motion vector is indicated in the bitstream at the video unit level, wherein the video unit includes slices, stripes, pictures, tiles, codec tree unit (CTU) rows, CTUs, codec tree blocks (CTBs), codec units (CUs), prediction units (PUs), or transform units (TUs). 44. The method of claim 26, wherein the initial motion vector of the at least one sub-block is different from another initial motion vector of the second sub-block of the current block. 45. The method of claim 43, wherein the initial motion vectors of the plurality of sub-blocks of the current block are determined differently according to the video unit. 46. The method of claim 45, wherein the video unit comprises a block, a slice, or a strip. 47. The method of claim 26, wherein, prior to identifying the reference block, the initial motion vector is converted to F-pixel integer precision, where F is a positive integer greater than or equal to 1. 48. The method of claim 47, wherein F is 1, 2 or 4. 49. The method of claim 47, wherein the initial motion vector is represented as (vx, vy), and wherein the transformed motion vector (vx’, vy’) is represented as (vx×F, vy×F). 50. The method of claim 49, wherein the top left position of the at least one sub-block is represented as (x, y), and the sub-block has a size of L×K, where L is the width of the current sub-block and K is the height of the sub-block, and wherein the reference block is identified as the region covering (x + offsetX + vx’, y + offsetY + vy’), where offsetX and offset are non-negative values. 51. The method of claim 50, wherein offsetX is 0 and/or offsetY is 0. 52. The method of claim 50, wherein offsetX is equal to L/2, L/2+1, or L/2-1. 53. The method of claim 50, wherein offsetY is equal to K/2, K/2+1, or K/2-1. 54. The method of claim 50, wherein offsetX and/or offsetY are limited to a range, said range including a picture, strip, slice, tile, or intra-frame block copy reference region. 55. The method of claim 26, wherein the motion vector of the at least one sub-block is further determined based on motion information of a reference block. 56. The method of claim 55, wherein, when the motion vector of the reference block points to the current image, the motion vector of the at least one sub-block is the same as the motion vector of the reference block. 57. The method of claim 55, wherein, when the motion vector of the reference block points to the current image, the motion vector of the at least one sub-block is determined based on the motion vector to which the initial motion vector is added to the reference block. 58. The method of claim 55, wherein the motion vector of the at least one sub-block is limited within a range such that the motion vector of the at least one sub-block points to the intra-block copy reference region. 59. The method of claim 55, wherein the motion vector of the at least one sub-block is a valid motion vector of an intra-block copy candidate of the at least one sub-block. 60. The method of claim 26, wherein one or more intra-block copy candidates of the at least one sub-block are determined for determining motion information of the at least one sub-block. 61. The method of claim 60, wherein the one or more intra-block copy candidates are added to a motion candidate list, wherein the motion candidate list includes one of the following: a sub-block merge candidate, a sub-block temporal motion vector prediction candidate, or a sub-block affine merge candidate. 62. The method of claim 61, wherein the one or more intra-block copy candidates are located before any Merge candidate of the at least one sub-block in the list. 63. The method of claim 61, wherein the one or more intra-block copy candidates are located after any sub-block temporal motion vector prediction candidates of the at least one sub-block in the list. 64. The method of claim 61, wherein the one or more intra-block copy candidates are located after any inherited or constructed affine candidates of the at least one sub-block in the list. 65. The method of claim 61, wherein whether the one or more intra-block copy candidates are added to the motion candidate list is based on the encoding/decoding mode of the current block. 66. The method of claim 62, wherein, when encoding and decoding the current block using the intra-block copy IBC sub-block temporal motion vector prediction mode, one or more intra-block copy candidates are excluded from the motion candidate list. 67. The method of claim 61, wherein whether the one or more intra-block copy candidates are added to the motion candidate list is based on the segmentation structure of the current block. 68. The method of claim 61, wherein the one or more intra-block copy candidates are added as merge candidates for the at least one sub-block in the list. 69. The method of claim 61, wherein the one or more intra-block copy candidates are added to the motion candidate list based on different initial motion vectors. 70. The method of claim 61, wherein whether the one or more intra-block copy candidates are added to the motion candidate list is indicated in the bitstream. 71. The method of claim 61, wherein the index of the motion candidate list is signaled in the bitstream based on the encoding/decoding mode of the current block. 72. The method of claim 71, wherein, when encoding or decoding the current block using the intra-block copy Merge mode, an index indicating that a motion candidate list including intra-block copy Merge candidates is signaled in the bitstream. 73. The method of claim 71, wherein, when encoding and decoding the current block using the intra-block copy sub-block temporal motion vector prediction mode, an index indicating a motion candidate list including intra-block copy sub-block temporal motion vector prediction candidates is signaled in the bitstream representation. 74. The method of claim 73, wherein the motion vector difference of the intra-block copy sub-block temporal motion vector prediction mode is applied to multiple sub-blocks. 75. The method of claim 26, wherein the reference block and the at least one sub-block are associated with the same color component of the video. 76. The method of claim 1, wherein whether the current block is divided into multiple sub-blocks is based on the encoding/decoding characteristics associated with the current block. 77. The method according to claim 76, wherein the encoding/decoding features include decoder parameter sets, sequence parameter sets, video parameter sets, picture parameter sets, APS, picture headers, strip headers, slice headers, maximum codec unit (LCU), codec unit (CU), LCU rows, LCU groups, transform units, prediction units, prediction unit blocks, or syntax flags in the bitstream representation of video codec units. 78. The method according to claim 76, wherein the encoding/decoding features include the positions of encoding/decoding units, prediction units, transform units, block or video encoding/decoding units. 79. The method of claim 76, wherein the encoding/decoding feature includes the dimension of the current block or a neighboring block of the current block. 80. The method of claim 76, wherein the encoding/decoding feature includes the shape of the current block or a neighboring block of the current block. 81. The method of claim 76, wherein the encoding/decoding features include the intra-frame encoding/decoding mode of the current block or a neighboring block of the current block. 82. The method of claim 76, wherein the encoding/decoding feature includes motion vectors of neighboring blocks of the current block. 83. The method of claim 76, wherein the encoding/decoding features include an indication of the color format of the video. 84. The method of claim 76, wherein the encoding/decoding features include the encoding/decoding tree structure of the current block. 85. The method of claim 76, wherein the encoding/decoding characteristics include a stripe type, slice group type, or picture type associated with the current block. 86. The method of claim 76, wherein the encoding/decoding features include color components associated with the current block. 87. The method of claim 76, wherein the encoding/decoding feature includes a temporal layer identifier associated with the current block. 88. The method of claim 76, wherein the encoding/decoding features include a summary table, level, or hierarchy of a standard bitstream representation. 89. The method according to any one of claims 1 to 88, wherein performing the conversion includes generating a bitstream based on the current block of the video. 90. The method according to any one of claims 1 to 88, wherein performing the conversion includes generating a current block of video from the bitstream. 91. A video processing apparatus comprising a processor configured to implement the method according to any one of claims 1 to 90. 92. A computer-readable medium having code stored thereon, which, when executed, causes a processor to perform the method according to any one of claims 1 to 90.