H.265/HEVC - Ittiam Systems

H.265/HEVC

H.265/HEVC

HEVC

(High Efficiency Video Coding) ratified as an International standard under ITU-T as

H.265

[1], is a new video compression technology that can provide up to 2x bandwidth savings over H.264 based compression. Broadcast segments like delivery of movies or videos over satellite, TV delivery over broadband and mobile networks and transmission of video for ENG, DMNG over 2G/3G networks will benefit from this bandwidth savings.

HEVC

achieves the stated goal of 2x bandwidth reduction over H.264 by using newer and more refined compression tools. Following are the significant additions and enhancements in

H.265

over the previous standard namely H.264:
  • Larger prediction and coding blocks –up to 64x64 coding units
  • Hierarchical coding structure of blocks for greater flexibility in dividing up the image
  • Larger as well as symmetric and asymmetric motion partitions
  • Larger transform sizes up to 32x32 with hierarchical coding structure
  • Up to 35 modes of intra prediction in

    H.265

    compared to 9 in H.264
  • 8-tap quarter pixel filtering for improved interpolation accuracy
  • Sample adaptive offset based in-loop filtering
  • Tiles and Wavefront parallel processing for higher efficiency parallel processing

Coding Tree Structure

Development of

H.265

standard also coincided with an increased resolution in displays and cameras enabling

UHD

(3840x2160),

4K

(4096x2048) and 8K (7680x4320) size videos. To enable optimal compression efficiency at these higher resolutions,

HEVC

supports larger coding and prediction block sizes.

HEVC

supports up to 64x64 coding units. In addition to this

HEVC

also supports coding tree blocks (CTBs) wherein the 64x64 coding unit can be further subdivided hierarchically into smaller coding units. This allows for maximum flexibility in breaking up a portion of image into different sizes that suit the large homogeneous regions as well as detailed smaller regions of an image.

The below images indicate how the large homogenous or less detailed areas are covered by large block sizes ranging from 64x64 to 8x8 and the detailed regions are hierarchically broken down to ensure the variations are divided into smaller areas to enable better prediction as well as residual coding.

Coding units in Ittiam HEVC codec

Figure 1 a: Part of a sample frame from a sequence of video frames

Coding structure of Ittiam HEVC codec

Figure 1 b: Frame from figure 1 a with Coding Units (CU) overlaid to indicate HEVC coding structure

Enhanced Prediction Modes

Higher motion prediction units up to 64x64 are also supported in

HEVC

. In addition to the higher prediction units,

HEVC

also supports square and rectangular symmetric and asymmetric motion partitions as shown in Figure 2 below
Inter prediction units in Ittiam HEVC codec

Figure 2: Types of Inter prediction units in HEVC

For example a 16x16 predict unit can be further split up into 12x16 and 4x16 prediction units. To complement these bigger coding units,

HEVC

also supports larger square transform sizes with transform size support of up to 32x32. An 8-tap quarter sample filter adds to the overall compression efficiency through accurate prediction of sub-pixel motions. In addition to this

HEVC

also supports a new mode of prediction called ‘prediction unit merge’ which provides more flexibility for inter prediction unlike ‘skip’ and ‘direct’ modes of H264.
Even in Intra prediction mode,

HEVC

provides 35 modes of prediction by enabling a finer angular prediction (of 5.625 degrees compared to 22.5 degrees in H.264) leading to a more refined orientation of prediction.

Additional in-loop filtering options

To handle larger transform induced block coding artifacts,

HEVC

standard has an enhanced deblocking filter and also supports a new mode of in-loop filtering called the sample adaptive offset (SAO) filter. It supports two modes of filtering namely edge offset and band offset. SAO filter typically adds an offset to de-blocked pixel values either based on edge orientation or pixel level based band offset. The edge offset mode is used to handle edges and computes an offset based on the current and neighboring pixels and the edge orientation. Band offset is determined based on the signal amplitude and is usually applied for breaking any banding that might arise due to quantization.

Improved Parallel Processing

Advent of higher SIMD (Single Instruction and Multiple Data) architectures like AVX/AVX2.0 from Intel and NEON from ARM along with 64 bit processing has enabled better throughputs for complex pixel processing operations. Larger transform and prediction sizes can be programmed to have higher throughput on both load/stores as well as for arithmetic operations, enabling better processing on these architectures.

To enable parallel processing on architectures with multiple cores, processing elements,

HEVC

supports wavefront parallel processing (WPP) which enables parallelization of arithmetic entropy encode/decode within a frame. With WPP, each CTB row can be decoded in parallel after the current row CTB0 synchronizes the entropy states with CTB1 of top row as shown in Figure 3 below. At a certain time snapshot, say tn+2 the various CTBs that have been processed are shaded in grey in Figure 3 indicating that CTB0 of bottom row can be processed in parallel with CTB2 of middle row and CTB4 of top row. However note that top right dependency for intra or inter prediction still needs to be handled with WPP.
CTB0 CTB1 CTB2 CTB3 CTB4 CTB5 CTB6 CTB7 CTB8 CTB9
tn-2 tn-1 tn tn+1 tn+2 tn+3 tn+4 tn+5 tn+6 tn+7
tn tn+1 tn+2 tn+3 tn+4 tn+5 tn+6 tn+7 tn+8 tn+9
tn+2 tn+3 tn+4 tn+5 tn+6 tn+7 tn+8 tn+9 tn+9 tn+10

Figure 3: Indicates staggering in time of CTB processing across rows to handle prediction dependencies

In addition to this the standard also supports tiles where the entire frame can be divided up to into rectangular regions which are coded independent of each other. However,

HEVC

does not allow both tiles and WPP to be enabled simultaneously. These tools have accelerated software based

HEVC

codec development and is one of the primary reasons for software based encoders to lead the hardware based encoders in terms of availability and scalability.

HDR, Range Extensions and Color Primaries

With increase in resolutions in monitors and televisions, industry experts have long advocated for higher bit depths to realize a true high dynamic range picture. With most movies and TV series being originally shot in higher bit and chroma depths, there is now a possibility to bring that experience to the viewers through HDR support on displays.

HEVC

standard has comprehensive support for format range extensions (RExt) enabling higher dynamic range and color gamut in the form of support for up to 4:4:4 12 bit profiles for video as well as 4:4:4 16 bit Intra profile for still picture coding. Support for multi-view coding (MVC) as well as scalable coding modes (SHVC) are also included in the version 2 profiles [1], [2]. In most cases

UHD

/

4K

delivery is now embracing 10bit format to enable a true high resolution image experience. By keeping most of the tools same across Main and Main 10 profiles, the additional complexity in coding video with 10 bits/pixel is kept to the least possible enabling wider adoption of Main 10 profile.

HEVC

standard has support for signaling the color primaries for a wide variety of devices through BT.601 to the latest REC 2020 [3]. These color primaries are tuned to provide best user experience for a variety of newer consumption devices like

UHD

TVs, tablets, smartphones, monitors in addition to legacy devices like standard and high definition televisions. Also due to the multiple consumption devices being used for a single source, say like movies which are consumed not just on televisions but also on mobile, tablet and laptop screens, color primaries signaling support is a very important feature in any signal compression standard today.

HEVC

Adoption
Emergence of newer mechanisms of media delivery like OTT, IPTV, VOD, SVOD has been a key driver in adoption of

HEVC

. These new distribution mechanisms do not rely on traditional decoding infrastructure and can instead be decoded using mobile devices, laptops and over the top decoding apps on TVs, enabling widespread consumer adoption of this compression technology. Additionally RVU alliance [4] has published specifications for

4K

/

UHD

HEVC

support in TVs.
Ittiam and

HEVC

With deep investments in

HEVC

compression technologies since 2012,

HEVC

is at the core of several next generation products at Ittiam, spanning multiple markets. Taking advantage of the higher processing capability of machines, Ittiam’s software

H.265

encoders include Rate Distortion Optimal (RDO) based mode decisions in all its encoding variants to realize the full potential of

H.265

compression technology. Ittiam also has an optimal implementation of

H.265

decoder on ARM and x86 platforms targeting a wide variety of consumption devices from Integrated Receiver Decoders (IRDs) to tablets and mobile processors.

References:

  1. H.265

    .1: Conformance specification for ITU-T

    H.265

    high efficiency video coding – available for download at http://www.itu.int/rec/T-REC-

    H.265

    .1-201410-I/en
  2. BT.2020: Parameter values for ultra-high definition television systems for production and international programme exchange – Recommendation BT.2020-1 (06/2014) -available for download at http://www.itu.int/rec/R-REC-BT.2020-1-201406-I
Request Information

Contact

Products

Knowledge Center

Whitepapers

The Dash for HEVC
Read More

Datasheets

Ittiam HEVC Decoder (iPhone4S)
Ittiam HEVC Decoder (iPhone5)
Read More

Twitter IconLinkedinLinkedin