Reflections on the First MSU HEVC Encoder Comparison Study Report

Ittiam’s i265 HEVC encoder recognized as ‘pareto optimal’ in MSU report

Moscow State University’s HEVC/H.265 video encoder comparison study report was released to the public recently. This is one of the first independent evaluations of HEVC encoder offerings in the marketplace with a focus on file based transcoding that aimed at studying 3 segments (fast transcoding, universal transcoding, and high quality ripping) on 2 different platforms (one, a high end Intel dual Xeon E5 v3 server, and, the second an Intel Core-i7 4770R desktop/NUC grade device).
We participated in the comparison and shared our CPU only and GPU accelerated HEVC encoders with MSU. The comparison study report rates Ittiam as ‘pareto optimal’ (none faster AND better) in 5 out of the 6 segments.

How to interpret the results?

MSU might have hoped that encoder vendors would attempt to provide the best quality at the minimum speed requirements specified for the fast transcode and universal use-cases. But, in reality, the granularity of encoder presets, which each vendor had tuned to, has resulted in a wide variation of average speed (across a range of bit-rates) and average quality that makes independent comparison on just quality or speed vectors misleading. Hence, the speed/quality trade-off comparison results give a more balanced view of how a particular encoder is placed.

For example, switching a service to HEVC from AVC is not justified if a particular preset of a HEVC encoder offers the same compression efficiency possible with AVC (say, with x264 at a certain preset), but at a slower fps. Another recent blog has also emphasized a similar point. In fact, as many may know, extracting the last few % points of BDRATE improvement from any standard results in a substantial speed degradation (e.g., x264 “very slow” to “placebo” is 4x slower to get <5% BDRATE gain). On the other hand, fundamental improvements to coding tools (as in HEVC/VP9) should allow a similar compression efficiency to be reached without such exponential increase in computational complexity.

How did we fare?

Our HEVC encoder offerings were rated as ‘pareto optimal’ (i.e. no other encoder in the comparison was faster AND better) in 5 out of 6 categories. This directly translates to having an excellent balance between speed and quality. Since the speed cut-off requirement for the comparison study was different for various segments, this also demonstrates Ittiam’s speed-quality balance across multiple operating points.
Further, the distinct speed advantage of Ittiam encoders is clearly demonstrated in some of the speed/quality trade-off graphs shown below from the report.

The summary of how we performed relative to other key participants can be found below.

The graphs and tables show that the Ittiam GPU accelerated encoder is significantly faster than x264, x265, Intel, and Strongene encoders while being fairly close to them on BDRATE, making it highly suitable for real-time or high density encoding/transcoding use-cases. Ittiam software HEVC encoder is considerably faster than all encoders other than Strongene and Intel. However, when speed is not important, it still has some gap to close with some of the competitors.
The results are in line with how our customers view our relative position among the various HEVC encoder options available today. Based on our success in several codec shootouts and design wins for several customer products, we feel that the balance of speed and quality is one of the most critical factors for selecting a HEVC encoder in the Broadcast industry.
NOTE: Ittiam’s encoder had been configured for subjective quality rather than for SSIM or PSNR. Also, in the universal and fast transcode categories, the encoder was configured to run single pass CBR (as compared to 2-pass VBR being used for x264). So the results are in spite of any BDRATE discrepancies that the above could have caused.

Improvements since then:

Since the submission of our codecs to MSU in the March 2015 timeframe, there has been significant progress on the following fronts:

• Our GPU accelerated HEVC encoder now covers a wider range of encoding presets while supporting up to Main-10 profile. Particularly, the slower presets have become faster by ~2x with minimal BDRATE hit, making them suitable for on-demand use-cases with quicker turn-around time requirement. The fastest preset has become 25% faster with nearly no BDRATE hit, making it suitable for live 1080p60 encoding on a single Core i7-4770R or Xeon E3 1285v4.
• Using the GPU accelerated HEVC encoder as a building block, we have realized an efficient Main-10 Profile UHD encoder that concurrently utilizes multiple nodes of Intel’s recently announced Visual Computing Accelerator (VCA) cards that are installed on an Intel server. We demonstrated UHD@60fps live transcoding using 2 VCA cards at IBC using the fastest preset, while the slower presets can be used for faster transcoding of UHD content for on-demand delivery.
• Ittiam software HEVC encoder runs 20% faster on servers with 20+ cores (such as the E5 2697v3 used in the study) through improved parallelization techniques.
• All variants of software-only and GPU accelerated encoders now support Ittiam TRAQO technology, which intelligently adapts the quantizer over the spatio-temporal volume to offer average bit-rate savings in the 5% range with tougher segments having up to 25% savings.

Observations on the methodology and presentation of results:

• To assess speed ratio between 2 encoders at the same quality (or) bit-rate savings between 2 encoders at the same speed, it is useful to have the compression gain and speed data on a given platform for multiple presets of those 2 encoders.
• Ensuring that all encoders are configured with similar intra period requirements (preferably, strict IDR period), and 2-pass VBR with uniform buffer-size and maximum bit-rate settings would make the comparisons fair.

We have shared some of the observations and feedback with MSU and we hope that future comparisons will address these.

Conclusions

In summary, the study has confirmed that Ittiam’s HEVC encoders are well tuned in most categories when it comes to speed vs. compression trade-off (pareto optimal) making them well suited for real-time and high density needs. The report further indicates that HEVC encoders have matured enough to surpass the golden reference of x264 (in terms of compression gain possible at a given computing power). With the improvements made since then, and with some of the methodology observations above, we feel that Ittiam offerings are well placed to fuel large scale adoption across live and on-demand segments.