Reflections on the First MSU HEVC Encoder Comparison Study Report
by Dr. Sriram Sethuraman (CTO)
Ittiam’s i265 HEVC encoder recognized as ‘pareto optimal’ in MSU report
Moscow State University’s HEVC/H.265 video encoder comparison study report was released to the public recently. This is one of the first independent evaluations of HEVC encoder offerings in the marketplace with a focus on file based transcoding that aimed at studying 3 segments (fast transcoding, universal transcoding, and high quality ripping) on 2 different platforms (one, a high end Intel dual Xeon E5 v3 server, and, the second an Intel Core-i7 4770R desktop/NUC grade device).
We participated in the comparison and shared our CPU only and GPU accelerated HEVC encoders with MSU. The comparison study report rates Ittiam as ‘pareto optimal’ (none faster AND better) in 5 out of the 6 segments.
MSU might have hoped that encoder vendors would attempt to provide the best quality at the minimum speed requirements specified for the fast transcode and universal use-cases. But, in reality, the granularity of encoder presets, which each vendor had tuned to, has resulted in a wide variation of average speed (across a range of bit-rates) and average quality that makes independent comparison on just quality or speed vectors misleading. Hence, the speed/quality trade-off comparison results give a more balanced view of how a particular encoder is placed.
For example, switching a service to HEVC from AVC is not justified if a particular preset of a HEVC encoder offers the same compression efficiency possible with AVC (say, with x264 at a certain preset), but at a slower fps. Another recent blog has also emphasized a similar point. In fact, as many may know, extracting the last few % points of BDRATE improvement from any standard results in a substantial speed degradation (e.g., x264 “very slow” to “placebo” is 4x slower to get <5% BDRATE gain). On the other hand, fundamental improvements to coding tools (as in HEVC/VP9) should allow a similar compression efficiency to be reached without such exponential increase in computational complexity.
Our HEVC encoder offerings were rated as ‘pareto optimal’ (i.e. no other encoder in the comparison was faster AND better) in 5 out of 6 categories. This directly translates to having an excellent balance between speed and quality. Since the speed cut-off requirement for the comparison study was different for various segments, this also demonstrates Ittiam’s speed-quality balance across multiple operating points.
Further, the distinct speed advantage of Ittiam encoders is clearly demonstrated in some of the speed/quality trade-off graphs shown below from the report.
The summary of how we performed relative to other key participants can be found below.
|Ittiam GPU accelerated
compared with others
|In Desktop Universal Category||In Desktop Fast Transcode Category|
was faster by
|@ BDRATE hit of||Ittiam Encoder
was faster by
|@ BDRATE hit of|
|Intel MSS HEVC Gacc||120%||13%||29%||20%|
compared with others
|In Desktop Ripping Category|
was faster by
|x264||100%||Savings of 1%|
|Strongene||0%||Savings of 4%|
|Intel MSS HEVC Software||240%||Increase of 14%|
|x265||550%||Increase of 21%|
|LibVP9||30%||Increase of 16%|
The graphs and tables show that the Ittiam GPU accelerated encoder is significantly faster than x264, x265, Intel, and Strongene encoders while being fairly close to them on BDRATE, making it highly suitable for real-time or high density encoding/transcoding use-cases. Ittiam software HEVC encoder is considerably faster than all encoders other than Strongene and Intel. However, when speed is not important, it still has some gap to close with some of the competitors.
The results are in line with how our customers view our relative position among the various HEVC encoder options available today. Based on our success in several codec shootouts and design wins for several customer products, we feel that the balance of speed and quality is one of the most critical factors for selecting a HEVC encoder in the Broadcast industry.
NOTE: Ittiam’s encoder had been configured for subjective quality rather than for SSIM or PSNR. Also, in the universal and fast transcode categories, the encoder was configured to run single pass CBR (as compared to 2-pass VBR being used for x264). So the results are in spite of any BDRATE discrepancies that the above could have caused.
Since the submission of our codecs to MSU in the March 2015 timeframe, there has been significant progress on the following fronts:
We have shared some of the observations and feedback with MSU and we hope that future comparisons will address these.
In summary, the study has confirmed that Ittiam’s HEVC encoders are well tuned in most categories when it comes to speed vs. compression trade-off (pareto optimal) making them well suited for real-time and high density needs. The report further indicates that HEVC encoders have matured enough to surpass the golden reference of x264 (in terms of compression gain possible at a given computing power). With the improvements made since then, and with some of the methodology observations above, we feel that Ittiam offerings are well placed to fuel large scale adoption across live and on-demand segments.